---
title: "IndicLLMSuite — paper by Khan et al. (2024)"
description: "The authors offer the recipe as a template that other under-resourced languages can reuse. 76 references, cited by 31 papers here."
authors: ["Mohammed Safi Ur Rahman Khan", "Priyam Mehta", "Ananth Sankar", "Umashankar Kumaravelan", "Sumanth Doddapaneni", "Suriyaprasaad B", "Varun G", "Sparsh Jain", "Anoop Kunchukuttan", "Pratyush Kumar", "Raj Dabre", "Mitesh M. Khapra"]
year: 2024
venue: "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2024.acl-long.843/"
doi: "10.18653/v1/2024.acl-long.843"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/khan-2024-indicllmsuite-blueprint-creating-pre-training/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/khan-2024-indicllmsuite-blueprint-creating-pre-training/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/8/) / IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languages

# IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languages

**Authors:** [Mohammed Safi Ur Rahman Khan](https://md.research.elamur.ai/free/en/authors/mohammed-safi-ur-rahman-khan/), Priyam Mehta, Ananth Sankar, Umashankar Kumaravelan, [Sumanth Doddapaneni](https://md.research.elamur.ai/free/en/authors/sumanth-doddapaneni/), Suriyaprasaad B, Varun G, Sparsh Jain, [Anoop Kunchukuttan](https://md.research.elamur.ai/free/en/authors/anoop-kunchukuttan/), [Pratyush Kumar](https://md.research.elamur.ai/free/en/authors/pratyush-kumar/), [Raj Dabre](https://md.research.elamur.ai/free/en/authors/raj-dabre/) and [Mitesh M. Khapra](https://md.research.elamur.ai/free/en/authors/mitesh-m-khapra/) · **Year:** 2024 · **Venue:** [Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2024](https://md.research.elamur.ai/free/en/venues/acl/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.acl-long.843/) · [PDF](https://aclanthology.org/2024.acl-long.843.pdf) · [DOI 10.18653/v1/2024.acl-long.843](https://doi.org/10.18653/v1/2024.acl-long.843) · [OpenAlex](https://openalex.org/W4402671529)

**Citations:** 17 citations (OpenAlex, 2026-10-14) · [31](https://md.research.elamur.ai/free/en/papers/khan-2024-indicllmsuite-blueprint-creating-pre-training/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Open LLMs have advanced quickly for English, but the 22 languages listed in India's 8th constitutional schedule have little presence in current training data and tokenizers. Few open pipelines exist to collect text in these languages from the web, scanned books or audio. Instruction data is also scarce, because there are no strong Indic LLMs to generate it. The authors respond with IndicLLMSuite, an openly licensed collection of resources:

* **Sangraha**, a pre-training corpus that mixes three kinds of text:
  * human-verified websites,
  * OCR-processed PDFs and transcribed speech,
  * existing multilingual corpora and large-scale machine translation.
* **Setu**, a Spark-based pipeline that handles extraction, cleaning, toxicity filtering and deduplication.
* **IndicAlign**, fine-tuning data assembled in four ways:
  * merging existing datasets,
  * translating English ones,
  * generating Wikipedia-grounded dialogues with LLaMa2 and Mixtral,
  * crowdsourcing prompts.

  It also includes a safety subset of toxic prompts paired with harmless replies.

In total the suite reaches 251B tokens and about 74.8M instruction-response pairs. The authors offer the recipe as a template that other under-resourced languages can reuse.

## Key points
- Sangraha holds roughly 251B tokens across 22 languages, split by origin:
  - Verified: about 64.3B tokens.
  - Synthetic (translated): about 162.7B tokens.
  - Unverified: about 24.3B tokens.
- Verified content comes from manually checked websites, PDFs digitized with GCP's Vision OCR, and transcripts such as OpenSubtitles, NPTEL and Mann ki Baat.
- IndicAlign-Instruct has about 74.7M prompt-response pairs, with English sets translated into 14 languages and crowdsourced prompts gathered via the Anudesh platform.
- IndicAlign-Toxic contains about 123K toxic-prompt/safe-response pairs, generated with an aligned LLaMa2 and translated into 14 languages.
- By MTLD score, the Wiki-Chat prompts show lexical diversity comparable to human-written sets such as Dolly. The code is released under the MIT License.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 378 works · 161,630 citations
- [Indian Institute of Technology Madras](https://md.research.elamur.ai/free/en/universities/indian-institute-of-technology-madras/) — University, Chennai, India · 16 works · 2,499 citations
- [National Institute of Information and Communications Technology](https://md.research.elamur.ai/free/en/labs/national-institute-of-information-and-communications-technology/) — Research institute, Tokyo, Japan · 24 works · 2,090 citations

Also on the byline: [Sant Longowal Institute of Engineering and Technology](https://ror.org/01n15vy71).

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Mohammed Safi Ur Rahman Khan, Priyam Mehta, Ananth Sankar, Umashankar Kumaravelan, Sumanth Doddapaneni, Suriyaprasaad B, Varun G, Sparsh Jain, Anoop Kunchukuttan, Pratyush Kumar, Raj Dabre and Mitesh M. Khapra. 2024. IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languages. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{khan2024indicllmsuite,
  title = {{IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languages}},
  author = {Mohammed Safi Ur Rahman Khan, Priyam Mehta, Ananth Sankar, Umashankar Kumaravelan, Sumanth Doddapaneni, Suriyaprasaad B, Varun G, Sparsh Jain, Anoop Kunchukuttan, Pratyush Kumar, Raj Dabre and Mitesh M. Khapra},
  booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.843},
  url = {https://doi.org/10.18653/v1/2024.acl-long.843}
}
```

- [← previous Attribute First, then Generate: Locally-attributable Grounded Text Generation](https://md.research.elamur.ai/free/en/papers/slobodkin-2024-attribute-first-then-generate-locally/)
- [next → Puzzle Solving using Reasoning of Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/giadikiaroglou-2024-puzzle-solving-reasoning-large-language/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/8/)

## References

76 references, 36 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Julien Abadji, Pedro Ortiz Suarez, Laurent Romary, and Benoît Sagot. 2022. [Towards a cleaner document-oriented multilingual crawled corpus](https://md.research.elamur.ai/free/en/papers/abadji-2022-towards-cleaner-document-oriented-multilingual/). In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 4344–4355, Marseille, France. European Language Resources Association.
2. Amro Abbas, Kushal Tirumala, Daniel Simig, Surya Ganguli, and Ari S. Morcos. 2023. [Semdedup: Data-efficient learning at web-scale through semantic deduplication](https://md.research.elamur.ai/free/en/papers/abbas-2023-semdedup-data-efficient-learning-web/). CoRR, abs/2303.09540.
3. Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. 2023. [The falcon series of open language mo…](https://md.research.elamur.ai/free/en/papers/almazrouei-2023-falcon-series-open-language-models/)
4. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Sh… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Saurav+Kadavath%2C+Jackson+Kernion%2C+Tom+Conerly%2C+)
5. Adrien Barbaresi. 2021a. [Trafilatura: A web scraping library and command-line tool for text discovery and extraction](https://md.research.elamur.ai/free/en/papers/barbaresi-2021-trafilatura-web-scraping-library-command/). In Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Pro…
6. Adrien Barbaresi. 2021b. [Trafilatura: A web scraping library and command-line tool for text discovery and extraction](https://md.research.elamur.ai/free/en/papers/barbaresi-2021-trafilatura-web-scraping-library-command/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstr…
7. Pushpak Bhattacharyya. 2010. [find](https://scholar.google.com/scholar?q=Pushpak+Bhattacharyya%2E+2010%2E)
8. Indowordnet. In Lexical Resources Engineering Conference 2010 (LREC 2010), Malta. [find](https://scholar.google.com/scholar?q=Indowordnet%2E+In+Lexical+Resources+Engineering+Conference+2010+%28LREC+2010%29%2C+Malta%2E)
9. Kaushal Bhogale, Deovrat Mehendale, and Tahir Javed. 2024. Mahadhwani. [https://github.com/](https://github.com/) AI4Bharat/Mahadhwani. [↗](https://github.com/)
10. Kaushal Bhogale, Abhigyan Raman, Tahir Javed, Sumanth Doddapaneni, Anoop Kunchukuttan, Pratyush Kumar, and Mitesh M. Khapra. 2023a. Effectiveness of mining audio and text pairs from public data for improving asr systems for low-resource languages. In ICASSP 2023 - 2023 IEEE International Conference… [find](https://scholar.google.com/scholar?q=Kaushal+Bhogale%2C+Abhigyan+Raman%2C+Tahir+Javed%2C+Sumanth+Doddapaneni%2C+Anoop+Kunchukuttan%2C+Pratyush+Kumar%2C+and+Mitesh+M%2E+Khapra%2E+2023a%2E+Effectiveness+of+mining+audio+and+text+pairs+from+public+data+for+im)
11. Kaushal Bhogale, Sairam Sundaresan, A. Raman, Tahir Javed, Mitesh M. Khapra, and Pratyush Kumar. 2023b. Vistaar: Diverse benchmarks and training sets for indian language asr. INTERSPEECH. [find](https://scholar.google.com/scholar?q=Kaushal+Bhogale%2C+Sairam+Sundaresan%2C+A%2E+Raman%2C+Tahir+Javed%2C+Mitesh+M%2E+Khapra%2C+and+Pratyush+Kumar%2E+2023b%2E+Vistaar%3A+Diverse+benchmarks+and+training+sets+for+indian+language+asr%2E+INTERSPEECH%2E)
12. Andrei Z. Broder. 1997. On the resemblance and containment of documents. In Compression and Complexity of SEQUENCES 1997, Positano, Amalfitan Coast, Salerno, Italy, June 11-13, 1997, Proceedings, pages 21–29. IEEE. [find](https://scholar.google.com/scholar?q=Andrei+Z%2E+Broder%2E+1997%2E+On+the+resemblance+and+containment+of+documents%2E+In+Compression+and+Complexity+of+SEQUENCES+1997%2C+Positano%2C+Amalfitan+Coast%2C+Salerno%2C+Italy%2C+June+11%2D13%2C+1997%2C+Proceedings%2C+page)
13. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
14. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramèr, and Chiyuan Zhang. 2023. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net. [find](https://scholar.google.com/scholar?q=Nicholas+Carlini%2C+Daphne+Ippolito%2C+Matthew+Jagielski%2C+Katherine+Lee%2C+Florian+Tram%C3%A8r%2C+and+Chiyuan+Zhang%2E+2023%2E+Quantifying+memorization+across+neural+language+models%2E+In+The+Eleventh+International+Conf)
15. Moses Charikar. 2002. [Similarity estimation techniques from rounding algorithms](https://md.research.elamur.ai/free/en/papers/charikar-2002-similarity-estimation-techniques-rounding/). In Proceedings on 34th Annual ACM Symposium on Theory of Computing, May 19-21, 2002, Montréal, Québec, Canada, pages 380–388. ACM.
16. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. [Unsupervised cross-lingual representation learning at scale](https://md.research.elamur.ai/free/en/papers/conneau-2020-unsupervised-cross-lingual-representation-learning/). In Proceedings of the 58th Annual Meeting of the Association fo…
17. Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. Free dolly: Introducing the world’s first truly open instruction-tuned llm. [find](https://scholar.google.com/scholar?q=Mike+Conover%2C+Matt+Hayes%2C+Ankit+Mathur%2C+Jianwei+Xie%2C+Jun+Wan%2C+Sam+Shah%2C+Ali+Ghodsi%2C+Patrick+Wendell%2C+Matei+Zaharia%2C+and+Reynold+Xin%2E+2023%2E+Free+dolly%3A+Introducing+the+world%E2%80%99s+first+truly+open+instruct)
18. Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loïc Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semar… [find](https://scholar.google.com/scholar?q=Marta+R%2E+Costa%2Djuss%C3%A0%2C+James+Cross%2C+Onur+%C3%87elebi%2C+Maha+Elbayad%2C+Kenneth+Heafield%2C+Kevin+Heffernan%2C+Elahe+Kalbassi%2C+Janice+Lam%2C+Daniel+Licht%2C+Jean+Maillard%2C+Anna+Sun%2C+Skyler+Wang%2C+Guillaume+Wenzek%2C+Al+Yo)
19. Cem Dilmegani. 2023. Ocr in 2024: Benchmarking text extraction/capture accuracy. [find](https://scholar.google.com/scholar?q=Cem+Dilmegani%2E+2023%2E+Ocr+in+2024%3A+Benchmarking+text+extraction%2Fcapture+accuracy%2E)
20. Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023a. [Enhancing chat language models by scaling high-quality instructional conversations](https://md.research.elamur.ai/free/en/papers/ding-2023-enhancing-chat-language-models-scaling/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singap…
21. Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023b. [Enhancing chat language models by scaling high-quality instructional conversations](https://md.research.elamur.ai/free/en/papers/ding-2023-enhancing-chat-language-models-scaling/).
22. Sumanth Doddapaneni, Rahul Aralikatte, Gowtham Ramesh, Shreya Goyal, Mitesh M. Khapra, Anoop Kunchukuttan, and Pratyush Kumar. 2023. [Towards leaving no Indic language behind: Building monolingual corpora, benchmark and models for Indic languages](https://md.research.elamur.ai/free/en/papers/doddapaneni-2023-leaving-no-indic-language-behind/). In Proceedings of the 61st Annual Meeting of the Asso…
23. István Endrédy and Attila Novák. 2013. More effective boilerplate removal - the goldminer algorithm. Polibits, 48:79–83. [find](https://scholar.google.com/scholar?q=Istv%C3%A1n+Endr%C3%A9dy+and+Attila+Nov%C3%A1k%2E+2013%2E+More+effective+boilerplate+removal+%2D+the+goldminer+algorithm%2E+Polibits%2C+48%3A79%E2%80%9383%2E)
24. Christiane Fellbaum, editor. 1998. WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Christiane+Fellbaum%2C+editor%2E+1998%2E+WordNet%3A+An+Electronic+Lexical+Database%2E+MIT+Press%2C+Cambridge%2C+MA%2E)
25. Jay Gala, Pranjal A Chitale, A K Raghavan, Varun Gumma, Sumanth Doddapaneni, Aswanth Kumar M, Janki Atul Nawale, Anupama Sujatha, Ratish Puduppully, Vivek Raghavan, Pratyush Kumar, Mitesh M Khapra, Raj Dabre, and Anoop Kunchukuttan. 2023. [Indictrans2: Towards high-quality and accessible machine tran…](https://md.research.elamur.ai/free/en/papers/gala-2023-indictrans2-towards-high-quality-accessible/)
26. Jay Gala, Thanmay Jayakumar, Jaavid Aktar Husain, Aswanth Kumar M, Mohammed Safi Ur Rahman Khan, Diptesh Kanojia, Ratish Puduppully, Mitesh M. Khapra, Raj Dabre, Rudra Murthy, and Anoop Kunchukuttan. 2024. [Airavata: Introducing hindi instruction-tuned llm](https://md.research.elamur.ai/free/en/papers/gala-2024-airavata-introducing-hindi-instruction-tuned/).
27. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2021. The pile: An 800gb dataset of diverse text for language modeling. CoRR, abs/2101.00027. [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Stella+Biderman%2C+Sid+Black%2C+Laurence+Golding%2C+Travis+Hoppe%2C+Charles+Foster%2C+Jason+Phang%2C+Horace+He%2C+Anish+Thite%2C+Noa+Nabeshima%2C+Shawn+Presser%2C+and+Connor+Leahy%2E+2021%2E+The+pile%3A+An+800gb+datas)
28. Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harkirat Singh Behl, Xin Wang, Sébastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, and Yuanzhi… [find](https://scholar.google.com/scholar?q=Suriya+Gunasekar%2C+Yi+Zhang%2C+Jyoti+Aneja%2C+Caio+C%C3%A9sar+Teodoro+Mendes%2C+Allie+Del+Giorno%2C+Sivakanth+Gopi%2C+Mojan+Javaheripi%2C+Piero+Kauffmann%2C+Gustavo+de+Rosa%2C+Olli+Saarikivi%2C+Adil+Salim%2C+Shital+Shah%2C+Harki)
29. Anirudh Gupta, Neeraj Chhimwal, Ankur Dhuriya, Rishabh Gaur, Priyanshi Shah, Harveen Singh Chadha, and Vivek Raghavan. 2022. indic-punct: An automatic punctuation restoration and inverse text normalization framework for indic languages. [find](https://scholar.google.com/scholar?q=Anirudh+Gupta%2C+Neeraj+Chhimwal%2C+Ankur+Dhuriya%2C+Rishabh+Gaur%2C+Priyanshi+Shah%2C+Harveen+Singh+Chadha%2C+and+Vivek+Raghavan%2E+2022%2E+indic%2Dpunct%3A+An+automatic+punctuation+restoration+and+inverse+text+normaliz)
30. Nizar Habash, Muhammed AbuOdeh, Dima Taji, Reem Faraj, Jamila El Gizuli, and Omar Kallas. 2022. [Camel treebank: An open multi-genre Arabic dependency treebank](https://md.research.elamur.ai/free/en/papers/habash-2022-camel-treebank-open-multi-genre/). In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 2672–2681, Marseille, France. European Language Resourc…
31. Kenneth Heafield. 2011. [KenLM: Faster and smaller language model queries](https://md.research.elamur.ai/free/en/papers/heafield-2011-kenlm-faster-smaller-language-model/). In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 187–197, Edinburgh, Scotland. Association for Computational Linguistics.
32. Danny Hernandez, Tom B. Brown, Tom Conerly, Nova DasSarma, Dawn Drain, Sheer El Showk, Nelson Elhage, Zac Hatfield-Dodds, Tom Henighan, Tristan Hume, Scott Johnston, Benjamin Mann, Chris Olah, Catherine Olsson, Dario Amodei, Nicholas Joseph, Jared Kaplan, and Sam McCandlish. 2022. [Scaling laws and i…](https://md.research.elamur.ai/free/en/papers/hernandez-2022-scaling-laws-interpretability-learning-repeated/)
33. Jaavid Aktar Husain, Raj Dabre, Aswanth Kumar, Ratish Puduppully, and Anoop Kunchukuttan. 2024. Romansetu: Efficiently unlocking multilingual capabilities of large language models models via romanization. [find](https://scholar.google.com/scholar?q=Jaavid+Aktar+Husain%2C+Raj+Dabre%2C+Aswanth+Kumar%2C+Ratish+Puduppully%2C+and+Anoop+Kunchukuttan%2E+2024%2E+Romansetu%3A+Efficiently+unlocking+multilingual+capabilities+of+large+language+models+models+via+romanizat)
34. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
35. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Antoine+Roux%2C+Arthur+Mensch%2C+Blanche+Savary%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+Las+Casas%2C+Emma+Bou+Hanna%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guill)
36. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Antoine+Roux%2C+Arthur+Mensch%2C+Blanche+Savary%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Emma+Bou+Hanna%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guill)
37. Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, Gokul N.C., Avik Bhattacharyya, Mitesh M. Khapra, and Pratyush Kumar. 2020. [IndicNLPSuite: Monolingual corpora, evaluation benchmarks and pre-trained multilingual language models for Indian languages](https://md.research.elamur.ai/free/en/papers/kakwani-2020-indicnlpsuite-monolingual-corpora-evaluation-benchmarks/). In Findings of the Association for Computation…
38. Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi-Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Richárd Nagyfi, Shahul ES, Sameer Suri, David Glushkov, Arnav Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, and Alexander Mattick. 2023.… — [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://md.research.elamur.ai/free/en/papers/kopf-2023-openassistant-conversations-democratizing-large-language/)
39. Julia Kreutzer, Isaac Caswell, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-Orshikh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, Monang Setyawan, Supheakmungkol Sarin, Sokhar Samb, Benoît Sagot, Clara Rivera, Annette Rios, Isabel Papadimitriou, Salomey Osei, Pedro… — [Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets](https://md.research.elamur.ai/free/en/papers/kreutzer-2021-quality-glance-audit-web-crawled/)
40. Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Christopher A. Choquette-Choo, Katherine Lee, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, and Orhan Firat. 2023. [Madlad-400: A multilingual and document-level large audited dataset](https://md.research.elamur.ai/free/en/papers/kudugunta-2023-madlad-400-multilingual-document-level/).
41. Anoop Kunchukuttan. 2020. [The indicnlp library](https://md.research.elamur.ai/free/en/papers/li-2023-camel-communicative-agents-mind/). [https://github.com/anoopkunchukuttan/](https://github.com/anoopkunchukuttan/) indic_nlp_library/blob/master/docs/ indicnlp.pdf. 2023a. CAMEL: communicative agents for "mind" exploration of large scale language model society. CoRR, abs/2303.17760. [↗](https://github.com/anoopkunchukuttan/)
42. Haonan Li, Fajri Koto, Minghao Wu, Alham Fikri Aji, and Timothy Baldwin. 2023b. [Bactrian-x: Multilingual replicable instruction-following models with low-rank adaptation](https://md.research.elamur.ai/free/en/papers/li-2023-bactrian-x-multilingual-replicable-instruction/).
43. Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. 2023c. [Textbooks are all you need ii: phi-1.5 technical report](https://md.research.elamur.ai/free/en/papers/li-2023-textbooks-all-need-ii-phi/).
44. Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, and Adam Roberts. 2023. [The flan collection: Designing data and methods for effective instruction tuning](https://md.research.elamur.ai/free/en/papers/longpre-2023-flan-collection-designing-data-methods/).
45. Edward Loper and Steven Bird. 2002. Nltk: The natural language toolkit. [find](https://scholar.google.com/scholar?q=Edward+Loper+and+Steven+Bird%2E+2002%2E+Nltk%3A+The+natural+language+toolkit%2E)
46. Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi-Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Richárd Nagyfi, Shahul ES, Sameer Suri, David Glushkov, Arnav Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, and Alexander Mattick. 2023.… — [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://md.research.elamur.ai/free/en/papers/kopf-2023-openassistant-conversations-democratizing-large-language/)
47. Yash Madhani, Mitesh M. Khapra, and Anoop Kunchukuttan. 2023a. Bhasa-abhijnaanam: Native-script and romanized language identification for 22 Indic languages. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 816–826, Toronto, C… [find](https://scholar.google.com/scholar?q=Yash+Madhani%2C+Mitesh+M%2E+Khapra%2C+and+Anoop+Kunchukuttan%2E+2023a%2E+Bhasa%2Dabhijnaanam%3A+Native%2Dscript+and+romanized+language+identification+for+22+Indic+languages%2E+In+Proceedings+of+the+61st+Annual+Meeting+)
48. Hugo Laurençon, Lucile Saulnier, Thomas Wang, Christopher Akiki, Albert Villanova del Moral, Teven Le Scao, Leandro Von Werra, Chenghao Mou, Eduardo González Ponferrada, Huu Nguyen, Jörg Frohberg, Mario Šaško, Quentin Lhoest, Angelina McMillan-Major, Gerard Dupont, Stella Biderman, Anna Rogers, Loub… — [The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset](https://md.research.elamur.ai/free/en/papers/lauren-ccon-2023-bigscience-roots-corpus-1-6tb/)
49. Yash Madhani, Sushane Parthan, Priyanka Bedekar, Gokul Nc, Ruchi Khapra, Anoop Kunchukuttan, Pratyush Kumar, and Mitesh Khapra. 2023b. Aksharantar: Open Indic-language transliteration datasets and models for the next billion users. In Findings of the Association for Computational Linguistics: EMNLP… [find](https://scholar.google.com/scholar?q=Yash+Madhani%2C+Sushane+Parthan%2C+Priyanka+Bedekar%2C+Gokul+Nc%2C+Ruchi+Khapra%2C+Anoop+Kunchukuttan%2C+Pratyush+Kumar%2C+and+Mitesh+Khapra%2E+2023b%2E+Aksharantar%3A+Open+Indic%2Dlanguage+transliteration+datasets+and+mod)
50. Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. 2022. Deduplicating training data makes language models better. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),… [find](https://scholar.google.com/scholar?q=Katherine+Lee%2C+Daphne+Ippolito%2C+Andrew+Nystrom%2C+Chiyuan+Zhang%2C+Douglas+Eck%2C+Chris+Callison%2DBurch%2C+and+Nicholas+Carlini%2E+2022%2E+Deduplicating+training+data+makes+language+models+better%2E+In+Proceedings+o)
51. Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. [find](https://scholar.google.com/scholar?q=Guohao+Li%2C+Hasan+Abed+Al+Kader+Hammoud%2C+Hani+Itani%2C+Dmitrii+Khizbullin%2C+and+Bernard+Ghanem%2E)
52. Philip M. McCarthy and Scott Jarvis. 2010. Mtld, vocdd, and hd-d: A validation study of sophisticated approaches to lexical diversity assessment. [find](https://scholar.google.com/scholar?q=Philip+M%2E+McCarthy+and+Scott+Jarvis%2E+2010%2E+Mtld%2C+vocdd%2C+and+hd%2Dd%3A+A+validation+study+of+sophisticated+approaches+to+lexical+diversity+assessment%2E)
53. Dipak Narayan, Debasri Chakrabarti, Prabhakar Pande, and Pushpak Bhattacharyya. 2002. An experience in building the indo wordnet-a wordnet for hindi. In First International Conference on Global WordNet, Mysore, India. [find](https://scholar.google.com/scholar?q=Dipak+Narayan%2C+Debasri+Chakrabarti%2C+Prabhakar+Pande%2C+and+Pushpak+Bhattacharyya%2E+2002%2E+An+experience+in+building+the+indo+wordnet%2Da+wordnet+for+hindi%2E+In+First+International+Conference+on+Global+WordNe)
54. Thuat Nguyen, Chien Van Nguyen, Viet Dac Lai, Hieu Man, Nghia Trung Ngo, Franck Dernoncourt, Ryan A. Rossi, and Thien Huu Nguyen. 2023a. [Culturax: A cleaned, enormous, and multilingual dataset for large language models in 167 languages](https://md.research.elamur.ai/free/en/papers/nguyen-2023-culturax-cleaned-enormous-multilingual-dataset/).
55. Xuan-Phi Nguyen, Wenxuan Zhang, Xin Li, Mahani Aljunied, Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang, Chaoqun Liu, Hang Zhang, and Lidong Bing. 2023b. [Seallms – large language models for southeast asia](https://md.research.elamur.ai/free/en/papers/nguyen-2023-seallms-large-language-models-southeast/).
56. Ritesh Panjwani, Diptesh Kanojia, and Pushpak Bhattacharyya. 2018. pyiwn: A python based API to access Indian language WordNets. In Proceedings of the 9th Global Wordnet Conference, pages 378–383, Nanyang Technological University (NTU), Singapore. Global Wordnet Association. [find](https://scholar.google.com/scholar?q=Ritesh+Panjwani%2C+Diptesh+Kanojia%2C+and+Pushpak+Bhattacharyya%2E+2018%2E+pyiwn%3A+A+python+based+API+to+access+Indian+language+WordNets%2E+In+Proceedings+of+the+9th+Global+Wordnet+Conference%2C+pages+378%E2%80%93383%2C+Nan)
57. Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023. The refinedweb dataset for falcon LLM: outperforming curated corpora with web data, and web data only. CoRR, abs/2306.01116. [find](https://scholar.google.com/scholar?q=Guilherme+Penedo%2C+Quentin+Malartic%2C+Daniel+Hesslow%2C+Ruxandra+Cojocaru%2C+Alessandro+Cappelli%2C+Hamza+Alobeidli%2C+Baptiste+Pannier%2C+Ebtesam+Almazrouei%2C+and+Julien+Launay%2E+2023%2E+The+refinedweb+dataset+for+f)
58. Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen… [find](https://scholar.google.com/scholar?q=Jack+W%2E+Rae%2C+Sebastian+Borgeaud%2C+Trevor+Cai%2C+Katie+Millican%2C+Jordan+Hoffmann%2C+Francis+Song%2C+John+Aslanides%2C+Sarah+Henderson%2C+Roman+Ring%2C+Susannah+Young%2C+Eliza+Rutherford%2C+Tom+Hennigan%2C+Jacob+Menick%2C+A)
59. Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, H. Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-S… [find](https://scholar.google.com/scholar?q=Jack+W%2E+Rae%2C+Sebastian+Borgeaud%2C+Trevor+Cai%2C+Katie+Millican%2C+Jordan+Hoffmann%2C+H%2E+Francis+Song%2C+John+Aslanides%2C+Sarah+Henderson%2C+Roman+Ring%2C+Susannah+Young%2C+Eliza+Rutherford%2C+Tom+Hennigan%2C+Jacob+Menick)
60. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). J. Mach. Learn. Res., 21:140:1–140:67.
61. Scrapinghub. 2021. Article extraction benchmark. GitHub repository. [find](https://scholar.google.com/scholar?q=Scrapinghub%2E+2021%2E+Article+extraction+benchmark%2E+GitHub+repository%2E)
62. Shashank Siripragada, Jerin Philip, Vinay P. Namboodiri, and C V Jawahar. 2020. A multilingual parallel corpora collection effort for Indian languages. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 3743–3751, Marseille, France. European Language Resources Associat… [find](https://scholar.google.com/scholar?q=Shashank+Siripragada%2C+Jerin+Philip%2C+Vinay+P%2E+Namboodiri%2C+and+C+V+Jawahar%2E+2020%2E+A+multilingual+parallel+corpora+collection+effort+for+Indian+languages%2E+In+Proceedings+of+the+Twelfth+Language+Resources)
63. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca). [↗](https://github.com/tatsu-lab/stanford_alpaca)
64. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023a. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/).
65. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
66. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023a. Self-instruct: Aligning language models with self-generated instructions. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A%2E+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2023a%2E+Self%2Dinstruct%3A+Aligning+language+models+with+self%2Dgenerated+instructions%2E)
67. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023b. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long… [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A%2E+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2023b%2E+Self%2Dinstruct%3A+Aligning+language+models+with+self%2Dgenerated+instructions%2E+In+Pro)
68. Xiangpeng Wei, Haoran Wei, Huan Lin, Tianhao Li, Pei Zhang, Xingzhang Ren, Mei Li, Yu Wan, Zhiwei Cao, Binbin Xie, Tianxiang Hu, Shangjie Li, Binyuan Hui, Bowen Yu, Dayiheng Liu, Baosong Yang, Fei Huang, and Jun Xie. 2023. [Polylm: An open source polyglot large language model](https://md.research.elamur.ai/free/en/papers/wei-2023-polylm-open-source-polyglot-large/). arXiv preprint arXiv: 2…
69. Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, and Edouard Grave. 2020. [CCNet: Extracting high quality monolingual datasets from web crawl data](https://md.research.elamur.ai/free/en/papers/wenzek-2020-ccnet-extracting-high-quality-monolingual/). In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4003–4012, Mar…
70. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023a. [Wizardlm: Empowering large language models to follow complex instructions](https://md.research.elamur.ai/free/en/papers/xu-2023-wizardlm-empowering-large-pre-trained/). CoRR, abs/2304.12244.
71. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023b. Wizardlm: Empowering large language models to follow complex instructions. [find](https://scholar.google.com/scholar?q=Can+Xu%2C+Qingfeng+Sun%2C+Kai+Zheng%2C+Xiubo+Geng%2C+Pu+Zhao%2C+Jiazhan+Feng%2C+Chongyang+Tao%2C+and+Daxin+Jiang%2E+2023b%2E+Wizardlm%3A+Empowering+large+language+models+to+follow+complex+instructions%2E)
72. Canwen Xu, Daya Guo, Nan Duan, and Julian J. McAuley. 2023c. [Baize: An open-source chat model with parameter-efficient tuning on self-chat data](https://md.research.elamur.ai/free/en/papers/xu-2023-baize-open-source-chat-model/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 6268– 6278. As…
73. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. [mT5: A massively multilingual pre-trained text-to-text transformer](https://md.research.elamur.ai/free/en/papers/xue-2021-mt5-massively-multilingual-pre-trained/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling…
74. Matei Zaharia, Reynold S. Xin, Patrick Wendell, Tathagata Das, Michael Armbrust, Ankur Dave, Xiangrui Meng, Josh Rosen, Shivaram Venkataraman, Michael J. Franklin, Ali Ghodsi, Joseph Gonzalez, Scott Shenker, and Ion Stoica. 2016. Apache spark: a unified engine for big data processing. Commun. ACM, 5… [find](https://scholar.google.com/scholar?q=Matei+Zaharia%2C+Reynold+S%2E+Xin%2C+Patrick+Wendell%2C+Tathagata+Das%2C+Michael+Armbrust%2C+Ankur+Dave%2C+Xiangrui+Meng%2C+Josh+Rosen%2C+Shivaram+Venkataraman%2C+Michael+J%2E+Franklin%2C+Ali+Ghodsi%2C+Joseph+Gonzalez%2C+Scott+S)
75. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. (inthe)wildchat: 570k chatGPT interaction logs in the wild. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Wenting+Zhao%2C+Xiang+Ren%2C+Jack+Hessel%2C+Claire+Cardie%2C+Yejin+Choi%2C+and+Yuntian+Deng%2E+2024%2E+%28inthe%29wildchat%3A+570k+chatGPT+interaction+logs+in+the+wild%2E+In+The+Twelfth+International+Conference+on+Learning)
76. Domain Number of Websites \\\#Pages 5926 817 446 250 81 75 445 Internet Archive eGyanKosh Indian Parliament AIR News Govt. Magazines School Books Miscellaneous 437225 5133 30964 74353 895 4315 27988 74M 88K 2.7M 148K 46K 359K 4.6M Total Total 82M *Table 5: Sangraha PDF sources - The final statistic of… [find](https://scholar.google.com/scholar?q=Domain+Number+of+Websites+%5C%5C%5C%23Pages+5926+817+446+250+81+75+445+Internet+Archive+eGyanKosh+Indian+Parliament+AIR+News+Govt%2E+Magazines+School+Books+Miscellaneous+437225+5133+30964+74353+895+4315+27988+7)

*Updated 2 November 2026*