---
title: "BHASA — paper by Leong et al. (2023) — summary and ideas"
description: "Most large-scale LLM benchmarks, such as HELM and BIG-bench, focus on English. 182 references, cited by 30 papers here."
authors: ["Wei Qi Leong", "Jian Gang Ngui", "Yosephine Susanto", "Hamsawardhini Rengarajan", "Kengatharaiyer Sarveswaran", "William Chandra Tjhi"]
year: 2023
venue: "arXiv:2309.06085"
source: "https://arxiv.org/abs/2309.06085"
doi: "10.48550/arxiv.2309.06085"
arxiv: "2309.06085"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/leong-2023-bhasa-holistic-southeast-asian-linguistic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/leong-2023-bhasa-holistic-southeast-asian-linguistic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/9/) / BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models

# BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models

**Authors:** [Wei Qi Leong](https://md.research.elamur.ai/free/en/authors/wei-qi-leong/), [Jian Gang Ngui](https://md.research.elamur.ai/free/en/authors/jian-gang-ngui/), [Yosephine Susanto](https://md.research.elamur.ai/free/en/authors/yosephine-susanto/), [Hamsawardhini Rengarajan](https://md.research.elamur.ai/free/en/authors/hamsawardhini-rengarajan/), [Kengatharaiyer Sarveswaran](https://md.research.elamur.ai/free/en/authors/kengatharaiyer-sarveswaran/) and [William Chandra Tjhi](https://md.research.elamur.ai/free/en/authors/william-chandra-tjhi/) · **Year:** 2023 · **Venue:** arXiv:2309.06085 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2309.06085) · [PDF](https://arxiv.org/pdf/2309.06085) · [DOI 10.48550/arxiv.2309.06085](https://doi.org/10.48550/arxiv.2309.06085) · [arXiv 2309.06085](https://arxiv.org/abs/2309.06085) · [OpenAlex](https://openalex.org/W4386721790)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [30](https://md.research.elamur.ai/free/en/papers/leong-2023-bhasa-holistic-southeast-asian-linguistic/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Most large-scale LLM benchmarks, such as HELM and BIG-bench, focus on English. Southeast Asian (SEA) languages get little systematic evaluation. BHASA addresses this gap with an evaluation suite in three parts:

- **NLP benchmark:** eight tasks spanning understanding, generation and reasoning, for Indonesian, Vietnamese, Thai and Tamil.
- **LINDSEA:** a diagnostic set handcrafted by linguists that probes syntax, semantics and pragmatics, currently for Indonesian and an early version of Tamil.
- **Cultural diagnostics:** a small set of questions on cultural representation and sensitivity in Indonesian and Tamil.

The authors tested GPT-4 and GPT-3.5-Turbo zero-shot, using English prompts and prompts manually translated by native speakers. GPT-4 clearly beats its predecessor and handles native-language instructions much better, sometimes even gaining from them. Even so, it still has notable weaknesses in several tasks, in language-specific grammatical phenomena, and in cultural knowledge and appropriateness. Tamil is the weakest language.

The suite is model-agnostic and still being extended. It gives the SEA region a reusable way to measure LLM quality, and it shows the need for better regional datasets or SEA-specific models.

## Key points
- The benchmark covers eight tasks: sentiment analysis, NER, machine translation, question answering, toxicity detection, abstractive summarization, causal reasoning and NLI.
- Results decline from Indonesian to Vietnamese to Thai to Tamil, and Tamil prompts yield markedly worse results.
- GPT-4 does reasonably well on QA, sentiment, summarization and causal reasoning. It lags on NER, toxicity detection, MT and NLI.
- LINDSEA uses minimal pairs, translation, information recovery and binary-choice formats. Because OpenAI output probabilities were unavailable, only the models' answers were scored.
- Code and data are at https://github.com/aisingapore/BHASA. Planned extensions include more models (e.g., PaLM-2), more languages (e.g., Filipino) and metrics beyond accuracy.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [National University of Singapore](https://md.research.elamur.ai/free/en/universities/national-university-of-singapore/) — University, Singapore, Singapore · 180 works · 34,304 citations
- [AI Singapore](https://md.research.elamur.ai/free/en/government/ai-singapore/) — Government body, Singapore, Singapore · 5 works · 383 citations
- [University of Jaffna](https://md.research.elamur.ai/free/en/universities/university-of-jaffna/) — University, Jaffna, Sri Lanka · 7 works · 70 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Wei Qi Leong, Jian Gang Ngui, Yosephine Susanto, Hamsawardhini Rengarajan, Kengatharaiyer Sarveswaran and William Chandra Tjhi. 2023. BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models. arXiv:2309.06085.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{leong2023bhasa,
  title = {{BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models}},
  author = {Wei Qi Leong, Jian Gang Ngui, Yosephine Susanto, Hamsawardhini Rengarajan, Kengatharaiyer Sarveswaran and William Chandra Tjhi},
  year = {2023},
  doi = {10.48550/arxiv.2309.06085},
  eprint = {2309.06085},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.06085}
}
```

- [← previous Xiezhi: An Ever-Updating Benchmark for Holistic Domain Knowledge Evaluation](https://md.research.elamur.ai/free/en/papers/gu-2023-xiezhi-ever-updating-benchmark/)
- [next → DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models](https://md.research.elamur.ai/free/en/papers/gong-2023-diffuseq-v2-bridging-discrete-continuous/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/9/)

## References

182 references, 73 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
2. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Sh… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Saurav+Kadavath%2C+Jackson+Kernion%2C+Tom+Conerly%2C+)
3. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emi… — [PaLM: Scaling Language Modeling with Pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
4. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. OPT: Open Pre-trained Tr… — [OPT: Open Pre-trained Transformer Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2022-opt-open-pre-trained-transformer/)
5. Junjie Ye, Xuanting Chen, Nuo Xu, Can Zu, Zekai Shao, Shichun Liu, Yuhan Cui, Zeyang Zhou, Chao Gong, Yang Shen, Jie Zhou, Siming Chen, Tao Gui, Qi Zhang, and Xuanjing Huang. A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models, 2023. arXiv:2303.10420 [cs]. [find](https://scholar.google.com/scholar?q=Junjie+Ye%2C+Xuanting+Chen%2C+Nuo+Xu%2C+Can+Zu%2C+Zekai+Shao%2C+Shichun+Liu%2C+Yuhan+Cui%2C+Zeyang+Zhou%2C+Chao+Gong%2C+Yang+Shen%2C+Jie+Zhou%2C+Siming+Chen%2C+Tao+Gui%2C+Qi+Zhang%2C+and+Xuanjing+Huang%2E+A+Comprehensive+Capabilit)
6. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/), January 2023. arXiv:2201.11903 [cs].
7. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Al… — [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/)
8. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric… [find](https://scholar.google.com/scholar?q=Percy+Liang%2C+Rishi+Bommasani%2C+Tony+Lee%2C+Dimitris+Tsipras%2C+Dilara+Soylu%2C+Michihiro+Yasunaga%2C+Yian+Zhang%2C+Deepak+Narayanan%2C+Yuhuai+Wu%2C+Ananya+Kumar%2C+Benjamin+Newman%2C+Binhang+Yuan%2C+Bobby+Yan%2C+Ce+Zhang%2C+C)
9. Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Maxamed Axmed, Kalika Bali, and Sunayana Sitaram. [MEGA: Multilingual Evaluation of Generative AI](https://md.research.elamur.ai/free/en/papers/ahuja-2023-mega-multilingual-evaluation-generative-ai/), May 2023. arXiv:2303.12528 [cs].
10. Akari Asai, Sneha Kudugunta, Xinyan Velocity Yu, Terra Blevins, Hila Gonen, Machel Reid, Yulia Tsvetkov, Sebastian Ruder, and Hannaneh Hajishirzi. [BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer](https://md.research.elamur.ai/free/en/papers/asai-2024-buffet-benchmarking-large-language-models/), May 2023. arXiv:2305.14857 [cs].
11. Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://md.research.elamur.ai/free/en/papers/bang-2023-multitask-multilingual-multimodal-evaluation-chatgpt/), February 2023. arXiv:230…
12. Viet Dac Lai, Nghia Trung Ngo, Amir Pouran Ben Veyseh, Hieu Man, Franck Dernoncourt, Trung Bui, and Thien Huu Nguyen. [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](https://md.research.elamur.ai/free/en/papers/lai-2023-chatgpt-beyond-english-comprehensive-evaluation/), April 2023. arXiv:2304.05613 [cs].
13. Wenxuan Zhang, Sharifah Mahani Aljunied, Chang Gao, Yew Ken Chia, and Lidong Bing. M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Models, June 2023. arXiv:2306.05179 [cs]. [find](https://scholar.google.com/scholar?q=Wenxuan+Zhang%2C+Sharifah+Mahani+Aljunied%2C+Chang+Gao%2C+Yew+Ken+Chia%2C+and+Lidong+Bing%2E+M3Exam%3A+A+Multilingual%2C+Multimodal%2C+Multilevel+Benchmark+for+Examining+Large+Language+Models%2C+June+2023%2E+arXiv%3A2306%2E0)
14. Rui Mao, Guanyi Chen, Xulang Zhang, Frank Guerin, and Erik Cambria. [GPTEval: A Survey on Assessments of ChatGPT and GPT-4](https://md.research.elamur.ai/free/en/papers/mao-2024-gpteval-survey-assessments-chatgpt-gpt/), August 2023. arXiv:2308.12488 [cs].
15. OpenAI. [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/), March 2023. arXiv:2303.08774 [cs].
16. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. [Attention is All You Need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing…
17. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. [Bert: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volum…
18. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355, Bruss…
19. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://md.research.elamur.ai/free/en/papers/wang-2019-superglue-stickier-benchmark-general-purpose/). In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors,…
20. Simone Tedeschi, Johan Bos, Thierry Declerck, Jan Hajic, Daniel Hershcovich, Eduard H. Hovy, Alexander Koller, Simon Krek, Steven Schockaert, Rico Sennrich, Ekaterina Shutova, and Roberto Navigli. What’s the Meaning of Superhuman Performance in Today’s NLU?, May 2023. arXiv:2305.08414 [cs]. [find](https://scholar.google.com/scholar?q=Simone+Tedeschi%2C+Johan+Bos%2C+Thierry+Declerck%2C+Jan+Hajic%2C+Daniel+Hershcovich%2C+Eduard+H%2E+Hovy%2C+Alexander+Koller%2C+Simon+Krek%2C+Steven+Schockaert%2C+Rico+Sennrich%2C+Ekaterina+Shutova%2C+and+Roberto+Navigli%2E+Wha)
21. Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://md.research.elamur.ai/free/en/papers/warstadt-2020-blimp-benchmark-linguistic-minimal-pairs/). Transactions of the Association for Computational Linguistics, 8:377–392, December 2020.
22. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://md.research.elamur.ai/free/en/papers/shen-2023-hugginggpt-solving-ai-tasks-chatgpt/), May 2023. arXiv:2303.17580 [cs].
23. Jessica López Espejel, El Hassane Ettifouri, Mahaman Sanoussi Yahaya Alassan, El Mehdi Chouham, and Walid Dahhane. GPT-3.5 vs GPT-4: Evaluating ChatGPT’s Reasoning Performance in Zero-shot Learning, 2023. arXiv:2305.12477 [cs]. [find](https://scholar.google.com/scholar?q=Jessica+L%C3%B3pez+Espejel%2C+El+Hassane+Ettifouri%2C+Mahaman+Sanoussi+Yahaya+Alassan%2C+El+Mehdi+Chouham%2C+and+Walid+Dahhane%2E+GPT%2D3%2E5+vs+GPT%2D4%3A+Evaluating+ChatGPT%E2%80%99s+Reasoning+Performance+in+Zero%2Dshot+Learning%2C+2)
24. Hanmeng Liu, Ruoxi Ning, Zhiyang Teng, Jian Liu, Qiji Zhou, and Yue Zhang. [Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4](https://md.research.elamur.ai/free/en/papers/liu-2023-evaluating-logical-reasoning-ability-chatgpt/), May 2023. arXiv:2304.03439 [cs].
25. Paloma Jeretič, Alex Warstadt, Suvrat Bhooshan, and Adina Williams. [Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition](https://md.research.elamur.ai/free/en/papers/jeretic-2020-are-natural-language-inference-models/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8690–8705, Online, 2020. Association for C…
26. Lalchand Pandia, Yan Cong, and Allyson Ettinger. [Pragmatic Competence of Pre-trained Language Models through the Lens of Discourse Connectives](https://md.research.elamur.ai/free/en/papers/pandia-2021-pragmatic-competence-pre-trained-language/), September 2021. arXiv:2109.12951 [cs].
27. Jennifer Hu, Roger Levy, Judith Degen, and Sebastian Schuster. [Expectations over Unspoken Alternatives Predict Pragmatic Inferences](https://md.research.elamur.ai/free/en/papers/hu-2023-expectations-over-unspoken-alternatives-predict/), April 2023. arXiv:2304.04758 [cs].
28. Wei Liu, Ming Xiang, and Nai Ding. Adjective Scale Probe: Can Language Models Encode Formal Semantics Information? Proceedings of the AAAI Conference on Artificial Intelligence, 37(11):13282– 13290, June 2023. [find](https://scholar.google.com/scholar?q=Wei+Liu%2C+Ming+Xiang%2C+and+Nai+Ding%2E+Adjective+Scale+Probe%3A+Can+Language+Models+Encode+Formal+Semantics+Information%3F+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+37%2811%29%3A13282%E2%80%93+13290%2C+J)
29. Alicia Parrish, Sebastian Schuster, Alex Warstadt, Omar Agha, Soo-Hwan Lee, Zhuoye Zhao, Samuel R. Bowman, and Tal Linzen. [NOPE: A Corpus of Naturally-Occurring Presuppositions in English](https://md.research.elamur.ai/free/en/papers/parrish-2021-nope-corpus-naturally-occurring-presuppositions/), September 2021. arXiv:2109.06987 [cs].
30. Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation, February 2023. arXiv:2302.09210 [cs]. [find](https://scholar.google.com/scholar?q=Amr+Hendy%2C+Mohamed+Abdelrehim%2C+Amr+Sharaf%2C+Vikas+Raunak%2C+Mohamed+Gabr%2C+Hitokazu+Matsushita%2C+Young+Jin+Kim%2C+Mohamed+Afify%2C+and+Hany+Hassan+Awadalla%2E+How+Good+Are+GPT+Models+at+Machine+Translation%3F+A+Co)
31. Maja Popović. [ChrF: Character n-gram F-score for Automatic MT Evaluation](https://md.research.elamur.ai/free/en/papers/popovic-2015-chrf-character-n-gram-f/). In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal, September 2015. Association for Computational Linguistics.
32. Ricardo Rei, José GC De Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and André FT Martins. [Comet-22: Unbabel-IST 2022 Submission for the Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/rei-2022-comet-22-unbabel-ist-2022/). In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 578–585, 202…
33. Ricardo Rei, Marcos Treviso, Nuno M. Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, José G. C. de Souza, Taisiya Glushkova, Duarte Alves, Luisa Coheur, Alon Lavie, and André F. T. Martins. [Cometkiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task](https://md.research.elamur.ai/free/en/papers/rei-2022-cometkiwi-ist-unbabel-2022-submission/). In Proceedings of…
34. Shuang Li, Jiangjie Chen, Siyu Yuan, Xinyi Wu, Hao Yang, Shimin Tao, and Yanghua Xiao. Translate Meanings, Not Just Words: IdiomKB’s Role in Optimizing Idiomatic Translation with Language Models, August 2023. arXiv:2308.13961 [cs]. — [arXiv:2308.13961](https://arxiv.org/abs/2308.13961) [find](https://scholar.google.com/scholar?q=Shuang+Li%2C+Jiangjie+Chen%2C+Siyu+Yuan%2C+Xinyi+Wu%2C+Hao+Yang%2C+Shimin+Tao%2C+and+Yanghua+Xiao%2E+Translate+Meanings%2C+Not+Just+Words%3A+IdiomKB%E2%80%99s+Role+in+Optimizing+Idiomatic+Translation+with+Language+Models%2C+Augu)
35. Tianyi Zhang, Faisal Ladhak, Esin Durmus, Percy Liang, Kathleen McKeown, and Tatsunori B. Hashimoto. [Benchmarking Large Language Models for News Summarization](https://md.research.elamur.ai/free/en/papers/zhang-2024-benchmarking-large-language-models-news/), Jan 2023. arXiv:2301.13848 [cs].
36. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training La… — [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
37. Alexander R. Fabbri, Wojciech Kryciski, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. SummEval: Re-evaluating Summarization Evaluation, February 2021. arXiv:2007.12626 [cs]. [find](https://scholar.google.com/scholar?q=Alexander+R%2E+Fabbri%2C+Wojciech+Kryciski%2C+Bryan+McCann%2C+Caiming+Xiong%2C+Richard+Socher%2C+and+Dragomir+Radev%2E+SummEval%3A+Re%2Devaluating+Summarization+Evaluation%2C+February+2021%2E+arXiv%3A2007%2E12626+%5Bcs%5D%2E)
38. Jordan Kodner, Sarah Payne, and Jeffrey Heinz. Why Linguistics Will Thrive in the 21st Century: A Reply to Piantadosi (2023), August 2023. arXiv:2308.03228 [cs]. — [arXiv:2308.03228](https://arxiv.org/abs/2308.03228) [find](https://scholar.google.com/scholar?q=Jordan+Kodner%2C+Sarah+Payne%2C+and+Jeffrey+Heinz%2E+Why+Linguistics+Will+Thrive+in+the+21st+Century%3A+A+Reply+to+Piantadosi+%282023%29%2C+August+2023%2E+arXiv%3A2308%2E03228+%5Bcs%5D%2E+%E2%80%94+%5BarXiv%3A2308%2E03228%5D%28https%3A%2F%2Farxiv%2Eorg)
39. Jennifer Hu, Jon Gauthier, Peng Qian, Ethan Wilcox, and Roger Levy. A Systematic Assessment of Syntactic Generalization in Neural Language Models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1725–1744, Online, 2020. Association for Computational… [find](https://scholar.google.com/scholar?q=Jennifer+Hu%2C+Jon+Gauthier%2C+Peng+Qian%2C+Ethan+Wilcox%2C+and+Roger+Levy%2E+A+Systematic+Assessment+of+Syntactic+Generalization+in+Neural+Language+Models%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Asso)
40. Jon Gauthier, Jennifer Hu, Ethan Wilcox, Peng Qian, and Roger Levy. [SyntaxGym: An Online Platform for Targeted Evaluation of Language Models](https://md.research.elamur.ai/free/en/papers/gauthier-2020-syntaxgym-online-platform-targeted-evaluation/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 70–76, Online, 2020. Association for…
41. Beilei Xiang, Changbing Yang, Yu Li, Alex Warstadt, and Katharina Kann. [CLiMP: A Benchmark for Chinese Language Model Evaluation](https://md.research.elamur.ai/free/en/papers/xiang-2021-climp-benchmark-chinese-language-model/). In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2784–2790, Online, 2021. Association fo…
42. Taiga Someya and Yohei Oseki. JBLiMP: Japanese Benchmark of Linguistic Minimal Pairs. In Findings of the Association for Computational Linguistics: EACL 2023, pages 1581–1594, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Taiga+Someya+and+Yohei+Oseki%2E+JBLiMP%3A+Japanese+Benchmark+of+Linguistic+Minimal+Pairs%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EACL+2023%2C+pages+1581%E2%80%931594%2C+Dubrovnik%2C+Croatia%2C+May+)
43. Daniel Hershcovich, Stella Frank, Heather Lent, Miryam De Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixiang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust, and Anders Søgaard. [Challenges and Strategies in Cross-Cultural NLP](https://md.research.elamur.ai/free/en/papers/hershcovich-2022-challenges-strategies-cross-cultural-nlp/). In…
44. Irene Solaiman, Zeerak Talat, William Agnew, Lama Ahmad, Dylan Baker, Su Lin Blodgett, Hal Daumé III, Jesse Dodge, Ellie Evans, Sara Hooker, Yacine Jernite, Alexandra Sasha Luccioni, Alberto Lusoli, Margaret Mitchell, Jessica Newman, Marie-Therese Png, Andrew Strait, and Apostol Vassilev. Evaluating… — [arXiv:2306.05949](https://arxiv.org/abs/2306.05949) [find](https://scholar.google.com/scholar?q=Irene+Solaiman%2C+Zeerak+Talat%2C+William+Agnew%2C+Lama+Ahmad%2C+Dylan+Baker%2C+Su+Lin+Blodgett%2C+Hal+Daum%C3%A9+III%2C+Jesse+Dodge%2C+Ellie+Evans%2C+Sara+Hooker%2C+Yacine+Jernite%2C+Alexandra+Sasha+Luccioni%2C+Alberto+Lusoli%2C+M)
45. Esin Durmus, Karina Nyugen, Thomas I. Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. Towards Measuring the Rep… — [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://md.research.elamur.ai/free/en/papers/durmus-2023-towards-measuring-representation-subjective-global/)
46. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. [Measuring Massive Multitask Language Understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/), January 2021. arXiv:2009.03300 [cs].
47. Jie Huang and Kevin Chen-Chuan Chang. [Towards Reasoning in Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/huang-2023-reasoning-large-language-models-survey/), May 2023. arXiv:2212.10403 [cs].
48. Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen. [Reasoning with Language Model Prompting: A Survey](https://md.research.elamur.ai/free/en/papers/qiao-2022-reasoning-language-model-prompting-survey/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5368–5393,…
49. Fangzhi Xu, Qika Lin, Jiawei Han, Tianzhe Zhao, Jun Liu, and Erik Cambria. [Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond](https://md.research.elamur.ai/free/en/papers/xu-2023-large-language-models-really-good/), August 2023. arXiv:2306.09841 [cs].
50. Fei Yu, Hongbo Zhang, Prayag Tiwari, and Benyou Wang. Natural Language Reasoning, A Survey, May 2023. arXiv:2303.14725 [cs]. [find](https://scholar.google.com/scholar?q=Fei+Yu%2C+Hongbo+Zhang%2C+Prayag+Tiwari%2C+and+Benyou+Wang%2E+Natural+Language+Reasoning%2C+A+Survey%2C+May+2023%2E+arXiv%3A2303%2E14725+%5Bcs%5D%2E)
51. Jifan Yu, Xiaozhi Wang, Shangqing Tu, Shulin Cao, Daniel Zhang-Li, Xin Lv, Hao Peng, Zijun Yao, Xiaohan Zhang, Hanming Li, Chunyang Li, Zheyuan Zhang, Yushi Bai, Yantao Liu, Amy Xin, Nianyi Lin, Kaifeng Yun, Linlu Gong, Jianhui Chen, Zhili Wu, Yunjia Qi, Weikai Li, Yong Guan, Kaisheng Zeng, Ji Qi, H… — [arXiv:2306.09296](https://arxiv.org/abs/2306.09296) [find](https://scholar.google.com/scholar?q=Jifan+Yu%2C+Xiaozhi+Wang%2C+Shangqing+Tu%2C+Shulin+Cao%2C+Daniel+Zhang%2DLi%2C+Xin+Lv%2C+Hao+Peng%2C+Zijun+Yao%2C+Xiaohan+Zhang%2C+Hanming+Li%2C+Chunyang+Li%2C+Zheyuan+Zhang%2C+Yushi+Bai%2C+Yantao+Liu%2C+Amy+Xin%2C+Nianyi+Lin%2C+Kaife)
52. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://md.research.elamur.ai/free/en/papers/joshi-2020-state-fate-linguistic-diversity-inclusion/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online, 2020. Association for Comp…
53. Martin Gellerstam. Translationese in Swedish Novels Translated from English. Translation studies in Scandinavia, 1:88–95, 1986. [find](https://scholar.google.com/scholar?q=Martin+Gellerstam%2E+Translationese+in+Swedish+Novels+Translated+from+English%2E+Translation+studies+in+Scandinavia%2C+1%3A88%E2%80%9395%2C+1986%2E)
54. Mona Baker. Corpus Linguistics and Translation Studies - Implications and Applications. In Mona Baker, Gill Francis, and Elena Tognini-Bonelli, editors, Text and Technology, page 233. John Benjamins Publishing Company, Amsterdam, 1993. [find](https://scholar.google.com/scholar?q=Mona+Baker%2E+Corpus+Linguistics+and+Translation+Studies+%2D+Implications+and+Applications%2E+In+Mona+Baker%2C+Gill+Francis%2C+and+Elena+Tognini%2DBonelli%2C+editors%2C+Text+and+Technology%2C+page+233%2E+John+Benjamins+P)
55. Gennadi Lembersky, Noam Ordan, and Shuly Wintner. Language Models for Machine Translation: Original vs. Translated Texts. Computational Linguistics, 38(4):799–825, December 2012. [find](https://scholar.google.com/scholar?q=Gennadi+Lembersky%2C+Noam+Ordan%2C+and+Shuly+Wintner%2E+Language+Models+for+Machine+Translation%3A+Original+vs%2E+Translated+Texts%2E+Computational+Linguistics%2C+38%284%29%3A799%E2%80%93825%2C+December+2012%2E)
56. V. Volansky, N. Ordan, and S. Wintner. On the Features of Translationese. Digital Scholarship in the Humanities, 30(1):98–118, April 2015. [find](https://scholar.google.com/scholar?q=V%2E+Volansky%2C+N%2E+Ordan%2C+and+S%2E+Wintner%2E+On+the+Features+of+Translationese%2E+Digital+Scholarship+in+the+Humanities%2C+30%281%29%3A98%E2%80%93118%2C+April+2015%2E)
57. Parker Riley, Isaac Caswell, Markus Freitag, and David Grangier. Translationese as a Language in Multilingual NMT. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7737–7746, Online, 2020. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Parker+Riley%2C+Isaac+Caswell%2C+Markus+Freitag%2C+and+David+Grangier%2E+Translationese+as+a+Language+in+Multilingual+NMT%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computational+Lingui)
58. Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Nothman, Kevin Knight, and Heng Ji. [Cross-lingual Name Tagging and Linking for 282 Languages](https://md.research.elamur.ai/free/en/papers/pan-2017-cross-lingual-name-tagging-linking/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1946–1958, Vancouver, Canada, 2017. A…
59. Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://md.research.elamur.ai/free/en/papers/hu-2020-xtreme-massively-multilingual-multi-task/), September 2020. arXiv:2003.11080 [cs].
60. Constantine Lignos, Nolan Holley, Chester Palen-Michel, and Jonne Sälevä. Toward More Meaningful Resources for Lower-resourced Languages. In Findings of the Association for Computational Linguistics: ACL 2022, pages 523–532, Dublin, Ireland, 2022. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Constantine+Lignos%2C+Nolan+Holley%2C+Chester+Palen%2DMichel%2C+and+Jonne+S%C3%A4lev%C3%A4%2E+Toward+More+Meaningful+Resources+for+Lower%2Dresourced+Languages%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+)
61. Bryan Wilie, Karissa Vincentio, Genta Indra Winata, Samuel Cahyawijaya, Xiaohong Li, Zhi Yuan Lim, Sidik Soleman, Rahmad Mahendra, Pascale Fung, Syafri Bahar, and Ayu Purwarianti. [IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding](https://md.research.elamur.ai/free/en/papers/wilie-2020-indonlu-benchmark-resources-evaluating-indonesian/). In Proceedings of the 1st Con…
62. Huyen T M Nguyen, Quyen T Ngo, Luong X Vu, Vu M Tran, and Hien T T Nguyen. VLSP Shared Task: Named Entity Recognition. Journal of Computer Science and Cybernetics, 34(4):283–294, January 2019. [find](https://scholar.google.com/scholar?q=Huyen+T+M+Nguyen%2C+Quyen+T+Ngo%2C+Luong+X+Vu%2C+Vu+M+Tran%2C+and+Hien+T+T+Nguyen%2E+VLSP+Shared+Task%3A+Named+Entity+Recognition%2E+Journal+of+Computer+Science+and+Cybernetics%2C+34%284%29%3A283%E2%80%93294%2C+January+2019%2E)
63. Wannaphong Phatthiyaphaibun. Thai NER 2.0, Sep 2022. [find](https://scholar.google.com/scholar?q=Wannaphong+Phatthiyaphaibun%2E+Thai+NER+2%2E0%2C+Sep+2022%2E)
64. Lalita Lowphansirikul, Charin Polpanumas, Nawat Jantrakulchai, and Sarana Nutanong. Wangchan-BERTa: Pretraining Transformer-based Thai Language Models, March 2021. arXiv:2101.09635 [cs]. — [arXiv:2101.09635](https://arxiv.org/abs/2101.09635) [find](https://scholar.google.com/scholar?q=Lalita+Lowphansirikul%2C+Charin+Polpanumas%2C+Nawat+Jantrakulchai%2C+and+Sarana+Nutanong%2E+Wangchan%2DBERTa%3A+Pretraining+Transformer%2Dbased+Thai+Language+Models%2C+March+2021%2E+arXiv%3A2101%2E09635+%5Bcs%5D%2E+%E2%80%94+%5BarXiv%3A2101)
65. Arnav Mhaske, Harshit Kedia, Sumanth Doddapaneni, Mitesh M. Khapra, Pratyush Kumar, Rudra Murthy, and Anoop Kunchukuttan. Naamapadam: A Large-Scale Named Entity Annotated Data for Indic Languages. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: L… [find](https://scholar.google.com/scholar?q=Arnav+Mhaske%2C+Harshit+Kedia%2C+Sumanth+Doddapaneni%2C+Mitesh+M%2E+Khapra%2C+Pratyush+Kumar%2C+Rudra+Murthy%2C+and+Anoop+Kunchukuttan%2E+Naamapadam%3A+A+Large%2DScale+Named+Entity+Annotated+Data+for+Indic+Languages%2E+In+)
66. Shuhe Wang, Xiaofei Sun, Xiaoya Li, Rongbin Ouyang, Fei Wu, Tianwei Zhang, Jiwei Li, and Guoyin Wang. GPT-NER: Named Entity Recognition via Large Language Models, May 2023. arXiv:2304.10428 [cs]. — [arXiv:2304.10428](https://arxiv.org/abs/2304.10428) [find](https://scholar.google.com/scholar?q=Shuhe+Wang%2C+Xiaofei+Sun%2C+Xiaoya+Li%2C+Rongbin+Ouyang%2C+Fei+Wu%2C+Tianwei+Zhang%2C+Jiwei+Li%2C+and+Guoyin+Wang%2E+GPT%2DNER%3A+Named+Entity+Recognition+via+Large+Language+Models%2C+May+2023%2E+arXiv%3A2304%2E10428+%5Bcs%5D%2E+%E2%80%94+%5Ba)
67. Anna Rogers, Matt Gardner, and Isabelle Augenstein. QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension. ACM Computing Surveys, 55(10):1– 45, October 2023. arXiv:2107.12708 [cs]. [find](https://scholar.google.com/scholar?q=Anna+Rogers%2C+Matt+Gardner%2C+and+Isabelle+Augenstein%2E+QA+Dataset+Explosion%3A+A+Taxonomy+of+NLP+Resources+for+Question+Answering+and+Reading+Comprehension%2E+ACM+Computing+Surveys%2C+55%2810%29%3A1%E2%80%93+45%2C+October+202)
68. Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. [TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages](https://md.research.elamur.ai/free/en/papers/clark-2020-tydi-qa-benchmark-information-seeking/). Transactions of the Association for Computational Linguistics, 8:454–470, 2020…
69. Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. [On the Cross-lingual Transferability of Monolingual Representations](https://md.research.elamur.ai/free/en/papers/artetxe-2020-cross-lingual-transferability-monolingual-representations/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623–4637, Online, 2020. Association for Computational Linguistics.
70. Sumanth Doddapaneni, Rahul Aralikatte, Gowtham Ramesh, Shreya Goyal, Mitesh M. Khapra, Anoop Kunchukuttan, and Pratyush Kumar. [Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languages](https://md.research.elamur.ai/free/en/papers/doddapaneni-2023-leaving-no-indic-language-behind/). In Proceedings of the 61st Annual Meeting of the Associatio…
71. Genta Indra Winata, Alham Fikri Aji, Samuel Cahyawijaya, Rahmad Mahendra, Fajri Koto, Ade Romadhony, Kemal Kurniawan, David Moeljadi, Radityo Eko Prasojo, Pascale Fung, Timothy Baldwin, Jey Han Lau, Rico Sennrich, and Sebastian Ruder. NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian… — [NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages](https://md.research.elamur.ai/free/en/papers/winata-2023-nusax-multilingual-parallel-sentiment-dataset/)
72. Kiet Van Nguyen, Vu Duc Nguyen, Phu X. V. Nguyen, Tham T. H. Truong, and Ngan Luu-Thuy Nguyen. UIT-VSFC: Vietnamese Students Feedback Corpus for Sentiment Analysis. In 2018 10th International Conference on Knowledge and Systems Engineering (KSE), pages 19–24, Ho Chi Minh City, November 2018. IEEE. [find](https://scholar.google.com/scholar?q=Kiet+Van+Nguyen%2C+Vu+Duc+Nguyen%2C+Phu+X%2E+V%2E+Nguyen%2C+Tham+T%2E+H%2E+Truong%2C+and+Ngan+Luu%2DThuy+Nguyen%2E+UIT%2DVSFC%3A+Vietnamese+Students+Feedback+Corpus+for+Sentiment+Analysis%2E+In+2018+10th+International+Conferen)
73. Bact’, Pattarawat Chormai, Charin, and ekapolc. Pythainlp/wisesight-sentiment: First release, sep 2019. [find](https://scholar.google.com/scholar?q=Bact%E2%80%99%2C+Pattarawat+Chormai%2C+Charin%2C+and+ekapolc%2E+Pythainlp%2Fwisesight%2Dsentiment%3A+First+release%2C+sep+2019%2E)
74. Yun Yu Chong and Haewoon Kwak. Understanding Toxicity Triggers on Reddit in the Context of Singapore. Proceedings of the International AAAI Conference on Web and Social Media, 16:1383– 1387, May 2022. [find](https://scholar.google.com/scholar?q=Yun+Yu+Chong+and+Haewoon+Kwak%2E+Understanding+Toxicity+Triggers+on+Reddit+in+the+Context+of+Singapore%2E+Proceedings+of+the+International+AAAI+Conference+on+Web+and+Social+Media%2C+16%3A1383%E2%80%93+1387%2C+May+2022%2E)
75. Muhammad Okky Ibrohim and Indra Budi. Multi-label Hate Speech and Abusive Language Detection in Indonesian Twitter. In Proceedings of the Third Workshop on Abusive Language Online, pages 46–57, Florence, Italy, 2019. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Muhammad+Okky+Ibrohim+and+Indra+Budi%2E+Multi%2Dlabel+Hate+Speech+and+Abusive+Language+Detection+in+Indonesian+Twitter%2E+In+Proceedings+of+the+Third+Workshop+on+Abusive+Language+Online%2C+pages+46%E2%80%9357%2C+Floren)
76. Son T Luu, Kiet Van Nguyen, and Ngan Luu-Thuy Nguyen. A Large-scale Dataset for Hate Speech Detection on Vietnamese Social Media Texts. In Advances and Trends in Artificial Intelligence. Artificial Intelligence Practices: 34th International Conference on Industrial, Engineering and Other Application… [find](https://scholar.google.com/scholar?q=Son+T+Luu%2C+Kiet+Van+Nguyen%2C+and+Ngan+Luu%2DThuy+Nguyen%2E+A+Large%2Dscale+Dataset+for+Hate+Speech+Detection+on+Vietnamese+Social+Media+Texts%2E+In+Advances+and+Trends+in+Artificial+Intelligence%2E+Artificial+In)
77. Sugan Sirihattasak, Mamoru Komachi, and Hiroshi Ishikawa. Annotation and Classification of Toxicity for Thai Twitter. In TA-COS 2018: 2nd Workshop on Text Analytics for Cybersecurity and Online Safety, pages 1–7, 2018. [find](https://scholar.google.com/scholar?q=Sugan+Sirihattasak%2C+Mamoru+Komachi%2C+and+Hiroshi+Ishikawa%2E+Annotation+and+Classification+of+Toxicity+for+Thai+Twitter%2E+In+TA%2DCOS+2018%3A+2nd+Workshop+on+Text+Analytics+for+Cybersecurity+and+Online+Safety)
78. Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. [Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis](https://md.research.elamur.ai/free/en/papers/zhu-2024-multilingual-machine-translation-large-language/), May 2023. arXiv:2304.04675 [cs].
79. NLLB Team, Marta R Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoff… [find](https://scholar.google.com/scholar?q=NLLB+Team%2C+Marta+R+Costa%2Djuss%C3%A0%2C+James+Cross%2C+Onur+%C3%87elebi%2C+Maha+Elbayad%2C+Kenneth+Heafield%2C+Kevin+Heffernan%2C+Elahe+Kalbassi%2C+Janice+Lam%2C+Daniel+Licht%2C+Jean+Maillard%2C+Anna+Sun%2C+Skyler+Wang%2C+Guillaume+Wen)
80. Tahmid Hasan, Abhik Bhattacharjee, Md. Saiful Islam, Kazi Mubasshir, Yuan-Fang Li, Yong-Bin Kang, M. Sohel Rahman, and Rifat Shahriyar. XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 46… [find](https://scholar.google.com/scholar?q=Tahmid+Hasan%2C+Abhik+Bhattacharjee%2C+Md%2E+Saiful+Islam%2C+Kazi+Mubasshir%2C+Yuan%2DFang+Li%2C+Yong%2DBin+Kang%2C+M%2E+Sohel+Rahman%2C+and+Rifat+Shahriyar%2E+XL%2DSum%3A+Large%2DScale+Multilingual+Abstractive+Summarization+for+4)
81. Christopher D Manning. Local Textual Inference: Its Hard to Circumscribe, But You Know It When You See It – And NLP Needs It. -, 2006. [find](https://scholar.google.com/scholar?q=Christopher+D+Manning%2E+Local+Textual+Inference%3A+Its+Hard+to+Circumscribe%2C+But+You+Know+It+When+You+See+It+%E2%80%93+And+NLP+Needs+It%2E+%2D%2C+2006%2E)
82. Adam Poliak. [A Survey on Recognizing Textual Entailment as an NLP Evaluation](https://md.research.elamur.ai/free/en/papers/poliak-2020-survey-recognizing-textual-entailment-as/). In Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems, pages 92–109, Online, 2020. Association for Computational Linguistics.
83. Rahmad Mahendra, Alham Fikri Aji, Samuel Louvan, Fahrurrozi Rahman, and Clara Vania. IndoNLI: A Natural Language Inference Dataset for Indonesian. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10511–10527, Online and Punta Cana, Dominican Republic,… [find](https://scholar.google.com/scholar?q=Rahmad+Mahendra%2C+Alham+Fikri+Aji%2C+Samuel+Louvan%2C+Fahrurrozi+Rahman%2C+and+Clara+Vania%2E+IndoNLI%3A+A+Natural+Language+Inference+Dataset+for+Indonesian%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Me)
84. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. XNLI: Evaluating Cross-lingual Sentence Representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2475–2485, Brussels, Belgium,… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Ruty+Rinott%2C+Guillaume+Lample%2C+Adina+Williams%2C+Samuel+Bowman%2C+Holger+Schwenk%2C+and+Veselin+Stoyanov%2E+XNLI%3A+Evaluating+Cross%2Dlingual+Sentence+Representations%2E+In+Proceedings+of+the+2018+)
85. Divyanshu Aggarwal, Vivek Gupta, and Anoop Kunchukuttan. Indic XNLI: Evaluating Multilingual Inference for Indian Languages. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10994–11006, Abu Dhabi, United Arab Emirate, December 2022. Association for Co… [find](https://scholar.google.com/scholar?q=Divyanshu+Aggarwal%2C+Vivek+Gupta%2C+and+Anoop+Kunchukuttan%2E+Indic+XNLI%3A+Evaluating+Multilingual+Inference+for+Indian+Languages%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in+Natural+Langu)
86. Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S Gordon. Choice of Plausible Alternatives: An Evaluation of Commonsense Causal Reasoning. In 2011 AAAI Spring Symposium Series, 2011. [find](https://scholar.google.com/scholar?q=Melissa+Roemmele%2C+Cosmin+Adrian+Bejan%2C+and+Andrew+S+Gordon%2E+Choice+of+Plausible+Alternatives%3A+An+Evaluation+of+Commonsense+Causal+Reasoning%2E+In+2011+AAAI+Spring+Symposium+Series%2C+2011%2E)
87. Edoardo Maria Ponti, Goran Glava, Olga Majewska, Qianchu Liu, Ivan Vuli, and Anna Korhonen. [XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning](https://md.research.elamur.ai/free/en/papers/ponti-2020-xcopa-multilingual-dataset-causal-commonsense/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2362–2376, Online, 2020. Association fo…
88. Jeff Wu, Long Ouyang, Daniel M. Ziegler, Nisan Stiennon, Ryan Lowe, Jan Leike, and Paul Christiano. [Recursively Summarizing Books with Human Feedback](https://md.research.elamur.ai/free/en/papers/wu-2021-recursively-summarizing-books-human-feedback/), September 2021. arXiv:2109.10862 [cs].
89. Keqin Peng, Liang Ding, Qihuang Zhong, Li Shen, Xuebo Liu, Min Zhang, Yuanxin Ouyang, and Dacheng Tao. [Towards Making the Most of ChatGPT for Machine Translation](https://md.research.elamur.ai/free/en/papers/peng-2023-making-most-chatgpt-machine-translation/), March 2023. arXiv:2303.13780 [cs].
90. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. [Large Language Models are Zero-Shot Reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/), 2023. arXiv:2205.11916 [cs].
91. Jieyi Long. Large Language Model Guided Tree-of-Thought, May 2023. arXiv:2305.08291 [cs]. [find](https://scholar.google.com/scholar?q=Jieyi+Long%2E+Large+Language+Model+Guided+Tree%2Dof%2DThought%2C+May+2023%2E+arXiv%3A2305%2E08291+%5Bcs%5D%2E)
92. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of Thoughts: Deliberate Problem Solving with Large Language Models, May 2023. arXiv:2305.10601 [cs]. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Thomas+L%2E+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+Tree+of+Thoughts%3A+Deliberate+Problem+Solving+with+Large+Language+Models%2C+May+2023%2E+arXiv%3A2305%2E1)
93. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. [Large Language Models Are Human-Level Prompt Engineers](https://md.research.elamur.ai/free/en/papers/zhou-2022-large-language-models-human-level/), March 2023. arXiv:2211.01910 [cs].
94. Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. [What Makes Good In-Context Examples for GPT-3?](https://md.research.elamur.ai/free/en/papers/liu-2022-what-makes-good-context-examples/) In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages 100–114, Dubl…
95. Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, and Lingpeng Kong. Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page… [find](https://scholar.google.com/scholar?q=Zhiyong+Wu%2C+Yaoxiang+Wang%2C+Jiacheng+Ye%2C+and+Lingpeng+Kong%2E+Self%2DAdaptive+In%2DContext+Learning%3A+An+Information+Compression+Perspective+for+In%2DContext+Example+Selection+and+Ordering%2E+In+Proceedings+of+th)
96. Yiming Zhang, Shi Feng, and Chenhao Tan. [Active Example Selection for In-Context Learning](https://md.research.elamur.ai/free/en/papers/zhang-2022-active-example-selection-context-learning/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9134– 9148, Abu Dhabi, United Arab Emirates, 2022. Association for Computational Linguistics.
97. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. [Calibrate Before Use: Improving Few-shot Performance of Language Models](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/). In International Conference on Machine Learning, pages 12697–12706. PMLR, 2021.
98. Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8…
99. Ananya B. Sai, Akash Kumar Mohankumar, and Mitesh M. Khapra. A Survey of Evaluation Metrics Used for NLG Systems. ACM Computing Surveys, 55(2):1–39, feb 2023. [find](https://scholar.google.com/scholar?q=Ananya+B%2E+Sai%2C+Akash+Kumar+Mohankumar%2C+and+Mitesh+M%2E+Khapra%2E+A+Survey+of+Evaluation+Metrics+Used+for+NLG+Systems%2E+ACM+Computing+Surveys%2C+55%282%29%3A1%E2%80%9339%2C+feb+2023%2E)
100. Chin-Yew Lin. [Rouge: A Package for Automatic Evaluation of Summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text summarization branches out, pages 74–81, 2004.
101. Max Grusky. Rogue Scores. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1914–1934, 2023. [find](https://scholar.google.com/scholar?q=Max+Grusky%2E+Rogue+Scores%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Long+Papers%29%2C+pages+1914%E2%80%931934%2C+2023%2E)
102. Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, Eleftherios Avramidis, Tom Kocmi, George Foster, Alon Lavie, and André FT Martins. [Results of WMT22 Metrics Shared Task: Stop Using BLEU – Neural Metrics are Better and More Robust](https://md.research.elamur.ai/free/en/papers/freitag-2022-results-wmt22-metrics-shared-task/). In Proceedings of the Seventh Conference on Mac…
103. Maja Popović. [ChrF++: Words Helping Character n-grams](https://md.research.elamur.ai/free/en/papers/popovic-2017-chrf-words-helping-character-n/). In Proceedings of the Second Conference on Machine Translation, pages 612–618, Copenhagen, Denmark, September 2017. Association for Computational Linguistics.
104. Tom Kocmi, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Thamme Gowda, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Rebecca Knowles, Philipp Koehn, Christof Monz, Makoto Morishita, Masaaki Nagata, Toshiaki Nakazawa, Michal Novák, Martin Popel, and Maja Popović.… [find](https://scholar.google.com/scholar?q=Tom+Kocmi%2C+Rachel+Bawden%2C+Ond%C5%99ej+Bojar%2C+Anton+Dvorkovich%2C+Christian+Federmann%2C+Mark+Fishel%2C+Thamme+Gowda%2C+Yvette+Graham%2C+Roman+Grundkiewicz%2C+Barry+Haddow%2C+Rebecca+Knowles%2C+Philipp+Koehn%2C+Christof+Monz)
105. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. [COMET: A Neural Framework for MT Evaluation](https://md.research.elamur.ai/free/en/papers/rei-2020-comet-neural-framework-mt-evaluation/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, Online, 2020. Association for Computational Linguistics.
106. Wojciech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. [Evaluating the Factual Consistency of Abstractive Text Summarization](https://md.research.elamur.ai/free/en/papers/kryscinski-2020-evaluating-factual-consistency-abstractive-text/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9332–9346, Online, 2020. Association for Computational…
107. Arle Richard Lommel, Aljoscha Burchardt, and Hans Uszkoreit. [Multidimensional Quality Metrics: A Flexible System for Assessing Translation Quality](https://md.research.elamur.ai/free/en/papers/lommel-2013-multidimensional-quality-metrics-flexible-system/). In Proceedings of Translating and the Computer 35, London, UK, November 2013. Aslib.
108. Markus Freitag, George Foster, David Grangier, Viresh Ratnakar, Qijun Tan, and Wolfgang Macherey. [Experts, Errors, and Context: A Large-scale Study of Human Evaluation for Machine Translation](https://md.research.elamur.ai/free/en/papers/freitag-2021-experts-errors-context-large-scale/). Transactions of the Association for Computational Linguistics, 9:1460–1474, 2021.
109. Yushi Bai, Jiahao Ying, Yixin Cao, Xin Lv, Yuze He, Xiaozhi Wang, Jifan Yu, Kaisheng Zeng, Yijia Xiao, Haozhe Lyu, Jiayin Zhang, Juanzi Li, and Lei Hou. Benchmarking Foundation Models with Language-Model-as-an-Examiner, June 2023. arXiv:2306.04181 [cs]. [find](https://scholar.google.com/scholar?q=Yushi+Bai%2C+Jiahao+Ying%2C+Yixin+Cao%2C+Xin+Lv%2C+Yuze+He%2C+Xiaozhi+Wang%2C+Jifan+Yu%2C+Kaisheng+Zeng%2C+Yijia+Xiao%2C+Haozhe+Lyu%2C+Jiayin+Zhang%2C+Juanzi+Li%2C+and+Lei+Hou%2E+Benchmarking+Foundation+Models+with+Language%2DMo)
110. Yi Chen, Rui Wang, Haiyun Jiang, Shuming Shi, and Ruifeng Xu. Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: A Preliminary Empirical Study, April 2023. arXiv:2304.00723 [cs]. — [arXiv:2304.00723](https://arxiv.org/abs/2304.00723) [find](https://scholar.google.com/scholar?q=Yi+Chen%2C+Rui+Wang%2C+Haiyun+Jiang%2C+Shuming+Shi%2C+and+Ruifeng+Xu%2E+Exploring+the+Use+of+Large+Language+Models+for+Reference%2DFree+Text+Quality+Evaluation%3A+A+Preliminary+Empirical+Study%2C+April+2023%2E+arXiv%3A23)
111. Cheng-Han Chiang and Hung-yi Lee. Can Large Language Models Be an Alternative to Human Evaluations? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15607–15631, Toronto, Canada, July 2023. Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Cheng%2DHan+Chiang+and+Hung%2Dyi+Lee%2E+Can+Large+Language+Models+Be+an+Alternative+to+Human+Evaluations%3F+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1)
112. Yunjie Ji, Yan Gong, Yiping Peng, Chao Ni, Peiyan Sun, Dongyu Pan, Baochang Ma, and Xiangang Li. Exploring ChatGPT’s Ability to Rank Content: A Preliminary Study on Consistency with Human Preferences, March 2023. arXiv:2303.07610 [cs]. — [arXiv:2303.07610](https://arxiv.org/abs/2303.07610) [find](https://scholar.google.com/scholar?q=Yunjie+Ji%2C+Yan+Gong%2C+Yiping+Peng%2C+Chao+Ni%2C+Peiyan+Sun%2C+Dongyu+Pan%2C+Baochang+Ma%2C+and+Xiangang+Li%2E+Exploring+ChatGPT%E2%80%99s+Ability+to+Rank+Content%3A+A+Preliminary+Study+on+Consistency+with+Human+Preferences%2C)
113. Tom Kocmi and Christian Federmann. Large Language Models Are State-of-the-Art Evaluators of Translation Quality, May 2023. arXiv:2302.14520 [cs]. [find](https://scholar.google.com/scholar?q=Tom+Kocmi+and+Christian+Federmann%2E+Large+Language+Models+Are+State%2Dof%2Dthe%2DArt+Evaluators+of+Translation+Quality%2C+May+2023%2E+arXiv%3A2302%2E14520+%5Bcs%5D%2E)
114. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. [G-Eval: NLG Evaluation Using GPT-4 with Better Human Alignment](https://md.research.elamur.ai/free/en/papers/liu-2023-g-eval-nlg-evaluation-gpt/), May 2023. arXiv:2303.16634 [cs].
115. Jiaan Wang, Yunlong Liang, Fandong Meng, Zengkui Sun, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou. [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://md.research.elamur.ai/free/en/papers/wang-2023-chatgpt-good-nlg-evaluator-preliminary/), April 2023. arXiv:2303.04048 [cs].
116. Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. Large Language Models are Not Fair Evaluators, August 2023. arXiv:2305.17926 [cs]. [find](https://scholar.google.com/scholar?q=Peiyi+Wang%2C+Lei+Li%2C+Liang+Chen%2C+Zefan+Cai%2C+Dawei+Zhu%2C+Binghuai+Lin%2C+Yunbo+Cao%2C+Qi+Liu%2C+Tianyu+Liu%2C+and+Zhifang+Sui%2E+Large+Language+Models+are+Not+Fair+Evaluators%2C+August+2023%2E+arXiv%3A2305%2E17926+%5Bcs%5D%2E)
117. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. [Judging LLM-as-a-judge with MT-Bench and Chatbot Arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/), July 2023. arXiv:2306.05685 [cs].
118. Kent K. Chang, Mackenzie Cramer, Sandeep Soni, and David Bamman. [Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4](https://md.research.elamur.ai/free/en/papers/chang-2023-speak-memory-archaeology-books-known/), April 2023. arXiv:2305.00118 [cs].
119. Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. [Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies](https://md.research.elamur.ai/free/en/papers/linzen-2016-assessing-ability-lstms-learn-syntax/). Transactions of the Association for Computational Linguistics, 4:521–535, dec 2016.
120. Rebecca Marvin and Tal Linzen. Targeted Syntactic Evaluation of Language Models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1192–1202, Brussels, Belgium, 2018. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Rebecca+Marvin+and+Tal+Linzen%2E+Targeted+Syntactic+Evaluation+of+Language+Models%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+1192%E2%80%931202%2C+Brussels%2C+B)
121. Jennifer Hu and Roger Levy. Prompt-based Methods May Underestimate Large Language Models’ Linguistic Generalizations, May 2023. arXiv:2305.13264 [cs]. — [arXiv:2305.13264](https://arxiv.org/abs/2305.13264) [find](https://scholar.google.com/scholar?q=Jennifer+Hu+and+Roger+Levy%2E+Prompt%2Dbased+Methods+May+Underestimate+Large+Language+Models%E2%80%99+Linguistic+Generalizations%2C+May+2023%2E+arXiv%3A2305%2E13264+%5Bcs%5D%2E+%E2%80%94+%5BarXiv%3A2305%2E13264%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2305%2E1)
122. Gasper Begus, Maksymilian Dabkowski, and Ryan Rhodes. Large Linguistic Models: Analyzing Theoretical Linguistic Abilities of LLMs, 2023. arXiv:2305.00948 [cs]. — [arXiv:2305.00948](https://arxiv.org/abs/2305.00948) [find](https://scholar.google.com/scholar?q=Gasper+Begus%2C+Maksymilian+Dabkowski%2C+and+Ryan+Rhodes%2E+Large+Linguistic+Models%3A+Analyzing+Theoretical+Linguistic+Abilities+of+LLMs%2C+2023%2E+arXiv%3A2305%2E00948+%5Bcs%5D%2E+%E2%80%94+%5BarXiv%3A2305%2E00948%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
123. Maksymilian Dabkowski and Gasper Begus. Large Language Models and (Non-)linguistic Recursion, 2023. arXiv:2306.07195 [cs]. — [arXiv:2306.07195](https://arxiv.org/abs/2306.07195) [find](https://scholar.google.com/scholar?q=Maksymilian+Dabkowski+and+Gasper+Begus%2E+Large+Language+Models+and+%28Non%2D%29linguistic+Recursion%2C+2023%2E+arXiv%3A2306%2E07195+%5Bcs%5D%2E+%E2%80%94+%5BarXiv%3A2306%2E07195%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2306%2E07195%29)
124. Noam Chomsky. Lectures on Government and Binding. Foris, 1981. [find](https://scholar.google.com/scholar?q=Noam+Chomsky%2E+Lectures+on+Government+and+Binding%2E+Foris%2C+1981%2E)
125. James Neil Sneddon, Alexander Adelaar, Dwi Noverini Djenar, and Michael C Ewing. Indonesian Reference Grammar. New South Wales: Allen and Unwin, 2010. [find](https://scholar.google.com/scholar?q=James+Neil+Sneddon%2C+Alexander+Adelaar%2C+Dwi+Noverini+Djenar%2C+and+Michael+C+Ewing%2E+Indonesian+Reference+Grammar%2E+New+South+Wales%3A+Allen+and+Unwin%2C+2010%2E)
126. Thomas Lehmann. A Grammar of Modern Tamil. Pondicherry Institute of Linguistics and Culture publication. Pondicherry Institute of Linguistics and Culture, Pondicherry, second edition, 1993. [find](https://scholar.google.com/scholar?q=Thomas+Lehmann%2E+A+Grammar+of+Modern+Tamil%2E+Pondicherry+Institute+of+Linguistics+and+Culture+publication%2E+Pondicherry+Institute+of+Linguistics+and+Culture%2C+Pondicherry%2C+second+edition%2C+1993%2E)
127. Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, and Colin Raffel.… — [Crosslingual Generalization through Multitask Finetuning](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-crosslingual-generalization-through-multitask-finetuning/)
128. Peter Cole, Gabriella Hermon, and Li-May Sung. Principles and Parameters of Long-distance Reflexives. Linguistic inquiry, pages 1–22, 1990. [find](https://scholar.google.com/scholar?q=Peter+Cole%2C+Gabriella+Hermon%2C+and+Li%2DMay+Sung%2E+Principles+and+Parameters+of+Long%2Ddistance+Reflexives%2E+Linguistic+inquiry%2C+pages+1%E2%80%9322%2C+1990%2E)
129. Peter Cole, Gabriella Hermon, and C-T James Huang. Long Distance Reflexives, volume 33. Brill, 2000. [find](https://scholar.google.com/scholar?q=Peter+Cole%2C+Gabriella+Hermon%2C+and+C%2DT+James+Huang%2E+Long+Distance+Reflexives%2C+volume+33%2E+Brill%2C+2000%2E)
130. Elisabet Engdahl. Parasitic Gaps, Resumptive Pronouns, and Subject Extractions. Linguistics, 23(1):3– 44, 1985. [find](https://scholar.google.com/scholar?q=Elisabet+Engdahl%2E+Parasitic+Gaps%2C+Resumptive+Pronouns%2C+and+Subject+Extractions%2E+Linguistics%2C+23%281%29%3A3%E2%80%93+44%2C+1985%2E)
131. James McCloskey. Resumption, Successive Cyclicity, and the Locality of Operations. Derivation and explanation in the minimalist program, pages 184–226, 2002. [find](https://scholar.google.com/scholar?q=James+McCloskey%2E+Resumption%2C+Successive+Cyclicity%2C+and+the+Locality+of+Operations%2E+Derivation+and+explanation+in+the+minimalist+program%2C+pages+184%E2%80%93226%2C+2002%2E)
132. James McCloskey. Resumption. The Blackwell companion to syntax, pages 94–117, 2006. [find](https://scholar.google.com/scholar?q=James+McCloskey%2E+Resumption%2E+The+Blackwell+companion+to+syntax%2C+pages+94%E2%80%93117%2C+2006%2E)
133. Ash Asudeh. Three Kinds of Resumption, June 2007. [find](https://scholar.google.com/scholar?q=Ash+Asudeh%2E+Three+Kinds+of+Resumption%2C+June+2007%2E)
134. Wynn Chao and Peter Sells. On the Interpretation of Resumptive Pronouns. In North East Linguistics Society Vol. 13, 1983. [find](https://scholar.google.com/scholar?q=Wynn+Chao+and+Peter+Sells%2E+On+the+Interpretation+of+Resumptive+Pronouns%2E+In+North+East+Linguistics+Society+Vol%2E+13%2C+1983%2E)
135. Ash Asudeh. Local Grammaticality in Syntactic Production. In Language from a Cognitive Perspective Grammar, Usage, and Processing Studies in honor of Tom Wasow. CSLI publications, 2011. [find](https://scholar.google.com/scholar?q=Ash+Asudeh%2E+Local+Grammaticality+in+Syntactic+Production%2E+In+Language+from+a+Cognitive+Perspective+Grammar%2C+Usage%2C+and+Processing+Studies+in+honor+of+Tom+Wasow%2E+CSLI+publications%2C+2011%2E)
136. Fernanda Ferreira and Benjamin Swets. The Production and Comprehension of Resumptive Pronouns in Relative Clause “Island” Contexts. Twenty-First Century Psycholinguistics: Four Cornerstones, January 2005. [find](https://scholar.google.com/scholar?q=Fernanda+Ferreira+and+Benjamin+Swets%2E+The+Production+and+Comprehension+of+Resumptive+Pronouns+in+Relative+Clause+%E2%80%9CIsland%E2%80%9D+Contexts%2E+Twenty%2DFirst+Century+Psycholinguistics%3A+Four+Cornerstones%2C+January+2)
137. Adam Milton Morgan and Matthew W. Wagers. English Resumptive Pronouns Are More Common Where Gaps Are Less Acceptable. Linguistic Inquiry, 49(4):861–876, October 2018. [find](https://scholar.google.com/scholar?q=Adam+Milton+Morgan+and+Matthew+W%2E+Wagers%2E+English+Resumptive+Pronouns+Are+More+Common+Where+Gaps+Are+Less+Acceptable%2E+Linguistic+Inquiry%2C+49%284%29%3A861%E2%80%93876%2C+October+2018%2E)
138. Xiaojuan Tang, Zilong Zheng, Jiaqi Li, Fanxu Meng, Song-Chun Zhu, Yitao Liang, and Muhan Zhang. [Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners](https://md.research.elamur.ai/free/en/papers/tang-2023-large-language-models-context-semantic/), June 2023. arXiv:2305.14825 [cs].
139. Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. [Toxicity in ChatGPT: Analyzing Persona-assigned Language Models](https://md.research.elamur.ai/free/en/papers/deshpande-2023-toxicity-chatgpt-analyzing-persona-assigned/), April 2023. arXiv:2304.05335 [cs].
140. Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. Unleashing Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration, July 2023. [arXiv:2307.05300 cs](https://md.research.elamur.ai/free/en/papers/wang-2023-unleashing-emergent-cognitive-synergy-large/).
141. Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Yongdong Zhang, and Zhendong Mao. ExpertPrompting: Instructing Large Language Models to be Distinguished Experts, May 2023. arXiv:2305.14688 [cs]. [find](https://scholar.google.com/scholar?q=Benfeng+Xu%2C+An+Yang%2C+Junyang+Lin%2C+Quan+Wang%2C+Chang+Zhou%2C+Yongdong+Zhang%2C+and+Zhendong+Mao%2E+ExpertPrompting%3A+Instructing+Large+Language+Models+to+be+Distinguished+Experts%2C+May+2023%2E+arXiv%3A2305%2E14688+%5Bc)
142. John Robert Ross. Constraints on Variables in Syntax. PhD thesis, Massachusetts Institute of Technology, 1967. [find](https://scholar.google.com/scholar?q=John+Robert+Ross%2E+Constraints+on+Variables+in+Syntax%2E+PhD+thesis%2C+Massachusetts+Institute+of+Technology%2C+1967%2E)
143. I Wayan Arka et al. On Modality and Finiteness in Indonesian: Complexities of =nya Nominalisation. In Workshop on TAM markers and evidentiality in Indonesian Languages, Tokyo University of Foreign Studies, pages 17–18, 2011. [find](https://scholar.google.com/scholar?q=I+Wayan+Arka+et+al%2E+On+Modality+and+Finiteness+in+Indonesian%3A+Complexities+of+%3Dnya+Nominalisation%2E+In+Workshop+on+TAM+markers+and+evidentiality+in+Indonesian+Languages%2C+Tokyo+University+of+Foreign+Stu)
144. Philippe Grangé. The Indonesian Verbal Suffix -nya – Nominalization or Subordination? Wacana, Journal of the Humanities of Indonesia, 16(1):133, April 2015. [find](https://scholar.google.com/scholar?q=Philippe+Grang%C3%A9%2E+The+Indonesian+Verbal+Suffix+%2Dnya+%E2%80%93+Nominalization+or+Subordination%3F+Wacana%2C+Journal+of+the+Humanities+of+Indonesia%2C+16%281%29%3A133%2C+April+2015%2E)
145. Julie Anne Legate. Voice and v: Lessons from Acehnese, volume 69. MIT Press, 2014. [find](https://scholar.google.com/scholar?q=Julie+Anne+Legate%2E+Voice+and+v%3A+Lessons+from+Acehnese%2C+volume+69%2E+MIT+Press%2C+2014%2E)
146. Yiyun Zhao, Jian Gang Ngui, Lucy Hall Hartley, and Steven Bethard. Do Pretrained Transformers Infer Telicity Like Humans? In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 72–81, Online, 2021. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yiyun+Zhao%2C+Jian+Gang+Ngui%2C+Lucy+Hall+Hartley%2C+and+Steven+Bethard%2E+Do+Pretrained+Transformers+Infer+Telicity+Like+Humans%3F+In+Proceedings+of+the+25th+Conference+on+Computational+Natural+Language+Learni)
147. Elay Annamalai. Modern Tamil. In Steever Sanford B, editor, The Dravidian Languages. Routledge, London, 2019. [find](https://scholar.google.com/scholar?q=Elay+Annamalai%2E+Modern+Tamil%2E+In+Steever+Sanford+B%2C+editor%2C+The+Dravidian+Languages%2E+Routledge%2C+London%2C+2019%2E)
148. Sandhya Sundaresan. A Plea for Syntax and a Return to First Principles: Monstrous Agreement in Tamil. In Semantics and Linguistic Theory, volume 21, pages 674–693, 2011. [find](https://scholar.google.com/scholar?q=Sandhya+Sundaresan%2E+A+Plea+for+Syntax+and+a+Return+to+First+Principles%3A+Monstrous+Agreement+in+Tamil%2E+In+Semantics+and+Linguistic+Theory%2C+volume+21%2C+pages+674%E2%80%93693%2C+2011%2E)
149. Herbert P Grice. Logic and Conversation. In Speech Acts, pages 41–58. Brill, 1975. [find](https://scholar.google.com/scholar?q=Herbert+P+Grice%2E+Logic+and+Conversation%2E+In+Speech+Acts%2C+pages+41%E2%80%9358%2E+Brill%2C+1975%2E)
150. Christopher Potts. Presupposition and Implicature. The Handbook of Contemporary Semantic Theory, pages 168–202, 2015. [find](https://scholar.google.com/scholar?q=Christopher+Potts%2E+Presupposition+and+Implicature%2E+The+Handbook+of+Contemporary+Semantic+Theory%2C+pages+168%E2%80%93202%2C+2015%2E)
151. Kai von Fintel. Would You Believe It? The King of France Is Back! (Presuppositions and Truth-value Intuitions). In Descriptions and Beyond. Oxford, New York, 2004. [find](https://scholar.google.com/scholar?q=Kai+von+Fintel%2E+Would+You+Believe+It%3F+The+King+of+France+Is+Back%21+%28Presuppositions+and+Truth%2Dvalue+Intuitions%29%2E+In+Descriptions+and+Beyond%2E+Oxford%2C+New+York%2C+2004%2E)
152. Mandy Simons, Judith Tonhauser, David Beaver, and Craige Roberts. What Projects and Why. In Semantics and Linguistic Theory, volume 20, pages 309–327, 2010. [find](https://scholar.google.com/scholar?q=Mandy+Simons%2C+Judith+Tonhauser%2C+David+Beaver%2C+and+Craige+Roberts%2E+What+Projects+and+Why%2E+In+Semantics+and+Linguistic+Theory%2C+volume+20%2C+pages+309%E2%80%93327%2C+2010%2E)
153. Yosuke Sato. Action/Result in Indonesian Accomplishment Verbs and the Agent Control Hypothesis. Oceanic Linguistics, 60(2):263–301, 2021. [find](https://scholar.google.com/scholar?q=Yosuke+Sato%2E+Action%2FResult+in+Indonesian+Accomplishment+Verbs+and+the+Agent+Control+Hypothesis%2E+Oceanic+Linguistics%2C+60%282%29%3A263%E2%80%93301%2C+2021%2E)
154. Tarek Naous, Michael J. Ryan, and Wei Xu. [Having Beer after Prayer? Measuring Cultural Bias in Large Language Models](https://md.research.elamur.ai/free/en/papers/naous-2023-having-beer-after-prayer-measuring/), May 2023. arXiv:2305.14456 [cs].
155. Krithika Ramesh, Sunayana Sitaram, and Monojit Choudhury. Fairness in Language Models Beyond English: Gaps and Challenges. In Findings of the Association for Computational Linguistics: EACL 2023, pages 2106–2119, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Krithika+Ramesh%2C+Sunayana+Sitaram%2C+and+Monojit+Choudhury%2E+Fairness+in+Language+Models+Beyond+English%3A+Gaps+and+Challenges%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EACL+2023%2C+page)
156. Aida Ramezani and Yang Xu. [Knowledge of Cultural Moral Norms in Large Language Models](https://md.research.elamur.ai/free/en/papers/ramezani-2023-knowledge-cultural-moral-norms-large/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 428–446, Toronto, Canada, 2023. Association for Computational Linguistics.
157. Brian Jude. The Best of Tenali Raman. Spider Books, 2005. [find](https://scholar.google.com/scholar?q=Brian+Jude%2E+The+Best+of+Tenali+Raman%2E+Spider+Books%2C+2005%2E)
158. Brian Jude. Famous Stories of Tenali Raman. Spider Books, 2007. [find](https://scholar.google.com/scholar?q=Brian+Jude%2E+Famous+Stories+of+Tenali+Raman%2E+Spider+Books%2C+2007%2E)
159. Brian Jude. Favourite Stories of Tenali Raman. Spider Books, 2008. [find](https://scholar.google.com/scholar?q=Brian+Jude%2E+Favourite+Stories+of+Tenali+Raman%2E+Spider+Books%2C+2008%2E)
160. Brian Jude. Popular Stories of Tenali Raman. Spider Books, 2008. [find](https://scholar.google.com/scholar?q=Brian+Jude%2E+Popular+Stories+of+Tenali+Raman%2E+Spider+Books%2C+2008%2E)
161. Brian Jude. Selected Stories of Tenali Raman. Spider Books, 2008. [find](https://scholar.google.com/scholar?q=Brian+Jude%2E+Selected+Stories+of+Tenali+Raman%2E+Spider+Books%2C+2008%2E)
162. Fitri Haryani Nasution. 70 Tradisi Unik Suku Bangsa di Indonesia. Bhuana Ilmu Populer, Jakarta, 2019. [find](https://scholar.google.com/scholar?q=Fitri+Haryani+Nasution%2E+70+Tradisi+Unik+Suku+Bangsa+di+Indonesia%2E+Bhuana+Ilmu+Populer%2C+Jakarta%2C+2019%2E)
163. Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Se… — [PaLM 2 Technical Report](https://md.research.elamur.ai/free/en/papers/anil-2023-palm-2-technical-report/)
164. Catherine Rose Fortin. We Need LF Copying: A Few Good Reasons Why. In Proceedings of the 28th West Coast Conference on Formal Linguistics, pages 87–95, 2011. [find](https://scholar.google.com/scholar?q=Catherine+Rose+Fortin%2E+We+Need+LF+Copying%3A+A+Few+Good+Reasons+Why%2E+In+Proceedings+of+the+28th+West+Coast+Conference+on+Formal+Linguistics%2C+pages+87%E2%80%9395%2C+2011%2E)
165. Yosuke Sato. P-stranding under Sluicing and Repair By Ellipsis: Why Is Indonesian (Not) Special? Journal of East Asian Linguistics, 20(4):339–382, November 2011. [find](https://scholar.google.com/scholar?q=Yosuke+Sato%2E+P%2Dstranding+under+Sluicing+and+Repair+By+Ellipsis%3A+Why+Is+Indonesian+%28Not%29+Special%3F+Journal+of+East+Asian+Linguistics%2C+20%284%29%3A339%E2%80%93382%2C+November+2011%2E)
166. Yosuke Sato. An In-situ Syntax of Sluicing in Indonesian. In The proceedings of AFLA, volume 23, pages 243–57, 2016. [find](https://scholar.google.com/scholar?q=Yosuke+Sato%2E+An+In%2Dsitu+Syntax+of+Sluicing+in+Indonesian%2E+In+The+proceedings+of+AFLA%2C+volume+23%2C+pages+243%E2%80%9357%2C+2016%2E)
167. Helen Jeoung. WH-agreement Across Three Domains in Indonesian. University of Pennsylvania Working Papers in Linguistics, 26(1):14, 2020. [find](https://scholar.google.com/scholar?q=Helen+Jeoung%2E+WH%2Dagreement+Across+Three+Domains+in+Indonesian%2E+University+of+Pennsylvania+Working+Papers+in+Linguistics%2C+26%281%29%3A14%2C+2020%2E)
168. Helen Jeoung. Categorial Ambiguity in mau, suka, and Other Indonesian Predicates. Language, 96(3):157–172, 2020. [find](https://scholar.google.com/scholar?q=Helen+Jeoung%2E+Categorial+Ambiguity+in+mau%2C+suka%2C+and+Other+Indonesian+Predicates%2E+Language%2C+96%283%29%3A157%E2%80%93172%2C+2020%2E)
169. Thomas Lehmann. Old Tamil. In The Dravidian Languages, pages 81–103. Routledge, 2019. [find](https://scholar.google.com/scholar?q=Thomas+Lehmann%2E+Old+Tamil%2E+In+The+Dravidian+Languages%2C+pages+81%E2%80%93103%2E+Routledge%2C+2019%2E)
170. Harold F Schiffman. The Tamil Case System. South Indian Horizons: Felicitation Volume for Francois Gros on the Occasion of his 70th Birthday, pages 293–322, 2004. [find](https://scholar.google.com/scholar?q=Harold+F+Schiffman%2E+The+Tamil+Case+System%2E+South+Indian+Horizons%3A+Felicitation+Volume+for+Francois+Gros+on+the+Occasion+of+his+70th+Birthday%2C+pages+293%E2%80%93322%2C+2004%2E)
171. Elai Annamalai. The Variable Relation of Verbs in Sequence in Tamil. In NINJAL International Symposium, 2013. [find](https://scholar.google.com/scholar?q=Elai+Annamalai%2E+The+Variable+Relation+of+Verbs+in+Sequence+in+Tamil%2E+In+NINJAL+International+Symposium%2C+2013%2E)
172. Tommi Leung. The Syntax of Two Types of Sluicing in Tamil. The Linguistic Review, 35(1):35–82, January 2018. [find](https://scholar.google.com/scholar?q=Tommi+Leung%2E+The+Syntax+of+Two+Types+of+Sluicing+in+Tamil%2E+The+Linguistic+Review%2C+35%281%29%3A35%E2%80%9382%2C+January+2018%2E)
173. I Wayan Arka, Christopher Manning, et al. Voice and Grammatical Relations in Indonesian: A New Perspective. CSLI Stanford, 1998. [find](https://scholar.google.com/scholar?q=I+Wayan+Arka%2C+Christopher+Manning%2C+et+al%2E+Voice+and+Grammatical+Relations+in+Indonesian%3A+A+New+Perspective%2E+CSLI+Stanford%2C+1998%2E)
174. Peter Cole, Gabriella Hermon, and Yanti. Voice in Malay/Indonesian. Lingua, 118(10):1500–1553, October 2008. [find](https://scholar.google.com/scholar?q=Peter+Cole%2C+Gabriella+Hermon%2C+and+Yanti%2E+Voice+in+Malay%2FIndonesian%2E+Lingua%2C+118%2810%29%3A1500%E2%80%931553%2C+October+2008%2E)
175. Bambang Purwo. Voice in Indonesian: A Discourse Study. Passive and voice, 16:195, 1988. [find](https://scholar.google.com/scholar?q=Bambang+Purwo%2E+Voice+in+Indonesian%3A+A+Discourse+Study%2E+Passive+and+voice%2C+16%3A195%2C+1988%2E)
176. Elai Annamalai. The Constituent Structure of Tamil. The Yearbook of South Asian Languages and Linguistics, pages 3–46, 2003. [find](https://scholar.google.com/scholar?q=Elai+Annamalai%2E+The+Constituent+Structure+of+Tamil%2E+The+Yearbook+of+South+Asian+Languages+and+Linguistics%2C+pages+3%E2%80%9346%2C+2003%2E)
177. Benjamin Wai Hoe Pong. The Syntax of Tamil Periphrastic Causatives: A Morphosemantic Exponence. Bachelor’s thesis, The National University of Singapore, 2022. [find](https://scholar.google.com/scholar?q=Benjamin+Wai+Hoe+Pong%2E+The+Syntax+of+Tamil+Periphrastic+Causatives%3A+A+Morphosemantic+Exponence%2E+Bachelor%E2%80%99s+thesis%2C+The+National+University+of+Singapore%2C+2022%2E)
178. Catherine Rose Fortin. On the Left Periphery in Indonesian. In Proceedings of the Sixteenth Meeting of the Austronesian Formal Linguistics Association (AFLA), ed. Sandy Chung, Daniel Finer, Ileana Paul, and Eric Postdam, pages 29–43. University of California, Santa Cruz, 2009. [find](https://scholar.google.com/scholar?q=Catherine+Rose+Fortin%2E+On+the+Left+Periphery+in+Indonesian%2E+In+Proceedings+of+the+Sixteenth+Meeting+of+the+Austronesian+Formal+Linguistics+Association+%28AFLA%29%2C+ed%2E+Sandy+Chung%2C+Daniel+Finer%2C+Ileana+Pau)
179. Catherine Rose Fortin. Indonesian. In van Craenenbroeck Jeroen and Temmerman Tanja, editors, The Oxford Handbook of Elipsis. Oxford University Press, New York, 2019. [find](https://scholar.google.com/scholar?q=Catherine+Rose+Fortin%2E+Indonesian%2E+In+van+Craenenbroeck+Jeroen+and+Temmerman+Tanja%2C+editors%2C+The+Oxford+Handbook+of+Elipsis%2E+Oxford+University+Press%2C+New+York%2C+2019%2E)
180. Peter Cole and Gabriella Hermon. Subject and Non-subject Relativisation in Indonesian. Journal of East Asian Linguistics, 14:59–88, January 2005. [find](https://scholar.google.com/scholar?q=Peter+Cole+and+Gabriella+Hermon%2E+Subject+and+Non%2Dsubject+Relativisation+in+Indonesian%2E+Journal+of+East+Asian+Linguistics%2C+14%3A59%E2%80%9388%2C+January+2005%2E)
181. Harold F Schiffman. A Reference Grammar of Spoken Tamil. Cambridge University Press, 1999. [find](https://scholar.google.com/scholar?q=Harold+F+Schiffman%2E+A+Reference+Grammar+of+Spoken+Tamil%2E+Cambridge+University+Press%2C+1999%2E)
182. Marwati Djoened Poesponegoro and Nugroho Notosusanto. Sejarah Nasional Indonesia, volume 2. Balai Pustaka, Jakarta, Indonesia, 2019. [find](https://scholar.google.com/scholar?q=Marwati+Djoened+Poesponegoro+and+Nugroho+Notosusanto%2E+Sejarah+Nasional+Indonesia%2C+volume+2%2E+Balai+Pustaka%2C+Jakarta%2C+Indonesia%2C+2019%2E)

*Updated 30 October 2026*