---
title: "SeaEval for Multilingual Foundation Models: From Cross-Lingual"
description: "SeaEval is a benchmark for multilingual foundation models. 80 references, cited by 71 papers here."
authors: ["Bin Wang", "Zhengyuan Liu", "Xin Huang", "Fangkai Jiao", "Yang Ding", "AiTi Aw", "Nancy Chen"]
year: 2024
venue: "Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)"
source: "https://aclanthology.org/2024.naacl-long.22/"
doi: "10.18653/v1/2024.naacl-long.22"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2024-seaeval-multilingual-foundation-models-cross/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2024-seaeval-multilingual-foundation-models-cross/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/) / SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning

# SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning

**Authors:** [Bin Wang](https://md.research.elamur.ai/free/en/authors/bin-wang/), [Zhengyuan Liu](https://md.research.elamur.ai/free/en/authors/zhengyuan-liu/), Xin Huang, Fangkai Jiao, Yang Ding, [AiTi Aw](https://md.research.elamur.ai/free/en/authors/aiti-aw/) and [Nancy Chen](https://md.research.elamur.ai/free/en/authors/nancy-chen/) · **Year:** 2024 · **Venue:** [Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2024](https://md.research.elamur.ai/free/en/venues/naacl/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.naacl-long.22/) · [PDF](https://aclanthology.org/2024.naacl-long.22.pdf) · [DOI 10.18653/v1/2024.naacl-long.22](https://doi.org/10.18653/v1/2024.naacl-long.22) · [OpenAlex](https://openalex.org/W4401042958)

**Citations:** 14 citations (OpenAlex, 2026-10-14) · [71](https://md.research.elamur.ai/free/en/papers/wang-2024-seaeval-multilingual-foundation-models-cross/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary

SeaEval is a benchmark for multilingual foundation models. It addresses gaps that monolingual and bilingual test suites leave open, namely cultural comprehension and whether a model's knowledge stays the same across languages. The benchmark covers four capability areas: classic NLP tasks, complex reasoning, cultural understanding and reasoning, and cross-lingual knowledge transfer.

The authors combine existing datasets with newly built ones. These include region-specific cultural sets such as SG-Eval, which features Singaporean local drinks and mixed Hokkien/Cantonese/Malay vocabulary, and parallel question sets for testing cross-lingual consistency. They also add metrics beyond accuracy, such as a consistency score and the combined AC3 measure. Open and closed models are evaluated, including GPT-4, ChatGPT, LLaMA and BLOOMZ.

The experiments surface four weaknesses:
- Outputs shift when instructions are paraphrased.
- Exposure biases persist, such as favouring certain answer positions or majority labels.
- Answers to the same factual question often differ depending on the query language.
- Proficiency is uneven across languages, typically strongest in English.

The work matters because it shows that multilingual training alone does not yield well-aligned, culturally aware models. It also provides a public toolkit and leaderboard for further study.

## Key points

- SeaEval has roughly 28–29 datasets. The paper is inconsistent, citing both 6 and 7 new ones, built for cultural reasoning and cross-lingual consistency. Named sets include Cross-MMLU, Cross-LogicQA, SG-Eval, US-Eval, CN-Eval and Singlish2English.
- On consistency, BLOOMZ has the lowest overall accuracy but still beats ChatGPT (54% vs. 47%). GPT-4 reaches 75%, falling to 64% under a stricter setting (s = 6).
- LLaMA models are trained on about 90% English data, which limits their multilingual and multicultural performance.
- Accuracy in English generally exceeds other languages, and low-resource languages lag further behind. The consistency phenomenon is illustrated with up to 16 languages.
- Datasets, evaluation toolkit and leaderboard are released at github.com/SeaEval/SeaEval. Stated limitations include limited language and culture coverage, no open-ended dialogue evaluation, and no safety or efficiency assessment.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Nanyang Technological University](https://md.research.elamur.ai/free/en/universities/nanyang-technological-university/) — University, Singapore, Singapore · 73 works · 24,390 citations
- [Institute for Infocomm Research](https://md.research.elamur.ai/free/en/labs/institute-for-infocomm-research/) — Research institute, Singapore, Singapore · 33 works · 14,373 citations
- [Agency for Science, Technology and Research](https://md.research.elamur.ai/free/en/government/agency-for-science-technology-and-research/) — Government body, Singapore, Singapore · 19 works · 13,974 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 1,282 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,343 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 510 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 467 works

## Cite

Cite as: Bin Wang, Zhengyuan Liu, Xin Huang, Fangkai Jiao, Yang Ding, AiTi Aw and Nancy Chen. 2024. SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2024seaeval,
  title = {{SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning}},
  author = {Bin Wang, Zhengyuan Liu, Xin Huang, Fangkai Jiao, Yang Ding, AiTi Aw and Nancy Chen},
  booktitle = {Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-long.22},
  url = {https://doi.org/10.18653/v1/2024.naacl-long.22}
}
```

- [← previous LLMs cannot find reasoning errors, but can correct them given the error location](https://md.research.elamur.ai/free/en/papers/tyen-2024-llms-cannot-find-reasoning-errors/)
- [next → When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages](https://md.research.elamur.ai/free/en/papers/chang-2024-when-is-multilinguality-curse-language/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/)

## References

80 references, 39 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Mohamed Ahmed, Kalika Bali, and Sunayana Sitaram. 2023. [MEGA: Multilingual evaluation of generative AI](https://md.research.elamur.ai/free/en/papers/ahuja-2023-mega-multilingual-evaluation-generative-ai/). In Proceedings of the 2023 Conference on Empirical Methods in N…
2. Yushi Bai, Jiahao Ying, Yixin Cao, Xin Lv, Yuze He, Xiaozhi Wang, Jifan Yu, Kaisheng Zeng, Yijia Xiao, Haozhe Lyu, et al. 2023. Benchmarking foundation models with language-model-as-an-examiner. arXiv preprint arXiv:2306.04181. [find](https://scholar.google.com/scholar?q=Yushi+Bai%2C+Jiahao+Ying%2C+Yixin+Cao%2C+Xin+Lv%2C+Yuze+He%2C+Xiaozhi+Wang%2C+Jifan+Yu%2C+Kaisheng+Zeng%2C+Yijia+Xiao%2C+Haozhe+Lyu%2C+et+al%2E+2023%2E+Benchmarking+foundation+models+with+language%2Dmodel%2Das%2Dan%2Dexaminer%2E+arXiv)
3. BIG bench authors. 2023. [Beyond the imitation game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/). Transactions on Machine Learning Research.
4. Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. [On the opportunities and risks of foundation models](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/). arXiv preprint arXiv:2108.07258.
5. Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Kaijie Zhu, Hao Chen, Linyi Yang, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. 2023. [A survey on evaluation of large language models](https://md.research.elamur.ai/free/en/papers/chang-2023-survey-evaluation-large-language-models/). arXiv preprint arXiv:2307.03109.
6. Fangzhou Chen and et al. 2023. Z-bench. [https://github.com/zhenbench/z-bench](https://github.com/zhenbench/z-bench). [↗](https://github.com/zhenbench/z-bench)
7. Yulong Chen, Yang Liu, Liang Chen, and Yue Zhang. 2021. [DialogSum: A real-life scenario dialogue summarization dataset](https://md.research.elamur.ai/free/en/papers/chen-2021-dialogsum-real-life-scenario-dialogue/). In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 5062–5074, Online. Association for Computational Linguistics.
8. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. [find](https://scholar.google.com/scholar?q=Wei%2DLin+Chiang%2C+Zhuohan+Li%2C+Zi+Lin%2C+Ying+Sheng%2C+Zhanghao+Wu%2C+Hao+Zhang%2C+Lianmin+Zheng%2C+Siyuan+Zhuang%2C+Yonghao+Zhuang%2C+Joseph+E%2E+Gonzalez%2C+Ion+Stoica%2C+and+Eric+P%2E+Xing%2E+2023%2E+Vicuna%3A+An+open%2Dsource+cha)
9. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. [PaLM: Scaling language modeling with pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/). arXiv preprint arXiv:2204.02311.
10. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022. [Scaling instruction-finetuned language models](https://md.research.elamur.ai/free/en/papers/chung-2022-scaling-instruction-finetuned-language-models/). arXiv preprint arXiv:2210.11416.
11. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? Try ARC, the AI2 reasoning challenge. arXiv preprint arXiv:1803.05457. [find](https://scholar.google.com/scholar?q=Peter+Clark%2C+Isaac+Cowhey%2C+Oren+Etzioni%2C+Tushar+Khot%2C+Ashish+Sabharwal%2C+Carissa+Schoenick%2C+and+Oyvind+Tafjord%2E+2018%2E+Think+you+have+solved+question+answering%3F+Try+ARC%2C+the+AI2+reasoning+challenge%2E+arX)
12. Yiming Cui, Ziqing Yang, and Xin Yao. 2023. [Efficient and effective text encoding for chinese llama and alpaca](https://md.research.elamur.ai/free/en/papers/cui-2023-efficient-effective-text-encoding-chinese/). arXiv preprint arXiv:2304.08177.
13. David Deterding. 2007. Singapore English. Edinburgh University Press. [find](https://scholar.google.com/scholar?q=David+Deterding%2E+2007%2E+Singapore+English%2E+Edinburgh+University+Press%2E)
14. Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2022. [GLM: General language model pretraining with autoregressive blank infilling](https://md.research.elamur.ai/free/en/papers/du-2022-glm-general-language-model-pretraining/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 320–335…
15. Zhouhong Gu, Xiaoxuan Zhu, Haoning Ye, Lin Zhang, Jianchen Wang, Sihang Jiang, Zhuozhi Xiong, Zihan Li, Qianyu He, Rui Xu, et al. 2023. Xiezhi: An ever-updating benchmark for holistic domain knowledge evaluation. arXiv preprint arXiv:2306.05783. [find](https://scholar.google.com/scholar?q=Zhouhong+Gu%2C+Xiaoxuan+Zhu%2C+Haoning+Ye%2C+Lin+Zhang%2C+Jianchen+Wang%2C+Sihang+Jiang%2C+Zhuozhi+Xiong%2C+Zihan+Li%2C+Qianyu+He%2C+Rui+Xu%2C+et+al%2E+2023%2E+Xiezhi%3A+An+ever%2Dupdating+benchmark+for+holistic+domain+knowledge)
16. Francisco Guzmán, Peng-Jen Chen, Myle Ott, Juan Pino, Guillaume Lample, Philipp Koehn, Vishrav Chaudhary, and Marc’Aurelio Ranzato. 2019. [The FLORES evaluation datasets for low-resource machine translation: Nepali–English and Sinhala– English](https://md.research.elamur.ai/free/en/papers/guzman-2019-flores-evaluation-datasets-low-resource/). In Proceedings of the EMNLP-IJCNLP 2019, pages 6098–6111…
17. Nikita Haduong, Alice Gao, and Noah A. Smith. 2023. Risks and NLP design: A case study on procedural document QA. In Findings of the Association for Computational Linguistics: ACL 2023, pages 1248–1269, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nikita+Haduong%2C+Alice+Gao%2C+and+Noah+A%2E+Smith%2E+2023%2E+Risks+and+NLP+design%3A+A+case+study+on+procedural+document+QA%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2023%2C+pages+1248%E2%80%93126)
18. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021a. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In International Conference on Learning Representations.
19. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021b. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). Proceedings of the International Conference on Learning Representations (ICLR).
20. Hai Hu, Kyle Richardson, Liang Xu, Lu Li, Sandra Kübler, and Lawrence Moss. 2020. [OCNLI: Original Chinese Natural Language Inference](https://md.research.elamur.ai/free/en/papers/hu-2020-ocnli-original-chinese-natural/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3512–3526, Online. Association for Computational Linguistics.
21. Haoyang Huang, Tianyi Tang, Dongdong Zhang, Wayne Xin Zhao, Ting Song, Yan Xia, and Furu Wei. 2023a. [Not all languages are created equal in llms: Improving multilingual capability by cross-lingual-thought prompting](https://md.research.elamur.ai/free/en/papers/huang-2023-all-languages-created-equal-llms/). arXiv preprint arXiv:2305.07004.
22. Yu Fei, Yifan Hou, Zeming Chen, and Antoine Bosselut. 2023. [Mitigating label biases for in-context learning](https://md.research.elamur.ai/free/en/papers/fei-2023-mitigating-label-biases-context-learning/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14014–14031, Toronto, Canada. Association for Computational Linguisti…
23. Lifu Huang, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2019. [Cosmos QA: machine reading comprehension with contextual commonsense reasoning](https://md.research.elamur.ai/free/en/papers/huang-2019-cosmos-qa-machine-reading-comprehension/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Langua…
24. Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. 2019. SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization. In Proceedings of the 2nd Workshop on New Frontiers in Summarization, pages 70–79, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Bogdan+Gliwa%2C+Iwona+Mochol%2C+Maciej+Biesek%2C+and+Aleksander+Wawer%2E+2019%2E+SAMSum+corpus%3A+A+human%2Dannotated+dialogue+dataset+for+abstractive+summarization%2E+In+Proceedings+of+the+2nd+Workshop+on+New+Fronti)
25. Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Jiayi Lei, Yao Fu, Maosong Sun, and Junxian He. 2023b. [C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models](https://md.research.elamur.ai/free/en/papers/huang-2023-c-eval-multi-level-multi/). In Advances in Neural Information Proc…
26. Li-Jun Ji, Zhiyong Zhang, and Richard E Nisbett. 2004. Is it culture or is it language? Examination of language effects in cross-cultural research on categorization. Journal of personality and social psychology, 87(1):57. [find](https://scholar.google.com/scholar?q=Li%2DJun+Ji%2C+Zhiyong+Zhang%2C+and+Richard+E+Nisbett%2E+2004%2E+Is+it+culture+or+is+it+language%3F+Examination+of+language+effects+in+cross%2Dcultural+research+on+categorization%2E+Journal+of+personality+and+social+)
27. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. 2022. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110. [find](https://scholar.google.com/scholar?q=Percy+Liang%2C+Rishi+Bommasani%2C+Tony+Lee%2C+Dimitris+Tsipras%2C+Dilara+Soylu%2C+Michihiro+Yasunaga%2C+Yian+Zhang%2C+Deepak+Narayanan%2C+Yuhuai+Wu%2C+Ananya+Kumar%2C+et+al%2E+2022%2E+Holistic+evaluation+of+language+models%2E+)
28. Fangkai Jiao, Yangyang Guo, Xuemeng Song, and Liqiang Nie. 2022. MERIt: Meta-Path Guided Contrastive Learning for Logical Reasoning. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3496–3509, Dublin, Ireland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Fangkai+Jiao%2C+Yangyang+Guo%2C+Xuemeng+Song%2C+and+Liqiang+Nie%2E+2022%2E+MERIt%3A+Meta%2DPath+Guided+Contrastive+Learning+for+Logical+Reasoning%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2)
29. Chin-Yew Lin. 2004. [ROUGE: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
30. Fangkai Jiao, Zhiyang Teng, Shafiq R. Joty, Bosheng Ding, Aixin Sun, Zhengyuan Liu, and Nancy F. Chen. 2023. Logicllm: Exploring self-supervised logic-enhanced training for large language models. CoRR, abs/2305.13718. [find](https://scholar.google.com/scholar?q=Fangkai+Jiao%2C+Zhiyang+Teng%2C+Shafiq+R%2E+Joty%2C+Bosheng+Ding%2C+Aixin+Sun%2C+Zhengyuan+Liu%2C+and+Nancy+F%2E+Chen%2E+2023%2E+Logicllm%3A+Exploring+self%2Dsupervised+logic%2Denhanced+training+for+large+language+models%2E+CoRR)
31. Claire Kramsch. 1991. Culture in language learning: A view from the united states. Foreign language research in cross-cultural perspective, 2:217–240. [find](https://scholar.google.com/scholar?q=Claire+Kramsch%2E+1991%2E+Culture+in+language+learning%3A+A+view+from+the+united+states%2E+Foreign+language+research+in+cross%2Dcultural+perspective%2C+2%3A217%E2%80%93240%2E)
32. Claire Kramsch. 2014. Language and culture. AILA review, 27(1):30–55. [find](https://scholar.google.com/scholar?q=Claire+Kramsch%2E+2014%2E+Language+and+culture%2E+AILA+review%2C+27%281%29%3A30%E2%80%9355%2E)
33. Saurabh Kulshreshtha, Jose Luis Redondo Garcia, and Ching-Yun Chang. 2020. Cross-lingual alignment methods for multilingual BERT: A comparative study. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 933–942, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Saurabh+Kulshreshtha%2C+Jose+Luis+Redondo+Garcia%2C+and+Ching%2DYun+Chang%2E+2020%2E+Cross%2Dlingual+alignment+methods+for+multilingual+BERT%3A+A+comparative+study%2E+In+Findings+of+the+Association+for+Computational+)
34. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard H. Hovy. 2017. [RACE: Large-scale reading comprehension dataset from examinations](https://md.research.elamur.ai/free/en/papers/lai-2017-race-large-scale-reading-comprehension/). In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pages 785–79…
35. Viet Dac Lai, Nghia Trung Ngo, Amir Pouran Ben Veyseh, Hieu Man, Franck Dernoncourt, Trung Bui, and Thien Huu Nguyen. 2023. [ChatGPT beyond English: Towards a comprehensive evaluation of large language models in multilingual learning](https://md.research.elamur.ai/free/en/papers/lai-2023-chatgpt-beyond-english-comprehensive-evaluation/). arXiv preprint arXiv:2304.05613.
36. Haonan Li, Yixuan Zhang, Fajri Koto, Yifei Yang, Hai Zhao, Yeyun Gong, Nan Duan, and Timothy Baldwin. 2023a. [CMMLU: Measuring massive multitask language understanding in chinese](https://md.research.elamur.ai/free/en/papers/li-2023-cmmlu-measuring-massive-multitask-language/).
37. Peggy Li and Lila Gleitman. 2002. Turning the tables: Language and spatial reasoning. Cognition, 83(3):265–294. [find](https://scholar.google.com/scholar?q=Peggy+Li+and+Lila+Gleitman%2E+2002%2E+Turning+the+tables%3A+Language+and+spatial+reasoning%2E+Cognition%2C+83%283%29%3A265%E2%80%93294%2E)
38. Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023b. AlpacaEval: An automatic evaluator of instruction-following models. [https://github.com/](https://github.com/) tatsu-lab/alpaca_eval. [↗](https://github.com/)
39. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022a. [TruthfulQA: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, Dublin, Ireland. Association for Computational Linguistic…
40. [Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyano…](https://md.research.elamur.ai/free/en/papers/lin-2021-few-shot-learning-multilingual-language/)
41. Hanmeng Liu, Jian Liu, Leyang Cui, Zhiyang Teng, Nan Duan, Ming Zhou, and Yue Zhang. 2023. Logiqa 2.0—an improved dataset for logical reasoning in natural language understanding. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:2947–2962. [find](https://scholar.google.com/scholar?q=Hanmeng+Liu%2C+Jian+Liu%2C+Leyang+Cui%2C+Zhiyang+Teng%2C+Nan+Duan%2C+Ming+Zhou%2C+and+Yue+Zhang%2E+2023%2E+Logiqa+2%2E0%E2%80%94an+improved+dataset+for+logical+reasoning+in+natural+language+understanding%2E+IEEE%2FACM+Transactions)
42. Zhengyuan Liu, Shikang Ni, Aiti Aw, and Nancy Chen. 2022. Singlish message paraphrasing: A joint task of creole translation and text normalization. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3924–3936. [find](https://scholar.google.com/scholar?q=Zhengyuan+Liu%2C+Shikang+Ni%2C+Aiti+Aw%2C+and+Nancy+Chen%2E+2022%2E+Singlish+message+paraphrasing%3A+A+joint+task+of+creole+translation+and+text+normalization%2E+In+Proceedings+of+the+29th+International+Conference+)
43. Robert K Logan. 1986. The alphabet effect. New York: Morrow. [find](https://scholar.google.com/scholar?q=Robert+K+Logan%2E+1986%2E+The+alphabet+effect%2E+New+York%3A+Morrow%2E)
44. John McCarthy. 2022. Artificial intelligence, logic, and formalising common sense. Machine Learning and the City: Applications in Architecture and Urban Design, pages 69–90. [find](https://scholar.google.com/scholar?q=John+McCarthy%2E+2022%2E+Artificial+intelligence%2C+logic%2C+and+formalising+common+sense%2E+Machine+Learning+and+the+City%3A+Applications+in+Architecture+and+Urban+Design%2C+pages+69%E2%80%9390%2E)
45. [Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, and Colin Raffel.…](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-crosslingual-generalization-through-multitask-finetuning/)
46. Nils J. Nilsson. 1991. Logic and artificial intelligence. Artif. Intell., 47:31–56. [find](https://scholar.google.com/scholar?q=Nils+J%2E+Nilsson%2E+1991%2E+Logic+and+artificial+intelligence%2E+Artif%2E+Intell%2E%2C+47%3A31%E2%80%9356%2E)
47. R OpenAI. 2023. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv, pages 2303–08774.
48. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in Neural Information Processing Systems, 35:27730–27744.
49. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
50. Alastair Pennycook. 2006. Global Englishes and transcultural flows. routledge. [find](https://scholar.google.com/scholar?q=Alastair+Pennycook%2E+2006%2E+Global+Englishes+and+transcultural+flows%2E+routledge%2E)
51. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. [SQuAD: 100,000+ questions for machine comprehension of text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics.
52. Mei Silviana Saputri, Rahmad Mahendra, and Mirna Adriani. 2018. Emotion classification on indonesian twitter dataset. In 2018 International Conference on Asian Language Processing (IALP), pages 90–95. IEEE. [find](https://scholar.google.com/scholar?q=Mei+Silviana+Saputri%2C+Rahmad+Mahendra%2C+and+Mirna+Adriani%2E+2018%2E+Emotion+classification+on+indonesian+twitter+dataset%2E+In+2018+International+Conference+on+Asian+Language+Processing+%28IALP%29%2C+pages+90%E2%80%9395%2E)
53. Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100. [find](https://scholar.google.com/scholar?q=Teven+Le+Scao%2C+Angela+Fan%2C+Christopher+Akiki%2C+Ellie+Pavlick%2C+Suzana+Ilic%CC%81%2C+Daniel+Hesslow%2C+Roman+Castagn%C3%A9%2C+Alexandra+Sasha+Luccioni%2C+Fran%C3%A7ois+Yvon%2C+Matthias+Gall%C3%A9%2C+et+al%2E+2022%2E+Bloom%3A+A+176b%2Dparameter)
54. Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, et al. 2022. [Language models are multilingual chain-of-thought reasoners](https://md.research.elamur.ai/free/en/papers/shi-2022-language-models-multilingual-chain-thought/). In The Eleventh International Conference on Learning Representations.
55. Kai Sun, Dian Yu, Jianshu Chen, Dong Yu, Yejin Choi, and Claire Cardie. 2019. [DREAM: A challenge data set and models for dialogue-based reading comprehension](https://md.research.elamur.ai/free/en/papers/sun-2019-dream-challenge-data-set-models/). Transactions of the Association for Computational Linguistics, 7:217–231.
56. Kai Sun, Dian Yu, Dong Yu, and Claire Cardie. 2020. [Investigating prior knowledge for challenging Chinese machine reading comprehension](https://md.research.elamur.ai/free/en/papers/sun-2019-investigating-prior-knowledge-challenging/). Transactions of the Association for Computational Linguistics, 8:141– 155.
57. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford Alpaca: An instruction-following llama model. [find](https://scholar.google.com/scholar?q=Rohan+Taori%2C+Ishaan+Gulrajani%2C+Tianyi+Zhang%2C+Yann+Dubois%2C+Xuechen+Li%2C+Carlos+Guestrin%2C+Percy+Liang%2C+and+Tatsunori+B%2E+Hashimoto%2E+2023%2E+Stanford+Alpaca%3A+An+instruction%2Dfollowing+llama+model%2E)
58. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023a. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971.
59. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023b. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Louis+Martin%2C+Kevin+Stone%2C+Peter+Albert%2C+Amjad+Almahairi%2C+Yasmine+Babaei%2C+Nikolay+Bashlykov%2C+Soumya+Batra%2C+Prajjwal+Bhargava%2C+Shruti+Bhosale%2C+et+al%2E+2023b%2E+Llama+2%3A+Open+foundation+and+f)
60. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. [GLUE: A multi-task benchmark and analysis platform for natural language understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). In International Conference on Learning Representations.
61. Bin Wang and Haizhou Li. 2023. Relational sentence embedding for flexible semantic matching. In Proceedings of the 8th Workshop on Representation Learning for NLP (RepL4NLP 2023), pages 238–252, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Bin+Wang+and+Haizhou+Li%2E+2023%2E+Relational+sentence+embedding+for+flexible+semantic+matching%2E+In+Proceedings+of+the+8th+Workshop+on+Representation+Learning+for+NLP+%28RepL4NLP+2023%29%2C+pages+238%E2%80%93252%2C+Toron)
62. Bin Wang, Geyu Lin, Zhengyuan Liu, Chengwei Wei, and Nancy F Chen. 2024a. Craft: Extracting and tuning cultural instructions from the wild. arXiv preprint arXiv:2405.03138. [find](https://scholar.google.com/scholar?q=Bin+Wang%2C+Geyu+Lin%2C+Zhengyuan+Liu%2C+Chengwei+Wei%2C+and+Nancy+F+Chen%2E+2024a%2E+Craft%3A+Extracting+and+tuning+cultural+instructions+from+the+wild%2E+arXiv+preprint+arXiv%3A2405%2E03138%2E)
63. Bin Wang, Zhengyuan Liu, and Nancy Chen. 2023a. Instructive dialogue summarization with query aggregations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7630–7653, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Bin+Wang%2C+Zhengyuan+Liu%2C+and+Nancy+Chen%2E+2023a%2E+Instructive+dialogue+summarization+with+query+aggregations%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+p)
64. Bin Wang, Chengwei Wei, Zhengyuan Liu, Geyu Lin, and Nancy F Chen. 2024b. Resilience of large language models for noisy instructions. arXiv preprint arXiv:2404.09754. [find](https://scholar.google.com/scholar?q=Bin+Wang%2C+Chengwei+Wei%2C+Zhengyuan+Liu%2C+Geyu+Lin%2C+and+Nancy+F+Chen%2E+2024b%2E+Resilience+of+large+language+models+for+noisy+instructions%2E+arXiv+preprint+arXiv%3A2404%2E09754%2E)
65. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023b. Voyager: An open-ended embodied agent with large language models. CoRR, abs/2305.16291. [find](https://scholar.google.com/scholar?q=Guanzhi+Wang%2C+Yuqi+Xie%2C+Yunfan+Jiang%2C+Ajay+Mandlekar%2C+Chaowei+Xiao%2C+Yuke+Zhu%2C+Linxi+Fan%2C+and+Anima+Anandkumar%2E+2023b%2E+Voyager%3A+An+open%2Dended+embodied+agent+with+large+language+models%2E+CoRR%2C+abs%2F2305%2E1)
66. Chengwei Wei, Yun-Cheng Wang, Bin Wang, and C.-C. Jay Kuo. 2024. An overview of language models: Recent developments and outlook. APSIPA Transactions on Signal and Information Processing, 13(2). [find](https://scholar.google.com/scholar?q=Chengwei+Wei%2C+Yun%2DCheng+Wang%2C+Bin+Wang%2C+and+C%2E%2DC%2E+Jay+Kuo%2E+2024%2E+An+overview+of+language+models%3A+Recent+developments+and+outlook%2E+APSIPA+Transactions+on+Signal+and+Information+Processing%2C+13%282%29%2E)
67. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In NeurIPS.
68. Bryan Wilie, Karissa Vincentio, Genta Indra Winata, Samuel Cahyawijaya, Xiaohong Li, Zhi Yuan Lim, Sidik Soleman, Rahmad Mahendra, Pascale Fung, Syafri Bahar, and Ayu Purwarianti. 2020. [IndoNLU: Benchmark and resources for evaluating Indonesian natural language understanding](https://md.research.elamur.ai/free/en/papers/wilie-2020-indonlu-benchmark-resources-evaluating-indonesian/). In Proceedings of the 1…
69. Lionel Wong, Gabriel Grand, Alexander K. Lew, Noah D. Goodman, Vikash K. Mansinghka, Jacob Andreas, and Joshua B. Tenenbaum. 2023. From word models to world models: Translating from natural language to the probabilistic language of thought. CoRR, abs/2306.12672. [find](https://scholar.google.com/scholar?q=Lionel+Wong%2C+Gabriel+Grand%2C+Alexander+K%2E+Lew%2C+Noah+D%2E+Goodman%2C+Vikash+K%2E+Mansinghka%2C+Jacob+Andreas%2C+and+Joshua+B%2E+Tenenbaum%2E+2023%2E+From+word+models+to+world+models%3A+Translating+from+natural+language+t)
70. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. [find](https://scholar.google.com/scholar?q=Yonghui+Wu%2C+Mike+Schuster%2C+Zhifeng+Chen%2C+Quoc+V+Le%2C+Mohammad+Norouzi%2C+Wolfgang+Macherey%2C+Maxim+Krikun%2C+Yuan+Cao%2C+Qin+Gao%2C+Klaus+Macherey%2C+et+al%2E+2016%2E+Google%E2%80%99s+neural+machine+translation+system%3A+Bridg)
71. Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, Fan Yang, et al. 2023. [Baichuan 2: Open large-scale language models](https://md.research.elamur.ai/free/en/papers/yang-2023-baichuan-2-open-large-scale/). arXiv preprint arXiv:2309.10305.
72. Jiacheng Ye, Xijia Tao, and Lingpeng Kong. 2023. Language versatilists vs. specialists: An empirical revisiting on multilingual transfer ability. arXiv preprint arXiv:2306.06688. — [arXiv:2306.06688](https://arxiv.org/abs/2306.06688) [find](https://scholar.google.com/scholar?q=Jiacheng+Ye%2C+Xijia+Tao%2C+and+Lingpeng+Kong%2E+2023%2E+Language+versatilists+vs%2E+specialists%3A+An+empirical+revisiting+on+multilingual+transfer+ability%2E+arXiv+preprint+arXiv%3A2306%2E06688%2E+%E2%80%94+%5BarXiv%3A2306%2E06688%5D%28)
73. Marcos Zampieri, Preslav Nakov, and Yves Scherrer. 2020. Natural language processing for similar languages, varieties, and dialects: A survey. Natural Language Engineering, 26(6):595–612. [find](https://scholar.google.com/scholar?q=Marcos+Zampieri%2C+Preslav+Nakov%2C+and+Yves+Scherrer%2E+2020%2E+Natural+language+processing+for+similar+languages%2C+varieties%2C+and+dialects%3A+A+survey%2E+Natural+Language+Engineering%2C+26%286%29%3A595%E2%80%93612%2E)
74. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Com)
75. Wenxuan Zhang, Sharifah Mahani Aljunied, Chang Gao, Yew Ken Chia, and Lidong Bing. 2023. M3exam: A multilingual, multimodal, multilevel benchmark for examining large language models. arXiv preprint arXiv:2306.05179. [find](https://scholar.google.com/scholar?q=Wenxuan+Zhang%2C+Sharifah+Mahani+Aljunied%2C+Chang+Gao%2C+Yew+Ken+Chia%2C+and+Lidong+Bing%2E+2023%2E+M3exam%3A+A+multilingual%2C+multimodal%2C+multilevel+benchmark+for+examining+large+language+models%2E+arXiv+preprint+ar)
76. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. [Calibrate before use: Improving few-shot performance of language models](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/). In International Conference on Machine Learning, pages 12697–12706. PMLR.
77. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. [find](https://scholar.google.com/scholar?q=Lianmin+Zheng%2C+Wei%2DLin+Chiang%2C+Ying+Sheng%2C+Siyuan+Zhuang%2C+Zhanghao+Wu%2C+Yonghao+Zhuang%2C+Zi+Lin%2C+Zhuohan+Li%2C+Dacheng+Li%2C+Eric%2E+P+Xing%2C+Hao+Zhang%2C+Joseph+E%2E+Gonzalez%2C+and+Ion+Stoica%2E+2023%2E+Judging+llm%2Das)
78. Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. 2023. [Agieval: A human-centric benchmark for evaluating foundation models](https://md.research.elamur.ai/free/en/papers/zhong-2024-agieval-human-centric-benchmark-evaluating/).
79. Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Lingpeng Kong, Jiajun Chen, Lei Li, and Shujian Huang. 2023a. [Multilingual machine translation with large language models: Empirical results and analysis](https://md.research.elamur.ai/free/en/papers/zhu-2024-multilingual-machine-translation-large-language/). arXiv preprint arXiv:2304.04675.
80. Wenhao Zhu, Yunzhe Lv, Qingxiu Dong, Fei Yuan, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. 2023b. Extrapolating large language models to non-english by aligning languages. arXiv preprint arXiv:2308.04948. [find](https://scholar.google.com/scholar?q=Wenhao+Zhu%2C+Yunzhe+Lv%2C+Qingxiu+Dong%2C+Fei+Yuan%2C+Jingjing+Xu%2C+Shujian+Huang%2C+Lingpeng+Kong%2C+Jiajun+Chen%2C+and+Lei+Li%2E+2023b%2E+Extrapolating+large+language+models+to+non%2Denglish+by+aligning+languages%2E+arXi)

*Updated 24 October 2026*