---
title: "OmniEval — paper by Wang et al. (2025) — summary and ideas"
description: "Existing RAG benchmarks tend to cover few scenarios and judge systems along few… 74.4% accuracy. 32 references, cited by 11 papers here."
authors: ["Shuting Wang", "Jiejun Tan", "Zhicheng Dou", "Ji-Rong Wen"]
year: 2025
venue: "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2025.emnlp-main.292/"
doi: "10.18653/v1/2025.emnlp-main.292"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2025-omnieval-omnidirectional-automatic-rag-evaluation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2025-omnieval-omnidirectional-automatic-rag-evaluation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/) / OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain

# OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain

**Authors:** [Shuting Wang](https://md.research.elamur.ai/free/en/authors/shuting-wang/), Jiejun Tan, [Zhicheng Dou](https://md.research.elamur.ai/free/en/authors/zhicheng-dou/) and [Ji-Rong Wen](https://md.research.elamur.ai/free/en/authors/ji-rong-wen/) · **Year:** 2025 · **Venue:** [Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2025](https://md.research.elamur.ai/free/en/venues/emnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.emnlp-main.292/) · [PDF](https://aclanthology.org/2025.emnlp-main.292.pdf) · [DOI 10.18653/v1/2025.emnlp-main.292](https://doi.org/10.18653/v1/2025.emnlp-main.292) · [OpenAlex](https://openalex.org/W4416037024)

**Citations:** 13 citations (OpenAlex, 2026-10-14) · [11](https://md.research.elamur.ai/free/en/papers/wang-2025-omnieval-omnidirectional-automatic-rag-evaluation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Existing RAG benchmarks tend to cover few scenarios and judge systems along few dimensions. This makes it hard to see how retrieval-augmented systems behave in specialised fields such as finance. OmniEval addresses this with a Chinese-language financial benchmark built on a two-axis grid that crosses five task types with 16 finance topics. The authors assembled a retrieval corpus from public financial challenge data, finance-related Wikipedia pages, a pretraining corpus and crawled official agency sites. They then used a chained pipeline in which GPT-4 classifies topics, generates question–answer instances and screens their quality, with human reviewers correcting a subset.

Evaluation covers both stages of the pipeline:
- **Retrieval** is scored with rule-based metrics such as MAP.
- **Generation** is scored with Rouge-style metrics plus learned judges that check aspects like hallucination and numerical correctness.

Experiments pair several retrievers with several large open LLMs. Scores vary noticeably across topic–task cells, and there is considerable headroom overall. The grid view helps locate specific weaknesses of a RAG system rather than hiding them in one average.

## Key points
- **Taxonomy:** five tasks (extractive, multi-hop reasoning, contrast, long-form, conversational QA) crossed with 16 financial subtopics, such as Fund, Stock Market and AML.
- **Data:**
  - 11.4k auto-generated test items;
  - 1.7k human-annotated test items;
  - 3k auto-generated training items.
- **Corpus:** drawn from BSCF, FinGLM, Wiki-Fin, BAAI-Fin and official web pages. It was chunked via LlamaIndex into 2048-length passages with 256 overlap.
- **Judge model:** a fine-tuned Qwen2.5-7B-Instruct reached 74.4% accuracy, outperforming prompted LLM judges.
- **Systems tested:**
  - Retrievers: BGE-M3, BGE-large-zh, GTE-Qwen2-1.5b and jina-zh.
  - Generators: Qwen2.5-72B-Instruct, Llama3.1-70B-Instruct, Deepseek-v2-chat and Yi15-34B.
  - Code is released at github.com/RUC-NLPIR/OmniEval.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Renmin University of China](https://md.research.elamur.ai/free/en/universities/renmin-university-of-china/) — University, Beijing, China · 139 works · 17,752 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (1): [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,341 works

## Cite

Cite as: Shuting Wang, Jiejun Tan, Zhicheng Dou and Ji-Rong Wen. 2025. OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2025omnieval,
  title = {{OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain}},
  author = {Shuting Wang, Jiejun Tan, Zhicheng Dou and Ji-Rong Wen},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.292},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.292}
}
```

- [← previous MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe](https://md.research.elamur.ai/free/en/papers/yu-2025-minicpm-v-4-5/)
- [next → Personalized Generation In Large Model Era: A Survey](https://md.research.elamur.ai/free/en/papers/xu-2025-personalized-generation-large-model-era/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/)

## References

32 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jian Chen, Peilin Zhou, Yining Hua, Loh Xin, Kehui Chen, Ziyuan Li, Bing Zhu, and Junwei Liang. 2024a. Fintextqa: A dataset for long-form financial question answering. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangko… [find](https://scholar.google.com/scholar?q=Jian+Chen%2C+Peilin+Zhou%2C+Yining+Hua%2C+Loh+Xin%2C+Kehui+Chen%2C+Ziyuan+Li%2C+Bing+Zhu%2C+and+Junwei+Liang%2E+2024a%2E+Fintextqa%3A+A+dataset+for+long%2Dform+financial+question+answering%2E+In+Proceedings+of+the+62nd+Annua)
2. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024b. [BGE m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/). CoRR, abs/2402.03216.
3. Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. 2024c. [Benchmarking large language models in retrieval-augmented generation](https://md.research.elamur.ai/free/en/papers/chen-2023-benchmarking-large-language-models-retrieval/). In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth…
4. Wei Chen, Qiushi Wang, Zefei Long, Xianyin Zhang, Zhongtian Lu, Bingxuan Li, Siyuan Wang, Jiarong Xu, Xiang Bai, Xuanjing Huang, and Zhongyu Wei. 2023. Disc-finllm: A chinese financial large language model based on multiple experts fine-tuning. CoRR, abs/2310.15205. — [arXiv:2310.15205](https://arxiv.org/abs/2310.15205) [find](https://scholar.google.com/scholar?q=Wei+Chen%2C+Qiushi+Wang%2C+Zefei+Long%2C+Xianyin+Zhang%2C+Zhongtian+Lu%2C+Bingxuan+Li%2C+Siyuan+Wang%2C+Jiarong+Xu%2C+Xiang+Bai%2C+Xuanjing+Huang%2C+and+Zhongyu+Wei%2E+2023%2E+Disc%2Dfinllm%3A+A+chinese+financial+large+language+)
5. Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. 2021. [Finqa: A dataset of numerical reasoning over financial data](https://md.research.elamur.ai/free/en/papers/chen-2021-finqa-dataset-numerical-reasoning-over/). Proceedings of EMNLP 2021.
6. Zhiyu Chen, Shiyang Li, Charese Smiley, Zhiqiang Ma, Sameena Shah, and William Yang Wang. 2022. [Convfinqa: Exploring the chain of numerical reasoning in conversational finance question answering](https://md.research.elamur.ai/free/en/papers/chen-2022-convfinqa-exploring-chain-numerical-reasoning/). Proceedings of EMNLP 2022.
7. DeepSeek-AI. 2024. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. Preprint, arXiv:2405.04434. [find](https://scholar.google.com/scholar?q=DeepSeek%2DAI%2E+2024%2E+Deepseek%2Dv2%3A+A+strong%2C+economical%2C+and+efficient+mixture%2Dof%2Dexperts+language+model%2E+Preprint%2C+arXiv%3A2405%2E04434%2E)
8. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurélien Rodriguez,… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
9. Michael Günther, Jackmin Ong, Isabelle Mohr, Alaeddine Abdessalem, Tanguy Abel, Mohammad Kalim Akram, Susana Guzman, Georgios Mastrapas, Saba Sturua, Bo Wang, Maximilian Werk, Nan Wang, and Han Xiao. 2023. [Jina embeddings 2: 8192-token general-purpose text embeddings for long documents](https://md.research.elamur.ai/free/en/papers/gunther-2023-jina-embeddings-2-8192-token/). CoRR, abs/23…
10. Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, and Bertie Vidgen. 2023. Financebench: A new benchmark for financial question answering. CoRR, abs/2311.11944. — [arXiv:2311.11944](https://arxiv.org/abs/2311.11944) [find](https://scholar.google.com/scholar?q=Pranab+Islam%2C+Anand+Kannappan%2C+Douwe+Kiela%2C+Rebecca+Qian%2C+Nino+Scherrer%2C+and+Bertie+Vidgen%2E+2023%2E+Financebench%3A+A+new+benchmark+for+financial+question+answering%2E+CoRR%2C+abs%2F2311%2E11944%2E+%E2%80%94+%5BarXiv%3A2311%2E11)
11. Xiang Li, Zhenyu Li, Chen Shi, Yong Xu, Qing Du, Mingkui Tan, and Jun Huang. 2024. Alphafin: Benchmarking financial analysis with retrieval-augmented stock-chain framework. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC… [find](https://scholar.google.com/scholar?q=Xiang+Li%2C+Zhenyu+Li%2C+Chen+Shi%2C+Yong+Xu%2C+Qing+Du%2C+Mingkui+Tan%2C+and+Jun+Huang%2E+2024%2E+Alphafin%3A+Benchmarking+financial+analysis+with+retrieval%2Daugmented+stock%2Dchain+framework%2E+In+Proceedings+of+the+2024+)
12. Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. [Towards general text embeddings with multi-stage contrastive learning](https://md.research.elamur.ai/free/en/papers/li-2023-towards-general-text-embeddings-multi/). arXiv preprint arXiv:2308.03281.
13. Yi Liu, Lianzhe Huang, Shicheng Li, Sishuo Chen, Hao Zhou, Fandong Meng, Jie Zhou, and Xu Sun. 2023. RECALL: A benchmark for llms robustness against external counterfactual knowledge. CoRR, abs/2311.08147. — [arXiv:2311.08147](https://arxiv.org/abs/2311.08147) [find](https://scholar.google.com/scholar?q=Yi+Liu%2C+Lianzhe+Huang%2C+Shicheng+Li%2C+Sishuo+Chen%2C+Hao+Zhou%2C+Fandong+Meng%2C+Jie+Zhou%2C+and+Xu+Sun%2E+2023%2E+RECALL%3A+A+benchmark+for+llms+robustness+against+external+counterfactual+knowledge%2E+CoRR%2C+abs%2F2311%2E0)
14. Yuanjie Lyu, Zhiyu Li, Simin Niu, Feiyu Xiong, Bo Tang, Wenjin Wang, Hao Wu, Huanyong Liu, Tong Xu, and Enhong Chen. 2024. CRUD-RAG: A comprehensive chinese benchmark for retrieval-augmented generation of large language models. CoRR, abs/2401.17043. — [arXiv:2401.17043](https://arxiv.org/abs/2401.17043) [find](https://scholar.google.com/scholar?q=Yuanjie+Lyu%2C+Zhiyu+Li%2C+Simin+Niu%2C+Feiyu+Xiong%2C+Bo+Tang%2C+Wenjin+Wang%2C+Hao+Wu%2C+Huanyong+Liu%2C+Tong+Xu%2C+and+Enhong+Chen%2E+2024%2E+CRUD%2DRAG%3A+A+comprehensive+chinese+benchmark+for+retrieval%2Daugmented+generatio)
15. Isabelle Mohr, Markus Krimmel, Saba Sturua, Mohammad Kalim Akram, Andreas Koukounas, Michael Günther, Georgios Mastrapas, Vinit Ravishankar, Joan Fontanals Martínez, Feng Wang, et al. 2024. Multi-task contrastive learning for 8192-token bilingual text embeddings. arXiv preprint arXiv:2402.17016. — [arXiv:2402.17016](https://arxiv.org/abs/2402.17016) [find](https://scholar.google.com/scholar?q=Isabelle+Mohr%2C+Markus+Krimmel%2C+Saba+Sturua%2C+Mohammad+Kalim+Akram%2C+Andreas+Koukounas%2C+Michael+G%C3%BCnther%2C+Georgios+Mastrapas%2C+Vinit+Ravishankar%2C+Joan+Fontanals+Mart%C3%ADnez%2C+Feng+Wang%2C+et+al%2E+2024%2E+Multi%2Dtask)
16. OpenAI. 2023. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). abs/2303.08774.
17. CoRR, Jon Saad-Falcon, Omar Khattab, Christopher Potts, and Matei Zaharia. 2024. ARES: an automated evaluation framework for retrieval-augmented generation systems. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Te… [find](https://scholar.google.com/scholar?q=CoRR%2C+Jon+Saad%2DFalcon%2C+Omar+Khattab%2C+Christopher+Potts%2C+and+Matei+Zaharia%2E+2024%2E+ARES%3A+an+automated+evaluation+framework+for+retrieval%2Daugmented+generation+systems%2E+In+Proceedings+of+the+2024+Conferen)
18. Julio Cesar Salinas Alvarado, Karin Verspoor, and Timothy Baldwin. 2015. Domain adaption of named entity recognition to support credit risk assessment. In Proceedings of the Australasian Language Technology Association Workshop 2015, pages 84–90, Parramatta, Australia. [find](https://scholar.google.com/scholar?q=Julio+Cesar+Salinas+Alvarado%2C+Karin+Verspoor%2C+and+Timothy+Baldwin%2E+2015%2E+Domain+adaption+of+named+entity+recognition+to+support+credit+risk+assessment%2E+In+Proceedings+of+the+Australasian+Language+Tech)
19. Raj Sanjay Shah, Kunal Chawla, Dheeraj Eidnani, Agam Shah, Wendi Du, Sudheer Chava, Natraj Raman, Charese Smiley, Jiaao Chen, and Diyi Yang. 2022. WHEN FLUE MEETS FLANG: benchmarks and large pre-trained language model for financial domain. CoRR, abs/2211.00083. [find](https://scholar.google.com/scholar?q=Raj+Sanjay+Shah%2C+Kunal+Chawla%2C+Dheeraj+Eidnani%2C+Agam+Shah%2C+Wendi+Du%2C+Sudheer+Chava%2C+Natraj+Raman%2C+Charese+Smiley%2C+Jiaao+Chen%2C+and+Diyi+Yang%2E+2022%2E+WHEN+FLUE+MEETS+FLANG%3A+benchmarks+and+large+pre%2Dtrain)
20. Ankur Sinha and Tanmay Khandait. 2020. Impact of news on the commodity market: Dataset and results. CoRR, abs/2009.04202. — [arXiv:2009.04202](https://arxiv.org/abs/2009.04202) [find](https://scholar.google.com/scholar?q=Ankur+Sinha+and+Tanmay+Khandait%2E+2020%2E+Impact+of+news+on+the+commodity+market%3A+Dataset+and+results%2E+CoRR%2C+abs%2F2009%2E04202%2E+%E2%80%94+%5BarXiv%3A2009%2E04202%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2009%2E04202%29)
21. Yejun Soun, Jaemin Yoo, Minyong Cho, Jihyeong Jeon, and U Kang. 2022. Accurate stock movement prediction with self-supervised learning from sparse noisy tweets. In 2022 IEEE International Conference on Big Data (Big Data), pages 1691–1700. IEEE Computer Society. [find](https://scholar.google.com/scholar?q=Yejun+Soun%2C+Jaemin+Yoo%2C+Minyong+Cho%2C+Jihyeong+Jeon%2C+and+U+Kang%2E+2022%2E+Accurate+stock+movement+prediction+with+self%2Dsupervised+learning+from+sparse+noisy+tweets%2E+In+2022+IEEE+International+Conference+o)
22. Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Mansooreh Karami, Jundong Li, Lu Cheng, and Huan Liu. 2024. [Large language models for data annotation and synthesis: A survey](https://md.research.elamur.ai/free/en/papers/tan-2024-large-language-models-data-annotation/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processin…
23. Qwen Team. 2024. Qwen2.5: A party of foundation models. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2024%2E+Qwen2%2E5%3A+A+party+of+foundation+models%2E)
24. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). [find](https://scholar.google.com/scholar?q=Nandan+Thakur%2C+Nils+Reimers%2C+Andreas+R%C3%BCckl%C3%A9%2C+Abhishek+Srivastava%2C+and+Iryna+Gurevych%2E+2021%2E+BEIR%3A+A+heterogeneous+benchmark+for+zero%2Dshot+evaluation+of+information+retrieval+models%2E+In+Thirty%2Dfifth+Co)
25. Shuting Wang, Jiongnan Liu, Shiren Song, Jiehan Cheng, Yuqi Fu, Peidong Guo, Kun Fang, Yutao Zhu, and Zhicheng Dou. 2024a. [Domainrag: A chinese Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff](https://md.research.elamur.ai/free/en/papers/xiao-2023-c-pack-packed-resources/). 2023b. C-pack: Packaged resources to advance general chinese embedding. Preprint, arXiv:2309.…
26. Qianqian Xie, Weiguang Han, Zhengyu Chen, Ruoyu Xiang, Xiao Zhang, Yueru He, Mengxi Xiao, Dong Li, Yongfu Dai, Duanyu Feng, Yijing Xu, Haoqiang Kang, Ziyan Kuang, Chenhan Yuan, Kailai Yang, Zheheng Luo, Tianlin Zhang, Zhiwei Liu, Guojun Xiong, Zhiyang Deng, Yuechen Jiang, Zhiyuan Yao, Haohang Li, Ya… — [arXiv:2402.12659](https://arxiv.org/abs/2402.12659) [find](https://scholar.google.com/scholar?q=Qianqian+Xie%2C+Weiguang+Han%2C+Zhengyu+Chen%2C+Ruoyu+Xiang%2C+Xiao+Zhang%2C+Yueru+He%2C+Mengxi+Xiao%2C+Dong+Li%2C+Yongfu+Dai%2C+Duanyu+Feng%2C+Yijing+Xu%2C+Haoqiang+Kang%2C+Ziyan+Kuang%2C+Chenhan+Yuan%2C+Kailai+Yang%2C+Zheheng+Lu)
27. Qianqian Xie, Weiguang Han, Xiao Zhang, Yanzhao Lai, Min Peng, Alejandro Lopez-Lira, and Jimin Huang. 2023. PIXIU: A large language model, instruction data and evaluation benchmark for finance. CoRR, abs/2306.05443. — [arXiv:2306.05443](https://arxiv.org/abs/2306.05443) [find](https://scholar.google.com/scholar?q=Qianqian+Xie%2C+Weiguang+Han%2C+Xiao+Zhang%2C+Yanzhao+Lai%2C+Min+Peng%2C+Alejandro+Lopez%2DLira%2C+and+Jimin+Huang%2E+2023%2E+PIXIU%3A+A+large+language+model%2C+instruction+data+and+evaluation+benchmark+for+finance%2E+CoRR%2C+)
28. Guangzhi Xiong, Qiao Jin, Zhiyong Lu, and Aidong Zhang. 2024. [Benchmarking retrieval-augmented generation for medicine](https://md.research.elamur.ai/free/en/papers/xiong-2024-benchmarking-retrieval-augmented-generation-medicine/). In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, pages 6233–6251. Association for Computational Li…
29. Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, Kaidong Yu, Peng Liu, Qiang Liu, Shawn Yue, Senbin Yang, Shiming Yang, Tao Yu, Wen Xie, Wenhao Huang, Xiaohui Hu, Xiaoyi Ren, Xinyao Niu, Pengcheng Nie, Yuchi Xu, Yudong Liu, Yue… — [Yi: Open Foundation Models by 01.AI](https://md.research.elamur.ai/free/en/papers/ai-2024-yi-open-foundation-models-01/)
30. Xiaodong Yu, Hao Cheng, Xiaodong Liu, Dan Roth, and Jianfeng Gao. 2024. Reeval: Automatic hallucination evaluation for retrieval-augmented large language models via transferable adversarial attacks. In Findings of the Association for Computational Linguistics: NAACL 2024, Mexico City, Mexico, June 1… [find](https://scholar.google.com/scholar?q=Xiaodong+Yu%2C+Hao+Cheng%2C+Xiaodong+Liu%2C+Dan+Roth%2C+and+Jianfeng+Gao%2E+2024%2E+Reeval%3A+Automatic+hallucination+evaluation+for+retrieval%2Daugmented+large+language+models+via+transferable+adversarial+attacks%2E+I)
31. Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023a. [C-pack: Packaged resources to advance general chinese embedding](https://md.research.elamur.ai/free/en/papers/xiao-2023-c-pack-packed-resources/). CoRR, abs/2309.07597.
32. Shuting Wang, Xin Yu, Mang Wang, Weipeng Chen, Yutao Zhu, and Zhicheng Dou. 2024b. [Richrag: Crafting rich responses for multi-faceted queries in retrieval-augmented generation](https://md.research.elamur.ai/free/en/papers/wang-2025-richrag-crafting-rich-responses-multi/). CoRR, abs/2406.12566. *Figure 8: Rouge-L scores of generators on topic-specific auto-generated subsets.* benchmark for eval…

*Updated 31 October 2026*