---
title: "CFQA — paper by Zhu et al. (2026) — summary and ideas"
description: "Most financial QA resources are in English, and few test question answering over full Chinese corporate annual reports. 18 references."
authors: ["Tianning Zhu", "Mo Liu", "Murathan Kurfali"]
year: 2026
venue: "The 7th Financial Narrative Processing Workshop"
source: "https://aclanthology.org/2026.fnp-1.3/"
doi: "10.63317/4e6fhrncfrbu"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhu-2026-cfqa-chinese-financial-question-answering/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhu-2026-cfqa-chinese-financial-question-answering/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/26/) / CFQA: A Chinese Financial Question Answering Benchmark from Corporate Annual Reports

# CFQA: A Chinese Financial Question Answering Benchmark from Corporate Annual Reports

**Authors:** Tianning Zhu, Mo Liu and [Murathan Kurfali](https://md.research.elamur.ai/free/en/authors/murathan-kurfali/) · **Year:** 2026 · **Venue:** [The 7th Financial Narrative Processing Workshop](https://md.research.elamur.ai/free/en/venues/fnp/) ([Financial Narrative Processing Workshop 2026](https://md.research.elamur.ai/free/en/venues/fnp/2026/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.fnp-1.3/) · [PDF](https://aclanthology.org/2026.fnp-1.3.pdf) · [DOI 10.63317/4e6fhrncfrbu](https://doi.org/10.63317/4e6fhrncfrbu) · [OpenAlex](https://openalex.org/W7166547097)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Most financial QA resources are in English, and few test question answering over full Chinese corporate annual reports. These documents are hard for models because they mix text, tables and charts, contain many numbers, and use dense accounting vocabulary. CFQA fills this gap. It contains 500 questions built from annual reports of 50 listed Chinese companies (fiscal years 2023–2025). Ten templates were applied to each report, producing five evenly split question types. A securities-industry investment advisor wrote and checked every gold answer against the source filing.

The authors also built a RAG pipeline that combines PyMuPDF and MinerU parsing, uses captions to stand in for visual content, and retrieves with BGE-M3. They compared it with a no-retrieval setup across six open-source LLMs. A rule-based detector then sorts wrong answers into fabrication, retrieval gaps or calculation mistakes.

Retrieval only slightly raised accuracy, and scores stayed very low overall. Still, the detector showed that RAG keeps genuine fabrication rare. Calculation and reasoning questions saw no significant benefit. CFQA therefore gives a reproducible way to measure where retrieval helps in Chinese financial QA and where it falls short.

## Key points
- **Dataset scale and balance:** 500 QA pairs from 50 annual-report PDFs (2023–2025), with 10 questions per report. Each of the five categories accounts for 20% of the questions: pulling out facts, listing items, computing comparisons, verifying whether a judgment holds, and analyzing reasoning.
- **Question generation:** A pool of 50 templates (10 per category) was filled with report metadata. Generation was assisted by Qwen3-8B-Instruct, and the results were then refined manually with real filenames and page references.
- **Document processing:** PyMuPDF text and MinerU layout output are aligned using dynamic programming. Qwen2.5-VL-32B-Instruct writes captions for uncaptioned tables and images. Chunks are embedded with BGE-M3 and retrieved by cosine similarity.
- **Error detector:** It combines three checks:
  - numerical matching with a relative tolerance of τ = 0.01;
  - text coverage;
  - reference (file/page) consistency.

  A hallucination is defined as either contradicting the evidence or asserting facts the evidence does not contain.
- **Results and availability:** Across six models, mean accuracy went from 7.53% without retrieval to 8.07% with RAG. Domain-adapted models gained most on fact extraction and judgment verification. The data, annotation guidelines and code are available on GitHub (zhutianning/Hallucination-detection-for-RAG).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Uppsala University](https://md.research.elamur.ai/free/en/universities/uppsala-university/) — University, Uppsala, Sweden · 65 works · 13,164 citations

Also on the byline: [CITIC Group](https://ror.org/037b6wy35).

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (2): [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,341 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 487 works

## Cite

Cite as: Tianning Zhu, Mo Liu and Murathan Kurfali. 2026. CFQA: A Chinese Financial Question Answering Benchmark from Corporate Annual Reports. The 7th Financial Narrative Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhu2026cfqa,
  title = {{CFQA: A Chinese Financial Question Answering Benchmark from Corporate Annual Reports}},
  author = {Tianning Zhu, Mo Liu and Murathan Kurfali},
  booktitle = {The 7th Financial Narrative Processing Workshop},
  year = {2026},
  doi = {10.63317/4e6fhrncfrbu},
  url = {https://doi.org/10.63317/4e6fhrncfrbu}
}
```

- [← previous CentaurTA: A Self-Improving Human-Agents Collaboration Framework for Thematic Analysis](https://md.research.elamur.ai/free/en/papers/wang-2026-centaurta-self-improving-human-agents/)
- [next → Chain-of-Memory: Lightweight Memory Construction with Dynamic Evolution for LLM Agents](https://md.research.elamur.ai/free/en/papers/xu-2026-chain-memory-lightweight-memory-construction/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/26/)

## References

18 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. [Factscore: Fine-grained atomic evaluation of factual precision in long form text generation](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/). In Proceedings of the 2023 Conference on Empirical Methods in Natural L…
2. Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda, and Pascale Fung. 2025. [Hallulens: Llm hallucination benchmark](https://md.research.elamur.ai/free/en/papers/bang-2025-hallulens-llm-hallucination-benchmark/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, pages 24128–24156, Vienna, Austria. Associati…
3. Xiangyu Peng, Can Qin, Zeyuan Chen, Ran Xu, Caiming Xiong, and Chien-Sheng Wu. 2025. Unidoc-bench: A unified benchmark for document-centric multimodal rag. arXiv preprint arXiv:2510.03663. — [arXiv:2510.03663](https://arxiv.org/abs/2510.03663) [find](https://scholar.google.com/scholar?q=Xiangyu+Peng%2C+Can+Qin%2C+Zeyuan+Chen%2C+Ran+Xu%2C+Caiming+Xiong%2C+and+Chien%2DSheng+Wu%2E+2025%2E+Unidoc%2Dbench%3A+A+unified+benchmark+for+document%2Dcentric+multimodal+rag%2E+arXiv+preprint+arXiv%3A2510%2E03663%2E+%E2%80%94+%5BarXiv%3A25)
4. Yiming Cui, Ting Liu, Wanxiang Che, Li Xiao, Zhipeng Chen, Wentao Ma, Shijin Wang, and Qwen Team. 2024. Qwen2.5 technical report. arXiv preprint arXiv:2407.10671. [find](https://scholar.google.com/scholar?q=Yiming+Cui%2C+Ting+Liu%2C+Wanxiang+Che%2C+Li+Xiao%2C+Zhipeng+Chen%2C+Wentao+Ma%2C+Shijin+Wang%2C+and+Qwen+Team%2E+2024%2E+Qwen2%2E5+technical+report%2E+arXiv+preprint+arXiv%3A2407%2E10671%2E)
5. Translation (English): What was the total operating revenue of PICC in 2022, in hundreds of millions of yuan? [find](https://scholar.google.com/scholar?q=Translation+%28English%29%3A+What+was+the+total+operating+revenue+of+PICC+in+2022%2C+in+hundreds+of+millions+of+yuan%3F)
6. Channdeth Sok, David Luz, and Yacine Haddam. 2025. Metarag: Metamorphic testing for hallucination detection in rag systems. arXiv preprint arXiv:2509.09360. — [arXiv:2509.09360](https://arxiv.org/abs/2509.09360) [find](https://scholar.google.com/scholar?q=Channdeth+Sok%2C+David+Luz%2C+and+Yacine+Haddam%2E+2025%2E+Metarag%3A+Metamorphic+testing+for+hallucination+detection+in+rag+systems%2E+arXiv+preprint+arXiv%3A2509%2E09360%2E+%E2%80%94+%5BarXiv%3A2509%2E09360%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
7. A.2. List Enumeration • Original (Chinese): 列举 2022 年中国人保前 五大客户的名称及其销售收入占比。 Manan Suri, Puneet Mathur, Franck Dernoncourt, Kanika Goswami, Ryan A. Rossi, and Dinesh Manocha. 2025. Visdom: Multi-document qa with visually rich elements using multimodal retrieval-augmented generation. In Proceedings of… [find](https://scholar.google.com/scholar?q=A%2E2%2E+List+Enumeration+%E2%80%A2+Original+%28Chinese%29%3A+%E5%88%97%E4%B8%BE+2022+%E5%B9%B4%E4%B8%AD%E5%9B%BD%E4%BA%BA%E4%BF%9D%E5%89%8D+%E4%BA%94%E5%A4%A7%E5%AE%A2%E6%88%B7%E7%9A%84%E5%90%8D%E7%A7%B0%E5%8F%8A%E5%85%B6%E9%94%80%E5%94%AE%E6%94%B6%E5%85%A5%E5%8D%A0%E6%AF%94%E3%80%82+Manan+Suri%2C+Puneet+Mathur%2C+Franck+Dernoncourt%2C+Kanika+Goswami%2C+Ryan+A%2E+Rossi%2C+and+Dinesh+Manocha%2E+2025%2E+Visdom%3A+Multi%2Ddocume)
8. Transliteration (Pinyin): Lièjǔ 2022 nián Zhōngguó Rénbǎo qián wǔ dà kèhù de míngchēng jí qí xiāoshòu shōurù zhànbǐ. [find](https://scholar.google.com/scholar?q=Transliteration+%28Pinyin%29%3A+Li%C3%A8j%C7%94+2022+ni%C3%A1n+Zh%C5%8Dnggu%C3%B3+R%C3%A9nb%C7%8Eo+qi%C3%A1n+w%C7%94+d%C3%A0+k%C3%A8h%C3%B9+de+m%C3%ADngch%C4%93ng+j%C3%AD+q%C3%AD+xi%C4%81osh%C3%B2u+sh%C5%8Dur%C3%B9+zh%C3%A0nb%C7%90%2E)
9. Translation (English): List the names and sales revenue percentages of PICC’s top five customers in 2022. [find](https://scholar.google.com/scholar?q=Translation+%28English%29%3A+List+the+names+and+sales+revenue+percentages+of+PICC%E2%80%99s+top+five+customers+in+2022%2E)
10. Shuting Wang, Jiejun Tan, Zhicheng Dou, and Ji-Rong Wen. 2025. [Omnieval: An omnidirectional and automatic rag evaluation benchmark in financial domain](https://md.research.elamur.ai/free/en/papers/wang-2025-omnieval-omnidirectional-automatic-rag-evaluation/). In Proceedings of the 2025 conference on empirical methods in natural language processing, pages 5737–5762.
11. A.3. Comparison & Calculation • Original (Chinese): 计算 2023 年中国神华营 业收入的同比增长率和增长金额。 • Transliteration (Pinyin): Jìsuàn 2023 nián Zhōngguó Shénhuá yíngyè shōurù de tóngbǐ zēngzhǎng lǜ hé zēngzhǎng jīn’é. [find](https://scholar.google.com/scholar?q=A%2E3%2E+Comparison+%26+Calculation+%E2%80%A2+Original+%28Chinese%29%3A+%E8%AE%A1%E7%AE%97+2023+%E5%B9%B4%E4%B8%AD%E5%9B%BD%E7%A5%9E%E5%8D%8E%E8%90%A5+%E4%B8%9A%E6%94%B6%E5%85%A5%E7%9A%84%E5%90%8C%E6%AF%94%E5%A2%9E%E9%95%BF%E7%8E%87%E5%92%8C%E5%A2%9E%E9%95%BF%E9%87%91%E9%A2%9D%E3%80%82+%E2%80%A2+Transliteration+%28Pinyin%29%3A+J%C3%ACsu%C3%A0n+2023+ni%C3%A1n+Zh%C5%8Dnggu%C3%B3+Sh%C3%A9nhu%C3%A1+y%C3%ADngy%C3%A8+sh%C5%8Dur%C3%B9+de+t%C3%B3ngb%C7%90+z%C4%93ngzh%C7%8Eng+l%C7%9C+h%C3%A9+z%C4%93ngzh%C7%8Eng+j%C4%ABn%E2%80%99%C3%A9)
12. Yuming Yang, Jiang Zhong, Li Jin, Jingwang Huang, Jingpeng Gao, Qing Liu, Yang Bai, Jingyuan Zhang, Rui Jiang, and Kaiwen Wei. 2025. Benchmarking multimodal rag through a chart-based document question-answering generation framework. arXiv preprint arXiv:2502.14864. — [arXiv:2502.14864](https://arxiv.org/abs/2502.14864) [find](https://scholar.google.com/scholar?q=Yuming+Yang%2C+Jiang+Zhong%2C+Li+Jin%2C+Jingwang+Huang%2C+Jingpeng+Gao%2C+Qing+Liu%2C+Yang+Bai%2C+Jingyuan+Zhang%2C+Rui+Jiang%2C+and+Kaiwen+Wei%2E+2025%2E+Benchmarking+multimodal+rag+through+a+chart%2Dbased+document+question)
13. Translation (English): Calculate the year-on-year growth rate and growth amount of China Shenhua’s operating revenue in 2023. [find](https://scholar.google.com/scholar?q=Translation+%28English%29%3A+Calculate+the+year%2Don%2Dyear+growth+rate+and+growth+amount+of+China+Shenhua%E2%80%99s+operating+revenue+in+2023%2E)
14. Chaofa Yuan, Yuhan Liu, Rongdi Yin, Jun Zhang, Qinling Zhu, Ruibin Mao, and Ruifeng Xu. 2020. Target-based sentiment annotation in chinese financial news. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 5040–5045, Marseille, France. European Language Resources Assoc… [find](https://scholar.google.com/scholar?q=Chaofa+Yuan%2C+Yuhan+Liu%2C+Rongdi+Yin%2C+Jun+Zhang%2C+Qinling+Zhu%2C+Ruibin+Mao%2C+and+Ruifeng+Xu%2E+2020%2E+Target%2Dbased+sentiment+annotation+in+chinese+financial+news%2E+In+Proceedings+of+the+Twelfth+Language+Resour)
15. Judgment & Verification • Original (Chinese): 2024 年建设银行是否存 在商誉减值？如果是，减值金额是多少？ • Transliteration (Pinyin): 2024 nián Jiàn-shè Yínháng shìfǒu cúnzài shāngyù jiǎnzhí? Rúguǒ shì, jiǎnzhí jīn’é shì duōshǎo? [find](https://scholar.google.com/scholar?q=Judgment+%26+Verification+%E2%80%A2+Original+%28Chinese%29%3A+2024+%E5%B9%B4%E5%BB%BA%E8%AE%BE%E9%93%B6%E8%A1%8C%E6%98%AF%E5%90%A6%E5%AD%98+%E5%9C%A8%E5%95%86%E8%AA%89%E5%87%8F%E5%80%BC%EF%BC%9F%E5%A6%82%E6%9E%9C%E6%98%AF%EF%BC%8C%E5%87%8F%E5%80%BC%E9%87%91%E9%A2%9D%E6%98%AF%E5%A4%9A%E5%B0%91%EF%BC%9F+%E2%80%A2+Transliteration+%28Pinyin%29%3A+2024+ni%C3%A1n+Ji%C3%A0n%2Dsh%C3%A8+Y%C3%ADnh%C3%A1ng+sh%C3%ACf%C7%92u+c%C3%BAnz%C3%A0i+sh%C4%81ngy%C3%B9+ji%C7%8Enzh%C3%AD%3F+R%C3%BAgu%C7%92+sh%C3%AC%2C+ji%C7%8Enzh%C3%AD+j%C4%ABn%E2%80%99%C3%A9+sh%C3%AC+du%C5%8Dsh%C7%8E)
16. Translation (English): Did CCB have goodwill impairment in 2024? If so, what is the impairment amount? [find](https://scholar.google.com/scholar?q=Translation+%28English%29%3A+Did+CCB+have+goodwill+impairment+in+2024%3F+If+so%2C+what+is+the+impairment+amount%3F)
17. Shun Zheng, Wei Cao, Wei Xu, and Jiang Bian. 2019. [Doc2edag: An end-to-end document-level framework for chinese financial event extraction](https://md.research.elamur.ai/free/en/papers/zheng-2019-doc2edag-end-end-document-level/). In Proceedings of EMNLP-IJCNLP, pages 337–346, Hong Kong, China. Association for Computational Linguistics.
18. A.5. Reasoning & Analysis • Original (Chinese): ROE 分解：分析 2024 年 杜邦三因素对工商银行 ROE 变动的贡献。 • Transliteration (Pinyin): ROE fēnjiě: fēnxī 2024 nián Dùbāng sān yīnsù duì Gōngshāng Yínháng ROE biàndòng de gòngxiàn. [find](https://scholar.google.com/scholar?q=A%2E5%2E+Reasoning+%26+Analysis+%E2%80%A2+Original+%28Chinese%29%3A+ROE+%E5%88%86%E8%A7%A3%EF%BC%9A%E5%88%86%E6%9E%90+2024+%E5%B9%B4+%E6%9D%9C%E9%82%A6%E4%B8%89%E5%9B%A0%E7%B4%A0%E5%AF%B9%E5%B7%A5%E5%95%86%E9%93%B6%E8%A1%8C+ROE+%E5%8F%98%E5%8A%A8%E7%9A%84%E8%B4%A1%E7%8C%AE%E3%80%82+%E2%80%A2+Transliteration+%28Pinyin%29%3A+ROE+f%C4%93nji%C4%9B%3A+f%C4%93nx%C4%AB+2024+ni%C3%A1n+D%C3%B9b%C4%81ng+s%C4%81n+y%C4%ABns%C3%B9+du%C3%AC+G%C5%8Dngsh%C4%81ng+Y%C3%ADnh%C3%A1ng+ROE+bi%C3%A0nd%C3%B2ng+de+g%C3%B2)

*Updated 2 November 2026*