---
title: "When Good OCR Is Not Enough — paper by Sun et al. (2026)"
description: "The authors present InduOCRBench to address this gap. 10.56 points. 21 references. Among them: 0."
authors: ["Lin Sun", "Wangdexian", "Jingang Huang", "Linglin Zhang", "Change Jia", "Zhengwei Cheng", "Xiangzheng Zhang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)"
source: "https://aclanthology.org/2026.acl-industry.60/"
doi: "10.18653/v1/2026.acl-industry.60"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sun-2026-when-good-ocr-is-not/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sun-2026-when-good-ocr-is-not/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/48/) / When Good OCR Is Not Enough: Benchmarking OCR Robustness for Retrieval-Augmented Generation

# When Good OCR Is Not Enough: Benchmarking OCR Robustness for Retrieval-Augmented Generation

**Authors:** [Lin Sun](https://md.research.elamur.ai/free/en/authors/lin-sun/), Wangdexian, Jingang Huang, [Linglin Zhang](https://md.research.elamur.ai/free/en/authors/linglin-zhang/), [Change Jia](https://md.research.elamur.ai/free/en/authors/change-jia/), Zhengwei Cheng and [Xiangzheng Zhang](https://md.research.elamur.ai/free/en/authors/xiangzheng-zhang/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-industry.60/) · [PDF](https://aclanthology.org/2026.acl-industry.60.pdf) · [DOI 10.18653/v1/2026.acl-industry.60](https://doi.org/10.18653/v1/2026.acl-industry.60) · [OpenAlex](https://openalex.org/W7166879473)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Industrial RAG systems depend on OCR to turn visual documents into text. OCR is usually judged by character- or word-level scores such as CER, WER and edit distance, and these say little about whether retrieval and answering still succeed afterwards. The authors present InduOCRBench to address this gap. They drew a stratified sample from 10,000 enterprise documents spanning 12 industries, yielding 570 documents (3,402 pages) in 11 difficult categories. These include watermarks, ultra-wide and ultra-long pages, historical books, cross-page tables and meaning carried by text styling. Annotations follow a hybrid Markdown scheme: HTML for tables, LaTeX for formulas, and explicit visual attributes. Ten OCR systems were tested, covering pipeline tools, general VLMs, OCR-specialized VLMs and a commercial API. Each feeds one fixed OCR-first RAG pipeline scored with RAGAS. Models that excel on OmniDocBench lose substantial accuracy here, with tables and formulas suffering most. Low transcription error does not ensure correct answers, because discarded structure and formatting break retrieval or generation. Teams choosing OCR for document QA therefore need downstream-aware evaluation.

## Key points
- **Score drops versus OmniDocBench:** average scores fall sharply on the new benchmark.
  - PP-StructureV3: 86.7 → 60.3
  - PaddleOCR-VL: 92.9 → 78.2
  - GPT-4o: 75.0 → 52.0
  - Hunyuan-OCR keeps 86.1 on text, but its table TEDS falls from 91.8 to 52.5.
- **Extreme layouts and reading order:** GPT-4o scores only 2.8% on UltraLong and 3.3% on UltraWide pages. On HistoryBooks, MinerU2 collapses to 0.1%, while Qwen3-VL-235B reaches 87.1%.
- **RAG accuracy against a ground-truth ceiling:** ground-truth text yields 100% RAG accuracy. UltraLong shows the largest shortfall, with the best system (Qwen3-VL-235B) at 59.50%. On HistoryBooks, RAG accuracy ranges from 8.63% (GPT-4o) to 83.45% (Hunyuan-OCR).
- **Where failures arise (PaddleOCR-VL vs. ground truth):** three patterns appear.
  - Retrieval-dominated: UltraWide and UltraLong lose roughly 20–22 points in both recall and accuracy.
  - Generation-sensitive: VisualStyle loses 10.56 points of recall but 41.19 of accuracy.
  - Compounded: MultiColumn and CrosspageTbl degrade at both stages.
- **Robustness of the findings:** the pattern holds when BGE-M3 is replaced by BM25 or Qwen3-8B-Embedding. It also holds when HTML-tree chunking is replaced by LLM semantic chunking, where the gap is 16.22 vs. 16.72.
- **Setup and availability:** metrics are Answer Accuracy and Context Recall, judged by GPT-OSS-120B. Annotation audits required 98% accuracy. Data and code are released at github.com/Qihoo360/InduOCRBench.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 473 works

## Cite

Cite as: Lin Sun, Wangdexian, Jingang Huang, Linglin Zhang, Change Jia, Zhengwei Cheng and Xiangzheng Zhang. 2026. When Good OCR Is Not Enough: Benchmarking OCR Robustness for Retrieval-Augmented Generation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{sun2026when,
  title = {{When Good OCR Is Not Enough: Benchmarking OCR Robustness for Retrieval-Augmented Generation}},
  author = {Lin Sun, Wangdexian, Jingang Huang, Linglin Zhang, Change Jia, Zhengwei Cheng and Xiangzheng Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-industry.60},
  url = {https://doi.org/10.18653/v1/2026.acl-industry.60}
}
```

- [← previous When Facts Change: Temporal Knowledge Conflict Resolution in LLMs](https://md.research.elamur.ai/free/en/papers/wallat-2026-when-facts-change-temporal-knowledge/)
- [next → When LLMs Disagree with Human Experts: Understanding LLM Annotation Failures in Nutrition Misinformation through Hierarchical Error Analysis using Seed Oil Narratives](https://md.research.elamur.ai/free/en/papers/shah-2026-when-llms-disagree-human-experts/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/48/)

## References

21 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. OpenAI Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K. Arora, Yu Bai, Bowen Baker, Hai-Biao Bao, Boaz Barak, Ally Bennett, Tyler Bertao, Nivedita Brett, Eugene Brevdo, Greg Brockman, Sébastien Bubeck, Cheng Chang, Kai Chen, and 105 others. 2025. [gpt-oss-120b…](https://md.research.elamur.ai/free/en/papers/agarwal-2025-gpt-oss-120b-gpt-oss/)
2. Avinash Anand, Raj Jaiswal, Pijush Bhuyan, Mohit Gupta, Siddhesh Bangar, Md. Modassir Imam, Rajiv Ratn Shah, and Shin’ichi Satoh. 2023. Tc-ocr: Tablecraft ocr for efficient detection & recognition of table structure & content. In Proceedings of the 1st International Workshop on Deep Multimodal Learn… [find](https://scholar.google.com/scholar?q=Avinash+Anand%2C+Raj+Jaiswal%2C+Pijush+Bhuyan%2C+Mohit+Gupta%2C+Siddhesh+Bangar%2C+Md%2E+Modassir+Imam%2C+Rajiv+Ratn+Shah%2C+and+Shin%E2%80%99ichi+Satoh%2E+2023%2E+Tc%2Docr%3A+Tablecraft+ocr+for+efficient+detection+%26+recognition+of+)
3. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xiong-Hui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Rongyao Fang, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Qidong Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, and 46 others. 2025. [Qwen3-vl technical report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen3-vl-technical-report/). ArXiv, a…
4. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. [Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/). Preprint, arXiv:2402.03216.
5. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, and 1 others.… — [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/)
6. Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Xueqing Wang, Changda Zhou, Hongen Liu, Manhui Lin, Yue Zhang, Yubo Zhang, Handong Zheng, Jing Zhang, Jun Zhang, Yi Liu, Dianhai Yu, and Yanjun Ma. 2025a. Paddleocr-vl: Boosting multilingual document parsing via a 0.9b ultra-c… — [arXiv:2510.14528](https://arxiv.org/abs/2510.14528) [find](https://scholar.google.com/scholar?q=Cheng+Cui%2C+Ting+Sun%2C+Suyin+Liang%2C+Tingquan+Gao%2C+Zelun+Zhang%2C+Jiaxuan+Liu%2C+Xueqing+Wang%2C+Changda+Zhou%2C+Hongen+Liu%2C+Manhui+Lin%2C+Yue+Zhang%2C+Yubo+Zhang%2C+Handong+Zheng%2C+Jing+Zhang%2C+Jun+Zhang%2C+Yi+Liu%2C+Dianh)
7. Cheng Cui, Ting Sun, Manhui Lin, Tingquan Gao, Yubo Zhang, Jiaxuan Liu, Xueqing Wang, Zelun Zhang, Changda Zhou, Hongen Liu, Yue Zhang, Wenyu Lv, Kui Huang, Yichao Zhang, Jing Zhang, Jun Zhang, Yi Liu, Dianhai Yu, and Yanjun Ma. 2025b. Paddleocr 3.0 technical report. Preprint, arXiv:2507.05595. — [arXiv:2507.05595](https://arxiv.org/abs/2507.05595) [find](https://scholar.google.com/scholar?q=Cheng+Cui%2C+Ting+Sun%2C+Manhui+Lin%2C+Tingquan+Gao%2C+Yubo+Zhang%2C+Jiaxuan+Liu%2C+Xueqing+Wang%2C+Zelun+Zhang%2C+Changda+Zhou%2C+Hongen+Liu%2C+Yue+Zhang%2C+Wenyu+Lv%2C+Kui+Huang%2C+Yichao+Zhang%2C+Jing+Zhang%2C+Jun+Zhang%2C+Yi+Liu)
8. Shahul Es, Jithin James, Luis Espinosa-Anke, and Steven Schockaert. 2025. [Ragas: Automated evaluation of retrieval augmented generation](https://md.research.elamur.ai/free/en/papers/es-2023-ragas-automated-evaluation-retrieval-augmented/). Preprint, arXiv:2309.15217.
9. OpenAI Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Mkadry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alexander Kirillov, Alex Nichol, Alex Paino, and 397 others. 2… — [GPT-4o System Card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/)
10. Jiajie Jin, Yutao Zhu, Zhicheng Dou, Guanting Dong, Xinyu Yang, Chenghao Zhang, Tong Zhao, Zhao Yang, and Ji-Rong Wen. 2025. Flashrag: A modular toolkit for efficient retrieval-augmented generation research. In Companion Proceedings of the ACM on Web Conference 2025, WWW 2025, Sydney, NSW, Australia… [find](https://scholar.google.com/scholar?q=Jiajie+Jin%2C+Yutao+Zhu%2C+Zhicheng+Dou%2C+Guanting+Dong%2C+Xinyu+Yang%2C+Chenghao+Zhang%2C+Tong+Zhao%2C+Zhao+Yang%2C+and+Ji%2DRong+Wen%2E+2025%2E+Flashrag%3A+A+modular+toolkit+for+efficient+retrieval%2Daugmented+generation+re)
11. Mahmoud Kasem, Abdelrahman Abdallah, Alexander Berendeyev, Ebrahem Elkady, Mahmoud Abdalla, Mohamed Mahmoud, Mohamed Hamada, Daniyar Nurseitov, and Islam Taj-Eddin. 2022. Deep learning for table detection and structure recognition: A survey. Preprint, arXiv:2211.08469. — [arXiv:2211.08469](https://arxiv.org/abs/2211.08469) [find](https://scholar.google.com/scholar?q=Mahmoud+Kasem%2C+Abdelrahman+Abdallah%2C+Alexander+Berendeyev%2C+Ebrahem+Elkady%2C+Mahmoud+Abdalla%2C+Mohamed+Mahmoud%2C+Mohamed+Hamada%2C+Daniyar+Nurseitov%2C+and+Islam+Taj%2DEddin%2E+2022%2E+Deep+learning+for+table+detec)
12. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2021. [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). Preprint, arXiv:2005.11401.
13. Chaofan Li, Zheng Liu, Shitao Xiao, and Yingxia Shao. 2023. Making large language models a better foundation for dense retrieval. Preprint, arXiv:2312.15503. — [arXiv:2312.15503](https://arxiv.org/abs/2312.15503) [find](https://scholar.google.com/scholar?q=Chaofan+Li%2C+Zheng+Liu%2C+Shitao+Xiao%2C+and+Yingxia+Shao%2E+2023%2E+Making+large+language+models+a+better+foundation+for+dense+retrieval%2E+Preprint%2C+arXiv%3A2312%2E15503%2E+%E2%80%94+%5BarXiv%3A2312%2E15503%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs)
14. Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. 2024. [Ocrbench: on the hidden mystery of ocr in large multimodal models](https://md.research.elamur.ai/free/en/papers/liu-2023-ocrbench-hidden-mystery-ocr-large/). Science China Information Sciences, 67(12).
15. Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Yuanhong Zheng, Dongsheng Ma, and 42 others. 2025. [Mineru2.5: A decoupled vision-language m…](https://md.research.elamur.ai/free/en/papers/niu-2026-mineru2-5-decoupled-vision-language/)
16. Linke Ouyang, Yuan Qu, Hongbin Zhou, Jiawei Zhu, Rui Zhang, Qunshu Lin, Bin Wang, Zhiyuan Zhao, Man Jiang, Xiaomeng Zhao, Jin Shi, Fan Wu, Pei Chu, Minghao Liu, Zhenxiang Li, Chao Xu, Bo Zhang, Botian Shi, Zhongying Tu, and Conghui He. 2025. Omnidocbench: Benchmarking diverse pdf document parsing wi… — [arXiv:2412.07626](https://arxiv.org/abs/2412.07626) [find](https://scholar.google.com/scholar?q=Linke+Ouyang%2C+Yuan+Qu%2C+Hongbin+Zhou%2C+Jiawei+Zhu%2C+Rui+Zhang%2C+Qunshu+Lin%2C+Bin+Wang%2C+Zhiyuan+Zhao%2C+Man+Jiang%2C+Xiaomeng+Zhao%2C+Jin+Shi%2C+Fan+Wu%2C+Pei+Chu%2C+Minghao+Liu%2C+Zhenxiang+Li%2C+Chao+Xu%2C+Bo+Zhang%2C+Botian)
17. Jon Saad-Falcon, Omar Khattab, Christopher Potts, and Matei Zaharia. 2024. Ares: An automated evaluation framework for retrieval-augmented generation systems. Preprint, arXiv:2311.09476. [find](https://scholar.google.com/scholar?q=Jon+Saad%2DFalcon%2C+Omar+Khattab%2C+Christopher+Potts%2C+and+Matei+Zaharia%2E+2024%2E+Ares%3A+An+automated+evaluation+framework+for+retrieval%2Daugmented+generation+systems%2E+Preprint%2C+arXiv%3A2311%2E09476%2E)
18. Hunyuan Vision Team, Pengyuan Lyu, Xingyu Wan, Gengluo Li, Shangpin Peng, Weinong Wang, Liang Wu, Huawen Shen, Yu Zhou, Canhui Tang, Qi Yang, Qiming Peng, Bin Luo, Hower Yang, Xinsong Zhang, Jinnian Zhang, Houwen Peng, Hongming Yang, Senhao Xie, and 13 others. 2025. [Hunyuanocr technical report](https://md.research.elamur.ai/free/en/papers/team-2025-vision-hunyuanocr-technical-report/). Prep…
19. Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, Bo Zhang, Liqun Wei, Zhihao Sui, Wei Li, Botian Shi, Yu Qiao, Dahua Lin, and Conghui He. 2024. Mineru: An open-source solution for precise document content extraction. Preprint, arXiv:2409… [find](https://scholar.google.com/scholar?q=Bin+Wang%2C+Chao+Xu%2C+Xiaomeng+Zhao%2C+Linke+Ouyang%2C+Fan+Wu%2C+Zhiyuan+Zhao%2C+Rui+Xu%2C+Kaiwen+Liu%2C+Yuan+Qu%2C+Fukai+Shang%2C+Bo+Zhang%2C+Liqun+Wei%2C+Zhihao+Sui%2C+Wei+Li%2C+Botian+Shi%2C+Yu+Qiao%2C+Dahua+Lin%2C+and+Conghui+He%2E)
20. Haoran Wei, Yaofeng Sun, and Yukun Li. 2025. [Deepseek-ocr: Contexts optical compression](https://md.research.elamur.ai/free/en/papers/wei-2025-deepseek-ocr-contexts-optical/). Preprint, arXiv:2510.18234.
21. Junyuan Zhang, Qintong Zhang, Bin Wang, Linke Ouyang, Zichen Wen, Ying Li, Ka-Ho Chow, Conghui He, and Wentao Zhang. 2025. Ocr hinders rag: Evaluating the cascading impact of ocr on retrieval-augmented generation. Preprint, arXiv:2412.02592. — [arXiv:2412.02592](https://arxiv.org/abs/2412.02592) [find](https://scholar.google.com/scholar?q=Junyuan+Zhang%2C+Qintong+Zhang%2C+Bin+Wang%2C+Linke+Ouyang%2C+Zichen+Wen%2C+Ying+Li%2C+Ka%2DHo+Chow%2C+Conghui+He%2C+and+Wentao+Zhang%2E+2025%2E+Ocr+hinders+rag%3A+Evaluating+the+cascading+impact+of+ocr+on+retrieval%2Daugmente)

*Updated 1 November 2026*