---
title: "Unveiling the Deficiencies of Pre-trained Text-and-Layout Models"
description: "Pre-trained text-and-layout models (PTLMs) score very highly on standard benchmarks… 100% accuracy. 54 references, cited by 1 paper here."
authors: ["Chong Zhang", "Yixi Zhao", "Yulu Xie", "Chenshu Yuan", "Yi Tu", "Ya Guo", "Mingxu Chai", "Ziyu Shen", "Yue Zhang", "Qi Zhang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: EACL 2026"
source: "https://aclanthology.org/2026.findings-eacl.1/"
doi: "10.18653/v1/2026.findings-eacl.1"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-unveiling-deficiencies-pre-trained-text/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-unveiling-deficiencies-pre-trained-text/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/19/) / Unveiling the Deficiencies of Pre-trained Text-and-Layout Models in Real-world Visually-rich Document Information Extraction

# Unveiling the Deficiencies of Pre-trained Text-and-Layout Models in Real-world Visually-rich Document Information Extraction

**Authors:** [Chong Zhang](https://md.research.elamur.ai/free/en/authors/chong-zhang/), Yixi Zhao, [Yulu Xie](https://md.research.elamur.ai/free/en/authors/yulu-xie/), Chenshu Yuan, Yi Tu, Ya Guo, Mingxu Chai, Ziyu Shen, [Yue Zhang](https://md.research.elamur.ai/free/en/authors/yue-zhang/) and [Qi Zhang](https://md.research.elamur.ai/free/en/authors/qi-zhang/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: EACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-eacl.1/) · [PDF](https://aclanthology.org/2026.findings-eacl.1.pdf) · [DOI 10.18653/v1/2026.findings-eacl.1](https://doi.org/10.18653/v1/2026.findings-eacl.1) · [OpenAlex](https://openalex.org/W7140073839)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Pre-trained text-and-layout models (PTLMs) score very highly on standard benchmarks for extracting information from visually-rich documents (VrD-IE), but they disappoint in practice. The authors trace this gap to two problems in how these models are evaluated. First, benchmarks such as FUNSD, XFUND and CORD annotate visual blocks rather than true semantic entities. All tokens in a block then share the same coordinates, so the layout input leaks the entity boundaries and test scores come out inflated. Second, evaluation looks only at benchmark accuracy.

To fix the first problem, they manually re-annotated FUNSD to build EC-FUNSD. This entity-centric dataset covers two tasks, identifying the semantic entities themselves (SER) and connecting related entities to each other (EL), and it keeps segment-level layout annotations apart from word/char-level entity annotations.

Using EC-FUNSD, they assessed popular PTLMs on absolute performance, generalization to unseen distributions, robustness to natural perturbations, and fairness on hard subsets. Every model scored clearly lower than on FUNSD and showed weaknesses on all three real-world dimensions. The authors conclude that progress in PTLMs may partly reflect overfitting to biased benchmarks.

## Key points
- EC-FUNSD keeps FUNSD's original 199 document images. It contains 10,662 segments and 8,398 entities, compared with 9,743 segments and 8,529 entities in FUNSD (after removing invalid links).
- Under identical settings, SER F1 falls by 7.48–8.55 points when baseline models move from FUNSD to EC-FUNSD.
- On complex entities, which span several rows or columns or overlap other entities, performance on EC-FUNSD drops by nearly 30%. FUNSD hides this fairness problem.
- On CORD's key-value EL task, GeoLayoutLM reaches 100% accuracy, an example of saturated benchmarks.
- The study covers only discriminative PTLMs, not generative models, and tests only vanilla adversarial training as a mitigation. Code is available at github.com/chongzhangFDU/ROOR.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 539 works · 179,533 citations
- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 333 works · 87,019 citations
- [Fudan University](https://md.research.elamur.ai/free/en/universities/fudan-university/) — University, Shanghai, China · 167 works · 20,987 citations
- [Ant Group](https://md.research.elamur.ai/free/en/companies/ant-group/) — Company, Hangzhou, China · 42 works · 1,131 citations
- [Nankai University](https://md.research.elamur.ai/free/en/universities/nankai-university/) — University, Tianjin, China · 24 works · 762 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🔎 Information extraction](https://md.research.elamur.ai/free/en/topics/information-extraction/) · 167 works

## Cite

Cite as: Chong Zhang, Yixi Zhao, Yulu Xie, Chenshu Yuan, Yi Tu, Ya Guo, Mingxu Chai, Ziyu Shen, Yue Zhang and Qi Zhang. 2026. Unveiling the Deficiencies of Pre-trained Text-and-Layout Models in Real-world Visually-rich Document Information Extraction. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2026unveiling,
  title = {{Unveiling the Deficiencies of Pre-trained Text-and-Layout Models in Real-world Visually-rich Document Information Extraction}},
  author = {Chong Zhang, Yixi Zhao, Yulu Xie, Chenshu Yuan, Yi Tu, Ya Guo, Mingxu Chai, Ziyu Shen, Yue Zhang and Qi Zhang},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.1},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.1}
}
```

- [← previous Unsupervised Labelling of Mutation Triggers in Welsh](https://md.research.elamur.ai/free/en/papers/rolon-2026-unsupervised-labelling-mutation-triggers-welsh/)
- [next → uogal at BEA 2026 Shared Task 1: Ensemble of Multilingual Encoders with NMT Augmentation for L1-Aware Vocabulary Difficulty Prediction](https://md.research.elamur.ai/free/en/papers/stearns-2026-uogal-bea-2026-shared-task/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/19/)

## References

54 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Srikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie, and R Manmatha. 2021. Docformer: End-to-end transformer for document understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 993–1003. [find](https://scholar.google.com/scholar?q=Srikar+Appalaraju%2C+Bhavan+Jasani%2C+Bhargava+Urala+Kota%2C+Yusheng+Xie%2C+and+R+Manmatha%2E+2021%2E+Docformer%3A+End%2Dto%2Dend+transformer+for+document+understanding%2E+In+Proceedings+of+the+IEEE%2FCVF+International+Con)
2. Srikar Appalaraju, Peng Tang, Qi Dong, Nishant Sankaran, Yichu Zhou, and R Manmatha. 2024. Docformerv2: Local features for document understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 709–718. [find](https://scholar.google.com/scholar?q=Srikar+Appalaraju%2C+Peng+Tang%2C+Qi+Dong%2C+Nishant+Sankaran%2C+Yichu+Zhou%2C+and+R+Manmatha%2E+2024%2E+Docformerv2%3A+Local+features+for+document+understanding%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+I)
3. Rajas Bansal. 2022. A survey on bias and fairness in natural language processing. arXiv preprint arXiv:2204.09591. — [arXiv:2204.09591](https://arxiv.org/abs/2204.09591) [find](https://scholar.google.com/scholar?q=Rajas+Bansal%2E+2022%2E+A+survey+on+bias+and+fairness+in+natural+language+processing%2E+arXiv+preprint+arXiv%3A2204%2E09591%2E+%E2%80%94+%5BarXiv%3A2204%2E09591%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2204%2E09591%29)
4. Simon Caton and Christian Haas. 2024. Fairness in machine learning: A survey. ACM Comput. Surv., 56(7). [find](https://scholar.google.com/scholar?q=Simon+Caton+and+Christian+Haas%2E+2024%2E+Fairness+in+machine+learning%3A+A+survey%2E+ACM+Comput%2E+Surv%2E%2C+56%287%29%2E)
5. Zheng Huang, Kai Chen, Jianhua He, Xiang Bai, Dimosthenis Karatzas, Shijian Lu, and CV Jawahar. 2019. Icdar2019 competition on scanned receipt ocr and information extraction. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 1516–1520. IEEE. [find](https://scholar.google.com/scholar?q=Zheng+Huang%2C+Kai+Chen%2C+Jianhua+He%2C+Xiang+Bai%2C+Dimosthenis+Karatzas%2C+Shijian+Lu%2C+and+CV+Jawahar%2E+2019%2E+Icdar2019+competition+on+scanned+receipt+ocr+and+information+extraction%2E+In+2019+International+Con)
6. Kai-Wei Chang, Vinodkumar Prabhakaran, and Vicente Ordonez. 2019. Bias and fairness in natural language processing. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): Tuto… [find](https://scholar.google.com/scholar?q=Kai%2DWei+Chang%2C+Vinodkumar+Prabhakaran%2C+and+Vicente+Ordonez%2E+2019%2E+Bias+and+fairness+in+natural+language+processing%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Proce)
7. Dieuwke Hupkes, Mario Giulianelli, Verna Dankers, Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Christos Christodoulopoulos, Karim Lasri, Naomi Saphra, Arabella Sinclair, Dennis Ulmer, Florian Schottmann, Khuyagbaatar Batsuren, Kaiser Sun, Koustuv Sinha, Leila Khalatbari, Maria Ryskina, Rita Frieske,… [find](https://scholar.google.com/scholar?q=Dieuwke+Hupkes%2C+Mario+Giulianelli%2C+Verna+Dankers%2C+Mikel+Artetxe%2C+Yanai+Elazar%2C+Tiago+Pimentel%2C+Christos+Christodoulopoulos%2C+Karim+Lasri%2C+Naomi+Saphra%2C+Arabella+Sinclair%2C+Dennis+Ulmer%2C+Florian+Schottma)
8. Lei Cui, Yiheng Xu, Tengchao Lv, and Furu Wei. 2021. Document ai: Benchmarks, models and applications. arXiv preprint arXiv:2111.08609. — [arXiv:2111.08609](https://arxiv.org/abs/2111.08609) [find](https://scholar.google.com/scholar?q=Lei+Cui%2C+Yiheng+Xu%2C+Tengchao+Lv%2C+and+Furu+Wei%2E+2021%2E+Document+ai%3A+Benchmarks%2C+models+and+applications%2E+arXiv+preprint+arXiv%3A2111%2E08609%2E+%E2%80%94+%5BarXiv%3A2111%2E08609%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2111%2E08609%29)
9. Guillaume Jaume, Hazim Kemal Ekenel, and Jean-Philippe Thiran. 2019. Funsd: A dataset for form understanding in noisy scanned documents. In 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW), volume 2, pages 1–6. IEEE. [find](https://scholar.google.com/scholar?q=Guillaume+Jaume%2C+Hazim+Kemal+Ekenel%2C+and+Jean%2DPhilippe+Thiran%2E+2019%2E+Funsd%3A+A+dataset+for+form+understanding+in+noisy+scanned+documents%2E+In+2019+International+Conference+on+Document+Analysis+and+Recog)
10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of NAACL-HLT, pages 4171–4186.
11. Yushun Dong, Jing Ma, Song Wang, Chen Chen, and Jundong Li. 2023. Fairness in graph mining: A survey. IEEE Transactions on Knowledge and Data Engineering, 35(10):10583–10602. [find](https://scholar.google.com/scholar?q=Yushun+Dong%2C+Jing+Ma%2C+Song+Wang%2C+Chen+Chen%2C+and+Jundong+Li%2E+2023%2E+Fairness+in+graph+mining%3A+A+survey%2E+IEEE+Transactions+on+Knowledge+and+Data+Engineering%2C+35%2810%29%3A10583%E2%80%9310602%2E)
12. Feihu Jiang, Chuan Qin, Jingshuai Zhang, Kaichun Yao, Xi Chen, Dazhong Shen, Chen Zhu, Hengshu Zhu, and Hui Xiong. 2024. Towards efficient resume understanding: A multi-granularity multi-modal pre-training approach. arXiv preprint arXiv:2404.13067. — [arXiv:2404.13067](https://arxiv.org/abs/2404.13067) [find](https://scholar.google.com/scholar?q=Feihu+Jiang%2C+Chuan+Qin%2C+Jingshuai+Zhang%2C+Kaichun+Yao%2C+Xi+Chen%2C+Dazhong+Shen%2C+Chen+Zhu%2C+Hengshu+Zhu%2C+and+Hui+Xiong%2E+2024%2E+Towards+efficient+resume+understanding%3A+A+multi%2Dgranularity+multi%2Dmodal+pre%2Dtra)
13. Aparna Elangovan, Jiayuan He, Yuan Li, and Karin Verspoor. 2024. Principles from clinical research for nlp model generalization. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers),… [find](https://scholar.google.com/scholar?q=Aparna+Elangovan%2C+Jiayuan+He%2C+Yuan+Li%2C+and+Karin+Verspoor%2E+2024%2E+Principles+from+clinical+research+for+nlp+model+generalization%2E+In+Proceedings+of+the+2024+Conference+of+the+North+American+Chapter+of+)
14. Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. 2022. [Ocr-free document understanding transformer](https://md.research.elamur.ai/free/en/papers/kim-2021-ocr-free-document-understanding-transformer/). In European Conference on Computer Vision, pages 498–517. Springer.
15. Zhangxuan Gu, Changhua Meng, Ke Wang, Jun Lan, Weiqiang Wang, Ming Gu, and Liqing Zhang. 2022. Xylayoutlm: Towards layout-aware multimodal networks for visually-rich document understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4583–4592. [find](https://scholar.google.com/scholar?q=Zhangxuan+Gu%2C+Changhua+Meng%2C+Ke+Wang%2C+Jun+Lan%2C+Weiqiang+Wang%2C+Ming+Gu%2C+and+Liqing+Zhang%2E+2022%2E+Xylayoutlm%3A+Towards+layout%2Daware+multimodal+networks+for+visually%2Drich+document+understanding%2E+In+Proceed)
16. Prashant Krishnan, Zilong Wang, Yangkun Wang, and Jingbo Shang. 2024. Towards few-shot entity recognition in document images: A graph neural network approach robust to image manipulation. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and E… [find](https://scholar.google.com/scholar?q=Prashant+Krishnan%2C+Zilong+Wang%2C+Yangkun+Wang%2C+and+Jingbo+Shang%2E+2024%2E+Towards+few%2Dshot+entity+recognition+in+document+images%3A+A+graph+neural+network+approach+robust+to+image+manipulation%2E+In+Proceedin)
17. He Guo, Xiameng Qin, Jiaming Liu, Junyu Han, Jingtuo Liu, and Errui Ding. 2019. Eaten: Entity-aware attention for single shot visual text extraction. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 254–259. IEEE. [find](https://scholar.google.com/scholar?q=He+Guo%2C+Xiameng+Qin%2C+Jiaming+Liu%2C+Junyu+Han%2C+Jingtuo+Liu%2C+and+Errui+Ding%2E+2019%2E+Eaten%3A+Entity%2Daware+attention+for+single+shot+visual+text+extraction%2E+In+2019+International+Conference+on+Document+Analy)
18. Xin Li, Yan Zheng, Yiqing Hu, Haoyu Cao, Yunfei Wu, Deqiang Jiang, Yinsong Liu, and Bo Ren. 2022. Relational representation learning in visually-rich documents. arXiv preprint arXiv:2205.02411. — [arXiv:2205.02411](https://arxiv.org/abs/2205.02411) [find](https://scholar.google.com/scholar?q=Xin+Li%2C+Yan+Zheng%2C+Yiqing+Hu%2C+Haoyu+Cao%2C+Yunfei+Wu%2C+Deqiang+Jiang%2C+Yinsong+Liu%2C+and+Bo+Ren%2E+2022%2E+Relational+representation+learning+in+visually%2Drich+documents%2E+arXiv+preprint+arXiv%3A2205%2E02411%2E+%E2%80%94+%5BarX)
19. Yulin Li, Yuxi Qian, Yuechen Yu, Xiameng Qin, Chengquan Zhang, Yan Liu, Kun Yao, Junyu Han, Jingtuo Liu, and Errui Ding. 2021. Structext: Structured text understanding with multi-modal transformers. In Proceedings of the 29th ACM International Conference on Multimedia, pages 1912– 1920. [find](https://scholar.google.com/scholar?q=Yulin+Li%2C+Yuxi+Qian%2C+Yuechen+Yu%2C+Xiameng+Qin%2C+Chengquan+Zhang%2C+Yan+Liu%2C+Kun+Yao%2C+Junyu+Han%2C+Jingtuo+Liu%2C+and+Errui+Ding%2E+2021%2E+Structext%3A+Structured+text+understanding+with+multi%2Dmodal+transformers%2E+I)
20. Teakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang, Daehyun Nam, and Sungrae Park. 2022. Bros: A pre-trained language model focusing on text and layout for better key information extraction from documents. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10767– 107… [find](https://scholar.google.com/scholar?q=Teakgyu+Hong%2C+Donghyun+Kim%2C+Mingi+Ji%2C+Wonseok+Hwang%2C+Daehyun+Nam%2C+and+Sungrae+Park%2E+2022%2E+Bros%3A+A+pre%2Dtrained+language+model+focusing+on+text+and+layout+for+better+key+information+extraction+from+docu)
21. Haofu Liao, Aruni RoyChowdhury, Weijian Li, Ankan Bansal, Yuting Zhang, Zhuowen Tu, Ravi Kumar Satzoda, R. Manmatha, and Vijay Mahadevan. 2023. Doctr: Document transformer for structured information extraction in documents. In Proceedings of the IEEE/CVF International Conference on Computer Vision (… [find](https://scholar.google.com/scholar?q=Haofu+Liao%2C+Aruni+RoyChowdhury%2C+Weijian+Li%2C+Ankan+Bansal%2C+Yuting+Zhang%2C+Zhuowen+Tu%2C+Ravi+Kumar+Satzoda%2C+R%2E+Manmatha%2C+and+Vijay+Mahadevan%2E+2023%2E+Doctr%3A+Document+transformer+for+structured+information+e)
22. Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. 2022. [Layoutlmv3: Pre-training for document AI with unified text and image masking](https://md.research.elamur.ai/free/en/papers/huang-2022-layoutlmv3-pre-training-document-ai/). In MM ’22: The 30th ACM International Conference on Multimedia, Lisboa, Portugal, October 10 - 14, 2022, pages 4083–4091. ACM.
23. Zening Lin, Jiapeng Wang, Teng Li, Wenhui Liao, Dayi Huang, Longfei Xiong, and Lianwen Jin. 2024. Peneo: unifying line extraction, line grouping, and entity linking for end-to-end document pair extraction. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 5171– 5180. other… [find](https://scholar.google.com/scholar?q=Zening+Lin%2C+Jiapeng+Wang%2C+Teng+Li%2C+Wenhui+Liao%2C+Dayi+Huang%2C+Longfei+Xiong%2C+and+Lianwen+Jin%2E+2024%2E+Peneo%3A+unifying+line+extraction%2C+line+grouping%2C+and+entity+linking+for+end%2Dto%2Dend+document+pair+extrac)
24. Hongbin Sun, Zhanghui Kuang, Xiaoyu Yue, Chenhao Lin, and Wayne Zhang. 2021. Spatial dual-modality graph reasoning for key information extraction. arXiv preprint arXiv:2103.14470. — [arXiv:2103.14470](https://arxiv.org/abs/2103.14470) [find](https://scholar.google.com/scholar?q=Hongbin+Sun%2C+Zhanghui+Kuang%2C+Xiaoyu+Yue%2C+Chenhao+Lin%2C+and+Wayne+Zhang%2E+2021%2E+Spatial+dual%2Dmodality+graph+reasoning+for+key+information+extraction%2E+arXiv+preprint+arXiv%3A2103%2E14470%2E+%E2%80%94+%5BarXiv%3A2103%2E14470%5D)
25. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692.
26. Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng, Xuyi Chen, Han Zhang, Xin Tian, Danxiang Zhu, Hao Tian, and Hua Wu. 2019. [Ernie: Enhanced representation through knowledge integration](https://md.research.elamur.ai/free/en/papers/sun-2019-ernie-enhanced-representation-through-knowledge/). arXiv preprint arXiv:1904.09223.
27. Jinghui Lu, Haiyang Yu, Yanjie Wang, Yongjie Ye, Jingqun Tang, Ziwei Yang, Binghong Wu, Qi Liu, Hao Feng, Han Wang, and 1 others. 2024. A bounding box is worth one token: Interleaving layout and text in a large language model for document understanding. arXiv preprint arXiv:2407.01976. [find](https://scholar.google.com/scholar?q=Jinghui+Lu%2C+Haiyang+Yu%2C+Yanjie+Wang%2C+Yongjie+Ye%2C+Jingqun+Tang%2C+Ziwei+Yang%2C+Binghong+Wu%2C+Qi+Liu%2C+Hao+Feng%2C+Han+Wang%2C+and+1+others%2E+2024%2E+A+bounding+box+is+worth+one+token%3A+Interleaving+layout+and+text+)
28. Yi Tu, Ya Guo, Huan Chen, and Jinyang Tang. 2023. Layoutmask: Enhance text-layout interaction in multi-modal pre-training for document understanding. arXiv preprint arXiv:2305.18721. [find](https://scholar.google.com/scholar?q=Yi+Tu%2C+Ya+Guo%2C+Huan+Chen%2C+and+Jinyang+Tang%2E+2023%2E+Layoutmask%3A+Enhance+text%2Dlayout+interaction+in+multi%2Dmodal+pre%2Dtraining+for+document+understanding%2E+arXiv+preprint+arXiv%3A2305%2E18721%2E)
29. Yi Tu, Chong Zhang, Ya Guo, Huan Chen, Jinyang Tang, Huijia Zhu, and Qi Zhang. 2024. Uner: A unified prediction head for named entity recognition in visually-rich documents. arXiv preprint arXiv:2408.01038. — [arXiv:2408.01038](https://arxiv.org/abs/2408.01038) [find](https://scholar.google.com/scholar?q=Yi+Tu%2C+Chong+Zhang%2C+Ya+Guo%2C+Huan+Chen%2C+Jinyang+Tang%2C+Huijia+Zhu%2C+and+Qi+Zhang%2E+2024%2E+Uner%3A+A+unified+prediction+head+for+named+entity+recognition+in+visually%2Drich+documents%2E+arXiv+preprint+arXiv%3A2408%2E)
30. Chuwei Luo, Changxu Cheng, Qi Zheng, and Cong Yao. 2023. Geolayoutlm: Geometric pre-training for visual information extraction. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). [find](https://scholar.google.com/scholar?q=Chuwei+Luo%2C+Changxu+Cheng%2C+Qi+Zheng%2C+and+Cong+Yao%2E+2023%2E+Geolayoutlm%3A+Geometric+pre%2Dtraining+for+visual+information+extraction%2E+2023+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVP)
31. Jiapeng Wang, Chongyu Liu, Lianwen Jin, Guozhi Tang, Jiaxin Zhang, Shuaitao Zhang, Qianying Wang, Yaqiang Wu, and Mingxiang Cai. 2021a. Towards robust visual information extraction in real world: new dataset and novel solution. In Proceedings of the AAAI Conference on Artificial Intelligence, volume… [find](https://scholar.google.com/scholar?q=Jiapeng+Wang%2C+Chongyu+Liu%2C+Lianwen+Jin%2C+Guozhi+Tang%2C+Jiaxin+Zhang%2C+Shuaitao+Zhang%2C+Qianying+Wang%2C+Yaqiang+Wu%2C+and+Mingxiang+Cai%2E+2021a%2E+Towards+robust+visual+information+extraction+in+real+world%3A+new+)
32. Chuwei Luo, Yufan Shen, Zhaoqing Zhu, Qi Zheng, Zhi Yu, and Cong Yao. 2024. Layoutllm: Layout instruction tuning with large language models for document understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15630–15640. [find](https://scholar.google.com/scholar?q=Chuwei+Luo%2C+Yufan+Shen%2C+Zhaoqing+Zhu%2C+Qi+Zheng%2C+Zhi+Yu%2C+and+Cong+Yao%2E+2024%2E+Layoutllm%3A+Layout+instruction+tuning+with+large+language+models+for+document+understanding%2E+In+Proceedings+of+the+IEEE%2FCVF+C)
33. Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, and 1 others. 2021b. [Textflint: Unified multilingual robustness evaluation toolkit for natural language processing](https://md.research.elamur.ai/free/en/papers/wang-2021-textflint-unified-multilingual-robustness-evaluation/). In Proceedings of the 59th Annual Meeting of the Association for Comp…
34. Pengyuan Lyu, Yulin Li, Hao Zhou, Weihong Ma, Xingyu Wan, Qunyi Xie, Liang Wu, Chengquan Zhang, Kun Yao, Errui Ding, and 1 others. 2024. Structextv3: An efficient vision-language model for text-rich image perception, comprehension, and beyond. arXiv preprint arXiv:2405.21013. — [arXiv:2405.21013](https://arxiv.org/abs/2405.21013) [find](https://scholar.google.com/scholar?q=Pengyuan+Lyu%2C+Yulin+Li%2C+Hao+Zhou%2C+Weihong+Ma%2C+Xingyu+Wan%2C+Qunyi+Xie%2C+Liang+Wu%2C+Chengquan+Zhang%2C+Kun+Yao%2C+Errui+Ding%2C+and+1+others%2E+2024%2E+Structextv3%3A+An+efficient+vision%2Dlanguage+model+for+text%2Drich+i)
35. Seunghyun Park, Seung Shin, Bado Lee, Junyeop Lee, Jaeheung Surh, Minjoon Seo, and Hwalsuk Lee. 2019. Cord: a consolidated receipt dataset for post-ocr parsing. In Workshop on Document Intelligence at NeurIPS 2019. [find](https://scholar.google.com/scholar?q=Seunghyun+Park%2C+Seung+Shin%2C+Bado+Lee%2C+Junyeop+Lee%2C+Jaeheung+Surh%2C+Minjoon+Seo%2C+and+Hwalsuk+Lee%2E+2019%2E+Cord%3A+a+consolidated+receipt+dataset+for+post%2Docr+parsing%2E+In+Workshop+on+Document+Intelligence+at)
36. Xuezhi Wang, Haohan Wang, and Diyi Yang. 2021c. [Measure and improve robustness in nlp models: A survey](https://md.research.elamur.ai/free/en/papers/wang-2022-measure-improve-robustness-nlp-models/). arXiv preprint arXiv:2112.08313.
37. Qiming Peng, Yinxu Pan, Wenjin Wang, Bin Luo, Zhenyu Zhang, Zhengjie Huang, Teng Hu, Weichong Yin, Yongfeng Chen, Yin Zhang, and 1 others. 2022. [Ernie-layout: Layout knowledge enhanced pre-training for visually-rich document understanding](https://md.research.elamur.ai/free/en/papers/peng-2022-ernie-layout-layout-knowledge/). arXiv preprint arXiv:2210.06155.
38. Zifeng Wang, Zizhao Zhang, Jacob Devlin, Chen-Yu Lee, Guolong Su, Hao Zhang, Jennifer Dy, Vincent Perot, and Tomas Pfister. 2022. Queryform: A simple zero-shot form entity query framework. arXiv preprint arXiv:2211.07730. [find](https://scholar.google.com/scholar?q=Zifeng+Wang%2C+Zizhao+Zhang%2C+Jacob+Devlin%2C+Chen%2DYu+Lee%2C+Guolong+Su%2C+Hao+Zhang%2C+Jennifer+Dy%2C+Vincent+Perot%2C+and+Tomas+Pfister%2E+2022%2E+Queryform%3A+A+simple+zero%2Dshot+form+entity+query+framework%2E+arXiv+prepr)
39. Zilong Wang, Yiheng Xu, Lei Cui, Jingbo Shang, and Furu Wei. 2021d. Layoutreader: Pre-training of text and layout for reading order detection. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4735–4744. [find](https://scholar.google.com/scholar?q=Zilong+Wang%2C+Yiheng+Xu%2C+Lei+Cui%2C+Jingbo+Shang%2C+and+Furu+Wei%2E+2021d%2E+Layoutreader%3A+Pre%2Dtraining+of+text+and+layout+for+reading+order+detection%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Method)
40. Abdellatif Sassioui, Rachid Benouini, Yasser El Ouar-gui, Mohamed El Kamili, Meriyem Chergui, and Mohammed Ouzzif. 2023. Visually-rich document understanding: Concepts, taxonomy and challenges. In 2023 10th International Conference on Wireless Networks and Mobile Communications (WINCOM), pages 1–7. [find](https://scholar.google.com/scholar?q=Abdellatif+Sassioui%2C+Rachid+Benouini%2C+Yasser+El+Ouar%2Dgui%2C+Mohamed+El+Kamili%2C+Meriyem+Chergui%2C+and+Mohammed+Ouzzif%2E+2023%2E+Visually%2Drich+document+understanding%3A+Concepts%2C+taxonomy+and+challenges%2E+In+202)
41. Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, and 1 others. 2021a. [Layoutlmv2: Multi-modal pre-training for visually-rich document understanding](https://md.research.elamur.ai/free/en/papers/xu-2021-layoutlmv2-multi-modal-pre-training/). In Proceedings of Štěpán Šimsa, Milan Šulc, Michal Uřičář, Yash Patel, Ahmed H…
42. Chong Zhang, Ya Guo, Yi Tu, Huan Chen, Jinyang Tang, Huijia Zhu, Qi Zhang, and Tao Gui. 2023. Reading order matters: Information extraction from visually-rich documents by token path prediction. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13716–13… [find](https://scholar.google.com/scholar?q=Chong+Zhang%2C+Ya+Guo%2C+Yi+Tu%2C+Huan+Chen%2C+Jinyang+Tang%2C+Huijia+Zhu%2C+Qi+Zhang%2C+and+Tao+Gui%2E+2023%2E+Reading+order+matters%3A+Information+extraction+from+visually%2Drich+documents+by+token+path+prediction%2E+In+Pr)
43. Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou. 2020. [Layoutlm: Pre-training of text and layout for document image understanding](https://md.research.elamur.ai/free/en/papers/xu-2019-layoutlm-pre-training-text-layout/). In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1192–1200.
44. Yiheng Xu, Tengchao Lv, Lei Cui, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, and Furu Wei. 2021b. Layoutxlm: Multimodal pre-training for multilingual visually-rich document understanding. arXiv preprint arXiv:2104.08836. — [arXiv:2104.08836](https://arxiv.org/abs/2104.08836) [find](https://scholar.google.com/scholar?q=Yiheng+Xu%2C+Tengchao+Lv%2C+Lei+Cui%2C+Guoxin+Wang%2C+Yijuan+Lu%2C+Dinei+Florencio%2C+Cha+Zhang%2C+and+Furu+Wei%2E+2021b%2E+Layoutxlm%3A+Multimodal+pre%2Dtraining+for+multilingual+visually%2Drich+document+understanding%2E+arXi)
45. Chong Zhang, Yi Tu, Yixi Zhao, Chenshu Yuan, Huan Chen, Yue Zhang, Mingxu Chai, Ya Guo, Huijia Zhu, Qi Zhang, and Tao Gui. 2024. Modeling layout reading order as ordering relations for visually-rich document understanding. In Proceedings of the 2024 Conference on Empirical Methods in Natural Languag… [find](https://scholar.google.com/scholar?q=Chong+Zhang%2C+Yi+Tu%2C+Yixi+Zhao%2C+Chenshu+Yuan%2C+Huan+Chen%2C+Yue+Zhang%2C+Mingxu+Chai%2C+Ya+Guo%2C+Huijia+Zhu%2C+Qi+Zhang%2C+and+Tao+Gui%2E+2024%2E+Modeling+layout+reading+order+as+ordering+relations+for+visually%2Drich+d)
46. Yiheng Xu, Tengchao Lv, Lei Cui, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, and Furu Wei. 2022. Xfund: A benchmark dataset for multilingual visually rich form understanding. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3214–3224. [find](https://scholar.google.com/scholar?q=Yiheng+Xu%2C+Tengchao+Lv%2C+Lei+Cui%2C+Guoxin+Wang%2C+Yijuan+Lu%2C+Dinei+Florencio%2C+Cha+Zhang%2C+and+Furu+Wei%2E+2022%2E+Xfund%3A+A+benchmark+dataset+for+multilingual+visually+rich+form+understanding%2E+In+Findings+of+th)
47. Huichen Yang and William Hsu. 2022. Transformer-based approach for document layout understanding. In 2022 IEEE International Conference on Image Processing (ICIP), pages 4043–4047. [find](https://scholar.google.com/scholar?q=Huichen+Yang+and+William+Hsu%2E+2022%2E+Transformer%2Dbased+approach+for+document+layout+understanding%2E+In+2022+IEEE+International+Conference+on+Image+Processing+%28ICIP%29%2C+pages+4043%E2%80%934047%2E)
48. Zhibo Yang, Rujiao Long, Pengfei Wang, Ro Song, Humen Zhong, Wenqing Cheng, Xiang Bai, and Cong Yao. 2023. Modeling entities as semantic points for visual information extraction in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15358–15367. [find](https://scholar.google.com/scholar?q=Zhibo+Yang%2C+Rujiao+Long%2C+Pengfei+Wang%2C+Ro+Song%2C+Humen+Zhong%2C+Wenqing+Cheng%2C+Xiang+Bai%2C+and+Cong+Yao%2E+2023%2E+Modeling+entities+as+semantic+points+for+visual+information+extraction+in+the+wild%2E+In+Procee)
49. Kaichun Yao, Jingshuai Zhang, Chuan Qin, Xin Song, Peng Wang, Hengshu Zhu, and Hui Xiong. 2023. Resuformer: Semantic structure understanding for resumes via multi-modal pre-training. In 2023 IEEE 39th International Conference on Data Engineering (ICDE), pages 3154–3167. IEEE. [find](https://scholar.google.com/scholar?q=Kaichun+Yao%2C+Jingshuai+Zhang%2C+Chuan+Qin%2C+Xin+Song%2C+Peng+Wang%2C+Hengshu+Zhu%2C+and+Hui+Xiong%2E+2023%2E+Resuformer%3A+Semantic+structure+understanding+for+resumes+via+multi%2Dmodal+pre%2Dtraining%2E+In+2023+IEEE+39th)
50. Wenwen Yu, Chengquan Zhang, Haoyu Cao, Wei Hua, Bohan Li, Huang Chen, Mingyu Liu, Mingrui Chen, Jianfeng Kuang, Mengjun Cheng, and 1 others. 2023a. [find](https://scholar.google.com/scholar?q=Wenwen+Yu%2C+Chengquan+Zhang%2C+Haoyu+Cao%2C+Wei+Hua%2C+Bohan+Li%2C+Huang+Chen%2C+Mingyu+Liu%2C+Mingrui+Chen%2C+Jianfeng+Kuang%2C+Mengjun+Cheng%2C+and+1+others%2E+2023a%2E)
51. Icdar 2023 competition on structured text extraction from visually-rich document images. arXiv preprint arXiv:2306.03287. — [arXiv:2306.03287](https://arxiv.org/abs/2306.03287) [find](https://scholar.google.com/scholar?q=Icdar+2023+competition+on+structured+text+extraction+from+visually%2Drich+document+images%2E+arXiv+preprint+arXiv%3A2306%2E03287%2E+%E2%80%94+%5BarXiv%3A2306%2E03287%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2306%2E03287%29)
52. Yuechen Yu, Yulin Li, Chengquan Zhang, Xiaoqiang Zhang, Zengyuan Guo, Xiameng Qin, Kun Yao, Junyu Han, Errui Ding, and Jingdong Wang. 2023b. Structextv2: Masked visual-textual prediction for document image pre-training. arXiv preprint arXiv:2303.00289. — [arXiv:2303.00289](https://arxiv.org/abs/2303.00289) [find](https://scholar.google.com/scholar?q=Yuechen+Yu%2C+Yulin+Li%2C+Chengquan+Zhang%2C+Xiaoqiang+Zhang%2C+Zengyuan+Guo%2C+Xiameng+Qin%2C+Kun+Yao%2C+Junyu+Han%2C+Errui+Ding%2C+and+Jingdong+Wang%2E+2023b%2E+Structextv2%3A+Masked+visual%2Dtextual+prediction+for+document+)
53. Tai-Ling Yuan, Zhe Zhu, Kun Xu, Cheng-Jun Li, and Shi-Min Hu. 2018. Chinese text in the wild. arXiv preprint arXiv:1803.00085. — [arXiv:1803.00085](https://arxiv.org/abs/1803.00085) [find](https://scholar.google.com/scholar?q=Tai%2DLing+Yuan%2C+Zhe+Zhu%2C+Kun+Xu%2C+Cheng%2DJun+Li%2C+and+Shi%2DMin+Hu%2E+2018%2E+Chinese+text+in+the+wild%2E+arXiv+preprint+arXiv%3A1803%2E00085%2E+%E2%80%94+%5BarXiv%3A1803%2E00085%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1803%2E00085%29)
54. Mingliang Zhai, Yulin Li, Xiameng Qin, Chen Yi, Qunyi Xie, Chengquan Zhang, Kun Yao, Yuwei Wu, and Yunde Jia. 2023. Fast-structext: An efficient [find](https://scholar.google.com/scholar?q=Mingliang+Zhai%2C+Yulin+Li%2C+Xiameng+Qin%2C+Chen+Yi%2C+Qunyi+Xie%2C+Chengquan+Zhang%2C+Kun+Yao%2C+Yuwei+Wu%2C+and+Yunde+Jia%2E+2023%2E+Fast%2Dstructext%3A+An+efficient)

*Updated 31 October 2026*