---
title: "-SEA³-QA — paper by Wang et al. (2025) — summary and ideas"
description: "The authors build two automated systems to address this. 34 references. Among them: 0."
authors: ["Kesen Wang", "Daulet Toibazar", "Pedro J Moreno Mengibar"]
year: 2025
venue: "Proceedings of The Third Arabic Natural Language Processing Conference"
source: "https://aclanthology.org/2025.arabicnlp-main.9/"
doi: "10.18653/v1/2025.arabicnlp-main.9"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2025-mathbfa-sea-3-mathbfl-qa/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2025-mathbfa-sea-3-mathbfl-qa/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/16/) / -SEA³-QA: A Fully Automated Self-Evolving, Adversarial Workflow for Arabic Long-Context Question-Answer Generation

# -SEA³-QA: A Fully Automated Self-Evolving, Adversarial Workflow for Arabic Long-Context Question-Answer Generation

**Authors:** [Kesen Wang](https://md.research.elamur.ai/free/en/authors/kesen-wang/), [Daulet Toibazar](https://md.research.elamur.ai/free/en/authors/daulet-toibazar/) and Pedro J Moreno Mengibar · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-main.9/) · [PDF](https://aclanthology.org/2025.arabicnlp-main.9.pdf) · [DOI 10.18653/v1/2025.arabicnlp-main.9](https://doi.org/10.18653/v1/2025.arabicnlp-main.9) · [OpenAlex](https://openalex.org/W4416037672)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Large vision-language models (LVLMs) do well on single-page document QA but perform poorly when answers depend on many pages. Arabic has little training or test data for this setting, and existing Arabic benchmarks such as KITAB-Bench and Camel mostly cover single pages. The authors build two automated systems to address this. The first is an agent-driven crawler that collects long Arabic documents. It handles native PDFs, scanned pages and right-to-left text, and it filters by license and length. The second is a closed-loop QA generator. In this loop, a question-generator LVLM writes questions about a document, a swarm of answer-generator LVLMs tries to answer them, and an evaluator LVLM scores the results. Weak items are sent back for regeneration, and the evaluator's quality threshold controls how hard the accepted questions are. The output is AraLongBench, a multi-page Arabic benchmark covering documents of up to hundreds of pages. Zero-shot tests show that both closed and open LVLMs lose about 20 accuracy points under the strictest difficulty gate. This suggests that Arabic long-document understanding is still far from solved.

## Key points
- The benchmark buckets documents by length: short-context (\<100 pages), medium-context (100–200) and long-context (>200). It also separates single-page (SP) questions from cross-page (CP) questions.
- Evaluation uses three gates: No Gate, 50% and 25%. Gemini-2.0 Flash drops from 93.0% to 84.1% on SC at the 50% gate. GPT-4o on SC falls from 87.2% to 65.7% at the 25% gate.
- Open models show the same pattern:
  - AIN: 78.5 → 69.1 → 58.2
  - Aya Vision: 77.0 → 68.7 → 57.3
  - Qwen 2.5 VL (SP): 91.5 → 82.0 → 71.5
- An evidence-source component was tested by auditing 100 random questions. Without it, 14% had evidence that did not match the answer. With it, the mismatch rate fell below 5%.
- The collection pipeline uses BeautifulSoup, pdfplumber, Tesseract OCR with binarization and deskewing, and a handler based on the Unicode Bidirectional Algorithm. Code is available on GitHub (Self_Improving_ARA_LONG_Doc).

## Interesting ideas
- The required quality score is treated as an adjustable knob, so one generator can produce anything from easy recall items to hard inference items, which makes curriculum-style datasets cheap to build.
- A group of different answering models stands in for difficulty: a question is hard if the group struggles with it, and weak or low-confidence items are regenerated automatically.
- Each generated answer must name its evidence location, and a small random sample is audited for mismatches, which gives a cheap, measurable fidelity check on synthetic data.
- Results are reported as a curve across strictness thresholds and split by local versus cross-page evidence, not as a single accuracy number.
- The crawler re-fetches a page when layout anomalies appear during extraction, so extraction feeds back into ingestion instead of passing broken text downstream.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (answer): Tale and fable QA sets could be generated at several difficulty levels, so we can see where the reader stops coping instead of reporting one pass rate.
- **I2** (learn): Our witness parsers and LLM teacher tiers could act as that group in self-play, keeping questions they disagree on as the most informative training material.
- **I3** (answer): Every answer could point to the tree node or frame it relies on, and periodic audits of a 100-item sample would catch silent grounding errors.
- **I4** (world): Splitting QA scores into questions answerable from one sentence versus questions needing cross-sentence event order, holdings or coreference would show whether the world model really links facts across the text.
- **I5** (tokens): If the tagger or lemmatizer flags garbled input (encoding or OCR noise), that input could be routed back for cleaning instead of being parsed into nonsense trees.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 487 works

## Cite

Cite as: Kesen Wang, Daulet Toibazar and Pedro J Moreno Mengibar. 2025. -SEA³-QA: A Fully Automated Self-Evolving, Adversarial Workflow for Arabic Long-Context Question-Answer Generation. Proceedings of The Third Arabic Natural Language Processing Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2025qa,
  title = {{-SEA³-QA: A Fully Automated Self-Evolving, Adversarial Workflow for Arabic Long-Context Question-Answer Generation}},
  author = {Kesen Wang, Daulet Toibazar and Pedro J Moreno Mengibar},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-main.9},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-main.9}
}
```

- [← previous «Are you Afraid of Ghosts?» A Proposal for Busting Predicate Ellipsis in Universal Dependencies](https://md.research.elamur.ai/free/en/papers/corbetta-2025-guillemotleftare-you-afraid-ghosts-guillemotright/)
- [next → A corpus-assisted metaphor analysis in portraying businesswomen: Diachronic changes in Hong Kong English news](https://md.research.elamur.ai/free/en/papers/li-2025-corpus-assisted-metaphor-analysis-portraying/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/16/)

## References

34 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774. [↗](https://arxiv.org/abs/2303.08774)
2. Anthropic. 2024. Claude 3 haiku: Our fastest model yet. Accessed: 2025-02-11. [find](https://scholar.google.com/scholar?q=Anthropic%2E+2024%2E+Claude+3+haiku%3A+Our+fastest+model+yet%2E+Accessed%3A+2025%2D02%2D11%2E)
3. Jeremy Belval. 2018. pdf2image: A python library to convert pdf pages to images using poppler. GitHub repository. [find](https://scholar.google.com/scholar?q=Jeremy+Belval%2E+2018%2E+pdf2image%3A+A+python+library+to+convert+pdf+pages+to+images+using+poppler%2E+GitHub+repository%2E)
4. Gagan Bhatia, El Moatez Billah Nagoudi, Fakhraddin Alwajih, and Muhammad Abdul-Mageed. 2024. [Qalam: A multimodal llm for arabic optical character and handwriting recognition](https://md.research.elamur.ai/free/en/papers/bhatia-2024-qalam-multimodal-llm-arabic-optical/). arXiv preprint arXiv:2407.13559. [↗](https://arxiv.org/abs/2407.13559)
5. Chao Deng, Jiale Yuan, Pi Bu, Peijie Wang, Zhong-Zhi Li, Jian Xu, Xiao-Hui Li, Yuan Gao, Jun Song, Bo Zheng, and 1 others. 2024. [Longdocurl: a comprehensive multimodal long document benchmark integrating understanding, reasoning, and locating](https://md.research.elamur.ai/free/en/papers/deng-2025-longdocurl-comprehensive-multimodal-long-document/). arXiv preprint arXiv:2412.18424. [↗](https://arxiv.org/abs/2412.18424)
6. Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Songyang Zhang, Haodong Duan, Wenwei Zhang, Yining Li, and 1 others. 2024. [Internlm-xcomposer2-4khd: A pioneering large vision-language model handling resolutions from 336 pixels to 4k hd](https://md.research.elamur.ai/free/en/papers/dong-2024-internlm-xcomposer2-4khd-pioneering/). arXiv preprint arXiv:2404.06512. [↗](https://arxiv.org/abs/2404.06512)
7. Gemini Team. 2024. [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-team-2024-gemini-1-5-unlocking-multimodal/). Accessed: 2025-02-11. [↗](https://doi.org/10.48550/arxiv.2403.05530)
8. Sara Ghaboura, Ahmed Heakl, Omkar Thawakar, Ali Alharthi, Ines Riahi, Abduljalil Saif, Jorma Laaksonen, Fahad S Khan, Salman Khan, and Rao M Anwer. 2024. [Camel-bench: A comprehensive arabic lmm benchmark](https://md.research.elamur.ai/free/en/papers/ghaboura-2025-camel-bench-comprehensive-arabic-lmm/). arXiv preprint arXiv:2410.18976. [↗](https://arxiv.org/abs/2410.18976)
9. Ahmed Heakl, Abdullah Sohail, Mukul Ranjan, Rania Hossam, Ghazi Ahmed, Mohamed El-Geish, Omar Maher, Zhiqiang Shen, Fahad Khan, and Salman Khan. 2025. [Kitab-bench: A comprehensive multi-domain benchmark for arabic ocr and document understanding](https://md.research.elamur.ai/free/en/papers/heakl-2025-kitab-bench-comprehensive-multi-domain/). arXiv preprint arXiv:2502.14949. [↗](https://arxiv.org/abs/2502.14949)
10. Benjamin Kiessling, Daniel Stökl Ben Ezra, and Matthew Thomas Miller. 2019. Badam: a public dataset for baseline detection in arabic-script manuscripts. In Proceedings of the 5th International Workshop on Historical Document Imaging and Processing, pages 13–18. [↗](https://doi.org/10.48550/arxiv.1907.04041)
11. Labellerr Inc. 2024. Labellerr: Automated activelearning and selfcorrection annotation pipeline. Proprietary product documentation. Available from Labellerr Inc. [find](https://scholar.google.com/scholar?q=Labellerr+Inc%2E+2024%2E+Labellerr%3A+Automated+activelearning+and+selfcorrection+annotation+pipeline%2E+Proprietary+product+documentation%2E+Available+from+Labellerr+Inc%2E)
12. LandingAI. 2025. Agentic document extraction. Proprietary product documentation. Available from LandingAI. [find](https://scholar.google.com/scholar?q=LandingAI%2E+2025%2E+Agentic+document+extraction%2E+Proprietary+product+documentation%2E+Available+from+LandingAI%2E)
13. Bo Li, Kaichen Zhang, Hao Zhang, Dong Guo, Renrui Zhang, Feng Li, Yuanhan Zhang, Ziwei Liu, and Chunyuan Li. 2024. Llava-next: Stronger llms supercharge multimodal capabilities in the wild. [find](https://scholar.google.com/scholar?q=Bo+Li%2C+Kaichen+Zhang%2C+Hao+Zhang%2C+Dong+Guo%2C+Renrui+Zhang%2C+Feng+Li%2C+Yuanhan+Zhang%2C+Ziwei+Liu%2C+and+Chunyuan+Li%2E+2024%2E+Llava%2Dnext%3A+Stronger+llms+supercharge+multimodal+capabilities+in+the+wild%2E)
14. Yubo Ma, Yuhang Zang, Liangyu Chen, Meiqi Chen, Yizhu Jiao, Xinze Li, Xinyuan Lu, Ziyu Liu, Yan Ma, Xiaoyi Dong, and 1 others. 2024. [Mmlongbench-doc: Benchmarking long-context document understanding with visualizations](https://md.research.elamur.ai/free/en/papers/ma-2024-mmlongbench-doc-benchmarking-long-context/). arXiv preprint arXiv:2407.01523. [↗](https://arxiv.org/abs/2407.01523)
15. Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. [Chartqa: A benchmark for question answering about charts with visual and logical reasoning](https://md.research.elamur.ai/free/en/papers/masry-2022-chartqa-benchmark-question-answering-about/). arXiv preprint arXiv:2203.10244. [↗](https://arxiv.org/abs/2203.10244)
16. Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawahar. 2022. Infographicvqa. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1697–1706. [find](https://scholar.google.com/scholar?q=Minesh+Mathew%2C+Viraj+Bagal%2C+Rub%C3%A8n+Tito%2C+Dimosthenis+Karatzas%2C+Ernest+Valveny%2C+and+CV+Jawahar%2E+2022%2E+Infographicvqa%2E+In+Proceedings+of+the+IEEE%2FCVF+Winter+Conference+on+Applications+of+Computer+Vision%2C)
17. Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. 2021. [Docvqa: A dataset for vqa on document images](https://md.research.elamur.ai/free/en/papers/mathew-2020-docvqa-dataset-vqa-document-images/). In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 2200–2209.
18. Omer Nacar, Yasser Al-Habashi, Serry Sibaee, Adel Ammar, and Wadii Boulila. 2025. Sard: A large-scale synthetic arabic ocr dataset for book-style text recognition. arXiv preprint arXiv:2505.24600. [↗](https://arxiv.org/abs/2505.24600)
19. OpenAI. 2024a. Advancing reasoning with o3. Accessed: 2025-02-12. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024a%2E+Advancing+reasoning+with+o3%2E+Accessed%3A+2025%2D02%2D12%2E)
20. OpenAI. 2024b. Introducing o1: A new era of reasoning models. Accessed: 2025-02-12. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024b%2E+Introducing+o1%3A+A+new+era+of+reasoning+models%2E+Accessed%3A+2025%2D02%2D12%2E)
21. Mohamed Rashad. 2024. [Arabic-nougat: Fine-tuning vision transformers for arabic ocr and markdown extraction](https://md.research.elamur.ai/free/en/papers/rashad-2024-arabic-nougat-fine-tuning-vision/). arXiv preprint arXiv:2411.17835. [↗](https://arxiv.org/abs/2411.17835)
22. Leonard Richardson. 2007. Beautiful soup documentation. images for people with visual impairments. In Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments, pages 1–8. [find](https://scholar.google.com/scholar?q=Leonard+Richardson%2E+2007%2E+Beautiful+soup+documentation%2E+images+for+people+with+visual+impairments%2E+In+Proceedings+of+the+9th+ACM+International+Conference+on+PErvasive+Technologies+Related+to+Assistive)
23. Zach Smiley. 2020. pdfplumber: A Python library for extracting text and tables from pdfs. [https://github.com/jsvine/pdfplumber](https://github.com/jsvine/pdfplumber). [↗](https://github.com/jsvine/pdfplumber)
24. Ray Smith. 2007. An overview of the tesseract ocr engine. In Proceedings of the International Conference on Document Analysis and Recognition, pages 629–633. [find](https://scholar.google.com/scholar?q=Ray+Smith%2E+2007%2E+An+overview+of+the+tesseract+ocr+engine%2E+In+Proceedings+of+the+International+Conference+on+Document+Analysis+and+Recognition%2C+pages+629%E2%80%93633%2E)
25. Tarjama (Arabic.AI). 2025. Arabic.ai agentic ecosystem. Proprietary product documentation. Available from Arabic.AI. [find](https://scholar.google.com/scholar?q=Tarjama+%28Arabic%2EAI%29%2E+2025%2E+Arabic%2Eai+agentic+ecosystem%2E+Proprietary+product+documentation%2E+Available+from+Arabic%2EAI%2E)
26. UiPath. 2025. Activelearning document understanding pipeline. Proprietary product documentation. Available from UiPath. [find](https://scholar.google.com/scholar?q=UiPath%2E+2025%2E+Activelearning+document+understanding+pipeline%2E+Proprietary+product+documentation%2E+Available+from+UiPath%2E)
27. Unicode Consortium. 1996. The Unicode Standard, Version 2.0. AddisonWesley. [find](https://scholar.google.com/scholar?q=Unicode+Consortium%2E+1996%2E+The+Unicode+Standard%2C+Version+2%2E0%2E+AddisonWesley%2E)
28. Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, and 1 others. 2023. [Cogvlm: Visual expert for pretrained language models](https://md.research.elamur.ai/free/en/papers/wang-2023-cogvlm-visual-expert-pretrained/). arXiv preprint arXiv:2311.03079. [↗](https://arxiv.org/abs/2311.03079)
29. Haoran Wei, Chenglong Liu, Jinyue Chen, Jia Wang, Lingyu Kong, Yanming Xu, Zheng Ge, Liang Zhao, Jianjian Sun, Yuang Peng, Chunrui Han, and Xiangyu Zhang. 2024a. General ocr theory: Towards ocr-2.0 via a unified end-to-end model. Preprint, arXiv:2409.01704. [↗](https://arxiv.org/abs/2409.01704)
30. Haoran Wei, Chenglong Liu, Jinyue Chen, Jia Wang, Lingyu Kong, Yanming Xu, Zheng Ge, Liang Zhao, Jianjian Sun, Yuang Peng, and 1 others. 2024b. General ocr theory: Towards ocr-2.0 via a unified end-to-end model. [find](https://scholar.google.com/scholar?q=Haoran+Wei%2C+Chenglong+Liu%2C+Jinyue+Chen%2C+Jia+Wang%2C+Lingyu+Kong%2C+Yanming+Xu%2C+Zheng+Ge%2C+Liang+Zhao%2C+Jianjian+Sun%2C+Yuang+Peng%2C+and+1+others%2E+2024b%2E+General+ocr+theory%3A+Towards+ocr%2D2%2E0+via+a+unified+end%2Dto)
31. Wenda Xu, Daniel Deutsch, Mara Finkelstein, Juraj Juraska, Biao Zhang, Zhongtao Liu, William Yang Wang, Lei Li, and Markus Freitag. 2023. Fine-grained llm agent: Pinpointing and refining large language models via fine-grained actionable feedback. arXiv preprint arXiv:2311.09336. [↗](https://arxiv.org/abs/2311.09336)
32. Zhiyuan Zhao, Hengrui Kang, Bin Wang, and Conghui He. 2024. Doclayout-yolo: Enhancing document layout analysis through diverse synthetic data and global-to-local adaptive perception. arXiv preprint arXiv:2410.12628. [↗](https://arxiv.org/abs/2410.12628)
33. Fengbin Zhu, Wenqiang Lei, Fuli Feng, Chao Wang, Haozhou Zhang, and Tat-Seng Chua. 2022. Towards complex document understanding by discrete reasoning. In Proceedings of the 30th ACM International Conference on Multimedia, pages 4857–4866. [find](https://scholar.google.com/scholar?q=Fengbin+Zhu%2C+Wenqiang+Lei%2C+Fuli+Feng%2C+Chao+Wang%2C+Haozhou+Zhang%2C+and+Tat%2DSeng+Chua%2E+2022%2E+Towards+complex+document+understanding+by+discrete+reasoning%2E+In+Proceedings+of+the+30th+ACM+International+Conf)
34. Rana SM Saad, Randa I Elanwar, NS Abdel Kader, Samia Mashali, and Margrit Betke. 2016. Bce-arabic-v1 dataset: Towards interpreting arabic document [find](https://scholar.google.com/scholar?q=Rana+SM+Saad%2C+Randa+I+Elanwar%2C+NS+Abdel+Kader%2C+Samia+Mashali%2C+and+Margrit+Betke%2E+2016%2E+Bce%2Darabic%2Dv1+dataset%3A+Towards+interpreting+arabic+document)

*Updated 2 November 2026*