---
title: "From Questions to Assessment Tuples: A Multi-Agent Framework"
description: "The authors present a multi-agent, staged pipeline that turns course documents into complete assessment tuples for two… 25 references."
authors: ["Gee-Lyle Wong", "Runcong Zhao", "Yulan He", "Jiazheng Li"]
year: 2026
venue: "Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)"
source: "https://aclanthology.org/2026.bea-1.22/"
doi: "10.18653/v1/2026.bea-1.22"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wong-2026-questions-assessment-tuples-multi-agent/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wong-2026-questions-assessment-tuples-multi-agent/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/) / From Questions to Assessment Tuples: A Multi-Agent Framework with Bloom-Specialized Agents and Automated Verification

# From Questions to Assessment Tuples: A Multi-Agent Framework with Bloom-Specialized Agents and Automated Verification

**Authors:** Gee-Lyle Wong, Runcong Zhao, [Yulan He](https://md.research.elamur.ai/free/en/authors/yulan-he/) and Jiazheng Li · **Year:** 2026 · **Venue:** [Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)](https://md.research.elamur.ai/free/en/venues/bea/) ([BEA 2026](https://md.research.elamur.ai/free/en/venues/bea/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.bea-1.22/) · [PDF](https://aclanthology.org/2026.bea-1.22.pdf) · [DOI 10.18653/v1/2026.bea-1.22](https://doi.org/10.18653/v1/2026.bea-1.22) · [OpenAlex](https://openalex.org/W7166811835)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Large language models can write exam questions, but a usable assessment item also needs a marking scheme and a model answer, and it should match an intended level of Bloom's Taxonomy. Hitting the upper levels (analysis, evaluation, synthesis) is especially unreliable. The authors present a multi-agent, staged pipeline that turns course documents into complete assessment tuples for two formats:
- **Short-answer questions (SAQs):** question, mark scheme and expected answer.
- **Scenario-based questions (SBQs):** the same, plus a scenario that is critiqued and refined.

The pipeline works in four steps:
1. It grounds and compresses the source content.
2. Agents specialised per Bloom level write the question.
3. Separate agents add the mark scheme and the answer.
4. A verifier discards candidates that fail checks on alignment, grounding or structure.

The authors also propose a rubric-based LLM-as-a-judge protocol with Bloom-specific alignment metrics. Tested against four other pipelines on university AI course material, their approach gave better Bloom alignment, mark schemes and valid-item yield. Adding Bloom labels to prompts alone was not enough. This matters for scaling the production of open-ended, gradable assessments.

## Key points
- **Outputs:** SAQs are produced as (q, m, a) and SBQs as (s, q, m, a). The target is one of the six levels of the original Bloom's Taxonomy.
- **Comparison:** Five generation pipelines were compared, covering zero-shot, Bloom-aware prompting and multi-stage approaches. The proposed pipeline performed best, most clearly at the higher-order levels.
- **Models:** GPT-4.1, GPT-4o-mini and Gemini-2.5 Flash served as generators. GPT-4o-mini, GPT-4.1-mini and Gemini-2.5 Flash served as judges.
- **SBQs are harder:** Scenario grounding improved, but showing that a scenario is actually needed, that answers can be graded objectively and that answers stay faithful remained harder. The scenario-necessity criterion also had weaker agreement among the judges.
- **Limitations:**
  - Only one domain (AI) was tested.
  - The human study covered just 18 matched comparisons.
  - No component ablations were run.
  - Only proprietary models were used.
  - Failed items are dropped rather than regenerated.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [King's College London](https://md.research.elamur.ai/free/en/universities/king-s-college-london/) — University, London, United Kingdom · 46 works · 1,714 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works · [🎓 Education](https://md.research.elamur.ai/free/en/shelves/social-sciences/education/) · 212 works

## Cite

Cite as: Gee-Lyle Wong, Runcong Zhao, Yulan He and Jiazheng Li. 2026. From Questions to Assessment Tuples: A Multi-Agent Framework with Bloom-Specialized Agents and Automated Verification. Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wong2026questions,
  title = {{From Questions to Assessment Tuples: A Multi-Agent Framework with Bloom-Specialized Agents and Automated Verification}},
  author = {Gee-Lyle Wong, Runcong Zhao, Yulan He and Jiazheng Li},
  booktitle = {Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.bea-1.22},
  url = {https://doi.org/10.18653/v1/2026.bea-1.22}
}
```

- [← previous From Query to Counsel: Structured Reasoning with a Multi-Agent Framework and Dataset for Legal Consultation](https://md.research.elamur.ai/free/en/papers/lu-2026-query-counsel-structured-reasoning-multi/)
- [next → From Recognition to Reasoning: Benchmarking and Enhancing MLLMs on Real-World Receipt Document Understanding](https://md.research.elamur.ai/free/en/papers/wang-2026-recognition-reasoning-benchmarking-enhancing-mllms/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/)

## References

25 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Kuang Wen Chan, Farhan Ali, Joonhyeong Park, Kah Shen Brandon Sham, Erdalyn Yeh Thong Tan, Francis Woon Chien Chong, Kun Qian, and Guan Kheng Sze. 2025. Automatic item generation in various STEM subjects using large language model (LLM) prompting. Computers and Education: Artificial Intelligence, 8:… [find](https://scholar.google.com/scholar?q=Kuang+Wen+Chan%2C+Farhan+Ali%2C+Joonhyeong+Park%2C+Kah+Shen+Brandon+Sham%2C+Erdalyn+Yeh+Thong+Tan%2C+Francis+Woon+Chien+Chong%2C+Kun+Qian%2C+and+Guan+Kheng+Sze%2E+2025%2E+Automatic+item+generation+in+various+STEM+subje)
2. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, and 3416 [other…](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/)
3. Bidyut Das, Mukta Majumder, Santanu Phadikar, and Arif Ahmed Sekh. 2021. Automatic question generation and answer assessment: a survey. Research and Practice in Technology Enhanced Learning, 16(1):5. [find](https://scholar.google.com/scholar?q=Bidyut+Das%2C+Mukta+Majumder%2C+Santanu+Phadikar%2C+and+Arif+Ahmed+Sekh%2E+2021%2E+Automatic+question+generation+and+answer+assessment%3A+a+survey%2E+Research+and+Practice+in+Technology+Enhanced+Learning%2C+16%281%29%3A5%2E)
4. Weiping Fu, Bifan Wei, Jianxiang Hu, Zhongmin Cai, and Jun Liu. 2024. Qgeval: Benchmarking multi-dimensional evaluation for question generation. Preprint, arXiv:2406.05707. [find](https://scholar.google.com/scholar?q=Weiping+Fu%2C+Bifan+Wei%2C+Jianxiang+Hu%2C+Zhongmin+Cai%2C+and+Jun+Liu%2E+2024%2E+Qgeval%3A+Benchmarking+multi%2Ddimensional+evaluation+for+question+generation%2E+Preprint%2C+arXiv%3A2406%2E05707%2E)
5. Helia Hashemi, Jason Eisner, Corby Rosset, Benjamin Van Durme, and Chris Kedzie. 2024. [Llm-rubric: A multidimensional, calibrated approach to automated evaluation of natural language texts](https://md.research.elamur.ai/free/en/papers/hashemi-2024-llm-rubric-multidimensional-calibrated-approach/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap…
6. Jiazheng Li, Yuxiang Zhou, Junru Lu, Gladys Tyen, Lin Gui, Cesare Aloisi, and Yulan He. 2025. Two heads are better than one: Dual-model verbal reflection at inference-time. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 3119–3140, Suzhou, China. Asso… [find](https://scholar.google.com/scholar?q=Jiazheng+Li%2C+Yuxiang+Zhou%2C+Junru+Lu%2C+Gladys+Tyen%2C+Lin+Gui%2C+Cesare+Aloisi%2C+and+Yulan+He%2E+2025%2E+Two+heads+are+better+than+one%3A+Dual%2Dmodel+verbal+reflection+at+inference%2Dtime%2E+In+Proceedings+of+the+2025+)
7. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. [G-eval: Nlg evaluation using gpt-4 with better human alignment](https://md.research.elamur.ai/free/en/papers/liu-2023-g-eval-nlg-evaluation-gpt/). Preprint, arXiv:2303.16634.
8. Anderson LW, Krathwohl DR, Airasian PW, Cruikshank KA, Richard Mayer, Pintrich PR, J. Raths, and Wittrock MC. 2001. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. [find](https://scholar.google.com/scholar?q=Anderson+LW%2C+Krathwohl+DR%2C+Airasian+PW%2C+Cruikshank+KA%2C+Richard+Mayer%2C+Pintrich+PR%2C+J%2E+Raths%2C+and+Wittrock+MC%2E+2001%2E+A+Taxonomy+for+Learning%2C+Teaching%2C+and+Assessing%3A+A+Revision+of+Bloom%E2%80%99s+Taxonomy+of+)
9. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with… [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+Shashank+Gupta%2C+Bodhisattwa+Prasad+Majumder%2C+Katherine+Her)
10. Subhankar Maity, Aniket Deroy, and Sudeshna Sarkar. 2024. How effective is gpt-4 turbo in generating school-level questions from textbooks based on bloom’s revised taxonomy? Preprint, arXiv:2406.15211. — [arXiv:2406.15211](https://arxiv.org/abs/2406.15211) [find](https://scholar.google.com/scholar?q=Subhankar+Maity%2C+Aniket+Deroy%2C+and+Sudeshna+Sarkar%2E+2024%2E+How+effective+is+gpt%2D4+turbo+in+generating+school%2Dlevel+questions+from+textbooks+based+on+bloom%E2%80%99s+revised+taxonomy%3F+Preprint%2C+arXiv%3A2406%2E15211)
11. Alireza Mohammadshahi, Thomas Scialom, Majid Yazdani, Pouya Yanki, Angela Fan, James Henderson, and Marzieh Saeidi. 2023. Rquge: Reference-free metric for evaluating question generation by answering the question. Preprint, arXiv:2211.01482. [find](https://scholar.google.com/scholar?q=Alireza+Mohammadshahi%2C+Thomas+Scialom%2C+Majid+Yazdani%2C+Pouya+Yanki%2C+Angela+Fan%2C+James+Henderson%2C+and+Marzieh+Saeidi%2E+2023%2E+Rquge%3A+Reference%2Dfree+metric+for+evaluating+question+generation+by+answering+t)
12. Ghader Kurdi, Jared Leo, Bijan Parsia, Uli Sattler, and Solafa Al-Emari. 2020. A systematic review of automatic question generation for educational purposes. International Journal of Artificial Intelligence in Education, 30:1–70. [find](https://scholar.google.com/scholar?q=Ghader+Kurdi%2C+Jared+Leo%2C+Bijan+Parsia%2C+Uli+Sattler%2C+and+Solafa+Al%2DEmari%2E+2020%2E+A+systematic+review+of+automatic+question+generation+for+educational+purposes%2E+International+Journal+of+Artificial+Intell)
13. Jiazheng Li, Artem Bobrov, David West, Cesare Aloisi, and Yulan He. 2024a. An automated explainable educational assessment system built on llms. Preprint, arXiv:2412.13381. — [arXiv:2412.13381](https://arxiv.org/abs/2412.13381) [find](https://scholar.google.com/scholar?q=Jiazheng+Li%2C+Artem+Bobrov%2C+David+West%2C+Cesare+Aloisi%2C+and+Yulan+He%2E+2024a%2E+An+automated+explainable+educational+assessment+system+built+on+llms%2E+Preprint%2C+arXiv%3A2412%2E13381%2E+%E2%80%94+%5BarXiv%3A2412%2E13381%5D%28https%3A)
14. Jiazheng Li, Lin Gui, Yuxiang Zhou, David West, Cesare Aloisi, and Yulan He. 2023. Distilling ChatGPT for explainable automated student answer assessment. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 6007–6026, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jiazheng+Li%2C+Lin+Gui%2C+Yuxiang+Zhou%2C+David+West%2C+Cesare+Aloisi%2C+and+Yulan+He%2E+2023%2E+Distilling+ChatGPT+for+explainable+automated+student+answer+assessment%2E+In+Findings+of+the+Association+for+Computatio)
15. Jiazheng Li, Hainiu Xu, Zhaoyue Sun, Yuxiang Zhou, David West, Cesare Aloisi, and Yulan He. 2024b. Calibrating LLMs with preference optimization on thought trees for generating rationale in science question scoring. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 5452… [find](https://scholar.google.com/scholar?q=Jiazheng+Li%2C+Hainiu+Xu%2C+Zhaoyue+Sun%2C+Yuxiang+Zhou%2C+David+West%2C+Cesare+Aloisi%2C+and+Yulan+He%2E+2024b%2E+Calibrating+LLMs+with+preference+optimization+on+thought+trees+for+generating+rationale+in+science+qu)
16. Bang Nguyen, Mengxia Yu, Yun Huang, and Meng Jiang. 2024. Reference-based metrics disprove themselves in question generation. Preprint, arXiv:2403.12242. [find](https://scholar.google.com/scholar?q=Bang+Nguyen%2C+Mengxia+Yu%2C+Yun+Huang%2C+and+Meng+Jiang%2E+2024%2E+Reference%2Dbased+metrics+disprove+themselves+in+question+generation%2E+Preprint%2C+arXiv%3A2403%2E12242%2E)
17. Shinhyeok Oh, Hyojun Go, Hyeongdon Moon, Yunsung Lee, Myeongho Jeong, Hyun Seung Lee, and Seungtaek Choi. 2023. Evaluation of question generation needs more references. Preprint, arXiv:2305.16626. [find](https://scholar.google.com/scholar?q=Shinhyeok+Oh%2C+Hyojun+Go%2C+Hyeongdon+Moon%2C+Yunsung+Lee%2C+Myeongho+Jeong%2C+Hyun+Seung+Lee%2C+and+Seungtaek+Choi%2E+2023%2E+Evaluation+of+question+generation+needs+more+references%2E+Preprint%2C+arXiv%3A2305%2E16626%2E)
18. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and 262 others.… — [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
19. Nicy Scaria, Suma Dharani Chenna, and Deepak Subramani. 2024. Automated educational question generation at different bloom’s skill levels using large language models: Strategies and evaluation. In Proceedings of the 25th International Conference on Artificial Intelligence in Education (AIED). ArXiv… — [arXiv:2408.04394](https://arxiv.org/abs/2408.04394) [find](https://scholar.google.com/scholar?q=Nicy+Scaria%2C+Suma+Dharani+Chenna%2C+and+Deepak+Subramani%2E+2024%2E+Automated+educational+question+generation+at+different+bloom%E2%80%99s+skill+levels+using+large+language+models%3A+Strategies+and+evaluation%2E+In+Pro)
20. Y. Song, J. Du, and Q. Zheng. 2025. Automatic item generation for educational assessments: a systematic literature review. Interactive Learning Environments, 33(9):5386–5405. [find](https://scholar.google.com/scholar?q=Y%2E+Song%2C+J%2E+Du%2C+and+Q%2E+Zheng%2E+2025%2E+Automatic+item+generation+for+educational+assessments%3A+a+systematic+literature+review%2E+Interactive+Learning+Environments%2C+33%289%29%3A5386%E2%80%935405%2E)
21. Bin Tan, Nour Armoush, Elisabetta Mazzullo, Okan Bulut, and Mark Gierl. 2025. A review of automatic item generation techniques leveraging large language models (LLMs). International Journal of Assessment Tools in Education, 12(2):317–340. [find](https://scholar.google.com/scholar?q=Bin+Tan%2C+Nour+Armoush%2C+Elisabetta+Mazzullo%2C+Okan+Bulut%2C+and+Mark+Gierl%2E+2025%2E+A+review+of+automatic+item+generation+techniques+leveraging+large+language+models+%28LLMs%29%2E+International+Journal+of+Assessm)
22. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy Lillicrap, Angeliki Lazari… — [Gemini: A Family of Highly Capable Multimodal Models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/)
23. Masaki Uto, Yuto Tomikawa, and Ayaka Suzuki. 2023. Difficulty-controllable neural question generation for reading comprehension using item response theory. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 119–129, Toronto, Canada. A… [find](https://scholar.google.com/scholar?q=Masaki+Uto%2C+Yuto+Tomikawa%2C+and+Ayaka+Suzuki%2E+2023%2E+Difficulty%2Dcontrollable+neural+question+generation+for+reading+comprehension+using+item+response+theory%2E+In+Proceedings+of+the+18th+Workshop+on+Innov)
24. Antoun Yaacoub, Jérôme Da-Rugna, and Zainab As-saghir. 2025. Assessing ai-generated questions’ alignment with cognitive frameworks in educational assessment. International Journal of Computer Theory and Engineering, 17(3):114–125. [find](https://scholar.google.com/scholar?q=Antoun+Yaacoub%2C+J%C3%A9r%C3%B4me+Da%2DRugna%2C+and+Zainab+As%2Dsaghir%2E+2025%2E+Assessing+ai%2Dgenerated+questions%E2%80%99+alignment+with+cognitive+frameworks+in+educational+assessment%2E+International+Journal+of+Computer+Theory+a)
25. Archana Yadav, Harshvivek Kashid, Medchalimi Sruthi, B JayaPrakash, Chintalapalli Raja Kullayappa, Mandala Jagadeesh Reddy, and Pushpak Bhattacharyya. 2025. From recall to creation: Generating follow-up questions using bloom’s taxonomy and grice’s maxims. In Proceedings of the 63rd Annual Meeting of… [find](https://scholar.google.com/scholar?q=Archana+Yadav%2C+Harshvivek+Kashid%2C+Medchalimi+Sruthi%2C+B+JayaPrakash%2C+Chintalapalli+Raja+Kullayappa%2C+Mandala+Jagadeesh+Reddy%2C+and+Pushpak+Bhattacharyya%2E+2025%2E+From+recall+to+creation%3A+Generating+follow%2D)

*Updated 2 November 2026*