---
title: "Author-in-the-Loop Response Generation and Evaluation:"
description: "The authors build Re³Align from conference (EMNLP24) and journal (PeerJ) records. 44 references, cited by 2 papers here."
authors: ["Qian Ruan", "Iryna Gurevych"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.565/"
doi: "10.18653/v1/2026.acl-long.565"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ruan-2026-author-loop-response-generation-evaluation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ruan-2026-author-loop-response-generation-evaluation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/7/) / Author-in-the-Loop Response Generation and Evaluation: Integrating Author Expertise and Intent in Responses to Peer Review

# Author-in-the-Loop Response Generation and Evaluation: Integrating Author Expertise and Intent in Responses to Peer Review

**Authors:** Qian Ruan and [Iryna Gurevych](https://md.research.elamur.ai/free/en/authors/iryna-gurevych/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.565/) · [PDF](https://aclanthology.org/2026.acl-long.565.pdf) · [DOI 10.18653/v1/2026.acl-long.565](https://doi.org/10.18653/v1/2026.acl-long.565) · [OpenAlex](https://openalex.org/W7166826830)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/ruan-2026-author-loop-response-generation-evaluation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Authors draw on knowledge only they have when answering peer reviewers, such as planned clarifications, new experiments and design reasons. They also choose deliberately how to respond. Earlier work on author response generation (ARG) gave models only the review text, which produced generic replies that were judged by surface overlap. This paper recasts ARG as an author-in-the-loop task.

The authors build Re³Align from conference (EMNLP24) and journal (PeerJ) records. Paper revisions serve as stand-ins for author signals and are linked to review comments and responses at the sentence-edit level. REspGen generates responses from author input of varying specificity. It controls response plans and length and refines drafts iteratively using evaluation feedback. REspEval scores outputs with more than 20 metrics.

Experiments with five state-of-the-art LLMs in nine settings show three things:
- Author input and evaluation-guided refinement improve responses.
- Richer context strengthens factual grounding but can dilute focus.
- Length limits trade off against quality.

The released resources support research on human–AI collaboration in peer review.

## Key points
- **Re³Align** contains 3,394 complete paper records (679 EMNLP24 from NLPEERv2/OpenReview, 2,715 PeerJ from MOPRD). It includes 16,071 review–response pairs, 439,798 annotated sentence-level edits and 15,521 aligned triplets. It is released under CC BY-NC 4.0.
- **REspGen** supports flexible author input, multi-attribute control (response action plan and length), optional retrieved paper context, and refinement guided by REspEval.
- **REspEval** covers four dimensions:
  - input utilization (factual grounding, input coverage);
  - controllability (plan label recall, plan order fidelity, length);
  - response quality (targeting, specificity, convincingness);
  - discourse (tone–stance profiles and transitions).
- **Models tested:** Llama-3.3, DeepSeek, Phi-4, Qwen3 and GPT-4o. Without author input, LLMs hedge less than humans (0.16–0.21 vs. 0.28). With author input, their tone–stance moves closer to human patterns and they become less overpolite.
- **Limitations:** the data are English-only scientific papers, and no fine-tuned or smaller models were evaluated. Code is on GitHub (UKPLab/acl2026-respgen-respeval), and the data are on TUdatalib.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🎭 History and Philosophy of Science](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/history-and-philosophy-of-science/) · 53 works

🧭 Topics (2): [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works · [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 624 works

## Cite

Cite as: Qian Ruan and Iryna Gurevych. 2026. Author-in-the-Loop Response Generation and Evaluation: Integrating Author Expertise and Intent in Responses to Peer Review. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ruan2026author,
  title = {{Author-in-the-Loop Response Generation and Evaluation: Integrating Author Expertise and Intent in Responses to Peer Review}},
  author = {Qian Ruan and Iryna Gurevych},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.565},
  url = {https://doi.org/10.18653/v1/2026.acl-long.565}
}
```

- [← previous Attention Basin: Why Contextual Position Matters in Large Language Models](https://md.research.elamur.ai/free/en/papers/yi-2026-attention-basin-why-contextual-position/)
- [next → Automatic Correction of Writing Anomalies in Hausa Texts](https://md.research.elamur.ai/free/en/papers/wali-2026-automatic-correction-writing-anomalies-hausa/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/7/)

## References

44 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marah Abdin, Sahaj Agarwal, Ahmed Awadallah, Vidhisha Balachandran, Harkirat Behl, Lingjiao Chen, Gustavo de Rosa, Suriya Gunasekar, Mojan Javaheripi, Neel Joshi, Piero Kauffmann, Yash Lara, Caio César Teodoro Mendes, Arindam Mitra, Besmira Nushi, Dimitris Papailiopoulos, Olli Saarikivi, Shital Shah… [find](https://scholar.google.com/scholar?q=Marah+Abdin%2C+Sahaj+Agarwal%2C+Ahmed+Awadallah%2C+Vidhisha+Balachandran%2C+Harkirat+Behl%2C+Lingjiao+Chen%2C+Gustavo+de+Rosa%2C+Suriya+Gunasekar%2C+Mojan+Javaheripi%2C+Neel+Joshi%2C+Piero+Kauffmann%2C+Yash+Lara%2C+Caio+C%C3%A9sa)
2. Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. [M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/). In Findings of the Association for Computational Linguistics: ACL 2024, pages 2318–2335, Bang…
3. Liying Cheng, Lidong Bing, Qian Yu, Wei Lu, and Luo Si. 2020. APE: Argument pair extraction from peer review and rebuttal via multi-task learning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7000–7011, Online. Association for Computational… [find](https://scholar.google.com/scholar?q=Liying+Cheng%2C+Lidong+Bing%2C+Qian+Yu%2C+Wei+Lu%2C+and+Luo+Si%2E+2020%2E+APE%3A+Argument+pair+extraction+from+peer+review+and+rebuttal+via+multi%2Dtask+learning%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Me)
4. Gordon V. Cormack, Charles L A Clarke, and Stefan Buettcher. 2009. [Reciprocal rank fusion outperforms condorcet and individual rank learning methods](https://md.research.elamur.ai/free/en/papers/cormack-2009-reciprocal-rank-fusion-outperforms/). In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’09, page 758–759, New https…
5. York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=York%2C+NY%2C+USA%2E+Association+for+Computing+Machinery%2E)
6. Mike D’Arcy, Alexis Ross, Erin Bransom, Bailey Kuehl, Jonathan Bragg, Tom Hope, and Doug Downey. 2024. [ARIES: A corpus of scientific paper edits made in response to peer reviews](https://md.research.elamur.ai/free/en/papers/arcy-2024-aries-corpus-scientific-paper-edits/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages…
7. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in ll…](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
8. Nils Dycke, Ilia Kuznetsov, and Iryna Gurevych. 2023. [NLPeer: A unified resource for the computational study of peer review](https://md.research.elamur.ai/free/en/papers/dycke-2023-nlpeer-unified-resource-computational-study/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5049– 5073, Toronto, Canada. Association for Computat…
9. Mauajama Firdaus, Hardik Chauhan, Asif Ekbal, and Pushpak Bhattacharyya. 2020. Emosen: Generating sentiment and emotion controlled responses in a multimodal dialogue system. IEEE Transactions on Affective Computing, PP:1–1. [find](https://scholar.google.com/scholar?q=Mauajama+Firdaus%2C+Hardik+Chauhan%2C+Asif+Ekbal%2C+and+Pushpak+Bhattacharyya%2E+2020%2E+Emosen%3A+Generating+sentiment+and+emotion+controlled+responses+in+a+multimodal+dialogue+system%2E+IEEE+Transactions+on+Affec)
10. Yang Gao, Steffen Eger, Ilia Kuznetsov, Iryna Gurevych, and Yusuke Miyao. 2019. Does my rebuttal matter? insights from a major NLP conference. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (… [find](https://scholar.google.com/scholar?q=Yang+Gao%2C+Steffen+Eger%2C+Ilia+Kuznetsov%2C+Iryna+Gurevych%2C+and+Yusuke+Miyao%2E+2019%2E+Does+my+rebuttal+matter%3F+insights+from+a+major+NLP+conference%2E+In+Proceedings+of+the+2019+Conference+of+the+North+Americ)
11. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
12. Junjie Huang, Win bin Huang, Yi Bu, Qi Cao, Huawei Shen, and Xueqi Cheng. 2023. What makes a successful rebuttal in computer science conferences?: A perspective on social interaction. Journal of Informetrics, 17(3):101427. [find](https://scholar.google.com/scholar?q=Junjie+Huang%2C+Win+bin+Huang%2C+Yi+Bu%2C+Qi+Cao%2C+Huawei+Shen%2C+and+Xueqi+Cheng%2E+2023%2E+What+makes+a+successful+rebuttal+in+computer+science+conferences%3F%3A+A+perspective+on+social+interaction%2E+Journal+of+Infor)
13. Neha Nayak Kennard, Tim O’Gorman, Rajarshi Das, Akshay Sharma, Chhandak Bagchi, Matthew Clinton, Pranay Kumar Yelugam, Hamed Zamani, and Andrew McCallum. 2022. DISAPERE: A dataset for discourse structure in peer review discussions. [find](https://scholar.google.com/scholar?q=Neha+Nayak+Kennard%2C+Tim+O%E2%80%99Gorman%2C+Rajarshi+Das%2C+Akshay+Sharma%2C+Chhandak+Bagchi%2C+Matthew+Clinton%2C+Pranay+Kumar+Yelugam%2C+Hamed+Zamani%2C+and+Andrew+McCallum%2E+2022%2E+DISAPERE%3A+A+dataset+for+discourse+struct)
14. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1234–1249, Seattle, United States. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+2022+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A+Human+Language+Technologies%2C+pages+1234%E2%80%931249%2C+Seattle%2C+United+States%2E+Association+)
15. Yuta Kikuchi, Graham Neubig, Ryohei Sasano, Hiroya Takamura, and Manabu Okumura. 2016. [Controlling output length in neural encoder-decoders](https://md.research.elamur.ai/free/en/papers/kikuchi-2016-controlling-output-length-neural-encoder/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1328– 1338, Austin, Texas. Association for Computational Lingu…
16. Terry K. Koo and Mae Y. Li. 2016. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2):155–163. [find](https://scholar.google.com/scholar?q=Terry+K%2E+Koo+and+Mae+Y%2E+Li%2E+2016%2E+A+guideline+of+selecting+and+reporting+intraclass+correlation+coefficients+for+reliability+research%2E+Journal+of+Chiropractic+Medicine%2C+15%282%29%3A155%E2%80%93163%2E)
17. Ilia Kuznetsov, Osama Mohammed Afzal, Koen Dercksen, Nils Dycke, Alexander Goldberg, Tom Hope, Dirk Hovy, Jonathan K. Kummerfeld, Anne Lauscher, Kevin Leyton-Brown, Sheng Lu, Mausam, Margot Mieskes, Aurélie Névéol, Danish Pruthi, Lizhen Qu, Roy Schwartz, Noah A. Smith, Thamar Solorio, and 5 others.… [find](https://scholar.google.com/scholar?q=Ilia+Kuznetsov%2C+Osama+Mohammed+Afzal%2C+Koen+Dercksen%2C+Nils+Dycke%2C+Alexander+Goldberg%2C+Tom+Hope%2C+Dirk+Hovy%2C+Jonathan+K%2E+Kummerfeld%2C+Anne+Lauscher%2C+Kevin+Leyton%2DBrown%2C+Sheng+Lu%2C+Mausam%2C+Margot+Mieskes%2C+A)
18. Ilia Kuznetsov, Jan Buchmann, Max Eichler, and Iryna Gurevych. 2022. [Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review](https://md.research.elamur.ai/free/en/papers/kuznetsov-2022-revise-resubmit-intertextual-model-text/). Computational Linguistics, 48(4):949–986.
19. J Richard Landis and Gary G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics, 33 1:159–74. [find](https://scholar.google.com/scholar?q=J+Richard+Landis+and+Gary+G%2E+Koch%2E+1977%2E+The+measurement+of+observer+agreement+for+categorical+data%2E+Biometrics%2C+33+1%3A159%E2%80%9374%2E)
20. Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. 2024. [Llms-as-judges: A comprehensive survey on llm-based evaluation methods](https://md.research.elamur.ai/free/en/papers/li-2024-llms-judges-comprehensive-survey-llm/). Preprint, arXiv:2412.05579.
21. Haoyuan Li, Somnath Basu Roy Chowdhury, and Snigdha Chaturvedi. 2023. Aspect-aware unsupervised extractive opinion summarization. In Findings of the Association for Computational Linguistics: ACL 2023, pages 12662–12678, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Haoyuan+Li%2C+Somnath+Basu+Roy+Chowdhury%2C+and+Snigdha+Chaturvedi%2E+2023%2E+Aspect%2Daware+unsupervised+extractive+opinion+summarization%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2023)
22. Xun Liang, Hanyu Wang, Yezhaohui Wang, Shichao Song, Jiawei Yang, Simin Niu, Jie Hu, Dan Liu, Shunyu Yao, Feiyu Xiong, and Zhiyu Li. 2024. Controllable text generation for [large language models: A survey](https://md.research.elamur.ai/free/en/papers/minaee-2024-large-language-models-survey/). Preprint, arXiv:2408.12599.
23. Jialiang Lin, Jiaxin Song, Zhangping Zhou, Yidong Chen, and Xiaodong Shi. 2023. Moprd: A multidisciplinary open peer review dataset. Neural Computing and Applications, 35(34):24191–24206. [find](https://scholar.google.com/scholar?q=Jialiang+Lin%2C+Jiaxin+Song%2C+Zhangping+Zhou%2C+Yidong+Chen%2C+and+Xiaodong+Shi%2E+2023%2E+Moprd%3A+A+multidisciplinary+open+peer+review+dataset%2E+Neural+Computing+and+Applications%2C+35%2834%29%3A24191%E2%80%9324206%2E)
24. Guillermo Macbeth, Eugenia Razumiejczyk, and Rubén Ledesma. 2011. Cliff’s delta calculator: A non-parametric effect size program for two groups of observations. Universitas Psychologica, 10:545–555. [find](https://scholar.google.com/scholar?q=Guillermo+Macbeth%2C+Eugenia+Razumiejczyk%2C+and+Rub%C3%A9n+Ledesma%2E+2011%2E+Cliff%E2%80%99s+delta+calculator%3A+A+non%2Dparametric+effect+size+program+for+two+groups+of+observations%2E+Universitas+Psychologica%2C+10%3A545%E2%80%93555%2E)
25. Mounica Maddela, Mayank Kulkarni, and Daniel Preotiuc-Pietro. 2022. EntSUM: A data set for entity-centric extractive summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3355–3366, Dublin, Ireland. Association for Co… [find](https://scholar.google.com/scholar?q=Mounica+Maddela%2C+Mayank+Kulkarni%2C+and+Daniel+Preotiuc%2DPietro%2E+2022%2E+EntSUM%3A+A+data+set+for+entity%2Dcentric+extractive+summarization%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+for+Com)
26. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. [FActScore: Fine-grained atomic evaluation of factual precision in long form text generation](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/). In Proceedings of the 2023 Conference on Empirical Methods in Natural L…
27. William Noble. 2017. Ten simple rules for writing a response to reviewers. PLOS Computational Biology, 13:e1005730. [find](https://scholar.google.com/scholar?q=William+Noble%2E+2017%2E+Ten+simple+rules+for+writing+a+response+to+reviewers%2E+PLOS+Computational+Biology%2C+13%3Ae1005730%2E)
28. Sukannya Purkayastha, Anne Lauscher, and Iryna Gurevych. 2023. Exploring jiu-jitsu argumentation for writing peer review rebuttals. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14479–14495, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sukannya+Purkayastha%2C+Anne+Lauscher%2C+and+Iryna+Gurevych%2E+2023%2E+Exploring+jiu%2Djitsu+argumentation+for+writing+peer+review+rebuttals%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natura)
29. Nils Reimers and Iryna Gurevych. 2019. [Sentence-BERT: Sentence embeddings using Siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
30. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends in Information Retrieval, 3(4):333–389. [find](https://scholar.google.com/scholar?q=Stephen+Robertson+and+Hugo+Zaragoza%2E+2009%2E+The+probabilistic+relevance+framework%3A+Bm25+and+beyond%2E+Foundations+and+Trends+in+Information+Retrieval%2C+3%284%29%3A333%E2%80%93389%2E)
31. Qian Ruan, Ilia Kuznetsov, and Iryna Gurevych. 2024a. Are large language models good classifiers? a study on edit intent classification in scientific document revisions. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 15049–15067, Miami, Florida, USA.… [find](https://scholar.google.com/scholar?q=Qian+Ruan%2C+Ilia+Kuznetsov%2C+and+Iryna+Gurevych%2E+2024a%2E+Are+large+language+models+good+classifiers%3F+a+study+on+edit+intent+classification+in+scientific+document+revisions%2E+In+Proceedings+of+the+2024+Con)
32. Qian Ruan, Ilia Kuznetsov, and Iryna Gurevych. 2024b. Re3: A holistic framework and dataset for modeling collaborative document revision. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4635–4655, Bangkok, Thailand. Associatio… [find](https://scholar.google.com/scholar?q=Qian+Ruan%2C+Ilia+Kuznetsov%2C+and+Iryna+Gurevych%2E+2024b%2E+Re3%3A+A+holistic+framework+and+dataset+for+modeling+collaborative+document+revision%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Association+f)
33. Patrick E. Shrout and Joseph L. Fleiss. 1979. Intraclass correlations: uses in assessing rater reliability. Psychological bulletin, 86 2:420–8. [find](https://scholar.google.com/scholar?q=Patrick+E%2E+Shrout+and+Joseph+L%2E+Fleiss%2E+1979%2E+Intraclass+correlations%3A+uses+in+assessing+rater+reliability%2E+Psychological+bulletin%2C+86+2%3A420%E2%80%938%2E)
34. Amanpreet Singh, Mike D’Arcy, Arman Cohan, Doug Downey, and Sergey Feldman. 2023. SciRepEval: A multi-format benchmark for scientific document representations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5548–5566, Singapore. Association for Compu… [find](https://scholar.google.com/scholar?q=Amanpreet+Singh%2C+Mike+D%E2%80%99Arcy%2C+Arman+Cohan%2C+Doug+Downey%2C+and+Sergey+Feldman%2E+2023%2E+SciRepEval%3A+A+multi%2Dformat+benchmark+for+scientific+document+representations%2E+In+Proceedings+of+the+2023+Conference+on)
35. Moritz Staudinger, Wojciech Kusa, Florina Piroi, and Allan Hanbury. 2024. An analysis of tasks and datasets in peer reviewing. In Proceedings of the Fourth Workshop on Scholarly Document Processing (SDP 2024), pages 257–268, Bangkok, Thailand. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Moritz+Staudinger%2C+Wojciech+Kusa%2C+Florina+Piroi%2C+and+Allan+Hanbury%2E+2024%2E+An+analysis+of+tasks+and+datasets+in+peer+reviewing%2E+In+Proceedings+of+the+Fourth+Workshop+on+Scholarly+Document+Processing+%28S)
36. Student. 1908. The probable error of a mean. Biometrika, 6(1):1–25. [find](https://scholar.google.com/scholar?q=Student%2E+1908%2E+The+probable+error+of+a+mean%2E+Biometrika%2C+6%281%29%3A1%E2%80%9325%2E)
37. Cheng Tan, Dongxin Lyu, Siyuan Li, Zhangyang Gao, Jingxuan Wei, Siqi Ma, Zicheng Liu, and Stan Z. Li. 2024. Peer review as a multi-turn and long-context dialogue with role-based interactions. Preprint, arXiv:2406.05688. — [arXiv:2406.05688](https://arxiv.org/abs/2406.05688) [find](https://scholar.google.com/scholar?q=Cheng+Tan%2C+Dongxin+Lyu%2C+Siyuan+Li%2C+Zhangyang+Gao%2C+Jingxuan+Wei%2C+Siqi+Ma%2C+Zicheng+Liu%2C+and+Stan+Z%2E+Li%2E+2024%2E+Peer+review+as+a+multi%2Dturn+and+long%2Dcontext+dialogue+with+role%2Dbased+interactions%2E+Preprint)
38. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, and 49 others… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
39. Wenlin Wang, Zhe Gan, Hongteng Xu, Ruiyi Zhang, Guoyin Wang, Dinghan Shen, Changyou Chen, and Lawrence Carin. 2019. Topic-guided variational auto-encoder for text generation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human L… [find](https://scholar.google.com/scholar?q=Wenlin+Wang%2C+Zhe+Gan%2C+Hongteng+Xu%2C+Ruiyi+Zhang%2C+Guoyin+Wang%2C+Dinghan+Shen%2C+Changyou+Chen%2C+and+Lawrence+Carin%2E+2019%2E+Topic%2Dguided+variational+auto%2Dencoder+for+text+generation%2E+In+Proceedings+of+the+201)
40. Frank Wilcoxon. 1945. Individual comparisons by ranking methods. Biometrics Bulletin, 1(6):80–83. [find](https://scholar.google.com/scholar?q=Frank+Wilcoxon%2E+1945%2E+Individual+comparisons+by+ranking+methods%2E+Biometrics+Bulletin%2C+1%286%29%3A80%E2%80%9383%2E)
41. Yumo Xu and Mirella Lapata. 2021. Generating query focused summaries from query-free resources. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6096– 6… [find](https://scholar.google.com/scholar?q=Yumo+Xu+and+Mirella+Lapata%2E+2021%2E+Generating+query+focused+summaries+from+query%2Dfree+resources%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+11th+)
42. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
43. Daoze Zhang, Zhijian Bao, Sihang Du, Zhiyi Zhao, Kuangling Zhang, Dezheng Bao, and Yang Yang. 2025. Re 2 : A consistency-ensured dataset for full-stage peer review and multi-turn rebuttal discussions. Preprint, arXiv:2505.07920. — [arXiv:2505.07920](https://arxiv.org/abs/2505.07920) [find](https://scholar.google.com/scholar?q=Daoze+Zhang%2C+Zhijian+Bao%2C+Sihang+Du%2C+Zhiyi+Zhao%2C+Kuangling+Zhang%2C+Dezheng+Bao%2C+and+Yang+Yang%2E+2025%2E+Re+2+%3A+A+consistency%2Densured+dataset+for+full%2Dstage+peer+review+and+multi%2Dturn+rebuttal+discussions%2E)
44. Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. 2023. A survey of controllable text generation using transformer-based pre-trained language models. ACM Computing Surveys, 56(3). high-quality basis for subsequent alignment with revision edits. [find](https://scholar.google.com/scholar?q=Hanqing+Zhang%2C+Haolin+Song%2C+Shaoyu+Li%2C+Ming+Zhou%2C+and+Dawei+Song%2E+2023%2E+A+survey+of+controllable+text+generation+using+transformer%2Dbased+pre%2Dtrained+language+models%2E+ACM+Computing+Surveys%2C+56%283%29%2E+high)

*Updated 1 November 2026*