---
title: "Hey, wait a minute: on at-issue sensitivity in Language Models"
description: "They propose DGRC (Divide, Generate, Recombine, Compare). 36 references, cited by 1 paper here."
authors: ["Sanghee J. Kim", "Kanishka Misra"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)"
source: "https://aclanthology.org/2026.eacl-short.15/"
doi: "10.18653/v1/2026.eacl-short.15"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/kim-2026-hey-wait-minute-issue-sensitivity/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/kim-2026-hey-wait-minute-issue-sensitivity/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/) / Hey, wait a minute: on at-issue sensitivity in Language Models

# Hey, wait a minute: on at-issue sensitivity in Language Models

**Authors:** Sanghee J. Kim and [Kanishka Misra](https://md.research.elamur.ai/free/en/authors/kanishka-misra/) · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-short.15/) · [PDF](https://aclanthology.org/2026.eacl-short.15.pdf) · [DOI 10.18653/v1/2026.eacl-short.15](https://doi.org/10.18653/v1/2026.eacl-short.15) · [OpenAlex](https://openalex.org/W7140082605)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Judging whether a language model's dialogue replies sound natural is difficult. Definitions of naturalness are vague, and human ratings disagree. The authors tie naturalness to at-issueness: a natural reply normally addresses the main point of the previous utterance, not its backgrounded asides, unless the reply explicitly signals a digression. They propose DGRC (Divide, Generate, Recombine, Compare). An utterance containing an appositive relative clause (ARC) is split into two sub-utterances. The model generates replies to each part. Those replies are then scored as continuations of the full original utterance using length-normalized log-probabilities. Using 300 English items and 10 Llama-3 and Qwen2.5 models (base and instruct), they find that models favour replies about the main clause. Instruct models show this more strongly. The preference is larger for ARCs than for coordination, so it is not just a recency effect. A digression header such as "Hey, wait a minute" weakens the preference. The method lets researchers evaluate discourse behaviour without human raters or hand-written answer pairs.

## Key points
- Stimuli: 300 English ARC items from Kim et al. (2022), each with a subject NP, an ARC VP (VP1, not-at-issue) and a main-clause VP (VP2, at-issue).
- Baselines and controls:
  - Coordination versions ("S VP1 and VP2") serve as a recency control, since both VPs are at-issue there.
  - VP order was swapped as a robustness check, and the swap had no effect (p = 0.42).
- Models: base and instruct versions of Llama-3-8B and the Qwen2.5 family, from 500M to 7B parameters, 10 LMs in total.
  - Replies were produced with greedy decoding and with temperature, top-p and top-k sampling.
  - The top-10 replies per VP were kept, giving 100 pairwise comparisons per item.
- Experiment 1 results, tested with a linear mixed-effects model:
  - VP2 preference is higher for ARC than for coordination (p \< .01).
  - Instruct models show a higher VP2 preference than base models (p \< .01).
  - Model size had no notable effect.
- Experiment 2 and follow-up:
  - Experiment 2 compared the header "Hey, wait a minute" against "No, that's not true". The digression header lowered at-issue preference, and instruction tuning did not strengthen this modulation.
  - A post-hoc study with nominal appositives replicated Experiment 1 but not Experiment 2.
  - Code and data: github.com/sangheek16/hey-wait-a-minute.

## Interesting ideas
- Instead of scoring a few researcher-written "correct" answers, sample many candidate replies from the model itself and compare their likelihoods pairwise; this avoids author bias and stays inside what the model would actually produce.
- Generate continuations for each isolated piece of a sentence, then re-score all of them against the full sentence, so each candidate is clearly linked to the part it addresses.
- Use a minimally different construction that removes the targeted distinction (coordination vs. appositive) as a control, so a positional effect like recency cannot be mistaken for real sensitivity.
- Treat the main clause as at-issue and appositive/parenthetical material as backgrounded, and expect replies to target the main clause unless a digression header flags the shift.
- Score sequences by log-probability per token so that replies of different lengths can be compared fairly.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Evaluation of LLM-teacher tiers could score sets of teacher-sampled candidate frames or answers pairwise, rather than checking a single hand-picked gold string.
- **I2** (answer): Test generators for fables could build questions from each clause separately and check that the reader ranks main-clause questions as more relevant than questions about parenthetical asides.
- **I3** (learn): This fits the negative-controls practice: every pragmatic metric could be paired with a coordination-style variant where the effect should vanish.
- **I4** (frames): Crate prag could tag frames from ARCs and appositives (UD `acl:relcl` between commas, `appos`) as not-at-issue, and recognise phrases like "Hey, wait a minute" as digression speech acts.
- **I5** (back): When generation produces several surface realisations of a meaning graph, length-normalized scoring could pick among them without favouring the shortest.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [The University of Texas at Austin](https://md.research.elamur.ai/free/en/universities/the-university-of-texas-at-austin/) — University, Austin, United States · 109 works · 56,942 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Sanghee J. Kim and Kanishka Misra. 2026. Hey, wait a minute: on at-issue sensitivity in Language Models. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{kim2026hey,
  title = {{Hey, wait a minute: on at-issue sensitivity in Language Models}},
  author = {Sanghee J. Kim and Kanishka Misra},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-short.15},
  url = {https://doi.org/10.18653/v1/2026.eacl-short.15}
}
```

- [← previous Hey, That's My Data! Token-Only Dataset Inference in Large Language Models](https://md.research.elamur.ai/free/en/papers/xiong-2026-hey-that-s-my-data/)
- [next → HFT at BEA 2026 Shared Task 2: Blunt-Edge Models for Hybrid Grading](https://md.research.elamur.ai/free/en/papers/pado-2026-hft-bea-2026-shared-task/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/)

## References

36 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Patricia Amaral, Craige Roberts, and E Allyn Smith. 2007. Review of the logic of conventional implicatures by Chris Potts. Linguistics and Philosophy, 30(6):707–749. [↗](https://doi.org/10.1007/s10988-008-9025-2)
2. Scott AnderBois, Adrian Brasoveanu, and Robert Henderson. 2010. Crossing the appositive/at-issue meaning boundary. In Semantics and Linguistic Theory, volume 20, pages 328–346. [↗](https://doi.org/10.3765/salt.v20i0.2551)
3. Scott AnderBois, Adrian Brasoveanu, and Robert Henderson. 2015. At-issue proposals and appositive impositions in discourse. Journal of Semantics, 32(1):93–138. [find](https://scholar.google.com/scholar?q=Scott+AnderBois%2C+Adrian+Brasoveanu%2C+and+Robert+Henderson%2E+2015%2E+At%2Dissue+proposals+and+appositive+impositions+in+discourse%2E+Journal+of+Semantics%2C+32%281%29%3A93%E2%80%93138%2E)
4. Khai Loong Aw, Syrielle Montariol, Badr AlKhamissi, Martin Schrimpf, and Antoine Bosselut. 2024. Instruction-tuning aligns LLMs to the human brain. In First Conference on Language Modeling. [↗](https://doi.org/10.48550/arxiv.2312.00575)
5. Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2015. [Fitting linear mixed-effects models using lme4](https://md.research.elamur.ai/free/en/papers/bates-2015-fitting-linear-mixed-effects-models/). Journal of statistical software, 67:1–48. [↗](https://doi.org/10.18637/jss.v067.i01)
6. Brian Dillon, Lyn Frazier, and Charles Clifton Jr. 2018. No longer an orphan: Evidence for appositive attachment from sentence comprehension. Glossa, 3(1). [↗](https://doi.org/10.5334/gjgl.379)
7. Amila Ferron, Amber Shore, Ekata Mitra, and Ameeta Agrawal. 2023. Meep: Is this engaging? prompting large language models for dialogue evaluation in multilingual settings. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2078–2100. [find](https://scholar.google.com/scholar?q=Amila+Ferron%2C+Amber+Shore%2C+Ekata+Mitra%2C+and+Ameeta+Agrawal%2E+2023%2E+Meep%3A+Is+this+engaging%3F+prompting+large+language+models+for+dialogue+evaluation+in+multilingual+settings%2E+In+Findings+of+the+Associati)
8. Lyn Frazier and Charles Clifton Jr. 2005. The syntax-discourse divide: Processing ellipsis. Syntax, 8(2):121–174. [↗](https://doi.org/10.1111/j.1467-9612.2005.00077.x)
9. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783. [↗](https://arxiv.org/abs/2407.21783)
10. Shengyue Guan, Haoyi Xiong, Jindong Wang, Jiang Bian, Bin Zhu, and Jian-guang Lou. 2025. Evaluating llm-based agents for multi-turn conversations: A survey. arXiv preprint arXiv:2503.22458. [↗](https://arxiv.org/abs/2503.22458)
11. Jennifer Hu, Roger Levy, and Sebastian Schuster. 2022. [Predicting scalar diversity with context-driven uncertainty over alternatives](https://md.research.elamur.ai/free/en/papers/hu-2022-predicting-scalar-diversity-context-driven/). In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pages 68–74, Dublin, Ireland. Association for Computational Linguistics.
12. Julie Hunter and Nicholas Asher. 2016. Shapes of conversation and at-issue content. In Semantics and Linguistic Theory, volume 26, pages 1022–1042. [↗](https://doi.org/10.3765/salt.v26i0.3946)
13. Katja Jasinskaja. 2016. Not at issue any more. Unpublished manuscript, University of Cologne. [find](https://scholar.google.com/scholar?q=Katja+Jasinskaja%2E+2016%2E+Not+at+issue+any+more%2E+Unpublished+manuscript%2C+University+of+Cologne%2E)
14. Carina Kauf, Emmanuele Chersoni, Alessandro Lenci, Evelina Fedorenko, and Anna A Ivanova. 2024. [Comparing plausibility estimates in base and instruction-tuned large language models](https://md.research.elamur.ai/free/en/papers/kauf-2024-log-probabilities-reliable-estimate-semantic/). arXiv preprint arXiv:2403.14859. [↗](https://arxiv.org/abs/2403.14859)
15. Sanghee J. Kim and Forrest Davis. 2025. Discourse sensitivity in attraction effects: The interplay between language model size and training data. Society for Computation in Linguistics, 8(1). [↗](https://doi.org/10.7275/scil.3156)
16. Sanghee J. Kim, Lang Yu, and Allyson Ettinger. 2022. “no, they did not”: Dialogue response dynamics in pre-trained language models. In Proceedings of the 29th International Conference on Computational Linguistics, pages 863–874, Gyeongju, Republic of Korea. International Committee on Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sanghee+J%2E+Kim%2C+Lang+Yu%2C+and+Allyson+Ettinger%2E+2022%2E+%E2%80%9Cno%2C+they+did+not%E2%80%9D%3A+Dialogue+response+dynamics+in+pre%2Dtrained+language+models%2E+In+Proceedings+of+the+29th+International+Conference+on+Computational)
17. Wei-Jen Ko, Yating Wu, Cutter Dalton, Dananjay Srinivas, Greg Durrett, and Junyi Jessy Li. 2022. [Discourse analysis via questions and answers: Parsing dependency structures of questions under discussion](https://md.research.elamur.ai/free/en/papers/ko-2023-discourse-analysis-questions-answers-parsing/). arXiv preprint arXiv:2210.05905. [↗](https://arxiv.org/abs/2210.05905)
18. Todor Koev. 2022. Parenthetical meaning. Oxford Studies in Semantics and Pragmatics. Oxford: Oxford University Press. [find](https://scholar.google.com/scholar?q=Todor+Koev%2E+2022%2E+Parenthetical+meaning%2E+Oxford+Studies+in+Semantics+and+Pragmatics%2E+Oxford%3A+Oxford+University+Press%2E)
19. Alexandra Kuznetsova, Per B Brockhoff, and Rune HB Christensen. 2017. lmertest package: tests in linear mixed effects models. Journal of statistical software, 82:1–26. [↗](https://doi.org/10.18637/jss.v082.i13)
20. Yen-Ting Lin and Yun-Nung Chen. 2023. [Llm-eval: Unified multi-dimensional automatic evaluation for open-domain conversations with large language models](https://md.research.elamur.ai/free/en/papers/lin-2023-llm-eval-unified-multi-dimensional/). arXiv preprint arXiv:2305.13711. [↗](https://arxiv.org/abs/2305.13711)
21. Rebecca Marvin and Tal Linzen. 2018. Targeted syntactic evaluation of language models. arXiv preprint arXiv:1808.09031. [↗](https://arxiv.org/abs/1808.09031)
22. Kanishka Misra. 2022. minicons: Enabling flexible behavioral and representational analyses of transformer language models. arXiv:2203.13112. [↗](https://arxiv.org/abs/2203.13112)
23. Kanishka Misra, Julia Rayz, and Allyson Ettinger. 2023. COMPS: Conceptual minimal pair sentences for testing robust property knowledge and its inheritance in pre-trained language models. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2928– 2949, Dubrovnik, Croatia. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2023.eacl-main.213)
24. Sarah E Murray. 2014. [Varieties of update](https://md.research.elamur.ai/free/en/papers/murray-2014-varieties-update/). Semantics and Pragmatics, 7(2):1–53. [↗](https://doi.org/10.3765/sp.7.2)
25. Rick Nouwen. 2014. A note on the projection of appositives. In Eric McCready, Kyoko Yabushita, and Koji Yoshimoto, editors, Formal Approaches to Semantics and Pragmatics: Japanese and Beyond, pages 205–222. Springer, Dordrecht, Netherlands. [↗](https://doi.org/10.1007/978-94-017-8813-7_10)
26. Christopher Potts. 2005. The logic of conventional implicatures. Oxford: Oxford University Press. [↗](https://doi.org/10.1093/acprof:oso/9780199273829.001.0001)
27. Sujan Reddy. 2022. Automating human evaluation of dialogue systems. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop, pages 229–234. [↗](https://doi.org/10.48448/v6ca-pf30)
28. Arndt Riester. 2019. Constructing QUD trees. In Malte Zimmermann, Klaus von Heusinger, and Edgar Onea, editors, Questions in discourse, volume 2, pages 163– 192. Leiden: Brill. [↗](https://doi.org/10.1163/9789004378322_007)
29. Sebastian Schuster and Tal Linzen. 2022. When a sentence does not introduce a discourse entity, transformer-based models still sometimes refer to it. arXiv preprint arXiv:2205.03472. [↗](https://arxiv.org/abs/2205.03472)
30. Benny Shanon. 1976. On the two kinds of presuppositions in natural language. Foundations of Language, 14(2):247–249. [find](https://scholar.google.com/scholar?q=Benny+Shanon%2E+1976%2E+On+the+two+kinds+of+presuppositions+in+natural+language%2E+Foundations+of+Language%2C+14%282%29%3A247%E2%80%93249%2E)
31. Mandy Simons, Judith Tonhauser, David Beaver, and Craige Roberts. 2010. What projects and why. In Semantics and Linguistic Theory, volume 20, pages 309–327. [↗](https://doi.org/10.3765/salt.v20i0.2584)
32. Kristen Syrett and Todor Koev. 2015. Experimental evidence for the truth conditional contribution and shifting information status of appositives. Journal of Semantics, 32(3):525–577. [↗](https://doi.org/10.1093/jos/ffu007)
33. Davide Testa, Emmanuele Chersoni, and Alessandro Lenci. 2023. [We understand elliptical sentences, and language models should too: A new dataset for studying ellipsis and its interaction with thematic fit](https://md.research.elamur.ai/free/en/papers/testa-2023-we-understand-elliptical-sentences-language/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3340–3353, Toronto, Canada. Association for Computational Linguistics.
34. Judith Tonhauser. 2012. Diagnosing (not-)at-issue content. In Proceedings of the Sixth Conference on the Semantics of Under-Represented Languages in the Americas (SULA), volume 6, pages 239–254. Amherst, MA: Graduate Linguistics Student Association. [find](https://scholar.google.com/scholar?q=Judith+Tonhauser%2E+2012%2E+Diagnosing+%28not%2D%29at%2Dissue+content%2E+In+Proceedings+of+the+Sixth+Conference+on+the+Semantics+of+Under%2DRepresented+Languages+in+the+Americas+%28SULA%29%2C+volume+6%2C+pages+239%E2%80%93254%2E+Amher)
35. Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. 2020. [BLiMP: The benchmark of linguistic minimal pairs for english](https://md.research.elamur.ai/free/en/papers/warstadt-2020-blimp-benchmark-linguistic-minimal-pairs/). Transactions of the Association for Computational Linguistics, 8:377– 392. [↗](https://doi.org/10.1162/tacl_a_00321)
36. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen2.5 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388. [↗](https://arxiv.org/abs/2505.09388)

*Updated 1 November 2026*