---
title: "Linguistic ambiguity analysis in ChatGPT — Ortega-Martin 2023"
description: "Ambiguity at many linguistic levels remains a core obstacle for NLP systems. 12 references, cited by 19 papers here."
authors: ["Miguel Ortega-Martin", "Oscar Garcia-Sierra", "Alfonso Ardoiz", "Jorge Alvarez", "Juan Carlos Armenteros", "Adrian Alonso"]
year: 2023
venue: "arXiv:2302.06426"
source: "https://arxiv.org/abs/2302.06426"
doi: "10.48550/arxiv.2302.06426"
arxiv: "2302.06426"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ortega-martin-2023-linguistic-ambiguity-analysis-chatgpt/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ortega-martin-2023-linguistic-ambiguity-analysis-chatgpt/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/8/) / Linguistic ambiguity analysis in ChatGPT

# Linguistic ambiguity analysis in ChatGPT

**Authors:** Miguel Ortega-Martin, Oscar Garcia-Sierra, Alfonso Ardoiz, Jorge Alvarez, Juan Carlos Armenteros and Adrian Alonso · **Year:** 2023 · **Venue:** arXiv:2302.06426 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2302.06426) · [PDF](https://arxiv.org/pdf/2302.06426) · [DOI 10.48550/arxiv.2302.06426](https://doi.org/10.48550/arxiv.2302.06426) · [arXiv 2302.06426](https://arxiv.org/abs/2302.06426) · [OpenAlex](https://openalex.org/W4320854779)

**Citations:** 29 citations (OpenAlex, 2026-10-14) · [19](https://md.research.elamur.ai/free/en/papers/ortega-martin-2023-linguistic-ambiguity-analysis-chatgpt/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Ambiguity at many linguistic levels remains a core obstacle for NLP systems. This paper reviews three kinds: lexical (homonymy and polysemy), syntactic (structural attachment) and semantic (mainly pronoun co-reference). It explains how contextual embeddings and attention have eased some of these problems, then tests how ChatGPT copes. The only public benchmark, BIG-bench Sentence-Ambiguity, targets discourse-level phenomena, so the authors built their own small English test sets. They prompted the model to say whether each sentence is ambiguous, then followed up with targeted questions about word meanings, definition classification, pronoun reference and generating an ambiguous sentence. Overall detection reached accuracy of about 0.61 and F1 of 0.48. ChatGPT resolves co-reference ambiguity well and often names missing context as the cause. However, it over-detects polysemy and needs careful prompting for syntactic cases. It also lets gender stereotypes override unambiguous pronouns and contradicts itself about sentences it generated. The work matters as an early, reproducible snapshot of LLM behaviour on ambiguity, with prompts and outputs in the appendices.

## Key points
- Detection scores over all ambiguity types are accuracy 0.6061 and F1 0.48. For homonymy, false positives dominate (7 versus 3 TP, 2 TN, 1 FN), mostly because the model flags polysemy inside homonyms.
- For polysemy, ChatGPT correctly rejected every non-ambiguous example but missed the ambiguous ones. In co-reference detection it scored perfectly.
- In the paper's motivation, the spaCy model en_core_web_lg misparses a sentence in which John is the one holding a telescope, wrongly linking the "telescope" to Mary, the subject.
- BIG-bench Sentence-Ambiguity was set aside because it covers discourse-level ambiguity. The basic prompt was: Is the sentence "[sentence]" ambiguous?
- The authors list several limitations: the study covers English only, ChatGPT keeps changing over time, and leftover conversation context sometimes pushed the model to agree with the user, which forced new sessions.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Universidad Complutense de Madrid](https://md.research.elamur.ai/free/en/universities/universidad-complutense-de-madrid/) — University, Madrid, Spain · 9 works · 63 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [📖 Lexical semantics & word embeddings](https://md.research.elamur.ai/free/en/topics/lexical-semantics/) · 63 works

## Cite

Cite as: Miguel Ortega-Martin, Oscar Garcia-Sierra, Alfonso Ardoiz, Jorge Alvarez, Juan Carlos Armenteros and Adrian Alonso. 2023. Linguistic ambiguity analysis in ChatGPT. arXiv:2302.06426.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{ortegamartin2023linguistic,
  title = {{Linguistic ambiguity analysis in ChatGPT}},
  author = {Miguel Ortega-Martin, Oscar Garcia-Sierra, Alfonso Ardoiz, Jorge Alvarez, Juan Carlos Armenteros and Adrian Alonso},
  year = {2023},
  doi = {10.48550/arxiv.2302.06426},
  eprint = {2302.06426},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2302.06426}
}
```

- [← previous GrammarGPT: Exploring Open-Source LLMs for Native Chinese Grammatical Error Correction with Supervised Fine-Tuning](https://md.research.elamur.ai/free/en/papers/fan-2023-grammargpt-exploring-open-source-llms/)
- [next → Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages](https://md.research.elamur.ai/free/en/papers/yong-2023-prompting-multilingual-large-language-models/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/8/)

## References

12 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. \#\# 3. Going to the right is the right choice. [find](https://scholar.google.com/scholar?q=%5C%23%5C%23+3%2E+Going+to+the+right+is+the+right+choice%2E)
2. Muneera Bano. 2015. Addressing the challenges of requirements ambiguity: A review of empirical literature. In 2015 IEEE Fifth International Workshop on Empirical Requirements Engineering (EmpiRE), pages 21–24. IEEE. [find](https://scholar.google.com/scholar?q=Muneera+Bano%2E+2015%2E+Addressing+the+challenges+of+requirements+ambiguity%3A+A+review+of+empirical+literature%2E+In+2015+IEEE+Fifth+International+Workshop+on+Empirical+Requirements+Engineering+%28EmpiRE%29%2C+pag)
3. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. [Language (technology) is power: A critical survey of" bias" in nlp](https://md.research.elamur.ai/free/en/papers/blodgett-2020-language-technology-is-power-critical/). arXiv preprint arXiv:2005.14050.
4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, and Amanda Askell. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
5. Dan Jurafsky and James H. Martin. 2020. Speech and language processing (3rd draft ed.). Stanford Univ. [find](https://scholar.google.com/scholar?q=Dan+Jurafsky+and+James+H%2E+Martin%2E+2020%2E+Speech+and+language+processing+%283rd+draft+ed%2E%29%2E+Stanford+Univ%2E)
6. Shaidah Jusoh. 2018. A study on NLP appliactions and ambiguity problems. Journal of Theoretical & Applied Information Technology, 96(6). ISBN: 1992-8645. [find](https://scholar.google.com/scholar?q=Shaidah+Jusoh%2E+2018%2E+A+study+on+NLP+appliactions+and+ambiguity+problems%2E+Journal+of+Theoretical+%26+Applied+Information+Technology%2C+96%286%29%2E+ISBN%3A+1992%2D8645%2E)
7. Qi Liu, Matt J Kusner, and Phil Blunsom. 2020. A survey on contextual embeddings. arXiv preprint arXiv:2003.07278. [find](https://scholar.google.com/scholar?q=Qi+Liu%2C+Matt+J+Kusner%2C+and+Phil+Blunsom%2E+2020%2E+A+survey+on+contextual+embeddings%2E+arXiv+preprint+arXiv%3A2003%2E07278%2E)
8. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. [Efficient estimation of word representations in vector space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/). arXiv preprint arXiv:1301.3781.
9. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, and Alex Ray. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). arXiv preprint arXiv:2203.02155.
10. Aarohi Srivastava et al. 2022. [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/). ArXiv:2206.04615 [cs, stat].
11. Huiqiang Xie, Zhijin Qin, Geoffrey Ye Li, and Biing-Hwang Juang. 2021. Deep learning enabled semantic communication systems. IEEE Transactions on Signal Processing, 69:2663–2675. [find](https://scholar.google.com/scholar?q=Huiqiang+Xie%2C+Zhijin+Qin%2C+Geoffrey+Ye+Li%2C+and+Biing%2DHwang+Juang%2E+2021%2E+Deep+learning+enabled+semantic+communication+systems%2E+IEEE+Transactions+on+Signal+Processing%2C+69%3A2663%E2%80%932675%2E)
12. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. [Gender bias in coreference resolution: Evaluation and debiasing methods](https://md.research.elamur.ai/free/en/papers/zhao-2018-gender-bias-coreference-resolution-evaluation/). arXiv preprint arXiv:1804.06876.

*Updated 30 October 2026*