---
title: "Evaluating LLM-Generated Explanations of Metaphors – A"
description: "This study asks whether large language models can explain Danish metaphors that have no English… 17 references, cited by 6 papers here."
authors: ["Bolette S. Pedersen", "Nathalie Sørensen", "Sanni Nimb", "Dorte Haltrup Hansen", "Sussi Olsen", "Ali Al-Laith"]
year: 2025
venue: "Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025)"
source: "https://aclanthology.org/2025.nodalida-1.51/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pedersen-2025-evaluating-llm-generated-explanations-metaphors/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pedersen-2025-evaluating-llm-generated-explanations-metaphors/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/11/) / Evaluating LLM-Generated Explanations of Metaphors – A Culture-Sensitive Study of Danish

# Evaluating LLM-Generated Explanations of Metaphors – A Culture-Sensitive Study of Danish

**Authors:** [Bolette S. Pedersen](https://md.research.elamur.ai/free/en/authors/bolette-pedersen/), Nathalie Sørensen, [Sanni Nimb](https://md.research.elamur.ai/free/en/authors/sanni-nimb/), Dorte Haltrup Hansen, [Sussi Olsen](https://md.research.elamur.ai/free/en/authors/sussi-olsen/) and [Ali Al-Laith](https://md.research.elamur.ai/free/en/authors/ali-al-laith/) · **Year:** 2025 · **Venue:** [Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025)](https://md.research.elamur.ai/free/en/venues/nodalida/) ([NoDaLiDa 2025](https://md.research.elamur.ai/free/en/venues/nodalida/2025/)) · **Publisher:** [University of Tartu Library](https://md.research.elamur.ai/free/en/publishers/university-of-tartu-library/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.nodalida-1.51/) · [PDF](https://aclanthology.org/2025.nodalida-1.51.pdf) · [OpenAlex](https://openalex.org/W4412210758)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [6](https://md.research.elamur.ai/free/en/papers/pedersen-2025-evaluating-llm-generated-explanations-metaphors/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This study asks whether large language models can explain Danish metaphors that have no English counterpart as well as those shared with English. The authors built a pilot set of 150 Danish metaphors and idioms drawn from The Danish Dictionary, using its main-sense/subsense structure and its metaphor labels. Each item was marked as culture-specific or cross-cultural, and English-speaking informants checked that split. The two strongest Danish chatbots according to Scandeval, ChatGPT and Llama, were then prompted to explain every item. Prompts were given in Danish and in English, with and without a textual context. Native speakers rated all outputs on a four-level scale. Both models did clearly better on metaphors that also exist in English. This suggests they carry English meanings over to Danish expressions, which produces fluent but wrong or invented explanations. The most frequent failure was connotation: negative, sarcastic or sexual undertones were softened or missed entirely. The authors see this as a risk to linguistic and cultural diversity as such chatbots spread through society.

## Key points
- The dataset has 150 items: 75 single-word metaphors (e.g. *sejle*) and 75 metaphorical idioms (e.g. *høste frugterne*). Within each type, 50 are culture-specific and 25 are cross-cultural.
- Items come from The Danish Dictionary (Den Danske Ordbog). Its 152 unpublished domain labels helped group items by source domain, such as farming and nautical terms. Idioms took the domain of their central lemmas.
- Unclear cases were labelled culture-specific whenever English had no exact match. Examples are *studehandel* (English uses "horse trading") and *myreflittig* (English uses "busy as a bee").
- All 1,200 model explanations were rated by humans: 150 items × 2 models × 2 prompt languages × with/without context.
- Typical failures were turning derogatory terms positive, such as *havelåge* and *skærveknuser* for old bikes, *tågehorn* and *sejle sin egen sø*, and ignoring sexual meanings. The data is released in the danish-semantic-reasoning-benchmark GitHub repository.

## Interesting ideas
- Pair the hard culture-specific items with a smaller set of items shared with English, so that any drop in quality can be traced to cultural distance rather than to metaphor difficulty in general.
- Harvest figurative senses directly from a dictionary's subsense hierarchy and domain tags, and let multiword idioms take the domain of their head lemmas.
- Judge connotation separately from literal gist, because models often get the meaning roughly right while reversing its sentiment.
- Cross prompt language with presence or absence of context in a small factorial design, so each factor's effect on explanation quality can be isolated.
- When the cultural status of an item is unclear, apply a strict rule (no exact equivalent means culture-specific) rather than leave it ambiguous.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Mova's evaluations could include a matched "easy twin" set as a negative control, showing whether a failure comes from the construction itself or from rare or idiomatic wording.
- **I2** (world): figurative senses could be compiled into the ontology with explicit links to their literal source sense and domain, so readers can tell when a word like "sail" is not about boats.
- **I3** (frames): the irony and implicature layer could mark evaluative polarity on figurative frames and test it on its own, since a positive reading of a derogatory idiom is a distinct error.
- **I4** (learn): LLM teacher tiers could be probed the same way before their silver labels are trusted, keeping only conditions where the teacher is stable.
- **I5** (learn): a fixed rule for borderline items keeps silver-data labels consistent and evaluation splits reproducible.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Copenhagen](https://md.research.elamur.ai/free/en/universities/university-of-copenhagen/) — University, Copenhagen, Denmark · 116 works · 15,292 citations

Also on the byline: [Det Danske Sprog- og Litteraturselskab](https://ror.org/00jn4mb53).

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,649 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 930 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,341 works · [🧷 Idioms & multiword expressions](https://md.research.elamur.ai/free/en/topics/multiword-expressions/) · 222 works

## Cite

Cite as: Bolette S. Pedersen, Nathalie Sørensen, Sanni Nimb, Dorte Haltrup Hansen, Sussi Olsen and Ali Al-Laith. 2025. Evaluating LLM-Generated Explanations of Metaphors – A Culture-Sensitive Study of Danish. Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{pedersen2025evaluating,
  title = {{Evaluating LLM-Generated Explanations of Metaphors – A Culture-Sensitive Study of Danish}},
  author = {Bolette S. Pedersen, Nathalie Sørensen, Sanni Nimb, Dorte Haltrup Hansen, Sussi Olsen and Ali Al-Laith},
  booktitle = {Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025)},
  year = {2025},
  url = {https://aclanthology.org/2025.nodalida-1.51/}
}
```

- [← previous Entailed Between the Lines: Incorporating Implication into NLI](https://md.research.elamur.ai/free/en/papers/havaldar-2025-entailed-between-lines-incorporating-implication/)
- [next → Evaluating RAG Pipelines for Arabic Lexical Information Retrieval: A Comparative Study of Embedding and Generation Models](https://md.research.elamur.ai/free/en/papers/rasheed-2025-evaluating-rag-pipelines-arabic-lexical/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/11/)

## References

17 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ehsan Aghazadeh, Mohsen Fayyaz, and Yadollah Yaghoobzadeh. 2022. Metaphors in pre-trained language models: Probing and generalization across datasets and languages. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2037–2050, Dublin, Ireland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ehsan+Aghazadeh%2C+Mohsen+Fayyaz%2C+and+Yadollah+Yaghoobzadeh%2E+2022%2E+Metaphors+in+pre%2Dtrained+language+models%3A+Probing+and+generalization+across+datasets+and+languages%2E+In+Proceedings+of+the+60th+Annual+M)
2. Junho Myung, Nayeon Lee, Yi Zhou, Jiho Jin, Rifki Afina Putri, Dimosthenis Antypas, Hsuvas Borkakoty, Eunsu Kim, Carla Perez-Almendros, Abinew Ali Ayele, Vı́ctor Gutiérrez-Basulto, Yazmı́n Ibáñez-Garcı́a, Hwaran Lee, Shamsuddeen Hassan Muhammad, Kiwoong Park, Anar Sabuhi Rzayev, Nina White, Seid Muhie Yimam, Mohammad Taher Pilehvar, Nedjma Ousidhoum, Jose Camacho-Collados, and Alice Oh. 2024. [BLEnD: A benchmark for LLMs on everyday knowledge in diverse cultures and languages](https://md.research.elamur.ai/free/en/papers/myung-2024-blend-benchmark-llms-everyday-knowledge/).
3. Naveen Badathala, Abisek Rajakumar Kalarani, Tejpalsingh Siledar, and Pushpak Bhattacharyya. 2023. A match made in heaven: A multi-task framework for hyperbole and metaphor detection. In Findings of the Association for Computational Linguistics: ACL 2023, pages 388–401, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Naveen+Badathala%2C+Abisek+Rajakumar+Kalarani%2C+Tejpalsingh+Siledar%2C+and+Pushpak+Bhattacharyya%2E+2023%2E+A+match+made+in+heaven%3A+A+multi%2Dtask+framework+for+hyperbole+and+metaphor+detection%2E+In+Findings+of+t)
4. Susan Nacey, W Gudrun Reijnierse, Tina Krennmayr, and Aletta G Dorst. 2019. Metaphor Identification in Multiple Languages. John Benjamins Publishing Company. [↗](https://doi.org/10.1075/celcr.22)
5. Valentina Bambini, Paolo Canal, Donatella Resta, and Mirko Grimaldi. 2019. Time course and neurophysiological underpinnings of metaphor in literary context. Discourse Processes, 56(1):77–97. [↗](https://doi.org/10.1080/0163853x.2017.1401876)
6. Dan Saattrup Nielsen. 2023. [Scandeval: A benchmark for Scandinavian natural language processing](https://md.research.elamur.ai/free/en/papers/nielsen-2023-scandeval-benchmark-scandinavian-natural-language/). Proceedings of Nodalida 2023, The Faroe Islands. [↗](https://doi.org/10.48550/arxiv.2304.00906)
7. Christian Burgers, Kiki Y Renardel de Lavalette, and Gerard J Steen. 2018. Metaphor, hyperbole, and irony: Uses in isolation and in combination in written discourse. Journal of Pragmatics, 127:71–83. [↗](https://doi.org/10.1016/j.pragma.2018.01.009)
8. Dan Saattrup Nielsen. 2024. Status på Scandinavian embedding benchmark (seb). Slides from Bench-markworkshop in The Danish Agency for Digital Government, September 20, 2024. [find](https://scholar.google.com/scholar?q=Dan+Saattrup+Nielsen%2E+2024%2E+Status+pa%CC%8A+Scandinavian+embedding+benchmark+%28seb%29%2E+Slides+from+Bench%2Dmarkworkshop+in+The+Danish+Agency+for+Digital+Government%2C+September+20%2C+2024%2E)
9. Ira A Noveck, Maryse Bianco, and Alain Castry. 2001. The costs and benefits of metaphor. Metaphor and Symbol, 16(1-2):109–121. [↗](https://doi.org/10.1080/10926488.2001.9678889)
10. Bolette Sandford Pedersen, Nathalie C Hau Sørensen, Sussi Olsen, and Sanni Nimb. 2024. Evaluering af sprogforståelsen i danske sprogmodeller - med udgangspunkt i semantiske ordbøger. NyS, Ny-danske Sprogstudier, pages 8–40. [find](https://scholar.google.com/scholar?q=Bolette+Sandford+Pedersen%2C+Nathalie+C+Hau+S%C3%B8rensen%2C+Sussi+Olsen%2C+and+Sanni+Nimb%2E+2024%2E+Evaluering+af+sprogforsta%CC%8Aelsen+i+danske+sprogmodeller+%2D+med+udgangspunkt+i+semantiske+ordb%C3%B8ger%2E+NyS%2C+Ny%2Ddanske+S)
11. Paolo Pedinotti, Eliana Di Palma, Ludovica Cerini, and Alessandro Lenci. 2021. A howling success or a working sea? Testing what BERT knows about metaphors. In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 192–204. [find](https://scholar.google.com/scholar?q=Paolo+Pedinotti%2C+Eliana+Di+Palma%2C+Ludovica+Cerini%2C+and+Alessandro+Lenci%2E+2021%2E+A+howling+success+or+a+working+sea%3F+Testing+what+BERT+knows+about+metaphors%2E+In+Proceedings+of+the+Fourth+BlackboxNLP+Wor)
12. Ciyuan Peng, Dang Thinh Vu, and Jason J Jung. 2021. Knowledge graph-based metaphor representation for literature understanding. Digital Scholarship in the Humanities, 36(3):698–711. [↗](https://doi.org/10.1093/llc/fqaa072)
13. Vinodkumar Prabhakaran, Marek Rei, and Ekaterina Shutova. 2021. How metaphors impact political discourse: A large-scale topic-agnostic study using neural metaphor detection. In Proceedings of the International AAAI Conference on Web and Social Media, volume 15, pages 503–512. [find](https://scholar.google.com/scholar?q=Vinodkumar+Prabhakaran%2C+Marek+Rei%2C+and+Ekaterina+Shutova%2E+2021%2E+How+metaphors+impact+political+discourse%3A+A+large%2Dscale+topic%2Dagnostic+study+using+neural+metaphor+detection%2E+In+Proceedings+of+the+Inte)
14. Elisa Sanchez-Bayona and Rodrigo Agerri. 2024. Meta4xnli: A crosslingual parallel corpus for metaphor detection and interpretation. arXiv preprint arXiv:2404.07053. [↗](https://arxiv.org/abs/2404.07053)
15. Andrew Szot, Bogdan Mazoure, Harsh Agrawal, R Devon Hjelm, Zsolt Kira, and Alexander T Toshev. 2024. Grounding multimodal large language models in actions. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Andrew+Szot%2C+Bogdan+Mazoure%2C+Harsh+Agrawal%2C+R+Devon+Hjelm%2C+Zsolt+Kira%2C+and+Alexander+T+Toshev%2E+2024%2E+Grounding+multimodal+large+language+models+in+actions%2E+In+The+Thirty%2Deighth+Annual+Conference+on+Ne)
16. Lennart Wachowiak and Dagmar Gromann. 2023. Does GPT-3 grasp metaphors? Identifying metaphor mappings with generative language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1018–1032. [find](https://scholar.google.com/scholar?q=Lennart+Wachowiak+and+Dagmar+Gromann%2E+2023%2E+Does+GPT%2D3+grasp+metaphors%3F+Identifying+metaphor+mappings+with+generative+language+models%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+for+)
17. Xiang Zhang, Senyu Li, Bradley Hauer, Ning Shi, and Grzegorz Kondrak. 2023. Don’t trust Chat-GPT when your question is not in English: A study of multilingual abilities and types of LLMs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7915–7927, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Xiang+Zhang%2C+Senyu+Li%2C+Bradley+Hauer%2C+Ning+Shi%2C+and+Grzegorz+Kondrak%2E+2023%2E+Don%E2%80%99t+trust+Chat%2DGPT+when+your+question+is+not+in+English%3A+A+study+of+multilingual+abilities+and+types+of+LLMs%2E+In+Proceedin)

*Updated 1 November 2026*