---
title: "Gender and Politeness Perception — Aljanaideh 2026"
description: "The paper examines whether men and women perceive the politeness of a text in the same way."
authors: ["Ahmad Aljanaideh"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.eacl-long.186/"
doi: "10.18653/v1/2026.eacl-long.186"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/aljanaideh-2026-gender-politeness-perception-novel-approach/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/aljanaideh-2026-gender-politeness-perception-novel-approach/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/page/2/) / Gender and Politeness Perception: A Novel Approach for Exploring Annotations Disagreement

# Gender and Politeness Perception: A Novel Approach for Exploring Annotations Disagreement

**Authors:** [Ahmad Aljanaideh](https://md.research.elamur.ai/free/authors/ahmad-aljanaideh/) · **Year:** 2026 · **Venue:** Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-long.186/) · [PDF](https://aclanthology.org/2026.eacl-long.186.pdf) · [DOI 10.18653/v1/2026.eacl-long.186](https://doi.org/10.18653/v1/2026.eacl-long.186) · [OpenAlex](https://openalex.org/W7140121495)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary

The paper examines whether men and women perceive the politeness of a text in the same way. The author proposes a method for automatically finding the linguistic patterns on which the ratings of the two groups of annotators diverge the most. The data come from POPQUORN: these are emails, each of which several people rated on a politeness scale, and the gender of every annotator is known. The method clusters contextual BERT embeddings to distinguish uses of the same word in different contexts. For each cluster, the divergence between the groups is measured with the Earth Mover's Distance (EMD) between the rating distributions of men and women. The result is as follows: women consider emails with formal markers more polite than men do, while men rate informal emails higher than women do. This hypothesis was tested quantitatively using formality ratings obtained from GPT-4. The discovered features also improve the prediction of whether an email will cause disagreement and the prediction of the full rating distribution. The paper shows that annotation disagreements contain sociolinguistic information and should be analyzed rather than averaged out.

## Key points

- **Data:** the politeness portion of POPQUORN contains 3,178 emails. Each email was rated by 5–7 annotators on a scale from 1 to 5. The annotators include 254 women, 236 men and 13 non-binary people; the latter were not analyzed because there were too few of them. The train/dev/test split is 70/10/20.
- **Method:** patterns are found by clustering BERT embeddings. The number of clusters k is chosen by the largest gain in cross-gender divergence. The baselines are unigrams and the 21 politeness strategies from Danescu-Niculescu-Mizil et al. (2013), denoted DNM.
- **Validation:** formality ratings for the emails were obtained from GPT-4. Politeness ratings from women correlate with formality considerably more strongly than ratings from men.
- **Disagreement classification:** an SVM was used to predict whether an email will cause high disagreement. The best accuracy, about 62.3%, is achieved by combining word clusters with DNM features. Fine-tuning BERT on such a small dataset performed poorly.
- **Rating distribution prediction:** the model is logistic regression with EMD as the loss function. For DNM+unigrams the EMD is 0.60, and for DNM+word clusters it is 0.56, i.e. 6.67% lower. The improvement is statistically significant according to the Wilcoxon test.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{aljanaideh2026gender,
  title = {{Gender and Politeness Perception: A Novel Approach for Exploring Annotations Disagreement}},
  author = {Aljanaideh, Ahmad},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.186},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.186}
}
```

- [← previous Gaze Behaviour & Conversation Unfolding in the HCRC Map Task Corpus](https://md.research.elamur.ai/free/papers/murat-2026-gaze-behaviour-conversation-unfolding-hcrc/)
- [next → Gender Disambiguation in Machine Translation: Diagnostic Evaluation in Decoder-Only Architectures](https://md.research.elamur.ai/free/papers/manna-2026-gender-disambiguation-machine-translation-diagnostic/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/page/2/)

## References

25 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774. [↗](https://arxiv.org/abs/2303.08774) [our card](https://md.research.elamur.ai/free/papers/openai-2023-gpt-4-technical-report/)
2. Ahmad Aljanaideh, Eric Fosler-Lussier, and Marie-Catherine de Marneffe. 2020. Contextualized Embeddings for Enriching Linguistic Analyses on Politeness. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2181– 2190. [↗](https://doi.org/10.18653/v1/2020.coling-main.198) [our card](https://md.research.elamur.ai/free/papers/aljanaideh-2020-contextualized-embeddings-enriching-linguistic-analyses/)
3. Nan-Jiang Jiang and Marie-Catherine de Marneffe. 2022. Investigating reasons for disagreement in natural language inference. Transactions of the Association for Computational Linguistics, 10:1357–1374. [↗](https://doi.org/10.1162/tacl_a_00523) [our card](https://md.research.elamur.ai/free/papers/jiang-2022-investigating-reasons-disagreement-natural-language/)
4. Sara Mills. 2003. Rethinking politeness, impoliteness and gender identity. In Gender identity and discourse analysis, pages 69–89. John Benjamins Publishing Company. [find](https://scholar.google.com/scholar?q=Sara+Mills%2E+2003%2E+Rethinking+politeness%2C+impoliteness+and+gender+identity%2E+In+Gender+identity+and+discourse+analysis%2C+pages+69%E2%80%9389%2E+John+Benjamins+Publishing+Company%2E)
5. Malika Aubakirova and Mohit Bansal. 2016. Interpreting Neural Networks to Improve Politeness Comprehension. In Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Malika+Aubakirova+and+Mohit+Bansal%2E+2016%2E+Interpreting+Neural+Networks+to+Improve+Politeness+Comprehension%2E+In+Empirical+Methods+in+Natural+Language+Processing%2E) [our card](https://md.research.elamur.ai/free/papers/aubakirova-2016-interpreting-neural-networks-improve-politeness/)
6. Shubhanshu Mishra, Sijun He, and Luca Belli. 2020. Assessing demographic bias in named entity recognition. arXiv preprint arXiv:2008.03415. [↗](https://arxiv.org/abs/2008.03415)
7. Penelope Brown and Stephen C Levinson. 1978. Universals in Language Usage: Politeness Phenomena. In Questions and politeness: Strategies in social interaction, pages 56–311. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Penelope+Brown+and+Stephen+C+Levinson%2E+1978%2E+Universals+in+Language+Usage%3A+Politeness+Phenomena%2E+In+Questions+and+politeness%3A+Strategies+in+social+interaction%2C+pages+56%E2%80%93311%2E+Cambridge+University+Press)
8. Tong Niu and Mohit Bansal. 2018. Polite dialogue generation without parallel data. Transactions of the Association for Computational Linguistics, 6:373– 389. [↗](https://doi.org/10.1162/tacl_a_00027)
9. Penelope Brown and Stephen C Levinson. 1987. Politeness: Some Universals in Language Usage, volume 4. Cambridge university press. [find](https://scholar.google.com/scholar?q=Penelope+Brown+and+Stephen+C+Levinson%2E+1987%2E+Politeness%3A+Some+Universals+in+Language+Usage%2C+volume+4%2E+Cambridge+university+press%2E)
10. Jiaxin Pei and David Jurgens. 2023. When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Dataset. The 17th Linguistic Annotation Workshop. [find](https://scholar.google.com/scholar?q=Jiaxin+Pei+and+David+Jurgens%2E+2023%2E+When+Do+Annotator+Demographics+Matter%3F+Measuring+the+Influence+of+Annotator+Demographics+with+the+POPQUORN+Dataset%2E+The+17th+Linguistic+Annotation+Workshop%2E)
11. Moira Burke and Robert Kraut. 2008. Mind your ps and qs: the impact of politeness and rudeness in online communities. In Proceedings of the 2008 ACM conference on Computer supported cooperative work, pages 281–284. [find](https://scholar.google.com/scholar?q=Moira+Burke+and+Robert+Kraut%2E+2008%2E+Mind+your+ps+and+qs%3A+the+impact+of+politeness+and+rudeness+in+online+communities%2E+In+Proceedings+of+the+2008+ACM+conference+on+Computer+supported+cooperative+work%2C+)
12. Malgorzata Chalupnik, Christine Christie, and Louise Mullany. 2017. (Im) Politeness and Gender. The Palgrave handbook of linguistic (im) politeness, pages 517–537. [↗](https://doi.org/10.1057/978-1-137-37508-7_20)
13. Jonathan P Chang, Caleb Chiam, Liye Fu, Andrew Z Wang, Justine Zhang, and Cristian Danescu-Niculescu-Mizil. 2020. Convokit: A toolkit for the analysis of conversations. arXiv preprint arXiv:2005.04246. [↗](https://arxiv.org/abs/2005.04246) [our card](https://md.research.elamur.ai/free/papers/chang-2020-convokit-toolkit-analysis-conversations/)
14. Herbert H Clark. 1979. Responding to Indirect Speech Acts. Cognitive psychology, 11(4):430–477. [↗](https://doi.org/10.1016/0010-0285(79)90020-3)
15. Herbert H Clark and Dale H Schunk. 1980. Polite responses to polite requests. Cognition, 8(2):111–143. [↗](https://doi.org/10.1016/0010-0277(80)90009-8)
16. Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A Computational Approach to Politeness with Application to Social Factors. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1306.6078) [our card](https://md.research.elamur.ai/free/papers/mizil-2013-computational-approach-politeness-application-social/)
17. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of The 2019 Conference of the North American Chapter of the Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
18. Nina Eliasoph. 1987. Politeness, power, and women’s language: Rethinking study in language and gender. Berkeley Journal of Sociology, 32:79–103. [find](https://scholar.google.com/scholar?q=Nina+Eliasoph%2E+1987%2E+Politeness%2C+power%2C+and+women%E2%80%99s+language%3A+Rethinking+study+in+language+and+gender%2E+Berkeley+Journal+of+Sociology%2C+32%3A79%E2%80%93103%2E)
19. Rebecca Qian, Candace Ross, Jude Fernandes, Eric Smith, Douwe Kiela, and Adina Williams. 2022. Perturbation augmentation for fairer nlp. arXiv preprint arXiv:2205.12586. [↗](https://arxiv.org/abs/2205.12586)
20. Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. 1998. A metric for distributions with applications to image databases. In Sixth international conference on computer vision (IEEE Cat. No. 98CH36271), pages 59–66. IEEE. [find](https://scholar.google.com/scholar?q=Yossi+Rubner%2C+Carlo+Tomasi%2C+and+Leonidas+J+Guibas%2E+1998%2E+A+metric+for+distributions+with+applications+to+image+databases%2E+In+Sixth+international+conference+on+computer+vision+%28IEEE+Cat%2E+No%2E+98CH36271%29)
21. Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. 2000. The Earth Mover’s Distance as a Metric for Image Retrieval. International Journal of Computer Vision, 40(2):99–121. [↗](https://doi.org/10.1023/a:1026543900054)
22. Jitesh Shetty and Jafar Adibi. 2004. The enron email dataset database schema and brief statistical report. Information sciences institute technical report, University of Southern California, 4(1):120–128. [find](https://scholar.google.com/scholar?q=Jitesh+Shetty+and+Jafar+Adibi%2E+2004%2E+The+enron+email+dataset+database+schema+and+brief+statistical+report%2E+Information+sciences+institute+technical+report%2C+University+of+Southern+California%2C+4%281%29%3A120%E2%80%93)
23. Huaman Sun, Jiaxin Pei, Minje Choi, and David Jurgens. 2023. Aligning with whom? Large Language Models have Gender and Racial Biases in Subjective NLP Tasks. arXiv preprint arXiv:2311.09730. [↗](https://arxiv.org/abs/2311.09730)
24. Charles Welch, Jonathan K Kummerfeld, Verónica Pérez-Rosas, and Rada Mihalcea. 2020. Compositional demographic word embeddings. Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Charles+Welch%2C+Jonathan+K+Kummerfeld%2C+Ver%C3%B3nica+P%C3%A9rez%2DRosas%2C+and+Rada+Mihalcea%2E+2020%2E+Compositional+demographic+word+embeddings%2E+Empirical+Methods+in+Natural+Language+Processing%2E)
25. Justine Zhang, Jonathan P Chang, Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Yiqing Hua, Nithum Thain, and Dario Taraborelli. 2018. Conversations Gone Awry: Detecting Early Signs of Conversational Failure. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1805.05345) [our card](https://md.research.elamur.ai/free/papers/zhang-2018-conversations-gone-awry-detecting-early/)