---
title: "Personality Expression in Spoken Dialogue Systems: From Text"
description: "Spoken dialogue systems often aim to show a stable personality, but users meeting a system for the first time may not notice… 18 references."
authors: ["Kenta Yamamoto", "Kazunori Komatani"]
year: 2026
venue: "Proceedings of the 16th International Workshop on Spoken Dialogue System Technology"
source: "https://aclanthology.org/2026.iwsds-1.3/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yamamoto-2026-personality-expression-spoken-dialogue-systems/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yamamoto-2026-personality-expression-spoken-dialogue-systems/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/44/) / Personality Expression in Spoken Dialogue Systems: From Text to Speech

# Personality Expression in Spoken Dialogue Systems: From Text to Speech

**Authors:** Kenta Yamamoto and [Kazunori Komatani](https://md.research.elamur.ai/free/en/authors/kazunori-komatani/) · **Year:** 2026 · **Venue:** [Proceedings of the 16th International Workshop on Spoken Dialogue System Technology](https://md.research.elamur.ai/free/en/venues/iwsds/) ([International Workshop on Spoken Dialogue System Technology 2026](https://md.research.elamur.ai/free/en/venues/iwsds/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.iwsds-1.3/) · [PDF](https://aclanthology.org/2026.iwsds-1.3.pdf)

## Summary
Spoken dialogue systems often aim to show a stable personality, but users meeting a system for the first time may not notice every intended Big Five trait. Yamamoto and Komatani ask which traits users actually perceive and how a system can convey them. They analysed third-party personality ratings from the Japanese multimodal Hazumi corpus with PCA and Varimax-rotated EFA. Three perceptual dimensions emerged: a combined Extraversion–Openness factor (EO), a factor joining Agreeableness with Conscientiousness (AC), and Neuroticism on its own (N).

They then ran crowd-sourced experiments in two settings. In the text setting, LLM-generated dialogues using the three-dimension scheme gave clearer, more consistent impressions than manipulating all five traits, though N was hard to convey in text. In the speech setting, backchannels, turn-taking pauses, fillers and speech rate controlled each dimension with little cross-trait interference, and N became clearly noticeable.

The main lesson is practical: designers should target the traits users can actually perceive, not the full Big Five, and should combine wording with speech cues.

## Key points
- **Data:** 155 Wizard-of-Oz dialogues from four Hazumi sub-corpora (1911, 2010, 2012, 2105). Each dialogue was rated by five third-party annotators with TIPI-J on a 7-point scale.
- **PCA result:** The first three components explained 86% of the variance in perceived personality (the first alone, 49%). This justified a three-factor EFA, which produced the EO / AC / N grouping.
- **Speech manipulations:**
  - EO: backchannels ("hai") inserted at 3–5 s intervals.
  - AC: turn-taking pause set to 3.0 s (high) or 0.5 s (low), versus the corpus default of 1.0 s.
  - N: utterance-initial fillers ("e–", "sono–") plus speech rate alternating between 80% and 110%.
- **Speech experiment:** 180 crowd workers each rated five dialogues, presented as ~1.5-minute videos with only icons on screen. Welch's t-tests showed significant differences only for the trait being manipulated. Spearman correlations between control setting and rating were 0.33 (EO), 0.33 (AC) and 0.28 (N).
- **Limitations:** Results are for Japanese only and depend on the specific TTS system used. The authors call for validation in other languages and cultures and with other TTS systems.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works

🧭 Topics (2): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 654 works · [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · 519 works

## Cite

Cite as: Kenta Yamamoto and Kazunori Komatani. 2026. Personality Expression in Spoken Dialogue Systems: From Text to Speech. Proceedings of the 16th International Workshop on Spoken Dialogue System Technology.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yamamoto2026personality,
  title = {{Personality Expression in Spoken Dialogue Systems: From Text to Speech}},
  author = {Kenta Yamamoto and Kazunori Komatani},
  booktitle = {Proceedings of the 16th International Workshop on Spoken Dialogue System Technology},
  year = {2026},
  url = {https://aclanthology.org/2026.iwsds-1.3/}
}
```

- [← previous PersonaForge: Psychology-Grounded Dual-Process Architecture for Personality-Consistent Role-Playing Agents](https://md.research.elamur.ai/free/en/papers/tong-2026-personaforge-psychology-grounded-dual-process/)
- [next → Personalizing LLMs with Binary Feedback: A Preference-Calibrated Optimization Framework](https://md.research.elamur.ai/free/en/papers/ma-2026-personalizing-llms-binary-feedback-preference/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/44/)

## References

18 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Graham Caron and Shashank Srivastava. 2023. Manipulating the perceived personality traits of language models. In Findings of the Association for Computational Linguistics (EMNLP), pages 2370–2386. [find](https://scholar.google.com/scholar?q=Graham+Caron+and+Shashank+Srivastava%2E+2023%2E+Manipulating+the+perceived+personality+traits+of+language+models%2E+In+Findings+of+the+Association+for+Computational+Linguistics+%28EMNLP%29%2C+pages+2370%E2%80%932386%2E)
2. Clifford Nass, Youngme Moon, B.J.Fogg, Byron Reeves, and D.Christopher Dryer. 1995. Can computer personalities be human personalities? Human-Computer studies, 43:223–239. [find](https://scholar.google.com/scholar?q=Clifford+Nass%2C+Youngme+Moon%2C+B%2EJ%2EFogg%2C+Byron+Reeves%2C+and+D%2EChristopher+Dryer%2E+1995%2E+Can+computer+personalities+be+human+personalities%3F+Human%2DComputer+studies%2C+43%3A223%E2%80%93239%2E)
3. Paul T. Costa and Robert R. McCrae. 1992. Normal personality assessment in clinical practice: The NEO personality inventory. Psychological Assessment, 4(1):5–13. [find](https://scholar.google.com/scholar?q=Paul+T%2E+Costa+and+Robert+R%2E+McCrae%2E+1992%2E+Normal+personality+assessment+in+clinical+practice%3A+The+NEO+personality+inventory%2E+Psychological+Assessment%2C+4%281%29%3A5%E2%80%9313%2E)
4. Shereen Oraby, Lena Reed, Shubhangi Tandon, Sharath T.S., Stephanie Lukin, and Marilyn Walker. 2018. Controlling personality-based stylistic variation with neural natural language generators. In Proceedings of the Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), page… [find](https://scholar.google.com/scholar?q=Shereen+Oraby%2C+Lena+Reed%2C+Shubhangi+Tandon%2C+Sharath+T%2ES%2E%2C+Stephanie+Lukin%2C+and+Marilyn+Walker%2E+2018%2E+Controlling+personality%2Dbased+stylistic+variation+with+neural+natural+language+generators%2E+In+Proce)
5. Lewis R. Goldberg. 1990. An alternative “description of personality”: the big-five factor structure. Personality and Social Psychology, 59(6):1216–1229. [find](https://scholar.google.com/scholar?q=Lewis+R%2E+Goldberg%2E+1990%2E+An+alternative+%E2%80%9Cdescription+of+personality%E2%80%9D%3A+the+big%2Dfive+factor+structure%2E+Personality+and+Social+Psychology%2C+59%286%29%3A1216%E2%80%931229%2E)
6. Atsushi Oshio, Shingo Abe, and Pino Cutrone. 2012. Development, reliability, and validity of the japanese version of ten item personality inventory (TIPI-J). The Japanese Journal of Personality, 21(1):42–52. [find](https://scholar.google.com/scholar?q=Atsushi+Oshio%2C+Shingo+Abe%2C+and+Pino+Cutrone%2E+2012%2E+Development%2C+reliability%2C+and+validity+of+the+japanese+version+of+ten+item+personality+inventory+%28TIPI%2DJ%29%2E+The+Japanese+Journal+of+Personality%2C+21%281%29)
7. Ji-Eun Han, Jun-Seok Koh, Hyeon-Tae Seo, Du-Seong Chang, and Kyung-Ah Sohn. 2024. Psydial: Personality-based synthetic dialogue generation using large language models. In Proceedings of the Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING),… [find](https://scholar.google.com/scholar?q=Ji%2DEun+Han%2C+Jun%2DSeok+Koh%2C+Hyeon%2DTae+Seo%2C+Du%2DSeong+Chang%2C+and+Kyung%2DAh+Sohn%2E+2024%2E+Psydial%3A+Personality%2Dbased+synthetic+dialogue+generation+using+large+language+models%2E+In+Proceedings+of+the+Joint+Inte)
8. Sougata Saha, Souvik Das, and Rohini Srihari. 2022. Stylistic response generation by controlling personality traits and intent. In Proceedings of the Workshop on NLP for Conversational AI, pages 197–211. [find](https://scholar.google.com/scholar?q=Sougata+Saha%2C+Souvik+Das%2C+and+Rohini+Srihari%2E+2022%2E+Stylistic+response+generation+by+controlling+personality+traits+and+intent%2E+In+Proceedings+of+the+Workshop+on+NLP+for+Conversational+AI%2C+pages+197%E2%80%932)
9. Ryuichiro Higashinaka, Masahiro Mizukami, Hidetoshi Kawabata, Emi Yamaguchi, Noritake Adachi, and Junji Tomita. 2018. Role play-based question-answering by real users for building chatbots with consistent personalities. In Proceedings of the Annual Meeting of the Special Interest Group on Discourse… [find](https://scholar.google.com/scholar?q=Ryuichiro+Higashinaka%2C+Masahiro+Mizukami%2C+Hidetoshi+Kawabata%2C+Emi+Yamaguchi%2C+Noritake+Adachi%2C+and+Junji+Tomita%2E+2018%2E+Role+play%2Dbased+question%2Danswering+by+real+users+for+building+chatbots+with+consis)
10. Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. 2023. [Character-LLM: A trainable agent for role-playing](https://md.research.elamur.ai/free/en/papers/shao-2023-character-llm-trainable-agent-role/). In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 13153–13187.
11. Kenta Yamamoto, Koji Inoue, and Tatsuya Kawahara. 2023a. Character expression for spoken dialogue systems with semi-supervised learning using variational auto-encoder. Computer Speech Language, 79:101469–101469. [find](https://scholar.google.com/scholar?q=Kenta+Yamamoto%2C+Koji+Inoue%2C+and+Tatsuya+Kawahara%2E+2023a%2E+Character+expression+for+spoken+dialogue+systems+with+semi%2Dsupervised+learning+using+variational+auto%2Dencoder%2E+Computer+Speech+Language%2C+79%3A101)
12. Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara. 2024. [PersonaLLM: Investigating the ability of large language models to express personality traits](https://md.research.elamur.ai/free/en/papers/jiang-2023-personallm-investigating-ability-large-language/). In Findings of the Association for Computational Linguistics (NAACL), pages 3605–3627.
13. Kenta Yamamoto, Koji Inoue, and Tatsuya Kawahara. 2023b. Character expression of a conversational robot for adapting to user personality. Advanced Robotics, 38(4):256–266. [find](https://scholar.google.com/scholar?q=Kenta+Yamamoto%2C+Koji+Inoue%2C+and+Tatsuya+Kawahara%2E+2023b%2E+Character+expression+of+a+conversational+robot+for+adapting+to+user+personality%2E+Advanced+Robotics%2C+38%284%29%3A256%E2%80%93266%2E)
14. Kazunori Komatani and Shogo Okada. 2021. Multimodal human-agent dialogue corpus with annotations at utterance and dialogue levels. In International Conference on Affective Computing and Intelligent Interaction (ACII), pages 1–8. [find](https://scholar.google.com/scholar?q=Kazunori+Komatani+and+Shogo+Okada%2E+2021%2E+Multimodal+human%2Dagent+dialogue+corpus+with+annotations+at+utterance+and+dialogue+levels%2E+In+International+Conference+on+Affective+Computing+and+Intelligent+In)
15. Sanae Yamashita, Koji Inoue, Ao Guo, Shota Mochizuki, Tatsuya Kawahara, and Ryuichiro Higashinaka. 2023. RealPersonaChat: A realistic persona chat corpus with interlocutors’ own personalities. In Proceedings of the Pacific Asia Conference on Language, Information and Computation (PACLIC), pages 852–… [find](https://scholar.google.com/scholar?q=Sanae+Yamashita%2C+Koji+Inoue%2C+Ao+Guo%2C+Shota+Mochizuki%2C+Tatsuya+Kawahara%2C+and+Ryuichiro+Higashinaka%2E+2023%2E+RealPersonaChat%3A+A+realistic+persona+chat+corpus+with+interlocutors%E2%80%99+own+personalities%2E+In+Proc)
16. Kazunori Komatani, Ryu Takeda, and Shogo Okada. 2023. Analyzing differences in subjective annotations by participants and third-party annotators in multimodal dialogue corpus. In Proceedings of the Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 1688–1692. [find](https://scholar.google.com/scholar?q=Kazunori+Komatani%2C+Ryu+Takeda%2C+and+Shogo+Okada%2E+2023%2E+Analyzing+differences+in+subjective+annotations+by+participants+and+third%2Dparty+annotators+in+multimodal+dialogue+corpus%2E+In+Proceedings+of+the+An)
17. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. [Personalizing dialogue agents: I have a dog, do you have pets too?](https://md.research.elamur.ai/free/en/papers/zhang-2018-personalizing-dialogue-agents-i-have/) In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 2204–2213.
18. Jiwei Li, Michel Galley, Chris Brockett, Georgios Spithourakis, Jianfeng Gao, and Bill Dolan. 2016. [A persona-based neural conversation model](https://md.research.elamur.ai/free/en/papers/li-2016-persona-based-neural-conversation-model/). In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 994–1003.

*Updated 2 November 2026*