---
title: "Translators' Perceptions and Edit Traces — Leal-Wyss 2026"
description: "Translators at the Parliament of Canada can draft House of Commons translations (French↔English) with Hawkeye, a custom NMT system built on the Sockeye…"
authors: ["Jeniffer Leal-Wyss", "Gabriel Bernier-Colborne", "Delaney Lothian", "Michel Simard", "Rebecca Knowles"]
year: 2026
venue: "Proceedings of the 17th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)"
source: "https://aclanthology.org/2026.amta-research.10/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wyss-2026-translators-perceptions-edit-traces-quality/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wyss-2026-translators-perceptions-edit-traces-quality/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/10/) / Translators' Perceptions and Edit Traces: Quality of MT as a Tool in Canadian Parliamentary Translation

# Translators' Perceptions and Edit Traces: Quality of MT as a Tool in Canadian Parliamentary Translation

**Authors:** [Jeniffer Leal-Wyss](https://md.research.elamur.ai/free/en/authors/jeniffer-leal-wyss/), [Gabriel Bernier-Colborne](https://md.research.elamur.ai/free/en/authors/gabriel-bernier-colborne/), [Delaney Lothian](https://md.research.elamur.ai/free/en/authors/delaney-lothian/), [Michel Simard](https://md.research.elamur.ai/free/en/authors/michel-simard/), [Rebecca Knowles](https://md.research.elamur.ai/free/en/authors/rebecca-knowles/) · **Year:** 2026 · **Venue:** Proceedings of the 17th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.amta-research.10/) · [PDF](https://aclanthology.org/2026.amta-research.10.pdf)

## Summary
Translators at the Parliament of Canada can draft House of Commons translations (French↔English) with Hawkeye, a custom NMT system built on the Sockeye toolkit and trained on parliamentary text. Earlier log analysis showed that Hawkeye-assisted drafts receive fewer reviser edits, but it did not show which errors occur or how translators react to them. This study fills that gap by combining edit-trace analysis with a user study. From Translation Bureau logs, the authors sampled House of Commons debate chunks and automatically aligned each draft with its final revised version to locate the changed spans. Two trained bilingual annotators then labelled every change with MQM-Core error categories and severities. The edits were examined in two ways: raw MT against the final text (PE+REV), and translator drafts with or without Hawkeye against the revised text (REV). A questionnaire followed by interviews captured how translators perceive MT quality. Hawkeye-assisted drafts needed fewer revisions and had slightly less severe errors. Translator feedback helped explain recurring problems, such as the adaptation that spoken source text requires. Together, these findings can guide improvements to a production MT system.

## Key points
- The sample covers 230 chunks: 1,393 segments and 26,197 words in REV. The PE+REV subset has 114 chunks, 784 segments and 14,737 words, all drawn only from House of Commons debates.
- In prior log analysis (Simard et al., 2026), the share of work done with Hawkeye rose from 16% in May 2023 to nearly 60% by December 2025.
- MQM-Core annotation was blind and cross-checked by the other annotator, with disagreements settled through discussion. Purely preferential changes were coded as Style – Unidiomatic with Neutral severity.
- The user study used a mixed-methods explanatory sequential design, with a questionnaire followed by semi-structured interviews. A plurality of respondents felt Hawkeye's output had improved over time.
- The study has several limitations:
  - The comparisons are observational, and Hawkeye users and non-users are largely different translators.
  - The annotators were not the actual users of the system.
  - The authors propose adding error explainability (identifying causes and possible remedies) in future work.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lealwyss2026translators,
  title = {{Translators' Perceptions and Edit Traces: Quality of MT as a Tool in Canadian Parliamentary Translation}},
  author = {Leal-Wyss, Jeniffer and Bernier-Colborne, Gabriel and Lothian, Delaney and Simard, Michel and Knowles, Rebecca},
  booktitle = {Proceedings of the 17th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)},
  year = {2026},
  url = {https://aclanthology.org/2026.amta-research.10/}
}
```

- [← previous Translation-CoT: A Human-Inspired Chain-of-Thought Framework for Multilingual LLM Translation](https://md.research.elamur.ai/free/en/papers/prama-2026-translation-cot-human-inspired-chain/)
- [next → Transparent Semantic Change Detection with Dependency-Based Profiles](https://md.research.elamur.ai/free/en/papers/tat-2026-transparent-semantic-change-detection-dependency/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/10/)

## References

24 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bowker, L. (2020). Fit-for-purpose translation. In O’Hagan, M., editor, The Routledge Handbook of Translation and Technology, volume 1, pages 453–468. Routledge, 1st edition. [find](https://scholar.google.com/scholar?q=Bowker%2C+L%2E+%282020%29%2E+Fit%2Dfor%2Dpurpose+translation%2E+In+O%E2%80%99Hagan%2C+M%2E%2C+editor%2C+The+Routledge+Handbook+of+Translation+and+Technology%2C+volume+1%2C+pages+453%E2%80%93468%2E+Routledge%2C+1st+edition%2E)
2. Cadwell, P., O’Brien, S., and Teixeira, C. (2018). Resistance and accommodation: factors for the (non-) adoption of machine translation among professional translators. Perspectives, pages 1–21. [find](https://scholar.google.com/scholar?q=Cadwell%2C+P%2E%2C+O%E2%80%99Brien%2C+S%2E%2C+and+Teixeira%2C+C%2E+%282018%29%2E+Resistance+and+accommodation%3A+factors+for+the+%28non%2D%29+adoption+of+machine+translation+among+professional+translators%2E+Perspectives%2C+pages+1%E2%80%9321%2E)
3. Creswell, J. W. and Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications. [find](https://scholar.google.com/scholar?q=Creswell%2C+J%2E+W%2E+and+Creswell%2C+J%2E+D%2E+%282017%29%2E+Research+design%3A+Qualitative%2C+quantitative%2C+and+mixed+methods+approaches%2E+Sage+publications%2E)
4. Freitag, M., Foster, G., Grangier, D., Ratnakar, V., Tan, Q., and Macherey, W. (2021). Experts, errors, and context: A large-scale study of human evaluation for machine translation. Transactions of the Association for Computational Linguistics, 9:1460–1474. [find](https://scholar.google.com/scholar?q=Freitag%2C+M%2E%2C+Foster%2C+G%2E%2C+Grangier%2C+D%2E%2C+Ratnakar%2C+V%2E%2C+Tan%2C+Q%2E%2C+and+Macherey%2C+W%2E+%282021%29%2E+Experts%2C+errors%2C+and+context%3A+A+large%2Dscale+study+of+human+evaluation+for+machine+translation%2E+Transactions+of+th) [our card](https://md.research.elamur.ai/free/en/papers/freitag-2021-experts-errors-context-large-scale/)
5. Graham, Y., Baldwin, T., Moffat, A., and Zobel, J. (2013). Continuous measurement scales in human evaluation of machine translation. In Pareja-Lora, A., Liakata, M., and Dipper, S., editors, Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, pages 33–41, Sofia… [find](https://scholar.google.com/scholar?q=Graham%2C+Y%2E%2C+Baldwin%2C+T%2E%2C+Moffat%2C+A%2E%2C+and+Zobel%2C+J%2E+%282013%29%2E+Continuous+measurement+scales+in+human+evaluation+of+machine+translation%2E+In+Pareja%2DLora%2C+A%2E%2C+Liakata%2C+M%2E%2C+and+Dipper%2C+S%2E%2C+editors%2C+Proceedin)
6. M. (2022). Sockeye 3: Fast neural machine translation with pytorch. [find](https://scholar.google.com/scholar?q=M%2E+%282022%29%2E+Sockeye+3%3A+Fast+neural+machine+translation+with+pytorch%2E)
7. Kocmi, T., Zouhar, V., Avramidis, E., Grundkiewicz, R., Karpinska, M., Popović, M., Sachan, M., and Shmatova, M. (2024). Error span annotation: A balanced approach for human evaluation of machine translation. In Haddow, B., Kocmi, T., Koehn, P., and Monz, C., editors, Proceedings of the Ninth Confe… [find](https://scholar.google.com/scholar?q=Kocmi%2C+T%2E%2C+Zouhar%2C+V%2E%2C+Avramidis%2C+E%2E%2C+Grundkiewicz%2C+R%2E%2C+Karpinska%2C+M%2E%2C+Popovic%CC%81%2C+M%2E%2C+Sachan%2C+M%2E%2C+and+Shmatova%2C+M%2E+%282024%29%2E+Error+span+annotation%3A+A+balanced+approach+for+human+evaluation+of+machine+tra) [our card](https://md.research.elamur.ai/free/en/papers/kocmi-2024-error-span-annotation-balanced-approach/)
8. Leal-Wyss, J., Lothian, D., Bernier-Colborne, G., Knowles, R., and Simard, M. (2026). “all those in favour will please say yea”’: Understanding the factors behind machine translation adoption at the Canadian Parliament. In Proceedings of the 26th Annual Conference of the European Association for Mac… [find](https://scholar.google.com/scholar?q=Leal%2DWyss%2C+J%2E%2C+Lothian%2C+D%2E%2C+Bernier%2DColborne%2C+G%2E%2C+Knowles%2C+R%2E%2C+and+Simard%2C+M%2E+%282026%29%2E+%E2%80%9Call+those+in+favour+will+please+say+yea%E2%80%9D%E2%80%99%3A+Understanding+the+factors+behind+machine+translation+adoption+at+the+C) [our card](https://md.research.elamur.ai/free/en/papers/wyss-2026-all-those-favour-will-please/)
9. Li, Y., Suzuki, J., Morishita, M., Abe, K., and Inui, K. (2025). MQM-chat: Multidimensional quality metrics for chat translation. In Rambow, O., Wanner, L., Apidianaki, M., Al-Khalifa, H., Eugenio, B. D., and Schockaert, S., editors, Proceedings of the 31st International Conference on Computational… [find](https://scholar.google.com/scholar?q=Li%2C+Y%2E%2C+Suzuki%2C+J%2E%2C+Morishita%2C+M%2E%2C+Abe%2C+K%2E%2C+and+Inui%2C+K%2E+%282025%29%2E+MQM%2Dchat%3A+Multidimensional+quality+metrics+for+chat+translation%2E+In+Rambow%2C+O%2E%2C+Wanner%2C+L%2E%2C+Apidianaki%2C+M%2E%2C+Al%2DKhalifa%2C+H%2E%2C+Eugenio%2C+B%2E)
10. Liu, T., Lo, C.-k., Marshman, E., and Knowles, R. (2024). Evaluation briefs: Drawing on translation studies for human evaluation of MT. In Knowles, R., Eriguchi, A., and Goel, S., editors, Proceedings of the 16th Conference of the Association for Machine Translation in the Americas (Volume 1: Resear… [find](https://scholar.google.com/scholar?q=Liu%2C+T%2E%2C+Lo%2C+C%2E%2Dk%2E%2C+Marshman%2C+E%2E%2C+and+Knowles%2C+R%2E+%282024%29%2E+Evaluation+briefs%3A+Drawing+on+translation+studies+for+human+evaluation+of+MT%2E+In+Knowles%2C+R%2E%2C+Eriguchi%2C+A%2E%2C+and+Goel%2C+S%2E%2C+editors%2C+Proceedings)
11. Lommel, A., Gladkoff, S., Melby, A., Wright, S. E., Strandvik, I., Gasova, K., Vaasa, A., Benzo, A., Marazzato Sparano, R., Foresi, M., Innis, J., Han, L., and Nenadic, G. (2024). The multi-range theory of translation quality measurement: MQM scoring models and statistical quality control. In Martin… [find](https://scholar.google.com/scholar?q=Lommel%2C+A%2E%2C+Gladkoff%2C+S%2E%2C+Melby%2C+A%2E%2C+Wright%2C+S%2E+E%2E%2C+Strandvik%2C+I%2E%2C+Gasova%2C+K%2E%2C+Vaasa%2C+A%2E%2C+Benzo%2C+A%2E%2C+Marazzato+Sparano%2C+R%2E%2C+Foresi%2C+M%2E%2C+Innis%2C+J%2E%2C+Han%2C+L%2E%2C+and+Nenadic%2C+G%2E+%282024%29%2E+The+multi%2Drange+theo)
12. Heinisch, B. and Lušicky, V. (2019). User expectations towards machine translation: A case study. In Forcada, M., Way, A., Tinsley, J., Shterionov, D., Rico, C., and Gaspari, F., editors, Proceedings of Machine Translation Summit XVII: Translator, Project and User Tracks, pages 42–48, Dublin, Irela… [find](https://scholar.google.com/scholar?q=Heinisch%2C+B%2E+and+Lus%CC%8Cicky%2C+V%2E+%282019%29%2E+User+expectations+towards+machine+translation%3A+A+case+study%2E+In+Forcada%2C+M%2E%2C+Way%2C+A%2E%2C+Tinsley%2C+J%2E%2C+Shterionov%2C+D%2E%2C+Rico%2C+C%2E%2C+and+Gaspari%2C+F%2E%2C+editors%2C+Proceedings)
13. Proceedings of the 17th Conference of the Association for Machine Translation in the Americas, Québec City, Canada, August 31 September 2, 2026. Volume 1: Research Papers Lommel, A. R., Burchardt, A., and Uszkoreit, H. (2013). Multidimensional quality metrics: a flexible system for assessing transl… [find](https://scholar.google.com/scholar?q=Proceedings+of+the+17th+Conference+of+the+Association+for+Machine+Translation+in+the+Americas%2C+Qu%C3%A9bec+City%2C+Canada%2C+August+31+September+2%2C+2026%2E+Volume+1%3A+Research+Papers+Lommel%2C+A%2E+R%2E%2C+Burchardt%2C+A%2E%2C) [our card](https://md.research.elamur.ai/free/en/papers/lommel-2013-multidimensional-quality-metrics-flexible-system/)
14. Magdy, S. M., Alwajih, F., Mekki, A. E., El-Sayed, W., and Abdul-Mageed, M. (2026). Lqm: Linguistically motivated multidimensional quality metrics for machine translation. [find](https://scholar.google.com/scholar?q=Magdy%2C+S%2E+M%2E%2C+Alwajih%2C+F%2E%2C+Mekki%2C+A%2E+E%2E%2C+El%2DSayed%2C+W%2E%2C+and+Abdul%2DMageed%2C+M%2E+%282026%29%2E+Lqm%3A+Linguistically+motivated+multidimensional+quality+metrics+for+machine+translation%2E)
15. Moorkens, J., Toral, A., Castilho, S., and Way, A. (2018). Translators’ perceptions of literary post-editing using statistical and neural machine translation. Translation Spaces, 7:240–262. 7th Conference of the Association for Machine Translation in the Americas: Technical Papers, pages 223– 231, C… [find](https://scholar.google.com/scholar?q=Moorkens%2C+J%2E%2C+Toral%2C+A%2E%2C+Castilho%2C+S%2E%2C+and+Way%2C+A%2E+%282018%29%2E+Translators%E2%80%99+perceptions+of+literary+post%2Dediting+using+statistical+and+neural+machine+translation%2E+Translation+Spaces%2C+7%3A240%E2%80%93262%2E+7th+Confer)
16. Vieira, L. N. and Alonso, E. (2020). Translating perceptions and managing expectations: an analysis of management and production perspectives on machine translation. Perspectives: Studies in Translatology, 28(2):163–184. [find](https://scholar.google.com/scholar?q=Vieira%2C+L%2E+N%2E+and+Alonso%2C+E%2E+%282020%29%2E+Translating+perceptions+and+managing+expectations%3A+an+analysis+of+management+and+production+perspectives+on+machine+translation%2E+Perspectives%3A+Studies+in+Translato)
17. Naeem, M., Ozuem, W., Howell, K., and Ranfagni, S. (2023). A step-by-step process of thematic analysis to develop a conceptual model in qualitative research. International Journal of Qualitative Methods, 22:1–18. [find](https://scholar.google.com/scholar?q=Naeem%2C+M%2E%2C+Ozuem%2C+W%2E%2C+Howell%2C+K%2E%2C+and+Ranfagni%2C+S%2E+%282023%29%2E+A+step%2Dby%2Dstep+process+of+thematic+analysis+to+develop+a+conceptual+model+in+qualitative+research%2E+International+Journal+of+Qualitative+Metho)
18. Yang, Y., Liu, R., Qian, X., and Ni, J. (2023). Performance and perception: machine translation post-editing in chinese-english news translation by novice translators. Humanities & Social Sciences Communications, 10(1):Article 798. [find](https://scholar.google.com/scholar?q=Yang%2C+Y%2E%2C+Liu%2C+R%2E%2C+Qian%2C+X%2E%2C+and+Ni%2C+J%2E+%282023%29%2E+Performance+and+perception%3A+machine+translation+post%2Dediting+in+chinese%2Denglish+news+translation+by+novice+translators%2E+Humanities+%26+Social+Sciences+Com)
19. Naveen, P. and Trojovský, P. (2024). Overview and challenges of machine translation for contextually appropriate translations. iScience, 27(10):110878. [find](https://scholar.google.com/scholar?q=Naveen%2C+P%2E+and+Trojovsky%CC%81%2C+P%2E+%282024%29%2E+Overview+and+challenges+of+machine+translation+for+contextually+appropriate+translations%2E+iScience%2C+27%2810%29%3A110878%2E)
20. A O’Brien, S. (2012). Towards a dynamic quality evaluation model for translation. The Journal of Specialised Translation, pages 55–77. [find](https://scholar.google.com/scholar?q=A+O%E2%80%99Brien%2C+S%2E+%282012%29%2E+Towards+a+dynamic+quality+evaluation+model+for+translation%2E+The+Journal+of+Specialised+Translation%2C+pages+55%E2%80%9377%2E)
21. Ortiz-Boix, C. and Matamala, A. (2017). Assessing the quality of post-edited wildlife documentaries. Perspectives, 25:1–23. [find](https://scholar.google.com/scholar?q=Ortiz%2DBoix%2C+C%2E+and+Matamala%2C+A%2E+%282017%29%2E+Assessing+the+quality+of+post%2Dedited+wildlife+documentaries%2E+Perspectives%2C+25%3A1%E2%80%9323%2E)
22. Park, D. and Padó, S. (2024). Multi-dimensional machine translation evaluation: Model evaluation and resource for Korean. In Calzolari, N., Kan, M.-Y., Hoste, V., Lenci, A., Sakti, S., and Xue, N., editors, Proceedings of the 2024 Joint International Conference on Computational Linguistics, Languag… [find](https://scholar.google.com/scholar?q=Park%2C+D%2E+and+Pado%CC%81%2C+S%2E+%282024%29%2E+Multi%2Ddimensional+machine+translation+evaluation%3A+Model+evaluation+and+resource+for+Korean%2E+In+Calzolari%2C+N%2E%2C+Kan%2C+M%2E%2DY%2E%2C+Hoste%2C+V%2E%2C+Lenci%2C+A%2E%2C+Sakti%2C+S%2E%2C+and+Xue%2C+N%2E%2C+e)
23. Simard, M., Leal-Wyss, J., Bernier-Colborne, G., and Knowles, R. a. (2026). A longitudinal study of the adoption of specialized mt systems in Canadian Parliamentary translation. In Proceedings of the 26th Annual Conference of the European Association for Machine Translation (EAMT 2026), pages 133–14… [find](https://scholar.google.com/scholar?q=Simard%2C+M%2E%2C+Leal%2DWyss%2C+J%2E%2C+Bernier%2DColborne%2C+G%2E%2C+and+Knowles%2C+R%2E+a%2E+%282026%29%2E+A+longitudinal+study+of+the+adoption+of+specialized+mt+systems+in+Canadian+Parliamentary+translation%2E+In+Proceedings+of+the+) [our card](https://md.research.elamur.ai/free/en/papers/simard-2026-longitudinal-study-adoption-specialized-mt/)
24. Snover, M., Dorr, B., Schwartz, R., Micciulla, L., and Makhoul, J. (2006). A study of translation edit rate with targeted human annotation. In Proceedings of the Details on Data Sampling [find](https://scholar.google.com/scholar?q=Snover%2C+M%2E%2C+Dorr%2C+B%2E%2C+Schwartz%2C+R%2E%2C+Micciulla%2C+L%2E%2C+and+Makhoul%2C+J%2E+%282006%29%2E+A+study+of+translation+edit+rate+with+targeted+human+annotation%2E+In+Proceedings+of+the+Details+on+Data+Sampling) [our card](https://md.research.elamur.ai/free/en/papers/snover-2006-study-translation-edit-rate-targeted/)