---
title: "CharacTer: Translation Edit Rate on Character — Wang 2016"
description: "CharacTER is a translation edit rate computed on characters. It keeps the word-level phrase shift of TER, so a moved phrase costs something, but the edit…"
authors: ["Weiyue Wang", "Jan-Thorsten Peter", "Hendrik Rosendahl", "Hermann Ney"]
year: 2016
venue: "Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers"
source: "https://aclanthology.org/W16-2342/"
doi: "10.18653/v1/w16-2342"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2016-character-translation-edit-rate-character/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2016-character-translation-edit-rate-character/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/2/) / CharacTer: Translation Edit Rate on Character Level

# CharacTer: Translation Edit Rate on Character Level

**Authors:** Weiyue Wang, [Jan-Thorsten Peter](https://md.research.elamur.ai/free/en/authors/jan-thorsten-peter/), Hendrik Rosendahl, [Hermann Ney](https://md.research.elamur.ai/free/en/authors/hermann-ney/) · **Year:** 2016 · **Venue:** First Conference on Machine Translation, Volume 2: Shared Task Papers (WMT 2016) **Link:** [aclanthology.org/W16-2342](https://aclanthology.org/W16-2342/) · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)

**Institutions:** [RWTH Aachen University](https://md.research.elamur.ai/free/en/universities/rwth-aachen-university/)

[**129** citations](https://openalex.org/works?filter=cites:W2512848817) OpenAlex, 2026-10-14 · [**85** citations in our library](https://md.research.elamur.ai/free/en/papers/wang-2016-character-translation-edit-rate-character/cited-by/) of the 140884 papers whose bibliographies we read · [Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers](https://aclanthology.org/volumes/W16-23/) · [licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/W16-2342/)
- [PDF](https://aclanthology.org/W16-2342.pdf)
- [DOI 10.18653/v1/w16-2342](https://doi.org/10.18653/v1/w16-2342)
- [OpenAlex](https://openalex.org/W2512848817)

**Authors (4)**

1. [Weiyue Wang](https://openalex.org/A5006737674) [ORCID](https://orcid.org/0000-0002-9127-7544) · [Semantic Scholar](https://www.semanticscholar.org/author/1976198) · h 11 · 534 citations · 41 works
2. [Jan-Thorsten Peter](https://openalex.org/A5103297461) [Semantic Scholar](https://www.semanticscholar.org/author/145407095) · h 11 · 563 citations · 37 works
3. [Hendrik Rosendahl](https://openalex.org/A5065302813) [Semantic Scholar](https://www.semanticscholar.org/author/38683081) · h 2 · 135 citations · 2 works
4. [Hermann Ney](https://openalex.org/A5112501010) [Semantic Scholar](https://www.semanticscholar.org/author/145322333) · h 93 · 47653 citations · 1461 works

Citations and author metrics: OpenAlex, 2026-10-14; authors with the same name are not always told apart.

## What it is about

CharacTER is a translation edit rate computed on characters. It keeps the word-level phrase shift of TER, so a moved phrase costs something, but the edit distance after the shifts is a character-level Levenshtein distance, with spaces counted as characters. Shifting characters directly was about 44 times slower, so the shift is done on words. Three choices make it work better than TER on morphologically rich languages: the cost of a shift is the average word length of the moved phrase, two words match when identical or within one edit (for stem variants such as code and codes), and the error count is normalised by the hypothesis length instead of the reference length. The last change removes the bias of TER towards short outputs and was the largest single gain, up to about nine points in correlation. On WMT13 and WMT14 system-level correlations, the variant beats chrF and BEER on several target languages, by up to about seven points, while costing roughly ten per cent more time than TER.

## Ideas

- I1. A word-shift and character-edit score for the translator's normal mode, so that *hath* against *has* or *thee* against *thou* costs a small edit and not a full miss. Compare with exact match on held-out verses, and run twice.
- I2. Test both normalisers on the Bible pairs. The hypothesis length rewards longer outputs, so adopt it only if it still wins once that bias is checked.
- I3. Charge a moved phrase by its average word length, so that displaced content words cost more than displaced function words. This is a count-based way to score the word-order residual class.
- I4. Use the one-edit stem rule as a near-match flag in the lexicon report, so that a plural or suffix variant is shown as a near merge and not as a failure.

## For Mova

The places are `tools/translate` (`measure.rs`) for the normal-mode score and `en/src/morph.rs` for suffix tolerance. It touches open problem 5 (number and person forms that a merge such as *thee/thou/ye* loses). The paper is about MT scoring, not about morphology, so only the shift cost, the normaliser and the stem tolerance are taken over.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/w16-2342](https://doi.org/10.18653/v1/w16-2342)

```bibtex
@inproceedings{wang2016character,
  title = {{CharacTer: Translation Edit Rate on Character Level}},
  author = {Wang, Weiyue and Peter, Jan-Thorsten and Rosendahl, Hendrik and Ney, Hermann},
  booktitle = {Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers},
  year = {2016},
  doi = {10.18653/v1/w16-2342},
  url = {https://doi.org/10.18653/v1/w16-2342}
}
```

- [← previous Automatic Sarcasm Detection: A Survey](https://md.research.elamur.ai/free/en/papers/joshi-2016-automatic-sarcasm-detection-survey/)
- [next → Are Word Embedding-based Features Useful for Sarcasm Detection?](https://md.research.elamur.ai/free/en/papers/joshi-2016-are-word-embedding-based-features/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/2/)

## References

11 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Miloš Stanojević and Khalil Sima’an. 2014. BEER: BEtter Evaluation as Ranking. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 414–419, Baltimore, MA, USA, June. [find](https://scholar.google.com/scholar?q=Milos%CC%8C+Stanojevic%CC%81+and+Khalil+Sima%E2%80%99an%2E+2014%2E+BEER%3A+BEtter+Evaluation+as+Ranking%2E+In+Proceedings+of+the+Ninth+Workshop+on+Statistical+Machine+Translation%2C+pages+414%E2%80%93419%2C+Baltimore%2C+MA%2C+USA%2C+June%2E)
2. Ondřej Bojar, Yvette Graham, Amir Kamran, and Miloš Stanojević. 2016. Results of the WMT16 Metrics Shared Task. In Proceedings of the First Conference on Statistical Machine Translation, Berlin, Germany, August. [find](https://scholar.google.com/scholar?q=Ondr%CC%8Cej+Bojar%2C+Yvette+Graham%2C+Amir+Kamran%2C+and+Milos%CC%8C+Stanojevic%CC%81%2E+2016%2E+Results+of+the+WMT16+Metrics+Shared+Task%2E+In+Proceedings+of+the+First+Conference+on+Statistical+Machine+Translation%2C+Berlin%2C+Ge) [our card](https://md.research.elamur.ai/free/en/papers/bojar-2016-results-wmt16-metrics-shared-task/)
3. Miloš Stanojević, Amir Kamran, Philipp Koehn, and Ondřej Bojar, 2015. Results of the WMT15 Metrics Shared Task. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Milos%CC%8C+Stanojevic%CC%81%2C+Amir+Kamran%2C+Philipp+Koehn%2C+and+Ondr%CC%8Cej+Bojar%2C+2015%2E+Results+of+the+WMT15+Metrics+Shared+Task%2E+Association+for+Computational+Linguistics%2E)
4. George Doddington. 2002. Automatic Evaluation of Machine Translation Quality Using N-gram Co-occurrence Statistics. In Proceedings of the Second International Conference on Human Language Technology Research, pages 128–132, San Diego, CA, USA, March. [find](https://scholar.google.com/scholar?q=George+Doddington%2E+2002%2E+Automatic+Evaluation+of+Machine+Translation+Quality+Using+N%2Dgram+Co%2Doccurrence+Statistics%2E+In+Proceedings+of+the+Second+International+Conference+on+Human+Language+Technology+R)
5. Alon Lavie and Abhaya Agarwal. 2007. METEOR: An automatic metric for mt evaluation with high levels of correlation with human judgments. In Proceedings of the ACL 2007 Workshop on Statistical Machine Translation, pages 228–231, Prague, Czech Republic, June. [find](https://scholar.google.com/scholar?q=Alon+Lavie+and+Abhaya+Agarwal%2E+2007%2E+METEOR%3A+An+automatic+metric+for+mt+evaluation+with+high+levels+of+correlation+with+human+judgments%2E+In+Proceedings+of+the+ACL+2007+Workshop+on+Statistical+Machine+) [our card](https://md.research.elamur.ai/free/en/papers/lavie-2007-meteor-automatic-metric-mt-evaluation/)
6. Matouš Macháček and Ondřej Bojar. 2013. Results of the WMT13 metrics shared task. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 45–51, Sofia, Bulgaria, August. [find](https://scholar.google.com/scholar?q=Matous%CC%8C+Macha%CC%81c%CC%8Cek+and+Ondr%CC%8Cej+Bojar%2E+2013%2E+Results+of+the+WMT13+metrics+shared+task%2E+In+Proceedings+of+the+Eighth+Workshop+on+Statistical+Machine+Translation%2C+pages+45%E2%80%9351%2C+Sofia%2C+Bulgaria%2C+August%2E)
7. Matouš Macháček and Ondřej Bojar. 2014. Results of the WMT14 metrics shared task. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 293–301, Baltimore, MA, USA, June. [find](https://scholar.google.com/scholar?q=Matous%CC%8C+Macha%CC%81c%CC%8Cek+and+Ondr%CC%8Cej+Bojar%2E+2014%2E+Results+of+the+WMT14+metrics+shared+task%2E+In+Proceedings+of+the+Ninth+Workshop+on+Statistical+Machine+Translation%2C+pages+293%E2%80%93301%2C+Baltimore%2C+MA%2C+USA%2C+June%2E) [our card](https://md.research.elamur.ai/free/en/papers/machacek-2014-results-wmt14-metrics-shared-task/)
8. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 311–318, Philadelphia, PA, USA, July. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+BLEU%3A+A+Method+for+Automatic+Evaluation+of+Machine+Translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+on+Association+for+Computat) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
9. Karl Pearson. 1895. Notes on Regression and Inheritance in the Case of Two Parents. In Proceedings of the Royal Society of London, volume 58, pages 240–242, London, UK, June. [find](https://scholar.google.com/scholar?q=Karl+Pearson%2E+1895%2E+Notes+on+Regression+and+Inheritance+in+the+Case+of+Two+Parents%2E+In+Proceedings+of+the+Royal+Society+of+London%2C+volume+58%2C+pages+240%E2%80%93242%2C+London%2C+UK%2C+June%2E)
10. Maja Popović. 2015. CHRF: Character n-gram F-Score for Automatic MT Evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisboa, Portugal, September. [find](https://scholar.google.com/scholar?q=Maja+Popovic%CC%81%2E+2015%2E+CHRF%3A+Character+n%2Dgram+F%2DScore+for+Automatic+MT+Evaluation%2E+In+Proceedings+of+the+Tenth+Workshop+on+Statistical+Machine+Translation%2C+pages+392%E2%80%93395%2C+Lisboa%2C+Portugal%2C+September%2E) [our card](https://md.research.elamur.ai/free/en/papers/popovic-2015-chrf-character-n-gram-f/)
11. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study [find](https://scholar.google.com/scholar?q=Matthew+Snover%2C+Bonnie+Dorr%2C+Richard+Schwartz%2C+Linnea+Micciulla%2C+and+John+Makhoul%2E+2006%2E+A+study)