---
title: "Building an Arabic Machine Translation Post-Edited Corpus:"
description: "The authors present the result as the largest manual post-edited corpus for this language pair. 18 references, cited by 8 papers here."
authors: ["Wajdi Zaghouani", "Nizar Habash", "Ossama Obeid", "Behrang Mohit", "Houda Bouamor", "Kemal Oflazer"]
year: 2016
venue: "Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)"
source: "https://aclanthology.org/L16-1295/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zaghouani-2016-building-arabic-machine-translation-post/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zaghouani-2016-building-arabic-machine-translation-post/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/4/) / Building an Arabic Machine Translation Post-Edited Corpus: Guidelines and Annotation

# Building an Arabic Machine Translation Post-Edited Corpus: Guidelines and Annotation

**Authors:** [Wajdi Zaghouani](https://md.research.elamur.ai/free/en/authors/wajdi-zaghouani/), [Nizar Habash](https://md.research.elamur.ai/free/en/authors/nizar-habash/), [Ossama Obeid](https://md.research.elamur.ai/free/en/authors/ossama-obeid/), [Behrang Mohit](https://md.research.elamur.ai/free/en/authors/behrang-mohit/), [Houda Bouamor](https://md.research.elamur.ai/free/en/authors/houda-bouamor/) and [Kemal Oflazer](https://md.research.elamur.ai/free/en/authors/kemal-oflazer/) · **Year:** 2016 · **Venue:** [Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)](https://md.research.elamur.ai/free/en/venues/lrec/) ([LREC 2016](https://md.research.elamur.ai/free/en/venues/lrec/2016/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/L16-1295/) · [PDF](https://aclanthology.org/L16-1295.pdf) · [OpenAlex](https://openalex.org/W2572792611)

**Citations:** 16 citations (OpenAlex, 2026-10-14) · [8](https://md.research.elamur.ai/free/en/papers/zaghouani-2016-building-arabic-machine-translation-post/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Automatic post-editing of machine translation needs training data: MT output paired with human corrections. No such resource existed for English→Arabic, so the authors built one within the QALB project. They took about 100K words of English Wikinews articles, mostly political news, and translated them into Modern Standard Arabic with the paid Google Translate API. Five annotators, supervised by a single lead annotator, then corrected the output using QAWI, a web-based tool.

The guidelines extend earlier QALB correction rules for native-speaker text with MT-specific policy:
- edit as little as possible,
- but put meaning preservation ahead of edit economy,
- fix grammar always,
- leave an odd but acceptable MT style untouched.

Quality was tracked with repeated inter-annotator agreement checks, measured as average WER between annotations, in two rounds. Annotators changed about a third of the MT text. Agreement on the first round was moderate, while agreement on refining already-corrected text was high. The authors present the result as the largest manual post-edited corpus for this language pair.

## Key points
- The corpus has 520 Wikinews articles (about 100K words, roughly 192 words per article), machine-translated into Arabic. It forms the MT portion of the 2M-word QALB corpus.
- Errors fall into seven categories: spelling (mainly Yaa/hamza), word choice, morphology, syntax (agreement, definiteness, case, tense), proper names, dialectal usage and punctuation.
- Measured as WER between raw and gold text, 31.75% of the MT text was changed. The figures for the other QALB corpora are 24.45 (L1) and 37.64 (L2).
- Agreement (average WER, punctuation ignored) was 16.87 in round 1 and 4.92 in round 2. It was computed on 20 files (2,980 words), each edited by at least three annotators.
- New annotators went through training and a monitored probation period before production. An online discussion group handled borderline questions, and WER-based agreement scores were built into the QAWI management interface.

## Interesting ideas
- **Two-stage agreement.** Agreement is measured once when annotators correct raw output independently, and again when a second pool reviews already-corrected text. This separates "many valid answers exist" from "the correction is genuinely contested."
- **Minimal edits with a meaning override.** Annotators make the smallest change that yields an acceptable result, but fidelity to the source always outranks edit count, and nothing may be added or dropped. This gives a principled way to judge any repair task.
- **Do not polish acceptable style.** Unusual but correct and faithful phrasing is explicitly left alone. This keeps corrections focused on real errors instead of annotator taste.
- **Cross-genre agreement baselines.** MT agreement numbers are interpreted against the same metric on native and learner corpora, which shows whether a score is high for this kind of text.
- **Typed error taxonomy for corrections.** Each fix belongs to a fixed category such as agreement, word choice, named entity or punctuation. This makes error statistics actionable rather than a single score.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Silver-data agreement among witness parsers could be scored both from scratch and as review of a draft tree, so that low agreement on legitimately ambiguous attachments is not mistaken for noise.
- **I2** (learn): Self-induced repair rules could be ranked by the fewest changes that restore a correct frame, while rejecting any rule that alters who-did-what-to-whom.
- **I3** (back): When evaluating text regenerated from the meaning graph, faithfulness and grammaticality could be the scored properties, and stylistic divergence from the original would not be penalized.
- **I4** (learn): Parser or frame agreement on a new genre (fables, math problems) could be reported next to the same metric on a familiar treebank as a reference point.
- **I5** (tree): Parser disagreements and repairs could be tagged by error class (attachment, label, agreement-driven, named-entity span) to show which component needs work.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 539 works · 179,533 citations
- [New York University Abu Dhabi](https://md.research.elamur.ai/free/en/universities/new-york-university-abu-dhabi/) — University, Abu Dhabi, United Arab Emirates · 122 works · 4,602 citations
- [Carnegie Mellon University Qatar](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university-qatar/) — University, Doha, Qatar · 47 works · 2,773 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 1,087 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Wajdi Zaghouani, Nizar Habash, Ossama Obeid, Behrang Mohit, Houda Bouamor and Kemal Oflazer. 2016. Building an Arabic Machine Translation Post-Edited Corpus: Guidelines and Annotation. Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zaghouani2016building,
  title = {{Building an Arabic Machine Translation Post-Edited Corpus: Guidelines and Annotation}},
  author = {Wajdi Zaghouani, Nizar Habash, Ossama Obeid, Behrang Mohit, Houda Bouamor and Kemal Oflazer},
  booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)},
  year = {2016},
  doi = {10.63317/4wiuxtfyzc54},
  url = {https://doi.org/10.63317/4wiuxtfyzc54}
}
```

- [← previous Arabic WordNet: New Content and New Applications](https://md.research.elamur.ai/free/en/papers/regragui-2016-arabic-wordnet-new-content-new/)
- [next → Comparing exhaustivity operators](https://md.research.elamur.ai/free/en/papers/spector-2016-comparing-exhaustivity-operators/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/4/)

## References

18 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Avramidis, E., Burchardt, A., Hunsicker, S., Popovic, M., Tscherwinka, C., Torres, D. V., and Uszkoreit, H. (2014). The taraxu corpus of human-annotated machine translations. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC 2014), pages 2679–2682. European Language Resources Association (ELRA), 5. [find](https://scholar.google.com/scholar?q=Avramidis%2C+E%2E%2C+Burchardt%2C+A%2E%2C+Hunsicker%2C+S%2E%2C+Popovic%2C+M%2E%2C+Tscherwinka%2C+C%2E%2C+Torres%2C+D%2E+V%2E%2C+and+Uszkoreit%2C+H%2E+%282014%29%2E+The+taraxu+corpus+of+human%2Dannotated+machine+translations%2E+In+Proceedings+of+the+Nin)
2. Bouamor, H., Alshikhabobakr, H., Mohit, B., and Oflazer, K. (2014). [A human judgement corpus and a metric for arabic MT evaluation](https://md.research.elamur.ai/free/en/papers/bouamor-2014-human-judgement-corpus-metric-arabic/). In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 207–213.
3. El Kholy, A. and Habash, N. (2012). Orthographic and morphological processing for english–arabic statistical machine translation. Machine Translation, 26(1-2):25– 45. [↗](https://doi.org/10.1007/s10590-011-9110-0)
4. Elming, J. (2006). Transformation-based corrections of rule-based mt. In Proceedings of the EAMT 11th Annual Conference. [find](https://scholar.google.com/scholar?q=Elming%2C+J%2E+%282006%29%2E+Transformation%2Dbased+corrections+of+rule%2Dbased+mt%2E+In+Proceedings+of+the+EAMT+11th+Annual+Conference%2E)
5. Fishel, M., Bojar, O., and Popovic, M. (2012). Terra: a collection of translation error-annotated corpora. In Proceedings of the 8th International Conference on Language Resources and Evaluation. International Conference on Language Resources and Evaluation (LREC-12), 8th, May 23-25, Istanbul, Turkey. European Language Resources Association (ELRA), 5. [find](https://scholar.google.com/scholar?q=Fishel%2C+M%2E%2C+Bojar%2C+O%2E%2C+and+Popovic%2C+M%2E+%282012%29%2E+Terra%3A+a+collection+of+translation+error%2Dannotated+corpora%2E+In+Proceedings+of+the+8th+International+Conference+on+Language+Resources+and+Evaluation%2E+Inte)
6. Habash, N., Soudi, A., and Buckwalter, T. (2007). On Arabic Transliteration. In A. van den Bosch et al., editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Springer. [↗](https://doi.org/10.1007/978-1-4020-6046-5_2)
7. Habash, N. (2010). Introduction to Arabic Natural Language Processing. Morgan & Claypool Publishers. [↗](https://doi.org/10.1007/978-3-031-02139-8)
8. Mohit, B., Rozovskaya, A., Habash, N., Zaghouani, W., and Obeid, O. (2014). [The first qalb shared task on automatic text correction for arabic](https://md.research.elamur.ai/free/en/papers/mohit-2014-first-qalb-shared-task-automatic/). In Proceedings of the EMNLP Workshop on Arabic Natural Language Processing, page 39.
9. Obeid, O., Zaghouani, W., Mohit, B., Habash, N., Oflazer, K., and Tomeh, N. (2013). A Web-based Annotation Framework For Large-Scale Text Correction. In The Companion Volume of the Proceedings of IJCNLP 2013: System Demonstrations, Nagoya, Japan, October. [find](https://scholar.google.com/scholar?q=Obeid%2C+O%2E%2C+Zaghouani%2C+W%2E%2C+Mohit%2C+B%2E%2C+Habash%2C+N%2E%2C+Oflazer%2C+K%2E%2C+and+Tomeh%2C+N%2E+%282013%29%2E+A+Web%2Dbased+Annotation+Framework+For+Large%2DScale+Text+Correction%2E+In+The+Companion+Volume+of+the+Proceedings+of+IJCN)
10. Pasha, A., Al-Badrashiny, M., Kholy, A. E., Eskander, R., Diab, M., Habash, N., Pooleery, M., Rambow, O., and Roth, R. (2014). [MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic](https://md.research.elamur.ai/free/en/papers/pasha-2014-madamira-fast-comprehensive-tool-morphological/). In Proceedings of the 9th International Conference on Language Resources and Evaluation, Reykjavik, Iceland.
11. Popescu-belis, A., King, M., and Benantar, H. (2002). Towards a corpus of corrected human translations. [find](https://scholar.google.com/scholar?q=Popescu%2Dbelis%2C+A%2E%2C+King%2C+M%2E%2C+and+Benantar%2C+H%2E+%282002%29%2E+Towards+a+corpus+of+corrected+human+translations%2E)
12. Rozovskaya, A., Bouamor, H., Habash, N., Zaghouani, W., Obeid, O., and Mohit, B. (2015). [The second qalb shared task on automatic text correction for arabic](https://md.research.elamur.ai/free/en/papers/rozovskaya-2015-second-qalb-shared-task-automatic/). In Proceedings of the ACL-IJCNLP Workshop on Arabic Natural Language Processing, page 26.
13. Simard, M., Goutte, C., and Isabelle, P. (2007). [Statistical phrase-based post-editing](https://md.research.elamur.ai/free/en/papers/simard-2007-statistical-phrase-based-post-editing/). In Proceedings of the North American Association for Computational Linguistics: Human Language Technologies Conference.
14. Snover, M., Dorr, B., Schwartz, R., Micciulla, L., and Makhoul, J. (2006). [A study of translation edit rate with targeted human annotation](https://md.research.elamur.ai/free/en/papers/snover-2006-study-translation-edit-rate-targeted/). In Proceedings of AMTA, pages 223–231.
15. Wisniewski, G., Kubler, N., and Yvon, F. (2014). A corpus of machine translation errors extracted from translation students exercises. In International Conference on Language Resources and Evaluation (LREC 2014), page 4 pages, Reykjavik, Iceland, may. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Wisniewski%2C+G%2E%2C+Kubler%2C+N%2E%2C+and+Yvon%2C+F%2E+%282014%29%2E+A+corpus+of+machine+translation+errors+extracted+from+translation+students+exercises%2E+In+International+Conference+on+Language+Resources+and+Evaluation+)
16. Zaghouani, W., Habash, N., and Mohit, B. (2014a). The qatar arabic language bank guidelines. Technical Report CMU-CS-QTR-124, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, September. [find](https://scholar.google.com/scholar?q=Zaghouani%2C+W%2E%2C+Habash%2C+N%2E%2C+and+Mohit%2C+B%2E+%282014a%29%2E+The+qatar+arabic+language+bank+guidelines%2E+Technical+Report+CMU%2DCS%2DQTR%2D124%2C+School+of+Computer+Science%2C+Carnegie+Mellon+University%2C+Pittsburgh%2C+PA%2C+Se)
17. Zaghouani, W., Mohit, B., Habash, N., Obeid, O., Tomeh, N., Rozovskaya, A., Farra, N., Alkuhlani, S., and Oflazer, K. (2014b). [Large scale arabic error annotation: Guidelines and framework](https://md.research.elamur.ai/free/en/papers/zaghouani-2014-large-scale-arabic-error-annotation/). In International Conference on Language Resources and Evaluation (LREC 2014).
18. Zaghouani, W., Habash, N., Bouamor, H., Rozovskaya, A., Mohit, B., Heider, A., and Oflazer, K. (2015). [Correction annotation for non-native arabic texts: Guidelines and corpus](https://md.research.elamur.ai/free/en/papers/zaghouani-2015-correction-annotation-non-native-arabic/). In Proceedings of the Association for Computational Linguistics Fourth Linguistic Annotation Workshop, pages 129–139.

*Updated 1 November 2026*