---
title: "Language Modelling with NMT Query Translation for"
description: "The paper tackles cross-language search between Amharic and Arabic, two morphologically rich Semitic languages with few… 34 references."
authors: ["Ibrahim Gashaw", "H L Shashirekha"]
year: 2019
venue: "Proceedings of the 16th International Conference on Natural Language Processing"
source: "https://aclanthology.org/2019.icon-1.7/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/gashaw-2019-language-modelling-nmt-query-translation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/gashaw-2019-language-modelling-nmt-query-translation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/8/) / Language Modelling with NMT Query Translation for Amharic-Arabic Cross-Language Information Retrieval

# Language Modelling with NMT Query Translation for Amharic-Arabic Cross-Language Information Retrieval

**Authors:** [Ibrahim Gashaw](https://md.research.elamur.ai/free/en/authors/ibrahim-gashaw/) and [H L Shashirekha](https://md.research.elamur.ai/free/en/authors/h-l-shashirekha/) · **Year:** 2019 · **Venue:** [Proceedings of the 16th International Conference on Natural Language Processing](https://md.research.elamur.ai/free/en/venues/icon/) ([ICON 2019](https://md.research.elamur.ai/free/en/venues/icon/2019/)) · **Publisher:** [NLP Association of India](https://md.research.elamur.ai/free/en/publishers/nlp-association-of-india/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2019.icon-1.7/) · [PDF](https://aclanthology.org/2019.icon-1.7.pdf)

## Summary
The paper tackles cross-language search between Amharic and Arabic, two morphologically rich Semitic languages with few resources. An earlier dictionary-based system for this pair was limited by out-of-dictionary words and unnormalized Arabic diacritics. The authors instead translate Amharic queries into Arabic with a pre-trained NMT model. Because no Amharic-Arabic parallel data existed, they assembled a small one from monolingual Arabic Quran text and its Amharic translation on Tanzil. Translated queries are run against a mixed collection of Amharic and Arabic Quran chapters using four classic retrieval models: unigram LM, bigram LM, a probabilistic model, and VSM. With no benchmark available, they built their own test collection, with relevance judgments derived from agreement between the four systems' rankings. The unigram LM performed best on both language collections. The bigram LM was strongest at rank 1 but lost recall. Query length strongly affected the LM results. The work is an early baseline and evaluation setup for NMT-driven CLIR in an underserved language pair.

## Key points
- Test collection: 75 Amharic queries and 228 documents, made of the 114 Quran chapters in Arabic plus their 114 Amharic translations. It is formatted TREC-style with a graded 0–4 relevance scale and is available from the authors on request.
- Relevance pseudo-judgments: the unions of pairwise intersections of top-ranked lists from Unigram, Bigram, Probabilistic, and VSM runs are used. If fewer than 10 documents result, the list is topped up with unigram results.
- Metrics: MAP, Recall, and NDCG at positions 1, 5, and 10. DCG is normalized by an ideal ordering so scores are comparable across queries.
- Results by model:
  - The unigram LM is best overall.
  - The bigram LM has the highest NDCG@1 but notably lower recall.
  - The probabilistic model and VSM perform roughly alike.
- Stated limitations and future work:
  - The collection comes from a single domain (the Quran).
  - The authors plan larger multi-domain data and neural IR methods.

## Interesting ideas
- Translating only the short query, rather than the whole document collection, keeps the costly MT step small and leaves the index monolingual.
- Relevance labels built from where several independent systems overlap substitute for missing human judgments, but they risk circularity because the evaluated systems also define the gold standard.
- Unigram and bigram scoring trade off differently: local word order helps put the single best hit on top, while independence assumptions recover more relevant items overall.
- The earlier dictionary system failed partly because diacritics were not normalized before lexicon lookup, so surface-form normalization directly decides lexical coverage.
- Reporting NDCG at several cutoffs (1, 5, 10) next to recall shows behaviour that a single score hides.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (back): Translation through the meta-world could be tested first on short question-like inputs, where errors are easy to inspect, before scaling up to full texts.
- **I2** (learn): This mirrors silver data agreed by witness parsers, and it argues for always keeping a held-out human-checked slice or negative controls so a witness is never graded only against its own vote.
- **I3** (answer): QA readers could use an order-sensitive matcher for picking the top answer passage and a looser bag-style recall stage only as a candidate filter, keeping the final step structural.
- **I4** (tokens): Normalizing accents, variant spellings, and foreign-name forms before lemma and lexicon lookup in crate en could cut avoidable out-of-vocabulary misses.
- **I5** (learn): Evaluation reports could pair a top-1 correctness measure with a coverage measure, so a change that improves one while hurting the other is noticed.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works

🧭 Topics (4): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 1,105 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 882 works · [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 552 works

## Cite

Cite as: Ibrahim Gashaw and H L Shashirekha. 2019. Language Modelling with NMT Query Translation for Amharic-Arabic Cross-Language Information Retrieval. Proceedings of the 16th International Conference on Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{gashaw2019language,
  title = {{Language Modelling with NMT Query Translation for Amharic-Arabic Cross-Language Information Retrieval}},
  author = {Ibrahim Gashaw and H L Shashirekha},
  booktitle = {Proceedings of the 16th International Conference on Natural Language Processing},
  year = {2019},
  url = {https://aclanthology.org/2019.icon-1.7/}
}
```

- [← previous Historical linguistics and linguistic dating of the Hebrew Bible 2015–2018: The Rezetko and Young assessment of the status quaestionis](https://md.research.elamur.ai/free/en/papers/winther-nielsen-2019-historical-linguistics-linguistic-dating-hebrew/)
- [next → Linguistic-functional Analysis of the Biblical Hebrew Lexemes ʿaṭarâ, keter and nezer](https://md.research.elamur.ai/free/en/papers/stornaiuolo-2019-linguistic-functional-analysis-biblical-hebrew/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/8/)

## References

34 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mazin Al-Shuaili and Marco Carvalho. 2016. Character Mapping for Cross-Language. International Journal of Future Computer and Communication, 5(1):18. [find](https://scholar.google.com/scholar?q=Mazin+Al%2DShuaili+and+Marco+Carvalho%2E+2016%2E+Character+Mapping+for+Cross%2DLanguage%2E+International+Journal+of+Future+Computer+and+Communication%2C+5%281%29%3A18%2E)
2. Atelach Alemu Argaw, Lars Asker, Rickard Coster, Jussi Karlgren, and Magnus Sahlgren. 2005. Dictionary-based amharic-french information retrieval. In Workshop of the Cross-Language Evaluation Forum for European Languages, pages 83–92. Springer. [find](https://scholar.google.com/scholar?q=Atelach+Alemu+Argaw%2C+Lars+Asker%2C+Rickard+Coster%2C+Jussi+Karlgren%2C+and+Magnus+Sahlgren%2E+2005%2E+Dictionary%2Dbased+amharic%2Dfrench+information+retrieval%2E+In+Workshop+of+the+Cross%2DLanguage+Evaluation+Forum+fo)
3. Olivier Chapelle and Mingrui Wu. 2010. Gradient descent optimization of smoothed information retrieval metrics. Information retrieval, 13(3):216–235. [find](https://scholar.google.com/scholar?q=Olivier+Chapelle+and+Mingrui+Wu%2E+2010%2E+Gradient+descent+optimization+of+smoothed+information+retrieval+metrics%2E+Information+retrieval%2C+13%283%29%3A216%E2%80%93235%2E)
4. Andres Duque, Juan Martinez-Romo, and Lourdes Araujo. 2015. Choosing the best dictionary for Cross-Lingual Word Sense Disambiguation. Knowledge-Based Systems, 81:65–75. *Table 3: Models Evaluation results* Models Am-uni-gram Am-probability Am-bi-gram Am-VSM NDCG@1 0.9933 0.97 0.9733 0.5233 NDCG@5 0.6969 0.6185 0.4581 0.5208 NDCG@10 0.7497 0.6279 0.4426 0.6038 MAP 0.7866 0.5059 0.2296 0.5264 RECALL 0.8556 0.5867 0.2681 0.6504 Ar-uni-gram Ar-probability Ar-bi-gram Ar-VSM 0.98 0.89 0.9667 0.3233 0.7896 0.6812 0.54 0.4483 0.8455 0.6883 0.4827 0.5257 0.8202 0.5698 0.2725 0.4148 0.8637 0.64 0.2844 0.5704 [find](https://scholar.google.com/scholar?q=Andres+Duque%2C+Juan+Martinez%2DRomo%2C+and+Lourdes+Araujo%2E+2015%2E+Choosing+the+best+dictionary+for+Cross%2DLingual+Word+Sense+Disambiguation%2E+Knowledge%2DBased+Systems%2C+81%3A65%E2%80%9375%2E+%2ATable+3%3A+Models+Evaluation+res)
5. Miles Efron. 2009. Using multiple query aspects to build test collections without human relevance judgments. In European Conference on Information Retrieval, pages 276–287. Springer. [find](https://scholar.google.com/scholar?q=Miles+Efron%2E+2009%2E+Using+multiple+query+aspects+to+build+test+collections+without+human+relevance+judgments%2E+In+European+Conference+on+Information+Retrieval%2C+pages+276%E2%80%93287%2E+Springer%2E)
6. Wei Gao, Cheng Niu, Jian-Yun Nie, Ming Zhou, Jian Hu, Kam-Fai Wong, and Hsiao-Wuen Hon. 2007. Cross-lingual query suggestion using query logs of different languages. In Proceedings of the 30th annual international ACM SI-GIR conference on Research and development in information retrieval, pages 463–470. ACM. [↗](https://arxiv.org/abs/1602.02410)
7. Yoon Kim, Yacine Jernite, David Sontag, and Alexander M Rush. 2016. [Character-Aware Neural Language Models](https://md.research.elamur.ai/free/en/papers/kim-2015-character-aware-neural-language-models/). In AAAI, pages 2741– 2749.
8. Parnia Samimi and Sri Devi Ravana. 2014. Creation of reliable relevance judgments in information retrieval systems evaluation experimentation through crowdsourcing: a review. [find](https://scholar.google.com/scholar?q=Parnia+Samimi+and+Sri+Devi+Ravana%2E+2014%2E+Creation+of+reliable+relevance+judgments+in+information+retrieval+systems+evaluation+experimentation+through+crowdsourcing%3A+a+review%2E)
9. The Scientific World Journal, 2014. [find](https://scholar.google.com/scholar?q=The+Scientific+World+Journal%2C+2014%2E)
10. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M Rush. 2017. [Opennmt: Open-source toolkit for neural machine translation](https://md.research.elamur.ai/free/en/papers/klein-2017-opennmt-open-source-toolkit-neural/). arXiv preprint arXiv:1701.02810. [↗](https://arxiv.org/abs/1701.02810)
11. Kazuhiro Seki. 2018. Exploring neural translation models for cross-lingual text similarity. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pages 1591–1594. ACM. [find](https://scholar.google.com/scholar?q=Kazuhiro+Seki%2E+2018%2E+Exploring+neural+translation+models+for+cross%2Dlingual+text+similarity%2E+In+Proceedings+of+the+27th+ACM+International+Conference+on+Information+and+Knowledge+Management%2C+pages+1591%E2%80%93)
12. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature, 521(7553):436. [find](https://scholar.google.com/scholar?q=Yann+LeCun%2C+Yoshua+Bengio%2C+and+Geoffrey+Hinton%2E+2015%2E+Deep+learning%2E+nature%2C+521%287553%29%3A436%2E)
13. HL Shashirekha and Ibrahim Gashaw. 2016. Dictionary based amharic-arabic cross language information retrieval. In International Conference on Advances in Computer Science and Information Technology, pages 49–60. [find](https://scholar.google.com/scholar?q=HL+Shashirekha+and+Ibrahim+Gashaw%2E+2016%2E+Dictionary+based+amharic%2Darabic+cross+language+information+retrieval%2E+In+International+Conference+on+Advances+in+Computer+Science+and+Information+Technology%2C+p)
14. Qing Liu. 2018. A Neural Approach to Cross-Lingual Information Retrieval. Ph.D. thesis, figshare. [find](https://scholar.google.com/scholar?q=Qing+Liu%2E+2018%2E+A+Neural+Approach+to+Cross%2DLingual+Information+Retrieval%2E+Ph%2ED%2E+thesis%2C+figshare%2E)
15. Mangala Madankar, MB Chandak, and Nekita Chavhan. 2016. Information retrieval system and machine translation: a review. Procedia Computer Science, 78:845–850. [find](https://scholar.google.com/scholar?q=Mangala+Madankar%2C+MB+Chandak%2C+and+Nekita+Chavhan%2E+2016%2E+Information+retrieval+system+and+machine+translation%3A+a+review%2E+Procedia+Computer+Science%2C+78%3A845%E2%80%93850%2E)
16. Kumar Sourabh. 2013. An extensive literature review on clir and mt activities in india. International Journal of Scientific & Engineering Research. [find](https://scholar.google.com/scholar?q=Kumar+Sourabh%2E+2013%2E+An+extensive+literature+review+on+clir+and+mt+activities+in+india%2E+International+Journal+of+Scientific+%26+Engineering+Research%2E)
17. Eddy Maddalena, Marco Basaldella, Dario De Nart, Dante Degl’Innocenti, Stefano Mizzaro, and Gianluca Demartini. 2016. Crowdsourcing relevance assessments: The unexpected benefits of limiting the time to judge. In Fourth AAAI Conference on Human Computation and Crowdsourcing. [find](https://scholar.google.com/scholar?q=Eddy+Maddalena%2C+Marco+Basaldella%2C+Dario+De+Nart%2C+Dante+Degl%E2%80%99Innocenti%2C+Stefano+Mizzaro%2C+and+Gianluca+Demartini%2E+2016%2E+Crowdsourcing+relevance+assessments%3A+The+unexpected+benefits+of+limiting+the+time+)
18. EAGLES SWLG. 1997. Types of language models. [find](https://scholar.google.com/scholar?q=EAGLES+SWLG%2E+1997%2E+Types+of+language+models%2E)
19. Aynalem Tesfaye and Kevin Scannell. 2012. Amharic–English Cross-lingual Information Retrieval: A Corpus Based Approach. Haramaya: Haramaya University. [find](https://scholar.google.com/scholar?q=Aynalem+Tesfaye+and+Kevin+Scannell%2E+2012%2E+Amharic%E2%80%93English+Cross%2Dlingual+Information+Retrieval%3A+A+Corpus+Based+Approach%2E+Haramaya%3A+Haramaya+University%2E)
20. MemoQ. 2019. 5 Translation Technology Trends to Watch Out for in 2019. [find](https://scholar.google.com/scholar?q=MemoQ%2E+2019%2E+5+Translation+Technology+Trends+to+Watch+Out+for+in+2019%2E)
21. Fasika Tesfaye. 2010. Phrasal Translation for Amharic English Cross Language Information Retrieval (Clir). Ph.D. thesis, AAU. [find](https://scholar.google.com/scholar?q=Fasika+Tesfaye%2E+2010%2E+Phrasal+Translation+for+Amharic+English+Cross+Language+Information+Retrieval+%28Clir%29%2E+Ph%2ED%2E+thesis%2C+AAU%2E)
22. Bhaskar Mitra and Nick Craswell. 2017. Neural models for information retrieval. arXiv preprint arXiv:1705.01509. [↗](https://arxiv.org/abs/1705.01509)
23. Jörg Tiedemann. 2012. Parallel Data, Tools and Interfaces in OPUS. In Lrec, volume 2012, pages 2214–2218. [find](https://scholar.google.com/scholar?q=J%C3%B6rg+Tiedemann%2E+2012%2E+Parallel+Data%2C+Tools+and+Interfaces+in+OPUS%2E+In+Lrec%2C+volume+2012%2C+pages+2214%E2%80%932218%2E)
24. Alistair Moffat and Justin Zobel. 2008. Rank-biased precision for measurement of retrieval effectiveness. ACM Transactions on Information Systems (TOIS), 27(1):2. [find](https://scholar.google.com/scholar?q=Alistair+Moffat+and+Justin+Zobel%2E+2008%2E+Rank%2Dbiased+precision+for+measurement+of+retrieval+effectiveness%2E+ACM+Transactions+on+Information+Systems+%28TOIS%29%2C+27%281%29%3A2%2E)
25. Kula Kekeba Tune. 2015. Development of Cross-Language Information Retrieval for Resource-Scarce African Languages. Ph.D. thesis, International Institute of Information Technology, Hyderabad. [find](https://scholar.google.com/scholar?q=Kula+Kekeba+Tune%2E+2015%2E+Development+of+Cross%2DLanguage+Information+Retrieval+for+Resource%2DScarce+African+Languages%2E+Ph%2ED%2E+thesis%2C+International+Institute+of+Information+Technology%2C+Hyderabad%2E)
26. Mequannint Munye and Solomon Atnafu. 2012. Amharic-English bilingual web search engine. In Proceedings of the International Conference on Management of Emergent Digital EcoSystems, pages 32–39. ACM. [find](https://scholar.google.com/scholar?q=Mequannint+Munye+and+Solomon+Atnafu%2E+2012%2E+Amharic%2DEnglish+bilingual+web+search+engine%2E+In+Proceedings+of+the+International+Conference+on+Management+of+Emergent+Digital+EcoSystems%2C+pages+32%E2%80%9339%2E+ACM%2E)
27. Ferhan Türe, Jimmy J Lin, and Douglas W Oard. 2012. Combining Statistical Translation Techniques for Cross-Language Information Retrieval. In COLING, pages 2685–2702. [find](https://scholar.google.com/scholar?q=Ferhan+T%C3%BCre%2C+Jimmy+J+Lin%2C+and+Douglas+W+Oard%2E+2012%2E+Combining+Statistical+Translation+Techniques+for+Cross%2DLanguage+Information+Retrieval%2E+In+COLING%2C+pages+2685%E2%80%932702%2E)
28. Jian-Yun Nie. 2010. Cross-language information retrieval. Synthesis Lectures on Human Language Technologies, 3(1):1–125. [find](https://scholar.google.com/scholar?q=Jian%2DYun+Nie%2E+2010%2E+Cross%2Dlanguage+information+retrieval%2E+Synthesis+Lectures+on+Human+Language+Technologies%2C+3%281%29%3A1%E2%80%93125%2E)
29. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. [↗](https://arxiv.org/abs/1609.08144)
30. Eyob Nigussie. 2013. Afaan Oromo–Amharic Cross Lingual Information Retrieval. Ph.D. thesis, AAU. [find](https://scholar.google.com/scholar?q=Eyob+Nigussie%2E+2013%2E+Afaan+Oromo%E2%80%93Amharic+Cross+Lingual+Information+Retrieval%2E+Ph%2ED%2E+thesis%2C+AAU%2E)
31. BNV Narasimha Raju, MSVS Bhadri Raju, and KVV Satyanarayana. 2014. Translation approaches in cross language information retrieval. In International Conference on Computing and Communication Technologies, pages 1–4. IEEE. [find](https://scholar.google.com/scholar?q=BNV+Narasimha+Raju%2C+MSVS+Bhadri+Raju%2C+and+KVV+Satyanarayana%2E+2014%2E+Translation+approaches+in+cross+language+information+retrieval%2E+In+International+Conference+on+Computing+and+Communication+Technologi)
32. Chengxiang Zhai and John Lafferty. 2017. A study of smoothing methods for language models applied to ad hoc information retrieval. In ACM SIGIR Forum, volume 51, pages 268–276. ACM. [find](https://scholar.google.com/scholar?q=Chengxiang+Zhai+and+John+Lafferty%2E+2017%2E+A+study+of+smoothing+methods+for+language+models+applied+to+ad+hoc+information+retrieval%2E+In+ACM+SIGIR+Forum%2C+volume+51%2C+pages+268%E2%80%93276%2E+ACM%2E)
33. Bing Zhou and Yiyu Yao. 2010. Evaluating information retrieval system performance based on user preference. Journal of Intelligent Information Systems, 34(3):227–248. [find](https://scholar.google.com/scholar?q=Bing+Zhou+and+Yiyu+Yao%2E+2010%2E+Evaluating+information+retrieval+system+performance+based+on+user+preference%2E+Journal+of+Intelligent+Information+Systems%2C+34%283%29%3A227%E2%80%93248%2E)
34. Sri Devi Ravana, Prabha Rajagopal, and Vimala Balakrishnan. 2015. Ranking retrieval systems using pseudo relevance judgments. Aslib Journal of Information Management, 67(6):700–714. [find](https://scholar.google.com/scholar?q=Sri+Devi+Ravana%2C+Prabha+Rajagopal%2C+and+Vimala+Balakrishnan%2E+2015%2E+Ranking+retrieval+systems+using+pseudo+relevance+judgments%2E+Aslib+Journal+of+Information+Management%2C+67%286%29%3A700%E2%80%93714%2E)

*Updated 2 November 2026*