---
title: "Answer Selection in Arabic Community Question Answering: A"
description: "The paper tackles answer selection in Arabic community question answering. 79.25 F1. 24 references, cited by 1 paper here."
authors: ["Yonatan Belinkov", "Alberto Barrón-Cedeño", "Hamdy Mubarak"]
year: 2015
venue: "Proceedings of the Second Workshop on Arabic Natural Language Processing"
source: "https://aclanthology.org/W15-3223/"
doi: "10.18653/v1/w15-3223"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/belinkov-2015-answer-selection-arabic-community-question/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/belinkov-2015-answer-selection-arabic-community-question/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/page/4/) / Answer Selection in Arabic Community Question Answering: A Feature-Rich Approach

# Answer Selection in Arabic Community Question Answering: A Feature-Rich Approach

**Authors:** [Yonatan Belinkov](https://md.research.elamur.ai/free/en/authors/yonatan-belinkov/), [Alberto Barrón-Cedeño](https://md.research.elamur.ai/free/en/authors/alberto-barron-cedeno/) and [Hamdy Mubarak](https://md.research.elamur.ai/free/en/authors/hamdy-mubarak/) · **Year:** 2015 · **Venue:** [Proceedings of the Second Workshop on Arabic Natural Language Processing](https://md.research.elamur.ai/free/en/venues/workshop-on-arabic-natural-language-processing/) ([Workshop on Arabic Natural Language Processing 2015](https://md.research.elamur.ai/free/en/venues/workshop-on-arabic-natural-language-processing/2015/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W15-3223/) · [PDF](https://aclanthology.org/W15-3223.pdf) · [DOI 10.18653/v1/w15-3223](https://doi.org/10.18653/v1/w15-3223) · [OpenAlex](https://openalex.org/W2251866374)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/belinkov-2015-answer-selection-arabic-community-question/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper tackles answer selection in Arabic community question answering. Given a forum question and its attached comments, each comment must be labelled DIRECT, RELATED or IRRELEVANT. The authors use the SemEval-2015 Task 3 Arabic setup, built on the Fatwa corpus of religious questions answered by scholars. Every question there carries five comments, and only one of them is the true answer. They treat each question–comment pair as one training instance for a linear SVM. The classifier combines four feature families:
- averaged lemma embeddings for question and comment;
- a battery of lexical n-gram similarity scores;
- binned scores from an SVM ranker;
- the output of their own rule-based ranking system.

The best mix, embeddings plus rule-based ranking, reaches 79.25 macro-F1 on the test set. This beats the competition winner's 78.55. An error analysis shows that most mistakes involve RELATED comments that share little or only indirect vocabulary with the question. The work matters because it shows that cheap, heterogeneous features and existing Arabic preprocessing tools can set a new state of the art.

## Key points
- **Data:** the Fatwa corpus (from fatwa.islamweb.net) has 1,300 training, 200 dev and 200 test questions, each with five comments. RELATED comments come from linked related questions, and IRRELEVANT ones are sampled at random.
- **Metrics:** the official measures are macro-averaged F1 and accuracy. Macro averaging stops the IRRELEVANT class, which is twice as frequent as the others, from dominating the score.
- **Embeddings:** 100-dimensional Word2Vec vectors were trained on Arabic Gigaword, lemmatized with MADAMIRA. These worked better than vectors from KSUCCA, a classical-Arabic corpus closer to the domain, probably because Gigaword offers wider coverage.
- **Similarity features:** computed on word 1–4-grams using greedy string tiling, longest common subsequence, Jaccard, containment and cosine. The rule-based ranker relies on QATARA stemming.
- **Results:** vectors combined with rule-based ranking give 79.25 F1 and 83.42 accuracy on test. A heuristic that turned a full-feature ranker's order into labels won on dev but did worse on test.

## Interesting ideas
- **Bin ranking scores.** A continuous ranker score is rescaled to [0,1] and split into 10 bins, each with its own binary feature. This lets a linear model learn a non-linear response to a confidence signal.
- **Use a rule system's output as features.** The hand-written rule ranker is not a competitor to the learned model; its binarized decision becomes input to the classifier, and that combination wins.
- **Concatenate instead of compressing to a cosine.** The question and answer representations are placed side by side rather than reduced to one similarity number, so the classifier can see which dimensions matter.
- **Watch for easy artificial negatives.** The authors point out that randomly sampled IRRELEVANT comments make that class almost trivial, so scores can look better than real difficulty warrants.
- **Label errors by overlap pattern.** Errors are sorted into under-specification, over-specification and indirect overlap through synonyms or lemmas. Each category points to a specific remedy.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): parser or witness confidence scores could be binned the same way before they feed a linear repair or arbitration model, while each bin stays readable as a rule condition.
- **I2** (answer): the output of a symbolic answer reader can be fed as an explicit, traceable feature into a learned answer selector rather than replacing it.
- **I3** (frames): when matching a question frame to candidate text frames, keep role-by-role comparison slots instead of one overall match score, so the explanation shows which role failed.
- **I4** (learn): negative controls in evaluation should include hard, topically close distractors, not only random ones, or the gains will be inflated.
- **I5** (world): QA failures can be tagged the same way, so that misses caused by synonymy are routed to the WordNet-based ontology layer rather than to the parser.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 243 works · 76,506 citations
- [MIT Computer Science and Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/mit-computer-science-and-artificial-intelligence-laboratory/) — Research institute, Cambridge, United States · 76 works · 32,207 citations
- [Hamad bin Khalifa University](https://md.research.elamur.ai/free/en/universities/hamad-bin-khalifa-university/) — University, Doha, Qatar · 144 works · 14,175 citations
- [Qatar Foundation](https://md.research.elamur.ai/free/en/ngos/qatar-foundation/) — Nonprofit, Doha, Qatar · 17 works · 2,111 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 487 works

## Cite

Cite as: Yonatan Belinkov, Alberto Barrón-Cedeño and Hamdy Mubarak. 2015. Answer Selection in Arabic Community Question Answering: A Feature-Rich Approach. Proceedings of the Second Workshop on Arabic Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{belinkov2015answer,
  title = {{Answer Selection in Arabic Community Question Answering: A Feature-Rich Approach}},
  author = {Yonatan Belinkov, Alberto Barrón-Cedeño and Hamdy Mubarak},
  booktitle = {Proceedings of the Second Workshop on Arabic Natural Language Processing},
  year = {2015},
  doi = {10.18653/v1/w15-3223},
  url = {https://doi.org/10.18653/v1/w15-3223}
}
```

- [← previous A Frobenius Model of Information Structure in Categorical Compositional Distributional Semantics](https://md.research.elamur.ai/free/en/papers/kartsaklis-2015-frobenius-model-information-structure-categorical/)
- [next → Generating acceptable Arabic Core Vocabularies and Symbols for AAC users](https://md.research.elamur.ai/free/en/papers/draffan-2015-generating-acceptable-arabic-core-vocabularies/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/page/4/)

## References

24 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Lloyd Allison and Trevor Dix. 1986. A bit-string longest-common-subsequence algorithm. Inf. Process. Lett., 23(6):305–310, December. [find](https://scholar.google.com/scholar?q=Lloyd+Allison+and+Trevor+Dix%2E+1986%2E+A+bit%2Dstring+longest%2Dcommon%2Dsubsequence+algorithm%2E+Inf%2E+Process%2E+Lett%2E%2C+23%286%29%3A305%E2%80%93310%2C+December%2E)
2. Maha Alrabiah, AbdulMalik Al-Salman, and Eric Atwell. 2013. The design and construction of the 50 million words KSUCCA King Saud University Corpus of Classical Arabic. In Proceedings of WACL-2. [find](https://scholar.google.com/scholar?q=Maha+Alrabiah%2C+AbdulMalik+Al%2DSalman%2C+and+Eric+Atwell%2E+2013%2E+The+design+and+construction+of+the+50+million+words+KSUCCA+King+Saud+University+Corpus+of+Classical+Arabic%2E+In+Proceedings+of+WACL%2D2%2E)
3. Yonatan Belinkov, Mitra Mohtarami, Scott Cyphers, and James Glass. 2015. VectorSLU: A continuous word vector approach to answer selection in community question answering systems. In Proceedings of the 9th International Workshop on Semantic Evaluation, SemEval ’15, Denver, Colorado, USA. [find](https://scholar.google.com/scholar?q=Yonatan+Belinkov%2C+Mitra+Mohtarami%2C+Scott+Cyphers%2C+and+James+Glass%2E+2015%2E+VectorSLU%3A+A+continuous+word+vector+approach+to+answer+selection+in+community+question+answering+systems%2E+In+Proceedings+of+the)
4. Jiang Bian, Yandong Liu, Eugene Agichtein, and Hongyuan Zha. 2008. Finding the Right Facts in the Crowd: Factoid Question Answering over Social Media. In Proceedings of the 17th International Conference on World Wide Web, WWW ’08, pages 467–476, New York, NY, USA. ACM. [find](https://scholar.google.com/scholar?q=Jiang+Bian%2C+Yandong+Liu%2C+Eugene+Agichtein%2C+and+Hongyuan+Zha%2E+2008%2E+Finding+the+Right+Facts+in+the+Crowd%3A+Factoid+Question+Answering+over+Social+Media%2E+In+Proceedings+of+the+17th+International+Conferen)
5. Kareem Darwish and Walid Magdy. 2013. Arabic information retrieval. Foundations and Trends R in Information Retrieval, 7(4):239–342. [find](https://scholar.google.com/scholar?q=Kareem+Darwish+and+Walid+Magdy%2E+2013%2E+Arabic+information+retrieval%2E+Foundations+and+Trends+R+in+Information+Retrieval%2C+7%284%29%3A239%E2%80%93342%2E)
6. Kareem Darwish, Ahmed Abdelali, and Hamdy Mubarak. 2014. [Using Stem-Templates to Improve Arabic POS and Gender/Number Tagging](https://md.research.elamur.ai/free/en/papers/darwish-2014-stem-templates-improve-arabic-pos/). In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014). European Language Resources Association (ELRA).
7. Ahmed Ezzeldin and Mohamed Shaheen. 2012. A survey of Arabic question answering: Challenges, tasks, approaches, tools, and future trends. In Proceedings of the International Arabic Conference on Information Technology (ACIT). [find](https://scholar.google.com/scholar?q=Ahmed+Ezzeldin+and+Mohamed+Shaheen%2E+2012%2E+A+survey+of+Arabic+question+answering%3A+Challenges%2C+tasks%2C+approaches%2C+tools%2C+and+future+trends%2E+In+Proceedings+of+the+International+Arabic+Conference+on+Infor)
8. Yongshuai Hou, Cong Tan, Xiaolong Wang, Yaoyun Zhang, Jun Xu, and Qingcai Chen. 2015. HITSZ-ICRC: Exploiting classification approach for answer selection in community question answering. In Proceedings of the 9th International Workshop on Semantic Evaluation, SemEval ’15, Denver, Colorado, USA. [find](https://scholar.google.com/scholar?q=Yongshuai+Hou%2C+Cong+Tan%2C+Xiaolong+Wang%2C+Yaoyun+Zhang%2C+Jun+Xu%2C+and+Qingcai+Chen%2E+2015%2E+HITSZ%2DICRC%3A+Exploiting+classification+approach+for+answer+selection+in+community+question+answering%2E+In+Proceeding)
9. Paul Jaccard. 1901. Étude comparative de la distribution florale dans une portion des Alpes et des Jura. Bulletin del la Société Vaudoise des Sciences Naturelles, 37:547–579. [find](https://scholar.google.com/scholar?q=Paul+Jaccard%2E+1901%2E+E%CC%81tude+comparative+de+la+distribution+florale+dans+une+portion+des+Alpes+et+des+Jura%2E+Bulletin+del+la+Socie%CC%81te%CC%81+Vaudoise+des+Sciences+Naturelles%2C+37%3A547%E2%80%93579%2E)
10. Thorsten Joachims. 1999. Making Large-scale Support Vector Machine Learning Practical. In Bernhard Schölkopf, Christopher J. C. Burges, and Alexander J. Smola, editors, Advances in Kernel Methods, pages 169–184. MIT Press, Cambridge, MA, USA. [find](https://scholar.google.com/scholar?q=Thorsten+Joachims%2E+1999%2E+Making+Large%2Dscale+Support+Vector+Machine+Learning+Practical%2E+In+Bernhard+Scho%CC%88lkopf%2C+Christopher+J%2E+C%2E+Burges%2C+and+Alexander+J%2E+Smola%2C+editors%2C+Advances+in+Kernel+Methods%2C+pa)
11. Thorsten Joachims. 2002. Optimizing search engines using clickthrough data. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’02, pages 133–142, New York, NY, USA. ACM. [find](https://scholar.google.com/scholar?q=Thorsten+Joachims%2E+2002%2E+Optimizing+search+engines+using+clickthrough+data%2E+In+Proceedings+of+the+Eighth+ACM+SIGKDD+International+Conference+on+Knowledge+Discovery+and+Data+Mining%2C+KDD+%E2%80%9902%2C+pages+133%E2%80%93)
12. Thorsten Joachims. 2006. Training linear svms in linear time. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’06, pages 217–226, New York, NY, USA. ACM. [find](https://scholar.google.com/scholar?q=Thorsten+Joachims%2E+2006%2E+Training+linear+svms+in+linear+time%2E+In+Proceedings+of+the+12th+ACM+SIGKDD+International+Conference+on+Knowledge+Discovery+and+Data+Mining%2C+KDD+%E2%80%9906%2C+pages+217%E2%80%93226%2C+New+York%2C+N)
13. Linguistic Data Consortium. 2011. Arabic Gigaword Fifth Edition. [https://catalog.ldc.upenn.edu/LDC2011T11](https://catalog.ldc.upenn.edu/LDC2011T11). [↗](https://catalog.ldc.upenn.edu/LDC2011T11)
14. Caroline Lyon, James Malcolm, and Bob Dickerson. 2001. Detecting short passages of similar text in large document collections. In Proceedings of the 2001 Conference on Empirical Methods in Natural Language Processing, EMNLP ’01, pages 118–125, Pittsburgh, PA, USA. [find](https://scholar.google.com/scholar?q=Caroline+Lyon%2C+James+Malcolm%2C+and+Bob+Dickerson%2E+2001%2E+Detecting+short+passages+of+similar+text+in+large+document+collections%2E+In+Proceedings+of+the+2001+Conference+on+Empirical+Methods+in+Natural+Lan)
15. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013a. [Distributed Representations of Words and Phrases and their Compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). In C.J.C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, pages 3111–3119. Curran Associates, Inc.
16. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013b. [Linguistic Regularities in Continuous Space Word Representations](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/). In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT ’13, pages 746–751, Atlanta, GA, USA.
17. Reham Mohamed, Maha Ragab, Heba Abdelnasser, Nagwa M. El-Makky, and Marwan Torki. 2015. [Al-Bayan: A knowledge-based system for Arabic answer selection](https://md.research.elamur.ai/free/en/papers/mohamed-2015-al-bayan-knowledge-based-system/). In Proceedings of the 9th International Workshop on Semantic Evaluation, SemEval ’15, Denver, Colorado, USA.
18. Preslav Nakov, Lluı́s Màrquez, Walid Magdy, Alessandro Moschitti, Jim Glass, and Bilal Randeree. 2015. Semeval-2015 task 3: Answer selection in community question answering. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 269–281, Denver, Colorado, June. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Preslav+Nakov%2C+Llu%C4%B1%CC%81s+Ma%CC%80rquez%2C+Walid+Magdy%2C+Alessandro+Moschitti%2C+Jim+Glass%2C+and+Bilal+Randeree%2E+2015%2E+Semeval%2D2015+task+3%3A+Answer+selection+in+community+question+answering%2E+In+Proceedings+of+the+9th)
19. Kevin K. Nam, Ackerman Mark S. , and Lada A. Adamic. 2009. Questions in, Knowledge iN?: A study of Navers question answering community. In Proceedings of the SIG CHI Conference on Human Factors in Computing Systems, Boston, MA. [find](https://scholar.google.com/scholar?q=Kevin+K%2E+Nam%2C+Ackerman+Mark+S%2E+%2C+and+Lada+A%2E+Adamic%2E+2009%2E+Questions+in%2C+Knowledge+iN%3F%3A+A+study+of+Navers+question+answering+community%2E+In+Proceedings+of+the+SIG+CHI+Conference+on+Human+Factors+in+Com)
20. Massimo Nicosia, Simone Filice, Alberto Barrón-Cedeño, Iman Saleh, Hamdy Mubarak, Wei Gao, Preslav Nakov, Giovanni Da San Martino, Alessandro Moschitti, Kareem Darwish, Lluı́s Màrquez, Shafiq Joty, and Walid Magdy. 2015. [QCRI: Answer selection for community question answering - experiments for Arabic and English](https://md.research.elamur.ai/free/en/papers/nicosia-2015-qcri-answer-selection-community-question/). In Proceedings of the 9th International Workshop on Semantic Evaluation, SemEval ’15, Denver, Colorado, USA.
21. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. [MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic](https://md.research.elamur.ai/free/en/papers/pasha-2014-madamira-fast-comprehensive-tool-morphological/). In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1094–1101, Reykjavik, Iceland.
22. Anselmo Peñas, Eduard Hovy, Pamela Forner, Álvaro Rodrigo, Richard Sutcliffe, Caroline Sporleder, Corina Forascu, Yassine Benajiba, and Petya Osenova. 2012. Overview of QA4MRE at CLEF 2012: Question Answering for Machine Reading Evaluation. Working Notes, CLEF. [find](https://scholar.google.com/scholar?q=Anselmo+Pen%CC%83as%2C+Eduard+Hovy%2C+Pamela+Forner%2C+A%CC%81lvaro+Rodrigo%2C+Richard+Sutcliffe%2C+Caroline+Sporleder%2C+Corina+Forascu%2C+Yassine+Benajiba%2C+and+Petya+Osenova%2E+2012%2E+Overview+of+QA4MRE+at+CLEF+2012%3A+Question)
23. Richard Sutcliffe, Anselmo Peñas, Eduard Hovy, Pamela Forner, Álvaro Rodrigo, Corina Forascu, Yassine Benajiba, and Petya Osenova. 2013. Overview of QA4MRE Main Task at CLEF 2013. Working Notes, CLEF. [find](https://scholar.google.com/scholar?q=Richard+Sutcliffe%2C+Anselmo+Pen%CC%83as%2C+Eduard+Hovy%2C+Pamela+Forner%2C+A%CC%81lvaro+Rodrigo%2C+Corina+Forascu%2C+Yassine+Benajiba%2C+and+Petya+Osenova%2E+2013%2E+Overview+of+QA4MRE+Main+Task+at+CLEF+2013%2E+Working+Notes%2C+CLE)
24. Michael Wise. 1996. Yap3: Improved detection of similarities in computer program and other texts. In Proceedings of the Twenty-seventh SIGCSE Technical Symposium on Computer Science Education, SIGCSE ’96, pages 130–134, New York, NY, USA. [find](https://scholar.google.com/scholar?q=Michael+Wise%2E+1996%2E+Yap3%3A+Improved+detection+of+similarities+in+computer+program+and+other+texts%2E+In+Proceedings+of+the+Twenty%2Dseventh+SIGCSE+Technical+Symposium+on+Computer+Science+Education%2C+SIGCSE+)

*Updated 2 November 2026*