---
title: "GW_QA at SemEval-2017 Task 3: Question Answer Re-ranking"
description: "The paper addresses Subtask D of SemEval-2017 Task 3. 14 references, cited by 1 paper here."
authors: ["Nada Almarwani", "Mona Diab"]
year: 2017
venue: "Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)"
source: "https://aclanthology.org/S17-2056/"
doi: "10.18653/v1/s17-2056"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/almarwani-2017-gw-qa-semeval-2017-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/almarwani-2017-gw-qa-semeval-2017-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/5/) / GW\_QA at SemEval-2017 Task 3: Question Answer Re-ranking on Arabic Fora

# GW\_QA at SemEval-2017 Task 3: Question Answer Re-ranking on Arabic Fora

**Authors:** [Nada Almarwani](https://md.research.elamur.ai/free/en/authors/nada-almarwani/) and [Mona Diab](https://md.research.elamur.ai/free/en/authors/mona-diab/) · **Year:** 2017 · **Venue:** [Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2017](https://md.research.elamur.ai/free/en/venues/semeval/2017/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/S17-2056/) · [PDF](https://aclanthology.org/S17-2056.pdf) · [DOI 10.18653/v1/s17-2056](https://doi.org/10.18653/v1/s17-2056) · [OpenAlex](https://openalex.org/W2752059660)

**Citations:** 9 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/almarwani-2017-gw-qa-semeval-2017-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses Subtask D of SemEval-2017 Task 3. Given a new question from an Arabic medical forum, a system must reorder a set of previously posted question–answer pairs so that the most related ones appear first. Arabic forum text is hard to process: the morphology is rich, word order is flexible, and posts mix Modern Standard Arabic with dialects and sometimes English. The GW_QA team cleaned the text with SPLIT and MADAMIRA and then built two kinds of similarity features. Latent semantic features came from a WTMF model trained on Arabic Gigaword plus unlabeled forum data. Lexical features measured set overlap and length differences. Both kinds were computed between the new question and the retrieved question, and between the new question and the retrieved answer. An SVM-rank model learned the ordering from these features. The submission placed first of three teams with a test MAP of 61.16. The result suggests that short-text latent models and simple matching features complement each other in low-resource, noisy question retrieval.

## Key points
- **Data:** the forum data has three relevance labels (Direct, Relevant, Irrelevant).
  - Training: 1,031 questions with 30,411 candidate pairs.
  - Development: 250 questions with 7,385 pairs.
  - Test: 1,400 questions with 8–9 candidates each.
- **Preprocessing:**
  - SPLIT detected URLs and punctuation, which were removed, and numbers and dates, which were replaced with Num/Date placeholders.
  - Alef and Yaa variants were unified.
  - MADAMIRA performed D3 tokenization, lemmatization and stemming, followed by stop-word removal.
- **WTMF model:** it used 500 dimensions over stems. Euclidean, Manhattan and cosine distances fed the ranker, together with overlap ratios at the token, lemma and stem levels.
- **Ranking results:**
  - The ranker was SVM-rank with a linear kernel and C=3, tuned on DEV.
  - WTMF+LEX reached DEV MAP 45.73, against 28.55 for the IR baseline.
  - On TEST it reached MAP 61.16, against 60.55 for the IR baseline. Each feature group alone scored about 59.2–59.3.
- **Unofficial binary run:** a LIBLINEAR logistic regression classifier scored F1 64.75 and accuracy 66.92. This was the best of all systems on every metric except precision.

## Interesting ideas
- WTMF suits sentence-length texts because it models absent vocabulary as well as present words, so a short text's profile reflects what it is not about too.
- Each candidate is scored by comparing the query separately with its question part and with its answer part, and the two feature sets are concatenated rather than merged.
- The same overlap measures are computed on surface tokens, lemmas and stems, which keeps matching robust to inflectional variation.
- Numbers and dates are replaced by placeholder classes before similarity is computed, so incidental values do not block a match.
- The error analysis traced failures to a genre mismatch between the latent model's training corpus and the test data, which raised out-of-vocabulary rates. It also traced failures to domain-irrelevant personal details, such as age and weight, acting as noise.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (world): Topic-level profiles of short passages could rank which knowledge-graph entries or fable passages are worth unfolding into a full world model, keeping the explainable pipeline for the final reasoning.
- **I2** (answer): A reader choosing supporting passages could keep separate match scores against a passage's stated question and its content, so the explanation shows which part justified the choice.
- **I3** (answer): Matching a question's frame arguments against the text at several normalization levels could recover answers phrased with different word forms, and each match level would be recorded as part of the explanation.
- **I4** (tokens): The tokenizer could attach a value-class tag (Num/Date) alongside the literal token, giving downstream frame matching a coarse slot while the exact value is kept for the world model.
- **I5** (learn): Evaluation reports could break errors down by OOV rate and by out-of-domain content to separate data-coverage problems from model faults.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [George Washington University](https://md.research.elamur.ai/free/en/universities/george-washington-university/) — University, Washington, United States · 51 works · 8,148 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (3): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 441 works · [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 473 works

## Cite

Cite as: Nada Almarwani and Mona Diab. 2017. GW\_QA at SemEval-2017 Task 3: Question Answer Re-ranking on Arabic Fora. Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{almarwani2017gw,
  title = {{GW\_QA at SemEval-2017 Task 3: Question Answer Re-ranking on Arabic Fora}},
  author = {Nada Almarwani and Mona Diab},
  booktitle = {Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)},
  year = {2017},
  doi = {10.18653/v1/s17-2056},
  url = {https://doi.org/10.18653/v1/s17-2056}
}
```

- [← previous Developing a concept-level knowledge base for sentiment analysis in Singlish](https://md.research.elamur.ai/free/en/papers/bajpai-2017-developing-concept-level-knowledge-base/)
- [next → Hierarchical Classification for Spoken Arabic Dialect Identification using Prosody: Case of Algerian Dialects](https://md.research.elamur.ai/free/en/papers/bougrine-2017-hierarchical-classification-spoken-arabic-dialect/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/5/)

## References

14 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. ldc2011t11. Philadelphia: Linguistic Data Consortium . [find](https://scholar.google.com/scholar?q=ldc2011t11%2E+Philadelphia%3A+Linguistic+Data+Consortium+%2E)
2. Mohamed Al-Badrashiny, Arfath Pasha, Mona Diab, Nizar Habash, Owen Rambow, Wael Salloum, and Ramy Eskander. 2016. Split: Smart preprocessing (quasi) language independent tool. In 10th International Conference on Language Resources and Evaluation (LREC’16). European Language Resources Association (ELRA), Portorož, Slovenia. [find](https://scholar.google.com/scholar?q=Mohamed+Al%2DBadrashiny%2C+Arfath+Pasha%2C+Mona+Diab%2C+Nizar+Habash%2C+Owen+Rambow%2C+Wael+Salloum%2C+and+Ramy+Eskander%2E+2016%2E+Split%3A+Smart+preprocessing+%28quasi%29+language+independent+tool%2E+In+10th+International+Co)
3. Arfath Pasha, Mohamed Al-Badrashiny, Mona T Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. [Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of arabic](https://md.research.elamur.ai/free/en/papers/pasha-2014-madamira-fast-comprehensive-tool-morphological/). In LREC. volume 14, pages 1094–1101.
4. PreslavNakov LluısMarquez WalidMagdy Alessandro-Moschitti, James Glass, and Bilal Randeree. 2015. Semeval-2015 task 3: Answer selection in community question answering. SemEval-2015 269. [find](https://scholar.google.com/scholar?q=PreslavNakov+Llu%C4%B1sMarquez+WalidMagdy+Alessandro%2DMoschitti%2C+James+Glass%2C+and+Bilal+Randeree%2E+2015%2E+Semeval%2D2015+task+3%3A+Answer+selection+in+community+question+answering%2E+SemEval%2D2015+269%2E)
5. Ian H Witten and Eibe Frank. 2005. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann. [find](https://scholar.google.com/scholar?q=Ian+H+Witten+and+Eibe+Frank%2E+2005%2E+Data+Mining%3A+Practical+machine+learning+tools+and+techniques%2E+Morgan+Kaufmann%2E)
6. Muhammad Asaduzzaman, Ahmed Shah Mashiyat, Chanchal K Roy, and Kevin A Schneider. 2013. Answering questions about unanswered questions of stack overflow. In Mining Software Repositories (MSR), 2013 10th IEEE Working Conference on. IEEE, pages 97–100. [↗](https://doi.org/10.11575/prism/35580)
7. Antoaneta Baltadzhieva and Grzegorz Chrupała. 2015. Question quality in community question answering forums: a survey. Acm Sigkdd Explorations Newsletter 17(1):8–13. [find](https://scholar.google.com/scholar?q=Antoaneta+Baltadzhieva+and+Grzegorz+Chrupa%C5%82a%2E+2015%2E+Question+quality+in+community+question+answering+forums%3A+a+survey%2E+Acm+Sigkdd+Explorations+Newsletter+17%281%29%3A8%E2%80%9313%2E)
8. David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. [Latent dirichlet allocation](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/). Journal of machine Learning research 3(Jan):993–1022. [↗](https://doi.org/10.5555/944919.944937)
9. Weiwei Guo and Mona Diab. 2012. Modeling sentences in the latent space. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-volume 1. Association for Computational Linguistics, pages 864–872. [↗](https://doi.org/10.7916/d81c256n)
10. Marti A. Hearst, Susan T Dumais, Edgar Osuna, John Platt, and Bernhard Scholkopf. 1998. Support vector machines. IEEE Intelligent Systems and their Applications 13(4):18–28. [↗](https://doi.org/10.1109/5254.708428)
11. Christopher D Manning, Prabhakar Raghavan, Hinrich Schütze, et al. 2008. Introduction to information retrieval, volume 1. Cambridge university press Cambridge. [find](https://scholar.google.com/scholar?q=Christopher+D+Manning%2C+Prabhakar+Raghavan%2C+Hinrich+Schu%CC%88tze%2C+et+al%2E+2008%2E+Introduction+to+information+retrieval%2C+volume+1%2E+Cambridge+university+press+Cambridge%2E)
12. Preslav Nakov, Doris Hoogeveen, Lluı́s Màrquez, Alessandro Moschitti, Hamdy Mubarak, Timothy Baldwin, and Karin Verspoor. 2017. [SemEval-2017 task 3: Community question answering](https://md.research.elamur.ai/free/en/papers/nakov-2017-semeval-2017-task-3-community/). In Proceedings of the 11th International Workshop on Semantic Evaluation. Association for Computational Linguistics, Vancouver, Canada, SemEval ’17.
13. Preslav Nakov, Lluı́s Màrquez, Alessandro Moschitti, Walid Magdy, Hamdy Mubarak, Abed Alhakim Freihat, Jim Glass, and Bilal Randeree. 2016. [SemEval-2016 task 3: Community question answering](https://md.research.elamur.ai/free/en/papers/nakov-2016-semeval-2016-task-3-community/). In Proceedings of the 10th International Workshop on Semantic Evaluation. Association for Computational Linguistics, San Diego, California, SemEval ’16.
14. Robert Parker, David Graff, Ke Chen, Junbo Kong, and Kazuaki Maeda. 2011. Arabic gigaword fifth edition [↗](https://doi.org/10.35111/p02g-rw14)

*Updated 1 November 2026*