---
title: "Effects of Stop Words Elimination for Arabic Information"
description: "The paper asks whether removing stop words helps Arabic information retrieval, and which kind of… 23 references, cited by 3 papers here."
authors: ["Ibrahim Abu El-Khair"]
year: 2006
venue: "arXiv:1702.01925 · Abu El-Khair, Ibrahim. (2006). Effects of stop words elimination for Arabic information retrieval: a comparative study. International Journal of Computing & Information Sciences 4 (3), 119-133"
source: "https://arxiv.org/abs/1702.01925"
doi: "10.48550/arxiv.1702.01925"
arxiv: "1702.01925"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/el-khair-2017-effects-stop-words-elimination-arabic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/el-khair-2017-effects-stop-words-elimination-arabic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2006](https://md.research.elamur.ai/free/en/papers/2006/page/2/) / Effects of Stop Words Elimination for Arabic Information Retrieval: A Comparative Study

# Effects of Stop Words Elimination for Arabic Information Retrieval: A Comparative Study

**Authors:** [Ibrahim Abu El-Khair](https://md.research.elamur.ai/free/en/authors/ibrahim-abu-el-khair/) · **Year:** 2006 · **Venue:** arXiv:1702.01925 · Abu El-Khair, Ibrahim. (2006). Effects of stop words elimination for Arabic information retrieval: a comparative study. International Journal of Computing & Information Sciences 4 (3), 119-133 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1702.01925) · [PDF](https://arxiv.org/pdf/1702.01925) · [DOI 10.48550/arxiv.1702.01925](https://doi.org/10.48550/arxiv.1702.01925) · [arXiv 1702.01925](https://arxiv.org/abs/1702.01925) · [OpenAlex](https://openalex.org/W4297672137)

**Citations:** 22 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/el-khair-2017-effects-stop-words-elimination-arabic/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks whether removing stop words helps Arabic information retrieval, and which kind of stoplist works best. Arabic IR research had no agreed-upon standard list. The author built three lists. A general list comes from Arabic grammatical classes. A corpus-based list comes from frequency counts. A third list combines the two. Each list was paired with three ranking models in the Lemur Toolkit: TF*IDF, Okapi BM25, and a KL-divergence language model with Dirichlet smoothing. Stemming was turned off on purpose, so that only the stopword effect would be measured. Tests ran on the LDC Arabic Newswire A collection with the 75 TREC 2001/2002 Arabic topics, scored by recall/precision and Wilcoxon tests. Without stoplists, the three models performed almost the same. Stoplists changed TF*IDF and KL very little, and KL actually lost some precision. BM25, by contrast, gained clearly, and the general list gave the largest boost. The author recommends the grammar-based list as a reusable, corpus-independent stoplist for Arabic and plans to add it to Lemur.

## Key points
- Corpus: LDC Arabic Newswire A, made of AFP articles from 1994 to 2000. It has 383,872 documents, about 76M tokens and roughly 666K distinct words. Queries are 75 TREC topics with pooled relevance judgments.
- General stoplist: 1,377 words drawn from Arabic syntactic categories such as prepositions, pronouns, adverbs, relative and interrogative pronouns, and "transformers".
  - Some entries were expanded with the prefixes ل ك ف ب أ.
  - The conjunction waw was deliberately left out, because it can attach to any word.
- Corpus-based stoplist: every word seen more than 25,000 times, giving 359 words. The author admits this cut-off is somewhat arbitrary.
- Settings: Lemur defaults were used, including BM25 k1=1.2, b=0.75 and a Dirichlet prior of 2000. Tuning attempts did not beat these defaults.
- Results: BM25 gained 7.67% with the corpus list, 9.49% with the combined list and 10.44% with the general list.
  - The best run, BM25 with the general list, reached a mean precision of 0.2496 against a TF*IDF baseline. It was significant at p=0.018 and beat the baseline on 47 of 75 queries.
  - For TF*IDF and KL, stoplists shifted results by only about 0.1–0.2%.

## Interesting ideas
- Choose function words from closed grammatical classes rather than from raw frequency, since frequency lists pull in frequent content words like "United", "States" or "Cairo"
- Switch off a strong confounding component (here, stemming) so that one factor's effect is measured cleanly
- The same preprocessing helped BM25 but hurt the KL model, so a component's value depends on what consumes it
- Results were reported as per-query win/tie/loss counts against a baseline plus a Wilcoxon signed-rank test, not just mean scores
- Clitic-like letters were not blindly stripped, because they can belong to the word itself; attachment was left to a separate morphological step

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): keep a curated closed-class lexicon keyed to UD categories (ADP, PRON, DET, SCONJ, AUX), and use it to flag tagger outputs that give function tags to frequent content words.
- **I2** (learn): run one-component ablations, for example the parser without morphology features, before crediting any single stage with a gain.
- **I3** (learn): evaluate components as a grid of component × downstream consumer (parser, frame builder, reader) instead of in isolation, since a tree "fix" may help frames but break answers.
- **I4** (answer): report per-question wins and losses with a paired test on school-course and fable QA sets, so small mean gains are not mistaken for real ones.
- **I5** (tokens): handle clitics such as 's and n't in the tokenizer/morphology rules rather than through word lists, so word identity never gets damaged by list-based deletion.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 473 works

## Cite

Cite as: Ibrahim Abu El-Khair. 2006. Effects of Stop Words Elimination for Arabic Information Retrieval: A Comparative Study. arXiv:1702.01925 · Abu El-Khair, Ibrahim. (2006). Effects of stop words elimination for Arabic information retrieval: a comparative study. International Journal of Computing & Information Sciences 4 (3), 119-133.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{elkhair2006effects,
  title = {{Effects of Stop Words Elimination for Arabic Information Retrieval: A Comparative Study}},
  author = {Ibrahim Abu El-Khair},
  year = {2006},
  doi = {10.48550/arxiv.1702.01925},
  eprint = {1702.01925},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1702.01925}
}
```

- [← previous Classification of Discourse Coherence Relations: An Exploratory Study using Multiple Knowledge Sources](https://md.research.elamur.ai/free/en/papers/wellner-2006-classification-discourse-coherence-relations-exploratory/)
- [next → Gesture Improves Coreference Resolution](https://md.research.elamur.ai/free/en/papers/eisenstein-2006-gesture-improves-coreference-resolution/)
- [↑ back to papers of 2006](https://md.research.elamur.ai/free/en/papers/2006/page/2/)

## References

23 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. M. Abdul-Rauf, Arabic for English Speaking Students, Al- Saadawi Publications, 1996 [find](https://scholar.google.com/scholar?q=M%2E+Abdul%2DRauf%2C+Arabic+for+English+Speaking+Students%2C+Al%2D+Saadawi+Publications%2C+1996)
2. A. Al-Raghi, Grammatical Application, Dar El-Nahda Al-Arabia Lelteba’ah Wa Al-nashr, 1981. [find](https://scholar.google.com/scholar?q=A%2E+Al%2DRaghi%2C+Grammatical+Application%2C+Dar+El%2DNahda+Al%2DArabia+Lelteba%E2%80%99ah+Wa+Al%2Dnashr%2C+1981%2E)
3. A. Alshehri, Optimization and Effectiveness of N-Grams Approach for Indexing and Retrieval in Arabic Information Retrieval Systems, Ph. D Thesis, School of Information Sciences University of Pittsburgh, 2002 [find](https://scholar.google.com/scholar?q=A%2E+Alshehri%2C+Optimization+and+Effectiveness+of+N%2DGrams+Approach+for+Indexing+and+Retrieval+in+Arabic+Information+Retrieval+Systems%2C+Ph%2E+D+Thesis%2C+School+of+Information+Sciences+University+of+Pittsburg)
4. B. Brunner (Ed.), Time Almanac 2005 with Information Please, [http://www.infoplease.com/ipa/A0775272.html](http://www.infoplease.com/ipa/A0775272.html), 2005 [↗](http://www.infoplease.com/ipa/A0775272.html)
5. A. Chen, F. Gey, Translation Term Weighting and Combining Translation Resources in Cross-Language Retrieval, Tenth Text REtrieval Conference (TREC 2001), [http://trec.nist.gov/pubs/trec10/papers/berkeley](http://trec.nist.gov/pubs/trec10/papers/berkeley) _trec10.pdf, 2002. [↗](http://trec.nist.gov/pubs/trec10/papers/berkeley)
6. C. Fox, A Stop List for General Text. SIGIR Forum, Vol. 24, No. 1-2, 1990, pp.19-35. [find](https://scholar.google.com/scholar?q=C%2E+Fox%2C+A+Stop+List+for+General+Text%2E+SIGIR+Forum%2C+Vol%2E+24%2C+No%2E+1%2D2%2C+1990%2C+pp%2E19%2D35%2E)
7. D. Hull, Using Statistical Testing in the Evaluation of Retrieval Experiments, Proceedings of the 16th annual international ACM SIGIR conference on Research and Development in Information Retrieval, 1993, pp. 329-338. [find](https://scholar.google.com/scholar?q=D%2E+Hull%2C+Using+Statistical+Testing+in+the+Evaluation+of+Retrieval+Experiments%2C+Proceedings+of+the+16th+annual+international+ACM+SIGIR+conference+on+Research+and+Development+in+Information+Retrieval%2C+1)
8. S. Khoja, R. Garside, Stemming Arabic Text, [http://www.comp.lancs.ac.uk/computing/users/khoja/stemmer.ps](http://www.comp.lancs.ac.uk/computing/users/khoja/stemmer.ps), 1999. [↗](http://www.comp.lancs.ac.uk/computing/users/khoja/stemmer.ps)
9. R. R. Korfhage, Information Storage and Retrieval, John Wiley, 1997. [find](https://scholar.google.com/scholar?q=R%2E+R%2E+Korfhage%2C+Information+Storage+and+Retrieval%2C+John+Wiley%2C+1997%2E)
10. L. S. Larkey, L. Ballesteros, M. E. Connell, Improving Stemming for Arabic Information Retrieval: Light Stemming and Co-occurrence Analysis, Proceedings of the 25 th Annual International AC SIGIR Conference on Research and Development in Information Retrieval, 2002, pp. 275-282. [find](https://scholar.google.com/scholar?q=L%2E+S%2E+Larkey%2C+L%2E+Ballesteros%2C+M%2E+E%2E+Connell%2C+Improving+Stemming+for+Arabic+Information+Retrieval%3A+Light+Stemming+and+Co%2Doccurrence+Analysis%2C+Proceedings+of+the+25+th+Annual+International+AC+SIGIR+Conf)
11. L. S. Larkey, M. E. Connell, Arabic Information Retrieval at UMass in TREC-10, Tenth Text REtrieval Conference (TREC 2001), [http://trec.nist.gov/pubs/trec10/papers/UMass](http://trec.nist.gov/pubs/trec10/papers/UMass) _TREC10_Final.pdf, 2002 [↗](http://trec.nist.gov/pubs/trec10/papers/UMass)
12. LDC, TREC 2002 Arabic Topic Development and Assessment, [http://www.ldc.upenn.edu/Projects/TREC/TR](http://www.ldc.upenn.edu/Projects/TREC/TR) EC_2002/index.html, 2002a. [↗](http://www.ldc.upenn.edu/Projects/TREC/TR)
13. LDC, Arabic Newswire Part 1, [http://www.ldc.upenn.edu/Catalog/LDC2001](http://www.ldc.upenn.edu/Catalog/LDC2001) T55.html, 2002b [↗](http://www.ldc.upenn.edu/Catalog/LDC2001)
14. Lemur Toolkit Team, Lemur Toolkit Overview & Documentation, [http://www.lemurproject.org](http://www.lemurproject.org), 2003 [↗](http://www.lemurproject.org)
15. G. Salton, C. Buckley, Term- Weighting Approaches in Automatic Text Retrieval. Information Processing & Management, Vol. 24, No. 4, 1998, pp.513-523. [find](https://scholar.google.com/scholar?q=G%2E+Salton%2C+C%2E+Buckley%2C+Term%2D+Weighting+Approaches+in+Automatic+Text+Retrieval%2E+Information+Processing+%26+Management%2C+Vol%2E+24%2C+No%2E+4%2C+1998%2C+pp%2E513%2D523%2E)
16. G. Salton, M. McGill, Introduction to Modern Information Retrieval, McGraw-Hill Book Company, 1983. [find](https://scholar.google.com/scholar?q=G%2E+Salton%2C+M%2E+McGill%2C+Introduction+to+Modern+Information+Retrieval%2C+McGraw%2DHill+Book+Company%2C+1983%2E)
17. J. Savoy, Y. Rasolofo, Report on the TREC-11 Experiment: Arabic, Named Page and Topic Distillation Searches. Eleventh Text REtrieval Conference (TREC 2002), [http://trec.nist.gov/pubs/trec11/papers/uneuch](http://trec.nist.gov/pubs/trec11/papers/uneuch) atel.pdf, 2003. [↗](http://trec.nist.gov/pubs/trec11/papers/uneuch)
18. P. Schauble, Multimedia Information Retrieval: Content-based Information Retrieval from Large Text and Audio Databases, Kluwer Academic Publishers, 1997. [find](https://scholar.google.com/scholar?q=P%2E+Schauble%2C+Multimedia+Information+Retrieval%3A+Content%2Dbased+Information+Retrieval+from+Large+Text+and+Audio+Databases%2C+Kluwer+Academic+Publishers%2C+1997%2E)
19. S. Siegel, N. J. Castellan, Nonparametric Statistics of the Behavioral Sciences, 2 nd edition, McGraw-Hill Book Company, 1988. [find](https://scholar.google.com/scholar?q=S%2E+Siegel%2C+N%2E+J%2E+Castellan%2C+Nonparametric+Statistics+of+the+Behavioral+Sciences%2C+2+nd+edition%2C+McGraw%2DHill+Book+Company%2C+1988%2E)
20. TREC, Query Set for Arabic Retrieval at the TREC-2001 Cross-Language Information Retrieval Track, [http://trec.nist.gov/data/topics_noneng/Arabic](http://trec.nist.gov/data/topics_noneng/Arabic) _topics.txt (Arabic); http://trec.nist.gov/data/topics_noneng/english_topics.txt (English), 2002a [↗](http://trec.nist.gov/data/topics_noneng/Arabic)
21. TREC, Relevance Judgments for Arabic Retrieval at the TREC-2001 Cross-Language Information Retrieval Track, [http://trec.nist.gov/data/qrels_noneng/xlingual_t10qrels.txt](http://trec.nist.gov/data/qrels_noneng/xlingual_t10qrels.txt), 2002b [↗](http://trec.nist.gov/data/qrels_noneng/xlingual_t10qrels.txt)
22. E. M. Voorhees, D. Harman, Overview of TREC 2001.Tenth Text Retrieval Conference, TREC 2001, 2002, pp. 13-16. [find](https://scholar.google.com/scholar?q=E%2E+M%2E+Voorhees%2C+D%2E+Harman%2C+Overview+of+TREC+2001%2ETenth+Text+Retrieval+Conference%2C+TREC+2001%2C+2002%2C+pp%2E+13%2D16%2E)
23. C. Zhai, Dirichlet Prior Smoothing Parameter. Personal Communication, 2003 Dr. Ibrahim Abu El-Khair Received MLIS, from the School of Library and Information Science (Currently School of Information Studies) - University of Wisconsin, Milwaukee – USA. 2000 Received Ph.D. in Information Science, School of Information Sciences, University of Pittsburgh – USA. 2003. Currently working as an assistant professor at the Dept. of Library and Information Science. Faculty of Arts, El-Minia University – Egypt. [find](https://scholar.google.com/scholar?q=C%2E+Zhai%2C+Dirichlet+Prior+Smoothing+Parameter%2E+Personal+Communication%2C+2003+Dr%2E+Ibrahim+Abu+El%2DKhair+Received+MLIS%2C+from+the+School+of+Library+and+Information+Science+%28Currently+School+of+Information+S)

*Updated 31 October 2026*