---
title: "Addax at WojoodNER 2024 — paper by Yahia et al. (2024)"
description: "The paper describes the Addax team's system for the flat subtask of the WojoodNER 2024… 89.16 F1. 27 references, cited by 2 papers here."
authors: ["Issam Yahia", "Houdaifa Atou", "Ismail Berrada"]
year: 2024
venue: "Proceedings of the Second Arabic Natural Language Processing Conference"
source: "https://aclanthology.org/2024.arabicnlp-1.103/"
doi: "10.18653/v1/2024.arabicnlp-1.103"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yahia-2024-addax-wojoodner-2024-attention-based/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yahia-2024-addax-wojoodner-2024-attention-based/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/12/) / Addax at WojoodNER 2024: Attention-Based Dual-Channel Neural Network for Arabic Named Entity Recognition

# Addax at WojoodNER 2024: Attention-Based Dual-Channel Neural Network for Arabic Named Entity Recognition

**Authors:** [Issam Yahia](https://md.research.elamur.ai/free/en/authors/issam-ait-yahia/), [Houdaifa Atou](https://md.research.elamur.ai/free/en/authors/houdaifa-atou/) and [Ismail Berrada](https://md.research.elamur.ai/free/en/authors/ismail-berrada/) · **Year:** 2024 · **Venue:** [Proceedings of the Second Arabic Natural Language Processing Conference](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2024](https://md.research.elamur.ai/free/en/venues/arabicnlp/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.arabicnlp-1.103/) · [PDF](https://aclanthology.org/2024.arabicnlp-1.103.pdf) · [DOI 10.18653/v1/2024.arabicnlp-1.103](https://doi.org/10.18653/v1/2024.arabicnlp-1.103) · [OpenAlex](https://openalex.org/W4402671358)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/yahia-2024-addax-wojoodner-2024-attention-based/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes the Addax team's system for the flat subtask of the WojoodNER 2024 shared task. The task is Arabic named entity recognition with fine-grained subtypes. The data is the Wojood Fine corpus, which is mostly Modern Standard Arabic with some dialectal social media text.

The authors make two main design choices:
- **Tagging.** Instead of predicting main types and subtypes as separate tag layers, they merge each pair into one BIO label.
- **Architecture.** A contextual encoder, AraBERTv0.2-Twitter, feeds two parallel branches. A stack of 1D convolutions captures local cues, and a bidirectional GRU captures sentence-wide context. Each branch passes through its own self-attention and normalization step, then the two are concatenated for token classification.

Stepwise comparisons show that every added component helps a little. The full model reaches 89.32 micro-F1 on the development set and 90.13 on the test set, which tied for second place.

The work shows that modest additions on top of a well-chosen pretrained encoder still pay off for a lower-resource language. The pretrained encoder's domain matters as much as the extra layers.

## Key points
- **Corpus:** Wojood Fine extends Wojood to about 550K tokens with 21 entity types.
  - It adds 31 subtypes under GPE, ORG, LOC and FAC.
  - Roughly 80% is MSA and about 10% is Palestinian/Lebanese dialect social media text, drawn from the Curras and Baladi corpora.
- **Encoder choice:** with a plain linear head, AraBERTv0.2-Twitter scored 89.16 F1 versus 86.74 for AraBERTv0.2.
  - The full Conv1D + BiGRU + Attention model gave the best dev F1, 89.32.
  - Swapping BiGRU for BiLSTM gave 89.30.
- **Ablation on convolution blocks:** three blocks scored 89.16 F1, ahead of one block (86.74) and four blocks (88.73).
- **Training setup:** 10 epochs, batch size 32, learning rate 1e-3, on a single RTX A6000, using PyTorch and HuggingFace Transformers.
- **Per-class results:**
  - PERCENT, PERS, ORDINAL, DATE and GPE all exceed 93 F1.
  - PRODUCT reaches only about 53 F1.
  - CURR and UNIT score 0.
- **Leaderboard:** notfine_tuning placed first with 91; Addax and muNERa tied at 90; the baseline scored 89.

## Interesting ideas
- Merging a coarse label and its subtype into one joint BIO tag turns a two-layer prediction problem into a single classification, trading a little granularity for simpler training.
- Two parallel encoders, one convolutional for short-range patterns and one recurrent for whole-sentence context, each get their own self-attention before being joined, so neither view drowns the other.
- Picking a pretrained encoder whose pretraining genre matches the target text (Twitter-trained for partly social-media data) gave a larger gain than most architectural extras.
- A per-class breakdown exposed rare categories like currency and units that scored zero even though the overall micro-F1 looked strong.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): the tagger could predict joint labels such as UPOS plus a key morphological feature as one tag, then split them back, to check whether this lowers inconsistent feature combinations.
- **I2** (tree): gparse could add a cheap local convolutional branch next to its transformer features and test whether attachment of short function-word arcs improves.
- **I3** (learn): when training on silver data, compare encoders by genre fit and measure that gain explicitly before adding model complexity.
- **I4** (world): measures and currency mentions should not rely on a learned tagger alone; compiled unit and value knowledge can back them up, and evaluation should report such rare classes separately.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Université Mohammed VI Polytechnique](https://md.research.elamur.ai/free/en/universities/universite-mohammed-vi-polytechnique/) — University, Marrakesh, Morocco · 18 works · 171 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🏷️ Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/) · 200 works

## Cite

Cite as: Issam Yahia, Houdaifa Atou and Ismail Berrada. 2024. Addax at WojoodNER 2024: Attention-Based Dual-Channel Neural Network for Arabic Named Entity Recognition. Proceedings of the Second Arabic Natural Language Processing Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yahia2024addax,
  title = {{Addax at WojoodNER 2024: Attention-Based Dual-Channel Neural Network for Arabic Named Entity Recognition}},
  author = {Issam Yahia, Houdaifa Atou and Ismail Berrada},
  booktitle = {Proceedings of the Second Arabic Natural Language Processing Conference},
  year = {2024},
  doi = {10.18653/v1/2024.arabicnlp-1.103},
  url = {https://doi.org/10.18653/v1/2024.arabicnlp-1.103}
}
```

- [← previous Accessible Communication: a systematic review and comparative analysis of official English Easy-to-Understand (E2U) language guidelines](https://md.research.elamur.ai/free/en/papers/deleanu-2024-accessible-communication-systematic-review-comparative/)
- [next → Ancient but Digitized: Developing Handwritten Optical Character Recognition for East Syriac Script Through Creating KHAMIS Dataset](https://md.research.elamur.ai/free/en/papers/majeed-2024-ancient-but-digitized-developing-handwritten/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/12/)

## References

27 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Karim Al-Haff, Mustafa Jarrar, Tymaa Hammouda, and Fadi Zaraket. 2022. Curras + baladi: Towards a Levantine corpus. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 769–778, Marseille, France. European Language Resources Association. [↗](https://doi.org/10.48550/arxiv.2205.09692)
2. Hamza Alami, Abdelkader El Mahdaouy, Abdessamad Benlahbib, Noureddine En-Nahnahi, Ismail Berrada, and Said El Alaoui Ouatik. 2023. Daqas: Deep arabic question answering system based on duplicate question detection and machine reading comprehension. Journal of King Saud University - Computer and Information Sciences, 35(8):101709. [↗](https://doi.org/10.1016/j.jksuci.2023.101709)
3. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [Arabert: Transformer-based model for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, page 9. [↗](https://doi.org/10.48550/arxiv.2003.00104)
4. Na Chen, Yanqiu Sun, and Yan Yan. 2023. Sentiment analysis and research based on two-channel parallel hybrid neural network model with attention mechanism. IET Control Theory & Applications, 17. [↗](https://doi.org/10.1049/cth2.12463)
5. Kareem Darwish, Nizar Habash, Mourad Abbas, Hend Al-Khalifa, Huseein T Al-Natsheh, Houda Bouamor, Karim Bouzoubaa, Violetta Cavalli-Sforza, Samhaa R El-Beltagy, Wassim El-Hajj, et al. 2021. A panoramic survey of natural language processing in the arab world. Communications of the ACM, 64(4):72–81. [↗](https://doi.org/10.1145/3447735)
6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [BERT: pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). CoRR, abs/1810.04805. [↗](https://arxiv.org/abs/1810.04805)
7. Abdellah El Mekki, Abdelkader El Mahdaouy, Ismail Berrada, and Ahmed Khoumsi. 2022. Adasl: An unsupervised domain adaptation framework for arabic multi-dialectal sequence labeling. Information Processing & Management, 59(4):102964. [↗](https://doi.org/10.1016/j.ipm.2022.102964)
8. Oren Etzioni, Michael Cafarella, Doug Downey, Ana-Maria Popescu, Tal Shaked, Stephen Soderland, Daniel S Weld, and Alexander Yates. 2005. Unsupervised named-entity extraction from the web: An experimental study. Artificial intelligence, 165(1):91– 134. [↗](https://doi.org/10.1016/j.artint.2005.03.001)
9. Mustafa Jarrar, Mohammed Khalilia, and Sana Ghanem. 2022. [Wojood: Nested arabic named entity corpus and recognition using bert](https://md.research.elamur.ai/free/en/papers/jarrar-2022-wojood-nested-arabic-named-entity/). arXiv preprint arXiv:2205.09651. [↗](https://arxiv.org/abs/2205.09651)
10. Kai Labusch, Preußischer Kulturbesitz, Clemens Neudecker, and David Zellhöfer. 2019. Bert for named entity recognition in contemporary and historical german. In Proceedings of the 15th conference on natural language processing, Erlangen, Germany, pages 8–11. [find](https://scholar.google.com/scholar?q=Kai+Labusch%2C+Preu%C3%9Fischer+Kulturbesitz%2C+Clemens+Neudecker%2C+and+David+Zellh%C3%B6fer%2E+2019%2E+Bert+for+named+entity+recognition+in+contemporary+and+historical+german%2E+In+Proceedings+of+the+15th+conference+on+n)
11. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. [Neural architectures for named entity recognition](https://md.research.elamur.ai/free/en/papers/lample-2016-neural-architectures-named-entity-recognition/). arXiv preprint arXiv:1603.01360. [↗](https://arxiv.org/abs/1603.01360)
12. Jing Li, Aixin Sun, Jianglei Han, and Chenliang Li. 2022. A survey on deep learning for named entity recognition. IEEE Transactions on Knowledge and Data Engineering, 34(1):50–70. [find](https://scholar.google.com/scholar?q=Jing+Li%2C+Aixin+Sun%2C+Jianglei+Han%2C+and+Chenliang+Li%2E+2022%2E+A+survey+on+deep+learning+for+named+entity+recognition%2E+IEEE+Transactions+on+Knowledge+and+Data+Engineering%2C+34%281%29%3A50%E2%80%9370%2E)
13. Xiao Ling and Daniel Weld. 2012. Fine-grained entity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 26, pages 94–100. [find](https://scholar.google.com/scholar?q=Xiao+Ling+and+Daniel+Weld%2E+2012%2E+Fine%2Dgrained+entity+recognition%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+26%2C+pages+94%E2%80%93100%2E)
14. Haneen Liqreina, Mustafa Jarrar, Mohammed Khalilia, Ahmed Oumar El-Shangiti, and Muhammad Abdul-Mageed. 2023. [Arabic Fine-Grained Entity Recognition](https://md.research.elamur.ai/free/en/papers/liqreina-2023-arabic-fine-grained-entity-recognition/). In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 310–323. ACL.
15. Tao Gui, Ruotian Ma, Qi Zhang, Lujun Zhao, Yu-Gang Jiang, and Xuanjing Huang. 2019. Cnn-based chinese ner with lexicon rethinking. In ijcai, volume 2019. [find](https://scholar.google.com/scholar?q=Tao+Gui%2C+Ruotian+Ma%2C+Qi+Zhang%2C+Lujun+Zhao%2C+Yu%2DGang+Jiang%2C+and+Xuanjing+Huang%2E+2019%2E+Cnn%2Dbased+chinese+ner+with+lexicon+rethinking%2E+In+ijcai%2C+volume+2019%2E)
16. Andrew McCallum and Wei Li. 2003. [Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons](https://md.research.elamur.ai/free/en/papers/mccallum-2003-early-results-named-entity-recognition/). In Proceedings of the Seventh Conference on Natural Language Learning, CoNLL 2003, Held in cooperation with HLT-NAACL 2003, Edmonton, Canada, May 31 - June 1, 2003, pages 188–191. ACL.
17. Daniel Hanisch, Katrin Fundel, Heinz-Theodor Mevis-sen, Ralf Zimmer, and Juliane Fluck. 2005. Prominer: rule-based protein and gene entity recognition. BMC bioinformatics, 6:1–9. [↗](https://doi.org/10.1186/1471-2105-6-s1-s14)
18. Paul McNamee and James Mayfield. 2002. Entity extraction without language-specific resources. In COLING-02: The 6th Conference on Natural Language Learning 2002 (CoNLL-2002). [find](https://scholar.google.com/scholar?q=Paul+McNamee+and+James+Mayfield%2E+2002%2E+Entity+extraction+without+language%2Dspecific+resources%2E+In+COLING%2D02%3A+The+6th+Conference+on+Natural+Language+Learning+2002+%28CoNLL%2D2002%29%2E)
19. Mustafa Jarrar, Muhammad Abdul-Mageed, Mohammed Khalilia, Bashar Talafha, AbdelRahim Elmadany, Nagham Hamad, and Alaa’ Omar. 2023. [Wojoodner 2023: The First Arabic Named Entity Recognition Shared Task](https://md.research.elamur.ai/free/en/papers/jarrar-2023-wojoodner-2023-first-arabic-named/). In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 748– 758. ACL.
20. Sudha Morwal, Nusrat Jahan, and Deepti Chopra. 2012. Named entity recognition using hidden markov model (hmm). International Journal on Natural Language Computing (IJNLC) Vol, 1. [↗](https://doi.org/10.5121/ijnlc.2012.1402)
21. Mustafa Jarrar, Nagham Hamad, Mohammed Khalilia, Bashar Talafha, and Muhammad Elmadany, Abdel-Rahim Abdul-Mageed. 2024. [WojoodNER 2024: The Second Arabic Named Entity Recognition Shared Task](https://md.research.elamur.ai/free/en/papers/jarrar-2024-wojoodner-2024-second-arabic-named/). In Proceedings of the 2nd Arabic Natural Language Processing Conference (Arabic-NLP), Part of the ACL 2024. Association for Computational Linguistics.
22. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. [Pytorch: An imperative style, high-performance deep learning library](https://md.research.elamur.ai/free/en/papers/paszke-2019-pytorch-imperative-style-high-performance/). Advances in neural information processing systems, 32. [↗](https://doi.org/10.48550/arxiv.1912.01703)
23. Khaled Shaalan and Hafsa Raza. 2007. [Person name entity recognition for arabic](https://md.research.elamur.ai/free/en/papers/shaalan-2007-person-name-entity-recognition-arabic/). In Proceedings of the 2007 workshop on computational approaches to semitic languages: common issues and resources, pages 17–24.
24. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
25. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019. [Huggingface’s transformers: State-ofthe-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). arXiv preprint arXiv:1910.03771. [↗](https://arxiv.org/abs/1910.03771)
26. Hang Yan, Bocao Deng, Xiaonan Li, and Xipeng Qiu. 2019. Tener: adapting transformer encoder for named entity recognition. arXiv preprint arXiv:1911.04474. [↗](https://arxiv.org/abs/1911.04474)
27. Shaodian Zhang and Noémie Elhadad. 2013. Unsupervised biomedical named entity recognition: Experiments with clinical and biological texts. Journal of biomedical informatics, 46(6):1088–1098. [↗](https://doi.org/10.1016/j.jbi.2013.08.004)

*Updated 1 November 2026*