---
title: "Fired_from_NLP at AraFinNLP 2024: Dual-Phase-BERT - A"
description: "The paper addresses intent detection for Arabic banking queries across dialects, a setting where… 12 references, cited by 1 paper here."
authors: ["Md. Sajid Alam Chowdhury", "Mostak Mahmud Chowdhury", "Anik Mahmud Shanto", "Hasan Murad", "Udoy Das"]
year: 2024
venue: "Proceedings of the Second Arabic Natural Language Processing Conference"
source: "https://aclanthology.org/2024.arabicnlp-1.36/"
doi: "10.18653/v1/2024.arabicnlp-1.36"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chowdhury-2024-fired-nlp-arafinnlp-2024-dual/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chowdhury-2024-fired-nlp-arafinnlp-2024-dual/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/12/) / Fired\_from\_NLP at AraFinNLP 2024: Dual-Phase-BERT - A Fine-Tuned Transformer-Based Model for Multi-Dialect Intent Detection in The Financial Domain for The Arabic Language

# Fired\_from\_NLP at AraFinNLP 2024: Dual-Phase-BERT - A Fine-Tuned Transformer-Based Model for Multi-Dialect Intent Detection in The Financial Domain for The Arabic Language

**Authors:** Md. Sajid Alam Chowdhury, Mostak Mahmud Chowdhury, [Anik Mahmud Shanto](https://md.research.elamur.ai/free/en/authors/anik-mahmud-shanto/), [Hasan Murad](https://md.research.elamur.ai/free/en/authors/hasan-murad/) and [Udoy Das](https://md.research.elamur.ai/free/en/authors/udoy-das/) · **Year:** 2024 · **Venue:** [Proceedings of the Second Arabic Natural Language Processing Conference](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2024](https://md.research.elamur.ai/free/en/venues/arabicnlp/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.arabicnlp-1.36/) · [PDF](https://aclanthology.org/2024.arabicnlp-1.36.pdf) · [DOI 10.18653/v1/2024.arabicnlp-1.36](https://doi.org/10.18653/v1/2024.arabicnlp-1.36) · [OpenAlex](https://openalex.org/W4402671509)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses intent detection for Arabic banking queries across dialects, a setting where financial NLP resources are scarce compared with English. It was written for Task 1 of the AraFinNLP 2024 shared task, which uses the ArBanking77 dataset of customer queries labelled with 77 banking intents. Training and development data cover only MSA and Palestinian (PAL), while the test set also includes other dialects such as Gulf, Levantine and North African. The authors compare TF-IDF-based classical classifiers (SVM, Random Forest, XGBoost), recurrent networks (LSTM, BiLSTM), a single fine-tuned AraBERT, and their own Dual-Phase-BERT. Dual-Phase-BERT works hierarchically: one AraBERT model first decides whether a query is MSA or PAL, then sends it to a dialect-specific AraBERT intent classifier. This system scored the best Micro-F1 among the models they tried, 0.801 on test, and placed 7th in the competition. The large drop from development to test performance shows how much unseen dialects and class imbalance hurt such systems.

## Key points
- Data: ArBanking77. Training has 10,733 MSA and 10,821 PAL queries, development has 1,230 MSA and 1,234 PAL, and the test set has 11,721 multi-dialect queries. There are 77 intent classes, and their distribution is strongly imbalanced.
- Dual-Phase-BERT combines three fine-tuned AraBERT models:
  - mpBERT routes each query by dialect (MSA vs PAL);
  - mBERT classifies intents for MSA queries;
  - pBERT classifies intents for PAL queries.
- Training setup: text is preprocessed with ArabertPreprocessor. The classification head is dropout, then a tanh dense layer, then dropout, then softmax. Training uses Adam with lr 5e-6 for 20–30 epochs.
- Test results reported in the text:
  - SVM, the best classical model: F1 0.617;
  - BiLSTM: 0.663;
  - plain AraBERT: 0.767;
  - Dual-Phase-BERT: 0.801.
- On the development set, Dual-Phase-BERT reached F1 0.984 but fell to 0.801 on test. The authors attribute the gap to dialects missing from training and to class imbalance.

## Interesting ideas
- A two-step "route, then specialise" design sends each input to a variety-specific model instead of forcing one model to cover every variety, which pays off when the varieties differ a lot.
- The near-perfect development score next to a much weaker test score warns that a validation split drawn from the same varieties as training hides how the model handles distribution shift.
- The authors measure every model on one ladder, from TF-IDF classifiers through LSTMs to transformers, which shows how much each extra level of complexity actually contributes.
- The error analysis connects mistakes to majority-class bias caused by imbalanced intent labels, a common failure for classifiers with many labels.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tree): A genre/register router in front of gparse could choose between parser heads trained on different treebanks, such as web, spoken and formal text, and record which head was used to keep the decision explainable.
- **I2** (learn): Evaluation should include a held-out split from a domain or dialect absent from training as a standing control, so in-domain gains are not mistaken for robustness.
- **I3** (learn): Each new component, such as a parser or role labeller, could be reported against a simple baseline on the same split, making it clear whether the added machinery earns its cost.
- **I4** (frames): When assigning speech acts or VerbNet-based roles, Mova could track per-class accuracy so that rare classes are not quietly absorbed by frequent ones.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Chittagong University of Engineering & Technology](https://md.research.elamur.ai/free/en/universities/chittagong-university-of-engineering-technology/) — University, Chittagong, Bangladesh · 25 works · 29 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Md. Sajid Alam Chowdhury, Mostak Mahmud Chowdhury, Anik Mahmud Shanto, Hasan Murad and Udoy Das. 2024. Fired\_from\_NLP at AraFinNLP 2024: Dual-Phase-BERT - A Fine-Tuned Transformer-Based Model for Multi-Dialect Intent Detection in The Financial Domain for The Arabic Language. Proceedings of the Second Arabic Natural Language Processing Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{chowdhury2024fired,
  title = {{Fired\_from\_NLP at AraFinNLP 2024: Dual-Phase-BERT - A Fine-Tuned Transformer-Based Model for Multi-Dialect Intent Detection in The Financial Domain for The Arabic Language}},
  author = {Md. Sajid Alam Chowdhury, Mostak Mahmud Chowdhury, Anik Mahmud Shanto, Hasan Murad and Udoy Das},
  booktitle = {Proceedings of the Second Arabic Natural Language Processing Conference},
  year = {2024},
  doi = {10.18653/v1/2024.arabicnlp-1.36},
  url = {https://doi.org/10.18653/v1/2024.arabicnlp-1.36}
}
```

- [← previous Experimental Pragmatics with Machines: Testing LLM Predictions for the Inferences of Plain and Embedded Disjunctions](https://md.research.elamur.ai/free/en/papers/tsvilodub-2024-experimental-pragmatics-machines-testing-llm/)
- [next → From Nile Sands to Digital Hands: Machine Translation of Coptic Texts](https://md.research.elamur.ai/free/en/papers/saeed-2024-nile-sands-digital-hands-machine/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/12/)

## References

12 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alaa T Al-Tuama and Dhamyaa A Nasrawi. 2022. Intent classification using machine learning algorithms and augmented data. In ICDSIC. IEEE. [find](https://scholar.google.com/scholar?q=Alaa+T+Al%2DTuama+and+Dhamyaa+A+Nasrawi%2E+2022%2E+Intent+classification+using+machine+learning+algorithms+and+augmented+data%2E+In+ICDSIC%2E+IEEE%2E)
2. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [AraBERT: Transformer-based model for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In OSACPT, Marseille, France. European Language Resource Association. [↗](https://doi.org/10.48550/arxiv.2003.00104)
3. Biodoumoye George Bokolo, Praise Onyehanere, Ebikela Ogegbene-Ise, Itunu Olufemi, and Josiah Nii Armah Tettey. 2023. Leveraging machine learning for crime intent detection in social media posts. In AIGC. Springer. [find](https://scholar.google.com/scholar?q=Biodoumoye+George+Bokolo%2C+Praise+Onyehanere%2C+Ebikela+Ogegbene%2DIse%2C+Itunu+Olufemi%2C+and+Josiah+Nii+Armah+Tettey%2E+2023%2E+Leveraging+machine+learning+for+crime+intent+detection+in+social+media+posts%2E+In+AI)
4. Osman Büyük, Mustafa Erden, and Levent M. Arslan. 2021. Leveraging the information in in-domain datasets for transformer-based intent detection. In 2021 Innovations in Intelligent Systems and Applications Conference (ASYU). [find](https://scholar.google.com/scholar?q=Osman+B%C3%BCy%C3%BCk%2C+Mustafa+Erden%2C+and+Levent+M%2E+Arslan%2E+2021%2E+Leveraging+the+information+in+in%2Ddomain+datasets+for+transformer%2Dbased+intent+detection%2E+In+2021+Innovations+in+Intelligent+Systems+and+Applicat)
5. Lisung Chen, Nuo Chen, Yuexian Zou, Yong Wang, and Xinzhong Sun. 2022. A transformer-based threshold-free framework for multi-intent nlu. In ICCL. [find](https://scholar.google.com/scholar?q=Lisung+Chen%2C+Nuo+Chen%2C+Yuexian+Zou%2C+Yong+Wang%2C+and+Xinzhong+Sun%2E+2022%2E+A+transformer%2Dbased+threshold%2Dfree+framework+for+multi%2Dintent+nlu%2E+In+ICCL%2E)
6. Qian Chen, Zhu Zhuo, and Wen Wang. 2019. Bert for joint intent classification and slot filling. arXiv. [find](https://scholar.google.com/scholar?q=Qian+Chen%2C+Zhu+Zhuo%2C+and+Wen+Wang%2E+2019%2E+Bert+for+joint+intent+classification+and+slot+filling%2E+arXiv%2E)
7. Mustafa Jarrar, Ahmet Birim, Mohammed Khalilia, Mustafa Erden, and Sana Ghanem. 2023. [Arbanking77: Intent detection neural model and a new dataset in modern and dialectical arabic](https://md.research.elamur.ai/free/en/papers/jarrar-2023-arbanking77-intent-detection-neural-model/). In Proceedings of ArabicNLP 2023, Singapore (Hybrid), December 7, 2023, pages 276–287. Association for Computational Linguistics.
8. Bing Liu and Ian Lane. 2016. Joint online spoken language understanding and language modeling with recurrent neural networks. arXiv. [find](https://scholar.google.com/scholar?q=Bing+Liu+and+Ian+Lane%2E+2016%2E+Joint+online+spoken+language+understanding+and+language+modeling+with+recurrent+neural+networks%2E+arXiv%2E)
9. Sanad Malaysha, Mo El-Haj, Saad Ezzini, Mohammad Khalilia, Mustafa Jarrar, Sultan Nasser, Ismail Berrada, and Houda Bouamor. 2024. [AraFinNlp 2024: The first arabic financial nlp shared task](https://md.research.elamur.ai/free/en/papers/malaysha-2024-arafinnlp-2024-first-arabic-financial/). In Proceedings of the 2nd Arabic Natural Language Processing Conference (Arabic-NLP), Part of the ACL 2024. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2407.09818)
10. Peiqing Niu, Zhongfu Chen, Meina Song, et al. 2019. [A novel bi-directional interrelated model for joint intent detection and slot filling](https://md.research.elamur.ai/free/en/papers/e-2019-novel-bi-directional-interrelated-model/). arXiv.
11. Libo Qin, Tailu Liu, Wanxiang Che, Bingbing Kang, Sendong Zhao, and Ting Liu. 2021. A co-interactive transformer for joint slot filling and intent detection. In ICASSP. IEEE. [↗](https://doi.org/10.48550/arxiv.2010.03880)
12. Yu Wang, Yilin Shen, and Hongxia Jin. 2018. A bi-model based rnn semantic frame parsing model for intent detection and slot filling. arXiv. [↗](https://doi.org/10.48550/arxiv.1812.10235)

*Updated 1 November 2026*