--- title: "Codezone Research Group at ImageEval Shared-Task 2: Arabic" description: "The paper presents a system built for ImageEval 2025's shared task, which focuses on generating Arabic… 30.82 ROUGE. 5 references." authors: ["Shehu Bichi, Abdulkadir"] year: 2025 venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks" source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.53/" doi: "10.18653/v1/2025.arabicnlp-sharedtasks.53" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/bichi-2025-codezone-research-group-imageeval-shared/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bichi-2025-codezone-research-group-imageeval-shared/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/17/) / Codezone Research Group at ImageEval Shared-Task 2: Arabic Image Captioning Using BLIP and M2M100: A Two-Stage Translation Approach for ImageEval 2025 # Codezone Research Group at ImageEval Shared-Task 2: Arabic Image Captioning Using BLIP and M2M100: A Two-Stage Translation Approach for ImageEval 2025 **Authors:** Shehu Bichi, Abdulkadir · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.53/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.53.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.53](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.53) · [OpenAlex](https://openalex.org/W4416037746) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary The paper presents a system built for ImageEval 2025's shared task, which focuses on generating Arabic captions for images. Arabic has few image–text resources, so the authors do not train an Arabic captioner. They chain two existing models without any fine-tuning. BLIP-base first writes an English caption using beam search. The best-scoring caption is then translated into Arabic with M2M100-418M. Before scoring, the Arabic output is normalized by stripping diacritics, Tatweel and punctuation and by standardizing whitespace. On the 752-image test set, the pipeline beats both zero-shot and fine-tuned Qwen 2.5-VL 7B on BLEU. It falls behind them on cosine similarity and on the LLM Judge score. Manual rating found weak cultural relevance and weak completeness. The authors trace the errors to idioms, culture-specific objects and Arabic morphology. The work shows that a translation-based pipeline is a cheap, workable baseline for a low-resource language. It also shows that such a pipeline tends to match the references word by word without capturing deeper meaning. ## Key points - The ImageEval 2025 dataset is reported as 3,471 images with Arabic captions. The stated splits are 2,718 train, 75 validation and 752 test, which add up to more than 3,471. - The models are Salesforce/blip-image-captioning-base and facebook/m2m100-418M, run with PyTorch and Transformers. Inference is zero-shot. - Test results for BLIP+M2M100 versus zero-shot and fine-tuned Qwen 2.5-VL 7B: | Metric | BLIP+M2M100 | Qwen zero-shot | Qwen fine-tuned | |---|---|---|---| | BLEU-1 | 0.2847 | 0.0992 | 0.1698 | | Cosine similarity | 0.383 | 0.5577 | 0.5846 | | LLM Judge | 15.14 | 27.11 | 30.82 | - ROUGE is reported as 0.0000 for every system, which suggests a scoring problem in the evaluation. - Qualitative ratings were cultural relevance 1.10, conciseness 2.03, completeness 1.47 and accuracy 2.03. Captioning plus translation took about 12.27 s per image. ## Interesting ideas - Generate in a high-resource language and then translate, so that each module can be replaced or debugged on its own. The weak point is that the system never learns how native speakers actually describe things. - Surface-overlap metrics and semantic metrics ranked the systems in opposite orders. Reporting only one family of metrics would therefore have been misleading. - The output is normalized before scoring (diacritics, elongation characters, punctuation, whitespace). This keeps purely orthographic noise from distorting the comparison with references. - Captions are rated on separate axes (cultural relevance, conciseness, completeness, accuracy) instead of one overall score. This shows exactly which quality is weak. - The error taxonomy names idioms with no target equivalent and culture-specific objects as recurring failures. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (back): Mova could produce English from the meaning graph first and treat translation as a separate, inspectable step, logging where target-language idioms are lost. - **I2** (learn): Mova's evaluation should pair exact-match scores (LAS, lemma accuracy) with meaning-level checks, so that a gain on one is not mistaken for overall progress. - **I3** (tokens): a shared normalizer for apostrophes, quotes, hyphens and case could be applied before comparing tokenizer and lemma output with treebank gold. - **I4** (answer): QA and school-test answers could be graded separately for correctness, completeness and brevity, to tell whether the reader missed a fact or only phrased the answer badly. - **I5** (frames): idioms could be detected before frame building and mapped to their figurative frame, so they are not read literally. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions Also on the byline: [Baba Ahmed University](https://ror.org/00ac4gz35). 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works 🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 811 works ## Cite Cite as: Shehu Bichi, Abdulkadir. 2025. Codezone Research Group at ImageEval Shared-Task 2: Arabic Image Captioning Using BLIP and M2M100: A Two-Stage Translation Approach for ImageEval 2025. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{abdulkadir2025codezone, title = {{Codezone Research Group at ImageEval Shared-Task 2: Arabic Image Captioning Using BLIP and M2M100: A Two-Stage Translation Approach for ImageEval 2025}}, author = {Shehu Bichi, Abdulkadir}, booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks}, year = {2025}, doi = {10.18653/v1/2025.arabicnlp-sharedtasks.53}, url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.53} } ``` - [← previous CNLP-NITS-PP at GenAI Detection Task 2: Leveraging DistilBERT and XLM-RoBERTa for Multilingual AI-Generated Text Detection](https://md.research.elamur.ai/free/en/papers/yadagiri-2025-cnlp-nits-pp-genai-detection-2025-genaidetect-1-34/) - [next → Cognate Detection for Historical Language Reconstruction of Proto-Sabean Languages: the Case of Ge'ez, Tigrinya, and Amharic](https://md.research.elamur.ai/free/en/papers/temesgen-2025-cognate-detection-historical-language-reconstruction/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/17/) ## References 5 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. A. Alami Chehbouni, A. Ouatik Said, and T. Rachidi. 2020. A proposed natural language processing preprocessing procedures for enhancing arabic text summarization. In Advances in Intelligent Systems and Computing, pages 25–34. Springer. [find](https://scholar.google.com/scholar?q=A%2E+Alami+Chehbouni%2C+A%2E+Ouatik+Said%2C+and+T%2E+Rachidi%2E+2020%2E+A+proposed+natural+language+processing+preprocessing+procedures+for+enhancing+arabic+text+summarization%2E+In+Advances+in+Intelligent+Systems+an) 2. M. Stefanini, M. Cornia, L. Baraldi, S. Cascianelli, G. Fiameni, and R. Cucchiara. 2023. From show to tell: A survey on deep learning-based image captioning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1):539–559. Ahlam Bashiti, Alaa Aljabari, Hadi Hamoud, Md. [find](https://scholar.google.com/scholar?q=M%2E+Stefanini%2C+M%2E+Cornia%2C+L%2E+Baraldi%2C+S%2E+Cascianelli%2C+G%2E+Fiameni%2C+and+R%2E+Cucchiara%2E+2023%2E+From+show+to+tell%3A+A+survey+on+deep+learning%2Dbased+image+captioning%2E+IEEE+Transactions+on+Pattern+Analysis+and+) 3. Rafiul Biswas, Bilal Shalash, Mustafa Jarrar, Fadi Zaraket, George Mikros, Ehsaneddin Asgari, and Wajdi Zaghouani. 2025. [ImageEval 2025: The First Arabic Image Captioning Shared Task](https://md.research.elamur.ai/free/en/papers/bashiti-2025-imageeval-2025-first-arabic-image/). In Proceedings of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics. 4. J. Elliott and A. P. de Vries. 2023. Evaluating image captioning systems via cross-modal retrieval. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12543–12559. [find](https://scholar.google.com/scholar?q=J%2E+Elliott+and+A%2E+P%2E+de+Vries%2E+2023%2E+Evaluating+image+captioning+systems+via+cross%2Dmodal+retrieval%2E+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+12543%E2%80%93) 5. A. Fan, S. Bhosale, H. Schwenk, Z. Ma, A. El-Kishky, S. Goyal, M. Baines, O. Celebi, G. Wenzek, V. Chaudhary, N. Goyal, T. Birch, V. Liptchinsky, S. Edunov, E. Grave, M. Auli, and A. Joulin. 2021. Beyond english-centric multilingual machine translation. Journal of Machine Learning Research, 22(107):1–48. [find](https://scholar.google.com/scholar?q=A%2E+Fan%2C+S%2E+Bhosale%2C+H%2E+Schwenk%2C+Z%2E+Ma%2C+A%2E+El%2DKishky%2C+S%2E+Goyal%2C+M%2E+Baines%2C+O%2E+Celebi%2C+G%2E+Wenzek%2C+V%2E+Chaudhary%2C+N%2E+Goyal%2C+T%2E+Birch%2C+V%2E+Liptchinsky%2C+S%2E+Edunov%2C+E%2E+Grave%2C+M%2E+Auli%2C+and+A%2E+Joulin%2E+2021%2E+Bey) *Updated 2 November 2026*