---
title: "VLCE — paper by Rahman et al. (2026) — summary and ideas"
description: "General-purpose VLMs such as LLaVA and QwenVL describe post-disaster satellite and drone imagery in vague, everyday… 25 references."
authors: ["Md. Mahfuzur Rahman", "Marufa Kamal", "Fahad Rahman", "Sunzida Siddique", "Ahmed Rafi Hasan", "Mohd Ariful Haque", "Kishor Datta Gupta", "Roy George"]
year: 2026
venue: "Proceedings of the 4th Workshop on Advances in Language and Vision Research (ALVR)"
source: "https://aclanthology.org/2026.alvr-main.15/"
doi: "10.18653/v1/2026.alvr-main.15"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/rahman-2026-vlce-knowledge-enhanced-framework-image/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rahman-2026-vlce-knowledge-enhanced-framework-image/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/47/) / VLCE: A Knowledge-Enhanced Framework for Image Description in Disaster Assessment

# VLCE: A Knowledge-Enhanced Framework for Image Description in Disaster Assessment

**Authors:** Md. Mahfuzur Rahman, Marufa Kamal, Fahad Rahman, Sunzida Siddique, Ahmed Rafi Hasan, Mohd Ariful Haque, Kishor Datta Gupta and Roy George · **Year:** 2026 · **Venue:** [Proceedings of the 4th Workshop on Advances in Language and Vision Research (ALVR)](https://md.research.elamur.ai/free/en/venues/alvr/) ([ALVR 2026](https://md.research.elamur.ai/free/en/venues/alvr/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.alvr-main.15/) · [PDF](https://aclanthology.org/2026.alvr-main.15.pdf) · [DOI 10.18653/v1/2026.alvr-main.15](https://doi.org/10.18653/v1/2026.alvr-main.15) · [OpenAlex](https://openalex.org/W4414667509)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
General-purpose VLMs such as LLaVA and QwenVL describe post-disaster satellite and drone imagery in vague, everyday language. They miss the damage-specific detail that emergency teams need. VLCE addresses this with a two-step pipeline. First, YOLOv8 detections are converted into a prompt, and a baseline VLM drafts a caption from it. Second, a trained sequential decoder rewrites that draft using a domain lexicon. The decoder is either a CNN-LSTM or a hierarchical cross-modal Transformer.

The lexicon is built in three steps. RAKE keywords are pulled from the training captions, then expanded with ConceptNet relations and WordNet synonyms. Frozen ConceptNet Numberbatch vectors serve as the embeddings. DistilBERT embeddings provide the no-KG ablation. Visual features come from domain-matched encoders: ResNet50-EuroSAT for satellite images and a ViT UAV classifier for drone images.

Evaluation uses xBD and RescueNet, scoring per-image preference against the baselines with CLIPScore and InfoMetIC. The Transformer with knowledge graph wins on most RescueNet images. Without the KG, UAV results collapse almost entirely. The work suggests that structured external knowledge is an inexpensive way to adapt captioning to specialized, high-stakes domains.

## Key points
- The enriched vocabulary has 3,195 tokens, of which 1,566 (49%) come only from ConceptNet/WordNet. Up to 10 ConceptNet terms are retrieved per keyword, and sequences are padded to length 192.
- Benchmarks: xBD (6,369 satellite images, 3 damage classes) and RescueNet (4,494 UAV images, 12 damage classes). CNN-LSTM variants are compared with LLaVA-7B v1.5, and Transformer variants with QwenVL-7B Instruct.
- Best result: Transformer + KG on RescueNet is preferred over QwenVL on 95.33% of images by InfoMetIC and 73.64% by CLIPScore. Without the KG, preference drops below 2% (0.08% and 0.22%).
- On xBD the KG helps less. The CNN-LSTM without KG even scores slightly higher on CLIPScore (55.34% vs 51.10%). The authors attribute this to the more uniform overhead imagery.
- VLCE captions mention all 272 UAV and 640 satellite reference objects, while baseline VLMs cover about 55–65%. Qualitatively, captions without the KG show hallucinations, repetition, and incoherence.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works · [⚙️ Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/) · 133 works · [⚙️ Media Technology](https://md.research.elamur.ai/free/en/shelves/engineering/media-technology/) · 34 works

## Cite

Cite as: Md. Mahfuzur Rahman, Marufa Kamal, Fahad Rahman, Sunzida Siddique, Ahmed Rafi Hasan, Mohd Ariful Haque, Kishor Datta Gupta and Roy George. 2026. VLCE: A Knowledge-Enhanced Framework for Image Description in Disaster Assessment. Proceedings of the 4th Workshop on Advances in Language and Vision Research (ALVR).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{rahman2026vlce,
  title = {{VLCE: A Knowledge-Enhanced Framework for Image Description in Disaster Assessment}},
  author = {Md. Mahfuzur Rahman, Marufa Kamal, Fahad Rahman, Sunzida Siddique, Ahmed Rafi Hasan, Mohd Ariful Haque, Kishor Datta Gupta and Roy George},
  booktitle = {Proceedings of the 4th Workshop on Advances in Language and Vision Research (ALVR)},
  year = {2026},
  doi = {10.18653/v1/2026.alvr-main.15},
  url = {https://doi.org/10.18653/v1/2026.alvr-main.15}
}
```

- [← previous ViX-Ray: A Vietnamese Chest X-Ray Dataset for Vision-Language Models](https://md.research.elamur.ai/free/en/papers/nguyen-2026-vix-ray-vietnamese-chest-x/)
- [next → Vocabulary Shapes Cross-Lingual Variation of Word-Order Learnability in Language Models](https://md.research.elamur.ai/free/en/papers/martins-2026-vocabulary-shapes-cross-lingual-variation/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/47/)

## References

25 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. O. Abbas and J. Dang. 2023. Using image captioning for automatic post-disaster damage detection. Intelligence, Informatics and Infrastructure, 4(2):66–74. [find](https://scholar.google.com/scholar?q=O%2E+Abbas+and+J%2E+Dang%2E+2023%2E+Using+image+captioning+for+automatic+post%2Ddisaster+damage+detection%2E+Intelligence%2C+Informatics+and+Infrastructure%2C+4%282%29%3A66%E2%80%9374%2E)
2. I. Alisjahbana and 1 others. 2024. DeepDama-geNet: A two-step deep-learning model for multi-disaster building damage segmentation. Preprint, arXiv:2405.04800. — [arXiv:2405.04800](https://arxiv.org/abs/2405.04800) [find](https://scholar.google.com/scholar?q=I%2E+Alisjahbana+and+1+others%2E+2024%2E+DeepDama%2DgeNet%3A+A+two%2Dstep+deep%2Dlearning+model+for+multi%2Ddisaster+building+damage+segmentation%2E+Preprint%2C+arXiv%3A2405%2E04800%2E+%E2%80%94+%5BarXiv%3A2405%2E04800%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
3. J. Bai and 1 others. 2023. [Qwen-VL: A versatile vision-language model for understanding, localization, text reading, and beyond](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/). Preprint, arXiv:2308.12966.
4. L. Chen and K. Li. 2025. Multi-modal graph aggregation transformer for image captioning. Neural Networks, 181:106813. [find](https://scholar.google.com/scholar?q=L%2E+Chen+and+K%2E+Li%2E+2025%2E+Multi%2Dmodal+graph+aggregation+transformer+for+image+captioning%2E+Neural+Networks%2C+181%3A106813%2E)
5. P.-J. Chun, T. Yamane, and Y. Maemura. 2022. A deep learning-based image captioning method for bridge damage. Computer-Aided Civil and Infrastructure Engineering, 37(11):1387–1401. [find](https://scholar.google.com/scholar?q=P%2E%2DJ%2E+Chun%2C+T%2E+Yamane%2C+and+Y%2E+Maemura%2E+2022%2E+A+deep+learning%2Dbased+image+captioning+method+for+bridge+damage%2E+Computer%2DAided+Civil+and+Infrastructure+Engineering%2C+37%2811%29%3A1387%E2%80%931401%2E)
6. Marcella Cornia, Matteo Stefanini, Lorenzo Baraldi, and Rita Cucchiara. 2020. Meshed-memory transformer for image captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10578–10587. [find](https://scholar.google.com/scholar?q=Marcella+Cornia%2C+Matteo+Stefanini%2C+Lorenzo+Baraldi%2C+and+Rita+Cucchiara%2E+2020%2E+Meshed%2Dmemory+transformer+for+image+captioning%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+R)
7. S. Elbedwehy and 1 others. 2023. Enhanced image captioning using features concatenation and efficient pre-trained word embedding. Computer Systems Science and Engineering, 46:3637–3652. [find](https://scholar.google.com/scholar?q=S%2E+Elbedwehy+and+1+others%2E+2023%2E+Enhanced+image+captioning+using+features+concatenation+and+efficient+pre%2Dtrained+word+embedding%2E+Computer+Systems+Science+and+Engineering%2C+46%3A3637%E2%80%933652%2E)
8. Ritwik Gupta and 1 others. 2019. xBD: A dataset for assessing building damage from satellite imagery. Preprint, arXiv:1911.09296. — [arXiv:1911.09296](https://arxiv.org/abs/1911.09296) [find](https://scholar.google.com/scholar?q=Ritwik+Gupta+and+1+others%2E+2019%2E+xBD%3A+A+dataset+for+assessing+building+damage+from+satellite+imagery%2E+Preprint%2C+arXiv%3A1911%2E09296%2E+%E2%80%94+%5BarXiv%3A1911%2E09296%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1911%2E09296%29)
9. C. Schön, S. Siebert, and F. Stolzenburg. 2019. Using ConceptNet to teach common sense to an automated theorem prover. Electronic Proceedings in Theoretical Computer Science, 311:19–24. [find](https://scholar.google.com/scholar?q=C%2E+Sch%C3%B6n%2C+S%2E+Siebert%2C+and+F%2E+Stolzenburg%2E+2019%2E+Using+ConceptNet+to+teach+common+sense+to+an+automated+theorem+prover%2E+Electronic+Proceedings+in+Theoretical+Computer+Science%2C+311%3A19%E2%80%9324%2E)
10. M. Talib, A. H. Y. Al-Noori, and J. Suad. 2024. YOLOv8-CAB: Improved YOLOv8 for real-time object detection. Karbala International Journal of Modern Science, 10(1):5. [find](https://scholar.google.com/scholar?q=M%2E+Talib%2C+A%2E+H%2E+Y%2E+Al%2DNoori%2C+and+J%2E+Suad%2E+2024%2E+YOLOv8%2DCAB%3A+Improved+YOLOv8+for+real%2Dtime+object+detection%2E+Karbala+International+Journal+of+Modern+Science%2C+10%281%29%3A5%2E)
11. J. Tang, P. Li, and M. Jiang. 2022. Image caption generation based on knowledge graph guidance and self-attention. Preprint. [find](https://scholar.google.com/scholar?q=J%2E+Tang%2C+P%2E+Li%2C+and+M%2E+Jiang%2E+2022%2E+Image+caption+generation+based+on+knowledge+graph+guidance+and+self%2Dattention%2E+Preprint%2E)
12. M. Toshevska, F. Stojanovska, and J. Kalajdjieski. 2020. Comparative analysis of word embeddings for capturing word similarities. Preprint, arXiv:2005.03812. — [arXiv:2005.03812](https://arxiv.org/abs/2005.03812) [find](https://scholar.google.com/scholar?q=M%2E+Toshevska%2C+F%2E+Stojanovska%2C+and+J%2E+Kalajdjieski%2E+2020%2E+Comparative+analysis+of+word+embeddings+for+capturing+word+similarities%2E+Preprint%2C+arXiv%3A2005%2E03812%2E+%E2%80%94+%5BarXiv%3A2005%2E03812%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs)
13. G. Wang and 1 others. 2023. UAV-YOLOv8: A small-object-detection model for UAV aerial photography. Sensors, 23(16):7190. [find](https://scholar.google.com/scholar?q=G%2E+Wang+and+1+others%2E+2023%2E+UAV%2DYOLOv8%3A+A+small%2Dobject%2Ddetection+model+for+UAV+aerial+photography%2E+Sensors%2C+23%2816%29%3A7190%2E)
14. X. Wang and 1 others. 2024. Knowledge enhancement strategies for remote sensing image captioning using CLIP and LLMs. In Proceedings of MIDA, pages 313–318. [find](https://scholar.google.com/scholar?q=X%2E+Wang+and+1+others%2E+2024%2E+Knowledge+enhancement+strategies+for+remote+sensing+image+captioning+using+CLIP+and+LLMs%2E+In+Proceedings+of+MIDA%2C+pages+313%E2%80%93318%2E)
15. X. Zhai and 1 others. 2023. YOLO-Drone: An optimized YOLOv8 for tiny UAV object detection. Electronics, 12(17):3664. [find](https://scholar.google.com/scholar?q=X%2E+Zhai+and+1+others%2E+2023%2E+YOLO%2DDrone%3A+An+optimized+YOLOv8+for+tiny+UAV+object+detection%2E+Electronics%2C+12%2817%29%3A3664%2E)
16. Wentian Zhao and Xinxiao Wu. 2024. Boosting entity-aware image captioning with multi-modal knowledge graph. IEEE Transactions on Multimedia, 26:2659– 2670. [find](https://scholar.google.com/scholar?q=Wentian+Zhao+and+Xinxiao+Wu%2E+2024%2E+Boosting+entity%2Daware+image+captioning+with+multi%2Dmodal+knowledge+graph%2E+IEEE+Transactions+on+Multimedia%2C+26%3A2659%E2%80%93+2670%2E)
17. J. Hessel and 1 others. 2021. [CLIPScore: A reference-free evaluation metric for image captioning](https://md.research.elamur.ai/free/en/papers/hessel-2021-clipscore-reference-free-evaluation-metric/). Preprint, arXiv:2104.08718.
18. Y. Zhou, Y. Sun, and V. Honavar. 2019. Improving image captioning by leveraging knowledge graphs. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision, pages 539–548. [find](https://scholar.google.com/scholar?q=Y%2E+Zhou%2C+Y%2E+Sun%2C+and+V%2E+Honavar%2E+2019%2E+Improving+image+captioning+by+leveraging+knowledge+graphs%2E+In+Proceedings+of+the+IEEE+Winter+Conference+on+Applications+of+Computer+Vision%2C+pages+539%E2%80%93548%2E)
19. A. Hu and 1 others. 2023. InfoMetIC: An informative metric for reference-free image caption evaluation. Preprint, arXiv:2305.06002. [find](https://scholar.google.com/scholar?q=A%2E+Hu+and+1+others%2E+2023%2E+InfoMetIC%3A+An+informative+metric+for+reference%2Dfree+image+caption+evaluation%2E+Preprint%2C+arXiv%3A2305%2E06002%2E)
20. Z. Zou and 1 others. 2021. Disaster image classification by fusing multimodal social media data. ISPRS International Journal of Geo-Information, 10(10):636. [find](https://scholar.google.com/scholar?q=Z%2E+Zou+and+1+others%2E+2021%2E+Disaster+image+classification+by+fusing+multimodal+social+media+data%2E+ISPRS+International+Journal+of+Geo%2DInformation%2C+10%2810%29%3A636%2E)
21. S. Kota and 1 others. 2022. Multimodal combination of text and image tweets for disaster response assessment. In D2R2. [find](https://scholar.google.com/scholar?q=S%2E+Kota+and+1+others%2E+2022%2E+Multimodal+combination+of+text+and+image+tweets+for+disaster+response+assessment%2E+In+D2R2%2E)
22. H. Liu and 1 others. 2023. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). In Advances in Neural Information Processing Systems, volume 36, pages 34892–34916.
23. Rebecca Mason and Eugene Charniak. 2014. Domain-specific image captioning. In Proceedings of the Conference on Computational Natural Language Learning, pages 28–36. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Rebecca+Mason+and+Eugene+Charniak%2E+2014%2E+Domain%2Dspecific+image+captioning%2E+In+Proceedings+of+the+Conference+on+Computational+Natural+Language+Learning%2C+pages+28%E2%80%9336%2E+Association+for+Computational+Lingu)
24. M. Rahnemoonfar, T. Chowdhury, and R. Murphy. 2023. RescueNet image classification dataset. Figshare Dataset. [find](https://scholar.google.com/scholar?q=M%2E+Rahnemoonfar%2C+T%2E+Chowdhury%2C+and+R%2E+Murphy%2E+2023%2E+RescueNet+image+classification+dataset%2E+Figshare+Dataset%2E)
25. A. F. Rasheed and M. Zarkoosh. 2025. Optimized YOLOv8 for multi-scale object detection. Journal of Real-Time Image Processing, 22(1):6. [find](https://scholar.google.com/scholar?q=A%2E+F%2E+Rasheed+and+M%2E+Zarkoosh%2E+2025%2E+Optimized+YOLOv8+for+multi%2Dscale+object+detection%2E+Journal+of+Real%2DTime+Image+Processing%2C+22%281%29%3A6%2E)

*Updated 1 November 2026*