A Comparative Evaluation of Open-Source Vision Language Models for Image Captioning
Küçük Resim Yok
Tarih
2025
Yazarlar
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
Institute of Electrical and Electronics Engineers Inc.
Erişim Hakkı
info:eu-repo/semantics/closedAccess
Özet
Vision Language Models (VLMs) have emerged as powerful architectures capable of jointly processing visual and textual inputs across a range of multimodal tasks. However, despite their growing adoption, a systematic evaluation of open-source VLMs on standard image captioning benchmarks remains limited. This study presents a comparative evaluation of three widely used open-source VLMs - LLaVA-7B, MiniCPM-V-8B, and Qwen2.5-VL-7B - on the task of image captioning using the Flickr8k dataset. We conduct a zero-shot assessment of each model's ability to generate descriptive captions for natural images and evaluate their outputs using a combination of lexical and semantic similarity metrics, including BLEU, METEOR, ROUGE-L, and BERTScore (F1). The results reveal notable differences in model behavior and output quality: Qwen2.5-VL-7B consistently achieves the highest scores across most metrics, while LLaVA-7B underperforms, likely due to its focus on instruction-following and visual question answering rather than structured caption generation. Despite Qwen2.5-VL-7B's relative advantage, all models fall short of human-level performance, underscoring the limitations of general-purpose VLMs in producing concise, reference-aligned captions without task-specific fine-tuning. Our findings highlight the importance of dataset alignment, prompt design, and semantic-aware evaluation methods in future research on open-source multimodal systems. © 2025 IEEE.
Açıklama
9th International Artificial Intelligence and Data Processing Symposium, IDAP 2025 -- 6 September 2025 through 7 September 2025 -- Malatya -- 215321
Anahtar Kelimeler
artificial intelligence, computer vision, language model, string similarity, vision language model
Kaynak
9th International Artificial Intelligence and Data Processing Symposium, IDAP 2025
WoS Q Değeri
Scopus Q Değeri
N/A
