Kabakus, Abdullah TalhaSenturk, Arafat2026-07-012026-07-012025979-833158990-5https://doi.org/10.1109/IDAP68205.2025.11222357https://hdl.handle.net/20.500.12684/230429th International Artificial Intelligence and Data Processing Symposium, IDAP 2025 -- 6 September 2025 through 7 September 2025 -- Malatya -- 215321Vision Language Models (VLMs) have emerged as powerful architectures capable of jointly processing visual and textual inputs across a range of multimodal tasks. However, despite their growing adoption, a systematic evaluation of open-source VLMs on standard image captioning benchmarks remains limited. This study presents a comparative evaluation of three widely used open-source VLMs - LLaVA-7B, MiniCPM-V-8B, and Qwen2.5-VL-7B - on the task of image captioning using the Flickr8k dataset. We conduct a zero-shot assessment of each model's ability to generate descriptive captions for natural images and evaluate their outputs using a combination of lexical and semantic similarity metrics, including BLEU, METEOR, ROUGE-L, and BERTScore (F1). The results reveal notable differences in model behavior and output quality: Qwen2.5-VL-7B consistently achieves the highest scores across most metrics, while LLaVA-7B underperforms, likely due to its focus on instruction-following and visual question answering rather than structured caption generation. Despite Qwen2.5-VL-7B's relative advantage, all models fall short of human-level performance, underscoring the limitations of general-purpose VLMs in producing concise, reference-aligned captions without task-specific fine-tuning. Our findings highlight the importance of dataset alignment, prompt design, and semantic-aware evaluation methods in future research on open-source multimodal systems. © 2025 IEEE.en10.1109/IDAP68205.2025.11222357info:eu-repo/semantics/closedAccessartificial intelligencecomputer visionlanguage modelstring similarityvision language modelA Comparative Evaluation of Open-Source Vision Language Models for Image CaptioningConference Object2-s2.0-105025032094N/A