From Generation to Detection: A Multimodal LLM-Based Framework for Realistic Vishing Attacks in Turkish

dc.contributor.authorAlbayrak, Ahmet
dc.date.accessioned2026-07-01T11:36:27Z
dc.date.available2026-07-01T11:36:27Z
dc.date.issued2026
dc.departmentDüzce Üniversitesi
dc.description.abstractThis study proposes a multimodal and AI-supported approach for both generation and detection of vishing threats, also known as audio social engineering attacks. By considering text and audio modalities together, fake contents specific to Turkish language were generated with GPT-2 and Tacotron-2 models; detection processes were carried out with discriminator structures such as BERT, CNN, LSTM, and Whisper. The developed system was designed with an integrated analysis (multimodal fusion) approach at feature and decision levels and was evaluated at both technical and psychosocial levels. In user tests conducted with 394 participants, the rate of perception of fake contents as real was measured in the range of 92.8-95.3%; it was observed that contents containing official institution imitations were the most convincing class. Experimental results show that the F1 scores obtained with single models remained in the range of 0.86-0.91%; however, 96.8% accuracy and 0.947 F1 score were achieved with the BERT + CNN + Whisper architecture. It has also been determined that GPT-4-supported content generation increases the plausibility of attack scenarios and challenges the performance of classifier models. The study is a pioneer in the field of fake audio content generation and detection in Turkish and reveals how LLM-supported multimodal systems can be used in both individual and institutional defense applications. © 2026 The Voice Foundation. Published by Elsevier Inc. All rights are reserved, including those for text and data mining, AI training, and similar technologies.
dc.identifier.doi10.1016/j.jvoice.2026.04.042
dc.identifier.issn0892-1997
dc.identifier.scopus2-s2.0-105039686722
dc.identifier.scopusqualityQ1
dc.identifier.urihttps://doi.org/10.1016/j.jvoice.2026.04.042
dc.identifier.urihttps://hdl.handle.net/20.500.12684/23034
dc.indekslendigikaynakScopus
dc.institutionauthorAlbayrak, Ahmet
dc.language.isoen
dc.publisherElsevier Inc.
dc.relation.ispartofJournal of Voice
dc.relation.publicationcategoryMakale - Uluslararası Hakemli Dergi - Kurum Öğretim Elemanı
dc.rightsinfo:eu-repo/semantics/closedAccess
dc.snmzKA_Scopus_20260623
dc.subjectGenerative AI
dc.subjectSocial engineering
dc.subjectText-to-speech fusion
dc.subjectVishing
dc.subjectVoice fraud
dc.titleFrom Generation to Detection: A Multimodal LLM-Based Framework for Realistic Vishing Attacks in Turkish
dc.typeArticle

Dosyalar