Optimizacija, prilagodba i procjena vizijsko-jezičnih modela

master's thesis

• Title (english): Optimization, fine tuning, and evaluation of vision-language models
• Author: Luka Majnarić
• Mentor: Petar Ćurković (mentor)
• Granter: University of Zagreb; Faculty of Mechanical Engineering and Naval Architecture; Zagreb
• Defense date and country: 2025-03-27, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Mechanical Engineering

Abstract

Ovaj rad istražuje optimizaciju vizijsko-jezičnih modela (VLM) putem kvantizacije i finog podešavanja kako bi se poboljšala učinkovitost na hardveru s ograničenim resursima. Metoda kvantizacije LLM.int8() smanjuje troškove memorije i računanja uz očuvanje točnosti modela. Fino podešavanje primjenjuje se na skup arhitektonskih podataka, poboljšavajući točnost klasifikacije s 24% na 83% i poboljšavajući sposobnost modela da generira detaljne opise stilova gradnje. Kombinirajući kvantizaciju s finim podešavanjem, pokazano je da VLM-ovi mogu postići visoke performanse uz smanjene računalne zahtjeve, što ih čini praktičnijima za specijalizirane aplikacije. Ovi rezultati pridonose širem cilju da se veliki modeli umjetne inteligencije učine pristupačnijim i učinkovitijim.

Abstract (english)

This thesis explores the optimization of vision-language models (VLMs) through quantization and fine-tuning for efficient deployment on resource-limited hardware. The LLM.int8() quantization method reduces memory and computation costs while preserving accuracy. Fine-tuning is applied to an architectural dataset, improving classification accuracy from 24% to 83% and enhancing the model’s ability to generate detailed descriptions of building styles. By combining quantization with fine-tuning, it is demonstrated that VLMs can achieve high performance with reduced computational requirements, making them more practical for specialized applications. These results contribute to the broader goal of making large-scale AI models more accessible and efficient.

• Keywords: Vizijsko-jezični modeli; Veliki jezični modeli; Kvantizacija; Fino podešavanje; Optimizacija modela
• Keywords (english): Vision-language models; Large language models; Quantization; Fine-tuning; Model optimization
• Language: croatian
• URN:NBN: urn:nbn:hr:235:550344
• Study programme: Title: Mechanical Engineering; specializations in: Design, Process and Energy Engineering, Production Engineering, Engineering Modelling and Computer Simulation, Marine Engineering, Industrial Engineering and Management, Materials Engineering, Mechanics and Robotics, Autonomous Systems, Mechatronics and Robotics; Course: Mechatronics and Robotics; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka strojarstva (magistar/magistra inženjer/inženjerka strojarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Open access
• Created on: 2025-03-26 11:26:21