Aplikacija afektivnog virtualnog agenta temeljena na višemodalnoj interakciji

master's thesis

• Title (english): An affective virtual agent application based on multimodal interaction
• Author: Andrija Ričko
• Mentor: Tomislav Stipančić (mentor)
• Granter: University of Zagreb; Faculty of Mechanical Engineering and Naval Architecture; Zagreb
• Defense date and country: 2022-07-20, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing; Program Engineering

Abstract

Tema ovog rada je izrada programske upravljačke platforme i vizualizaciju afektivnog
virtualnog robota PLEA koji se razvija u sklopu Laboratorija za projektiranje izradbenih i
montažnih sustava. Takva platforma krajnjem korisniku omogućuje kreiranje vlastitog profila,
konfiguracija spremanja podataka te pregledavanje i brisanje spremljenih podataka te
komunikaciju s virtualnim agentom. Ako je krajnji korisnik administrator, platforma
omogućuje dodatne opcije kao što su uživo pregledavanje svih povezanih korisnika,
konfiguriranje virtualnog agenta, daljinsko otvaranje komunikacije s određenim klijentima i
pregledavanje povijesti svih konekcija.
Vizualizacija virtualnog agenta izrađena je u programu Unreal Engine 4 pomoću
programskog dodatka Metahuman Creator. Izrađen je set animacija lica koje se aktiviraju kao
odgovor na emociju korisnika s kojim PLEA komunicira. Prijenos podataka (slika i zvuk)
između korisnika i virtualnog agenta ostvaruje se pomoću istorazinske (eng. Peer to Peer)
komunikacije koja omogućuje vizualnu interakciju realnom vremenu.

Abstract (english)

The topic of this paper is the creation of a program management platform and
visualization of the affective virtual robot PLEA, which is being developed as part of the
Laboratory for Designing Manufactured and Assembled Systems. Such a platform enables the
end user to create his own profile, configure data storage, view and delete saved data, and
communicate with a virtual agent. If the administrator is the end user, the platform offers
additional options such as directly viewing all connected users, configuring a virtual agent,
remotely opening communication with specific clients and viewing the history of all
connections.
The visualization of the virtual agent was created in Unreal Engine 4 using the Metahuman
Creator plugin. A set of facial animations was created that are activated in response to the
emotion of the user with whom PLEA communicates. Data transfer (image and sound) between
the user and the virtual agent is achieved using peer-to-peer communication, which enables
visual interaction in real time.

• Keywords: afektivna robotika; Unreal Engine 4; WebRTC; internetska platforma; imitacija
• Keywords (english): affective robotics; Unreal Engine 4; WebRTC; web platform; imitation
• Language: croatian
• URN:NBN: urn:nbn:hr:235:959860
• Study programme: Title: Mechanical Engineering; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka strojarstva (magistar/magistra inženjer/inženjerka strojarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Open access
• Created on: 2022-07-18 18:58:54