Vizijski sustav za upravljanje industrijskim robotom pokretima ruke

master's thesis

• Title (english): A vision system for controlling an industrial robot with hand movements
• Author: Matija Zidarić
• Mentor: Tomislav Stipančić (mentor)
• Granter: University of Zagreb; Faculty of Mechanical Engineering and Naval Architecture; Zagreb
• Defense date and country: 2024-07-15, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Mechanical Engineering; Production Mechanical Engineering

Abstract

Moćne kamere i suvremeni modeli umjetne inteligencije omogućuju zanimljivu sintezu i
različite aplikacije. Fokus rada jest izrada vizijskog sustava koji prepoznaje geste ruke te
upravlja industrijskim robotom u realnome vremenu. U Python programskome jeziku koristi
se MediaPipe Hands model za prepoznavanje i praćenje ruke koju snima Intel RealSense
D435 kamera, a RoboDK je posrednik za komunikaciju s robotima. Zbog šuma sustava
izvršeno je digitalno filtriranje koordinata. Robotske hvataljke izrađene su 3D tiskanjem za
potrebe eksperimentalne evaluacije. Aplikacija je testirana na UR5 i Fanuc LR Mate 200iC
industrijskim robotima.

Abstract (english)

Powerful cameras and modern artificial intelligence models allow for interesting synthesis
and different applications. The focus of this thesis is the development of a vision system that
recognizes hand gestures and controls the industrial robot in real time. In the Python
programming language, a MediaPipe hands model is used to recognize and track the hand
being captured by the Intel RealSense D435 camera, withRoboDK as the intermediary for
communication with robots. Due to system noise, digital filtering of coordinates was carried
out. Robot grips have been made by 3D printing for the purpose of experimental evaluation.
The application was tested on UR5 and Fanuc LR Mate 200ic industrial robots.

• Keywords: Vizijski sustav; industrijska robotika; Python; MediaPipe Hands; RealSense D435; RoboDK; digitalni filter; 3D tiskanje; UR; Fanuc
• Keywords (english): Vision system; industrial robotics; Python; MediaPipe Hands; RealSense D435; digital filter; 3D printing; UR; Fanuc
• Language: croatian
• URN:NBN: urn:nbn:hr:235:108157
• Study programme: Title: Mechanical Engineering; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka strojarstva (magistar/magistra inženjer/inženjerka strojarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Open access
• Created on: 2024-07-12 09:01:35