Inteligentno robotsko rukovanje i vizualna detekcija prehrambenih proizvoda

master's thesis

• Title (english): Intelligent robotic handling and visual detection of food products
• Author: Dominik Levanić
• Mentor: Marko Švaco (mentor)
• Mentor: Filip Šuligoj (komentor)
• Granter: University of Zagreb; Faculty of Mechanical Engineering and Naval Architecture; Zagreb
• Defense date and country: 2022-01-20, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Mechanical Engineering

Abstract

U današnje vrijeme proizvodnja postaje sve fleksibilnija, zbog čega se kod robotskog rukovanja javlja sve veća potreba za brzom prilagodljivošću i invarijantnošću prema svojstvima proizvoda. Zadatak ovog diplomskog rada je osmisliti i integrirati sustav za inteligentno robotsko rukovanje prehrambenim proizvodima, temeljen na prikupljanju njihovih slika i oblaka točaka, detekciji na slikama neuronskom mrežom, određivanju položaja u prostoru te planiranju i realiziranju hvatanja na temelju geometrije. Korištene se slike i oblaci točaka prikupljaju 3D vizijskim sustavom Intel RealSense D435 postavljenim na kolaborativnog robota Universal Robots UR5, uz provođenje kalibracije 3D vizijskog sustava naspram robotske hvataljke. Za detekciju proizvoda na slikama koristi se neuronska mreža YOLO v5, trenirana na posebno izrađenom skupu označenih slika, dok se za određivanje položaja proizvoda u prostoru koristi knjižnica Open3D za obradu oblaka točaka. Kao temelj za planiranje hvatanja proizvoda koristi se algoritam za analizu glavnih komponenti (eng. Principal component analysis – PCA). Navedeni se elementi sustava povezuju grafičkim korisničkim sučeljem, dok se komunikacija između robota i računala ostvaruje TCP/IP protokolom. U radu se najprije daje uvod u navedeno područje s detaljima zadatka, nakon čega se opisuje način integracije svakog od elemenata sustava, uz zasebne analize njihovih rezultata. Zadatak se zaokružuje analizom rezultata i evaluacijom sustava u cjelini, nakon čega se izvodi odgovarajući zaključak.

Abstract (english)

Nowadays, production is becoming more flexible, which is why robotic handling is increasingly in the need of rapid adaptability and invariance to product properties. The goal of this master thesis is to design and integrate a system for intelligent robotic handling of food products, based on collecting their images and point clouds, detection in images by a neural network, determining spatial position and planning and realizing grasping based on geometry. The images and point clouds used are collected by an Intel RealSense D435 3D vision system mounted on a Universal Robots UR5 collaborative robot, with an eye-in-hand calibration of the 3D vision system against his gripper. The YOLO v5 neural network, trained on a specially designed set of annotated images is used to detect the product in images, while the Open3D library for point cloud processing is used to determine the spatial position of the product. The Principal component analysis (PCA) algorithm is used as the basis for planning of product grasping. The listed elements of the system are connected by a graphical user interface, while the communication between the robot and computer is realized by the TCP/IP protocol. The thesis first gives an introduction to stated area with details of the task, after which the method of integration for each element of the system is described, with separate analyses of their results. The task is completed by analyzing the results and evaluating the complete system, after which anappropriate conclusion is given.

• Keywords: robotsko rukovanje; detekcija objekata; oblak točaka; 3D vizijski sustav
• Keywords (english): robotic handling; object detection; point cloud; 3D vision system
• Language: croatian
• URN:NBN: urn:nbn:hr:235:227988
• Study programme: Title: Mechanical Engineering; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka strojarstva (magistar/magistra inženjer/inženjerka strojarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Open access
• Created on: 2022-01-12 22:18:35