Razvoj sustava s više dubinskih kamera koji generira skupove podataka za treniranje neuronskih mreža

master's thesis

• Title (english): Development of a multiple depth-camera system for generating datasets to train neural networks
• Author: Vlatko Novak
• Mentor: Marko Švaco (mentor)
• Mentor: Filip Šuligoj (komentor)
• Granter: University of Zagreb; Faculty of Mechanical Engineering and Naval Architecture; Zagreb
• Defense date and country: 2022-07-18, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Electrical Engineering; Automation and Robotics

Abstract

U ovom radu je izrađen sustav za rekonstrukciju 3D geometrije proizvoda i za automatsko generiranje seta 2D slika spremnih za treniranje neuronskih mreţa. Prvo je simuliran postav s 3 kamere i konstruiran nosač za kameru u programskom CAD paketu CATIA V5R20. Nosač za kamere je zatim napravljen aditivnom tehnologijom 3D printanja. Korišten je postav od aluminijskih profila na kojima su bile montirane 3 kamere Intel Realsense D435. Kamerama se upravljalo kodom u C++ programskom jeziku pisanim u integriranom razvojnom okruţenju (engl. IDE) QT. Korištene su biblioteke: Open3D koja sluţi za dohvaćanje i obradu oblaka točaka, Eigen za operacije s linearnom algebrom, OpenCV za 2D obradu slika, te biblioteke dane od strane proizvoĎača kamere pod nazivom Librealsense za konfiguriranje i upravljanje kamerama. Za 3D rekonstrukciju predmeta korišteni su koncepti stereo kalibracije kamera, smanjenje uzorka oblaka točaka, izrezivanje, odabir i izbacivanje po indeksu oblaka točaka te algoritam za minimiziranje razlike između dva oblaka točaka ICP (Iterative Closest Point). Za generiranje 2D slika spremnih za treniranje neuronskih mreţa koristile su se mnoge funkcije dostupne u biblioteci OpenCV. Neke od funkcija su: pretvorba slike u boji u slike u nijansama sive, pretvorba u binarnu sliku, segmentacija boje iz slike, pronalazak kontura na slici, iscrtavanje pravokutnika oko predmeta. Nakon što je generiran set slika, one su dane neuronskoj mreži YOLOv5 na treniranje kako bi neuronska mreţa bila spremna za kasniju detekciju predmeta. YOLOv5 je pokazao visok postatak uspješnosti detekcije naučenih predmeta - točnost od oko 90%.

Abstract (english)

In this thesis, a system for 3D reconstruction of product’s geometry and automatically generated dataset of 2D images fit to train neural networks was developed. Firstly, a setup with three cameras was simulated, and a camera carrier was design in software package CATIA V5R20. The camera carrier was made via 3D printing additive manufacturing process. The setup was made out of aluminum profiles, on top of which three Intel Realsense D435 cameras were montaged. Cameras were managed by a code written in C++ programming language in integrated development environment (IDE) QT. There were used libraries: Open3D for capturing and processing point cloud, Eigen for operations with linear algebra, OpenCV for 2D image processing and libraries supplied by camera manufacturer named Liberalsense for configuring and managing cameras. Concepts like stereo camera calibration, reducing the number of points, cropping, selecting by indices and algorithm for minimizing the difference between two point clouds ICP (Iterative Closest Point) were used for 3D reconstruction of product’s geometry. A lot of functions from an OpenCV library were used for generating 2D images ready to train a neural network. Some used functions are: color image conversion to grayscale, conversion to binary image color segmentation, contours detection, drawing a rectangle on image. After the dataset of images is generated, they are forwarded to neural network YOLOv5 for training so that the neural network is ready for object detection. YOLOv5 showed a high rate of detection of learned objects – an accuracy of about 90%.

• Keywords: stereo kamera; 3D rekonstrukcija geometrije proizvoda; automatsko generiranje slika za treniranje neuronske mreže; C++; Open3D; Eigen; OpenCV; ICP; YOLOv5; detekcija predmeta.
• Keywords (english): stereo camera; 3D reconstruction of product’s geometry; automatic generation of images for neural network training; C++; Open3D; Eigen; OpenCV; ICP; YOLOv5; object detection.
• Language: croatian
• URN:NBN: urn:nbn:hr:235:037035
• Study programme: Title: Mechanical Engineering; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka strojarstva (magistar/magistra inženjer/inženjerka strojarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Open access
• Created on: 2022-07-07 06:38:22