TY - GEN
T1 - ToF/Radar early feature-based fusion system for human detection and tracking
AU - Zoghlami, Feryel
AU - Sen, Okan Kamil
AU - Heinrich, Harald
AU - Schneider, Germar
AU - Ercelik, Emec
AU - Knoll, Alois
AU - Villmann, Thomas
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021/3/10
Y1 - 2021/3/10
N2 - Industry 4.0 has become a general keyword over the last years. It is based on the inclusion of automation by increasing connectivity in various tasks during the production process. This fact did not exclude the human's effort whose presence remains important, especially the interaction between humans and robots will be a key element in the future manufacturing. In automated production lines, we find both humans and robots operating side-by-side in hybrid workplaces. The major focus for this workplaces today and in the future is to establish a safe work environment. However, what if safety meets "collaborative efficiency"? The system presented in this paper relies on the fusion of data coming from a Time of Flight (ToF) sensor and a 60 GHz radar sensor. The data are analyzed and evaluated using deep learning (DL) algorithms. The purpose is to detect humans and track their movements in the observed area. The resulted perception system can be installed somewhere in a room or on a moving system. A first demonstrator has been developed, tested and evaluated. An additional graphical interface was developed to show in real time the capability of the data fusion system. The system can detect up to 5 persons in a selected area with 98% confidentiality. The so-described system is able as well to estimate each person DoM and the person's instantaneous speed and position. Based on the output of our developed system, it is possible to define industrial use cases as well as many other different applications in different fields.
AB - Industry 4.0 has become a general keyword over the last years. It is based on the inclusion of automation by increasing connectivity in various tasks during the production process. This fact did not exclude the human's effort whose presence remains important, especially the interaction between humans and robots will be a key element in the future manufacturing. In automated production lines, we find both humans and robots operating side-by-side in hybrid workplaces. The major focus for this workplaces today and in the future is to establish a safe work environment. However, what if safety meets "collaborative efficiency"? The system presented in this paper relies on the fusion of data coming from a Time of Flight (ToF) sensor and a 60 GHz radar sensor. The data are analyzed and evaluated using deep learning (DL) algorithms. The purpose is to detect humans and track their movements in the observed area. The resulted perception system can be installed somewhere in a room or on a moving system. A first demonstrator has been developed, tested and evaluated. An additional graphical interface was developed to show in real time the capability of the data fusion system. The system can detect up to 5 persons in a selected area with 98% confidentiality. The so-described system is able as well to estimate each person DoM and the person's instantaneous speed and position. Based on the output of our developed system, it is possible to define industrial use cases as well as many other different applications in different fields.
KW - automated fabrication
KW - deep learning
KW - human/robotic collaboration
KW - industry 4.0
KW - machine learning
KW - radar sensor
KW - sensor fusion
KW - time of flight camera
UR - http://www.scopus.com/inward/record.url?scp=85112561346&partnerID=8YFLogxK
U2 - 10.1109/ICIT46573.2021.9453703
DO - 10.1109/ICIT46573.2021.9453703
M3 - Conference contribution
AN - SCOPUS:85112561346
T3 - Proceedings of the IEEE International Conference on Industrial Technology
SP - 942
EP - 949
BT - Proceedings - 2021 22nd IEEE International Conference on Industrial Technology, ICIT 2021
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 22nd IEEE International Conference on Industrial Technology, ICIT 2021
Y2 - 10 March 2021 through 12 March 2021
ER -