TY - GEN
T1 - Self-Supervised Object Recognition Based on Repeated Re-Capturing of Dynamic Indoor Environments
AU - Piccolrovazzi, Martin
AU - Adam, Michael G.
AU - Eger, Sebastian
AU - Zakour, Marsil
AU - Steinbach, Eckehard
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - Capturing a digital replica of an environment using hand held devices or mobile mapping systems has become increasingly easy in recent years. However, leveraging large amounts of data for various semantic applications is usually impaired by a costly data annotation process. In this paper, we investigate the topic of object recognition in indoor environments without supervision. We approach the problem from a remapping perspective, where we capture RGB images from the same environment at different times and use the naturally occurring changes to identify single objects. In the first step, we create pairs of images from different recordings and generate object candidates using optical flow and an off-the-shelf region proposal algorithm. Then, we use a self-supervised representation learning framework and cluster the extracted objects. We evaluate the performance of several existing clustering methods in an over-clustering setting, since the number of object classes is unknown in an unsupervised setup. Our experimental validation on a real-world dataset shows that the proposed system can successfully recognize objects and pre-annotate a dataset by exploiting a recapturing process.
AB - Capturing a digital replica of an environment using hand held devices or mobile mapping systems has become increasingly easy in recent years. However, leveraging large amounts of data for various semantic applications is usually impaired by a costly data annotation process. In this paper, we investigate the topic of object recognition in indoor environments without supervision. We approach the problem from a remapping perspective, where we capture RGB images from the same environment at different times and use the naturally occurring changes to identify single objects. In the first step, we create pairs of images from different recordings and generate object candidates using optical flow and an off-the-shelf region proposal algorithm. Then, we use a self-supervised representation learning framework and cluster the extracted objects. We evaluate the performance of several existing clustering methods in an over-clustering setting, since the number of object classes is unknown in an unsupervised setup. Our experimental validation on a real-world dataset shows that the proposed system can successfully recognize objects and pre-annotate a dataset by exploiting a recapturing process.
KW - Clustering
KW - Convolutional Neural Networks
KW - Deep Learning
KW - Object Recognition
KW - Self-Supervised Learning
UR - http://www.scopus.com/inward/record.url?scp=85147549161&partnerID=8YFLogxK
U2 - 10.1109/ISM55400.2022.00023
DO - 10.1109/ISM55400.2022.00023
M3 - Conference contribution
AN - SCOPUS:85147549161
T3 - Proceedings - 2022 IEEE International Symposium on Multimedia, ISM 2022
SP - 105
EP - 112
BT - Proceedings - 2022 IEEE International Symposium on Multimedia, ISM 2022
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 24th IEEE International Symposium on Multimedia, ISM 2022
Y2 - 5 December 2022 through 7 December 2022
ER -