Interactive RGB Image Segmentation via Depth-modified Click Encoding and Estimated Depth

Interactive image segmentation separates the object of interest in the scene from the background, using the help of human annotations. Click-based interactive segmentation methods typically receive positive and negative clicks from the user indicating the object of interest and the background. These clicks are encoded into click-maps to be processed further by a neural network. Although the depth information is known to improve the image segmentation accuracy, the previous work encodes only the locations of the clicks into the click-maps. We propose two novel click-map generation methods that modify the conventional click-maps using relative depth information. This depth information is estimated from the monocular RGB image. After retraining the baseline interactive segmentation method with our novel click-maps, the segmentation accuracy improved without requiring any additional input or increasing the network size. Experimental evaluations showed that our method yields a better mean segmentation accuracy on the Berkeley and DAVIS datasets than the baseline using conventional click-maps, and a comparable performance on the GrabCut dataset.