Visual repetition sampling for robot manipulation planning

One of the main challenges in sampling-based motion planners is to find an efficient sampling strategy. While methods such as Rapidly-exploring Random Tree (RRT) have shown to be more reliable in complex environments than optimization-based methods, they often require longer planning times, which reduces their usability for real-time applications. Recently, biased sampling methods have shown to remedy this issue. For example Gaussian Mixture Models (GMMs) have been used to sample more efficiently in feasible regions of the configuration space. Once the GMM is learned, however, this approach does not adapt its biases to individual planning scene during inference. Hence, we propose in this work a more efficient sampling strategy to further bias the GMM based on visual input upon query. We employ an autoencoder trained entirely in simulation to extract features from depth images and use the latent representation to adjust the weights of each mixture components in the GMM. We show empirically that this improves the sampling efficiency of an RRT motion planner in both real and simulated scenes.