End-to-End Visuomotor Learning from Virtual Environment to Real Robot

Robots can acquire skills to accomplish a target task by learning human manipulations. In this study, we build an end-to-end visuomotor learning system for a robot to learn multiple tasks in a virtual environment, and then perform the same tasks in a real environment without re-training. We use domain randomization to improve the generalization performance of the learning model. To effectively tackle this challenge, we build an integrated learning system that jointly learns robot motions and visual features of the task. The experimental results show that our system can perform multiple tasks with a high success rate and is able to successfully bridge the Sim2Real gap, compared to learning motion and visual features separately.