Can we reach human expert programming performance? A tactile manipulation case study in learning time and task performance

Reaching human-level performance in tactile manipulation is one of the grand challenges in nowadays robotics research. Over the past decade significant progress in both skill control and learning was made. However, the achievable execution speed still falls behind the human ability, without clearly understanding whether the specific shortcomings are mainly in the control, skill learning, or motion planning layer. For gaining a better understanding of this complex problem, we draw an experimental side-by-side comparative case study. First, given a task program for a challenging benchmarking task, the goal is to objectify the achievable task performance from a human expert programmer against autonomously learning these assembly behaviors with a state-of-the-art skill learning framework. Second, we compare the manually tuned and learned robot skills to the performance of an adult human solving the task manually. For the former, it could be shown that despite longer learning duration, the task execution speed of the machine learning-based solution is equivalent to the one programmed by the human expert. For the latter, the identified performance gap remained significantly larger, where only for some specific isolated skills the system was able to reach comparable or even faster than human execution speeds. The overall analysis gave also useful hints where in particular manipulation policies and arm-hand coordination still need significant improvements in the future.