Estimating Virtual Fixture Parameters in Digital Twin Environments for Robot Manipulation Tasks using Reinforcement Learning

In recent years, the idea of teleoperating robots to perform manipulation tasks has gained popularity. Being able to take remote control of a robotic system performing an assembly task provides exciting possibilities for industry, not only operating the robot in hazardous environments, but also facilitating the accessibility of human workers, requesting help from a remote expert, or even teaching skills. With the rise of teleoperated manipulators, it is expected that tools that facilitate the interaction of humans with the remote environment will be more prevalent in industry, leading to a greater availability of them. This is the case with Virtual Fixtures (VFs), which are collections of abstract sensory information overlaid on top of reflected sensory feedback from a remote environment. Other ever more prevalent tools are Digital Twins (DTs), virtual representations of systems that facilitate bidirectional communication between the real and the virtual worlds. Being more likely than ever that factories have both an implemented Digital Twin of a system and VFs for a particular manipulation system, we propose a method to leverage both tools. In this paper, methods to integrate already existing VFs in a Reinforcement Learning (RL) pipeline and to use RL to construct an optimal VF to aid the human user in a telemanipulation task are proposed, tackling the cumbersome problems of reward function and VF design. The results show that this approach is able to correctly estimate the right parameters of predefined VFs and open the possibility to future work in this topic.