Sensor and Actuator Latency during Teleoperation of Automated Vehicles

Due to the challenges of autonomous driving, backup options like teleoperation become a relevant solution for critical scenarios an automated vehicle might face. To enable teleoperated systems, two main problems have to be solved: Safely controlling the vehicle under latency, and presenting the sensor data from the vehicle to the operator in such a way, that the operator can easily understand the vehicles environment and the vehicles current state. While most of the teleoperation systems face similar challenges, the teleoperation of automated vehicles is unique in its scale, safety requirements and system constraints. Two major constraints are the round-trip-latency and the maximum upload-bandwidth. While the latency mainly influences the controllability and safety of the vehicle, the upload-bandwidth affects the amount of transmittable sensor data and therefore operators situation awareness, as well as the running costs of the whole system. The focus of this paper is measuring and reducing the end-to-end latency for a teleoperation setup. Therefore the latency is separated into actuator and sensor latency. For each part the different components and settings are analyzed in order to find a realistic minimal end-to-end latency for the teleoperation of automated vehicles. Therefore new measurement methods are developed and existing methods adapted.