MirrARbilitation: A clinically-related gesture recognition interactive tool for an AR rehabilitation system

Background and Objective Interactive systems for rehabilitation have been widely investigated for motivational purposes. However, more attention should be given to the manner in which user movements are recognized and categorized. This paper aims to evaluate the efficacy of using a clinically-related gesture recognition tool, based on the international biomechanical standards (ISB) for the reporting of human joint motion, for the development of an interactive augmented reality (AR) rehabilitation system —mirrARbilitation. Methods This work presents an AR rehabilitation system based on ISB standards, which enables the system to interact and to be configured according to therapeutic needs. The Kinect^TM skeleton tracking technology was exploited and a new movement recognition method was developed to recognize and classify biomechanical movements. Further, our mirrARbilitation system provides exercise instructions while simultaneously motivating the patient. The system was evaluated on a cohort of 33 patients, physiotherapists, and software developers when performing shoulder abduction therapy exercises. Tests were performed in three moments: (i) users performed the exercise until they feel tired without the help of the system, (ii) the same however using the mirrARbilitation for motivation and guidance, and (iii) users performed the exercise again without the system. Users performing the movement without the help of the system worked as baseline reference. Results We demonstrated that the percentage of correct exercises, measured by the movement analysis method we developed, improved from 69.02% to 93.73% when users interacted with the mirrARbilitation. The number of exercise repetitions also improved from 34.06 to 66.09 signifying that our system increased motivation of the users. The system also prevented the users from performing the exercises in a completely wrong manner. Finally, with the help of our system the users' worst result was performing 73.68% of the rehabilitation movements correctly. Besides the engagement, these results suggest that the use of biomechanical standards to recognize movements is valuable in guiding users during rehabilitation exercises. Conclusion The proposed system proved to be efficient by improving the user engagement and exercise performance outcomes. The results also suggest that the use of biomechanical standards to recognize movements is valuable in guiding users during rehabilitation exercises.