Sistema intelligente di tele-riabilitazione e tele-monitoraggio basato su sensori 3D e tecniche avanzate di computer vision

Telerehabilitation is important for the restoration of hand mobility in patients with after-stroke and after-surgery problems or neurological conditions. In this work, we aim to create a remote hand rehabilitation system employing the Virtual Glove (VG). The VG is a computer vision-based structure that simultaneously utilizes two synchronized LEAP Motion Controllers (LMCs) to track the motion of the hand and reconstruct a numerical hand model in real time. It has been conceptualized for telerehabilitation purposes, but it can also be used for human-system interaction. We have analyzed the use of computer vision methods that employ more than one sensor and focused on optimizing the function of the VG for detailed hand and finger movement tracking. Several methods have been developed and tested to efficiently combine the data from the two infrared (IR) sensors. The problem of occlusions, frequent and common in computer vision systems, has been examined, and methods for minimizing such a problem, while maintaining real-time, have been used. The results have been implemented to a complete framework for at-home hand rehabilitation, in a system made for the use of the patient as well as the therapist. The total of the data analysis procedure has been implemented in an interactive application designated to be used by a therapist to control and analyze hand telerehabilitation sessions, numerically control joints' mobility, as well as to organize personalized exercises for each patient. Then, to allow the system to be used together with Machine Learning techniques, a humanoid actuator in the form of a human hand has been 3D printed and used to produce controlled wrist and finger movements, recorded by the VG. A complete pipeline for data analysis and a method to map spatial VG information to the motor movements of a mechanical actuator have been established. Also, a method for moving to a sensor-independent, hand-based Cartesian coordinate reference system has been developed. Finally, the pipeline has been developed to create a database of hand and finger movement data with ground truth information using the 3D printed actuator.