Compact, Accurate and Low-cost Hand Tracking System based on LEAP Motion Controllers and Raspberry Pi

The large diffusion of low-cost computer vision (CV) hand tracking sensors used for hand gesture recognition, has allowed the development of precise and low cost touchless tracking systems. The main problem with CV solutions is how to cope with occlusions, very frequent when the hand has to grasp a tool, and self-occlusions occurring when some joint obscures some other. In most cases occlusions are solved by using synchronized multiple stereo sensors. Virtual Glove (VG) is one of the CV-based systems that uses two orthogonal LEAP sensors integrated into a single system. The VG system is driven by a Personal Computer in which both a master operating system (OS) and a virtual machine have to be installed in order to drive the two sensors (just one sensor at a time can be driven by a single OS instance). This is a strong limitation because VG has to run on a powerful PC, thus resulting in a not properly low-cost and portable solution. We propose a VG architecture based on three Raspberry Pi (RP), each consisting of a cheap single board computer with Linux OS. The proposed architecture assigns an RPi to each LEAP and a third RP to collect data from the other two. The third RP merges, in real time, data into a single hand model and makes it available, through an API, to be rendered in a web application or inside a Virtual Reality (VR) interface. The detailed design is proposed, the architecture is implemented and experimental benchmark measurements, demonstrating the RPi-based VG real-time behaviour while containing costs and power consumption, are presented and discussed. The proposed architecture could open the way to develop modular hand tracking systems based on more than two LEAPs, each associated to one RP, in order to further improve robustness.