Integration of a BCI with a Hand Tracking System and a Motorized Robotic Arm to Improve Decoding of Brain Signals Related to Hand and Finger Movements

Nowadays, most of the functional relationships between brain signals and the corresponding complex hand/fingers movements (cause, effect, feedback) are not yet completely understood. In the last years it has been assisted to important advances in Brain Computer Interfaces (BCI), computer vision (CV)-based tracking systems and Robots, especially due to: hardware improvements and miniaturization; increasing pursuit of intelligent and real time tracking systems; fast design, prototyping and production of consumer robots facilitated by 3D printing technologies. We present an integrated system composed by a BCI, a CV-based hand tracking system and a motorized robotic arm reproducing the hand and the forearm of a person (scale 1:1). The proposal is to synchronize and to monitor the brain activity during complex hand and fingers movements (interpreted and reproduced in real time on a numerical hand model by the tracking system). Further, we aim at recognizing the brain signals which give rise to specific movements and, finally, at using them for producing the corresponding movements on the robotic arm. Different scenarios and potential use-cases are reported and their usefulness discussed.