The hand is one of the most complex and beautiful pieces of natural engineering in the human body. Much research in academia and industry has focused on fast and accurate finger motion tracking. For tasks that are difficult to accomplish with traditional interactive devices, this significantly improves the efficiency of human-computer interaction. It can be widely used in military, entertainment, sports, medical applications, robot technology, and even the metaverse. Still, existing technology for finger motion tracking often requires wearing a heavy device and a laborious calibration process to track the bending angle of each joint.
To address this issue, researchers have published an article in IEEE Transactions on Human-Machine Systems proposing a compressed sensing-based tracking (CST) framework that enables the estimation of the bending angle of all hand joints using sensors smaller than the number of hand joints. The motion-tracking gloves developed because of this study are user-friendly and particularly suitable for human-computer interaction applications in robot control, the metaverse, and other fields.
Motion Tracking Technology
Motion-capture gloves are a standard tool for finger movement tracking, which can provide hand and finger tracking data by implementing various types of sensors inside or outside a glove. According to the researchers, they can be divided into four categories: inertial measurement unit (IMU)-based schemes, optical-based finger motion tracking, flexible sensor-based schemes, and multicategory sensor-based schemes. During hand motion, the hand muscles produce linkages, which are collected by a data glove.
The researchers have proposed a finger motion tracking method based on compressed sensing. This method uses a compressed sensing-based tracking (CST) framework to track a small number of sensors on gloves and maps the bending angle information of all joints to an anatomical hand constraint model.
CST framework system and applications.
The framework reduces hardware costs and enhances real-time performance. The compressed sensing algorithm included in our CST framework can also perform automatic calibration and filter noise reduction, solve the differences in gloves worn by different users and the errors caused by glove looseness, and coordinate joint activities according to the anatomical constraints of the hands. Specifically, the researchers built a CST framework for finger motion tracking using flexible sensor gloves. These gloves are equipped with liquid metal sensors, a novel type of sensor that can accurately detect and measure the bending angle of the finger joints. The liquid metal sensors are integrated with an IMU powered by batteries and transmit data through Bluetooth without cables.
Measuring Success
The CST framework includes models of the bending model of finger joints according to the anatomical constraints of fingers and applies the liquid metal sensor layout according to the constraints. A compressed sensing decoupling algorithm tracks the finger motion from the sensor data, and the resulting estimated finger motion data are further processed using a Kalman filter. The researchers review the constraints, materials, and algorithm construction before covering the quantitative and qualitative experiments.
For a comprehensive evaluation, participants wore gloves with reflective markers and clenched their fists. The researchers meticulously traced the influence of glove size, algorithm delay evaluation, and tracking accuracy of different numbers of sensors. The qualitative experiment followed real-use processing, measuring finger tracking outdoors, self-occlusion, object interaction, and waterproof performance. In challenging scenarios, the performance of the gloves was significantly better than that of other hand pose estimation schemes, demonstrating the robustness of the CST framework.
The figure shows the data values of the 12 liquid metal sensors and the finger motion results calculated by the CST framework.
Discussion and Conclusion
Through the quantitative and qualitative evaluation of our CST framework, the researchers have proven the scheme's excellent performance and robustness. The scheme can track the finger motion of users with different hand lengths in the real world and their interaction with real objects.
The practical implications of this research are far-reaching, spanning across various fields such as virtual reality, human-computer interaction, and healthcare. By providing a lightweight, cost-effective, and highly accurate solution for finger motion tracking, the CST framework paves the way for more immersive experiences in virtual environments, more intuitive interactions with digital devices, and innovative clinical applications. This research is not just theoretical; it has the potential to transform the way we interact with technology in our daily lives.
The authors note the limitations of the chosen approach, such as the CST framework cannot directly spatially locate hands, relies on the assistance of other tracking devices, the gloves are susceptible to damaging the material encapsulating the liquid metal during the process of putting them on and taking them off—and offers areas for future research.
Interested in learning more about the Human-Computer Interaction? IEEE offers continuing education with the IEEE eLearning Library course programs to smartly implement digital tools into your organization.
Interested in acquiring full-text access to this collection for your entire organization? Request a free demo and trial subscription for your organization.
