New York: In a potential breakthrough in wearable sensing technology, scientists have developed a wrist-mounted bracelet, called FingerTrak that continuously tracks the entire human hand in 3D.
The device can sense and translate into 3D the many positions of the human hand, including 20 finger joint positions, using three or four miniature, low-resolution thermal cameras that read contours on the wrist.
The device could be used in sign language translation, virtual reality, mobile health, human-robot interaction, and other areas, according to the study, published in the journal Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Tech.
“This was a major discovery by our team – that by looking at your wrist contours, the technology could reconstruct in 3D, with keen accuracy, where your fingers are,” said study researcher Cheng Zhang from Cornell University in the US.
“It’s the first system to reconstruct your full hand posture based on the contours of the wrist, Zhang added.
Past wrist-mounted cameras have been considered too bulky and obtrusive for everyday use, and most could reconstruct only a few discrete hand gestures.
FingerTrak’s breakthrough is a lightweight bracelet, allowing for free movement. Instead of using cameras to directly capture the position of the fingers, the focus of most prior research, FingerTrak uses a combination of thermal imaging and machine learning to virtually reconstruct the hand. The bracelet’s four miniature, thermal cameras – each about the size of a pea – snap multiple “silhouette” images to form an outline of the hand.
According to the researchers, a deep neural network then stitches these silhouette images together and reconstructs the virtual hand in 3D. Through this method, they were able to capture the entire hand pose, even when the hand is holding an object. While the technology has a wide range of possible uses, Zhang said the most promising is its potential application in sign language translation.
“Current sign language translation technology requires the user to either wear a glove or have a camera in the environment, both of which are cumbersome. This could really push the current technology into new areas,” he said.
“A device like this might be used to better understand how the elderly use their hands in daily life, helping to detect early signs of diseases like Parkinson’s and Alzheimer’s,” the study authors wrote.