Washington :Researchers have developed a new method that teaches computers to understand what humans do in a typical day, an advance that may lead to devices that predict human activities and even suggest alternatives.
The researchers at Georgia Institute of Technology used the technique to gather more than 40,000 pictures taken every 30 to 60 seconds, over a 6 month period, by a wearable camera and predicted with 83 per cent accuracy what activity that person was doing.
Researchers taught the computer to categorise images across 19 activity classes.
The test subject wearing the camera could review and annotate the photos at the end of each day to ensure that they were correctly categorised.
“It was surprising how the method’s ability to correctly classify images could be generalised to another person after just two more days of annotation,” said Steven Hickson, a PhD candidate in Computer Science at Georgia Institute of Technology’s School of Interactive Computing and the Institute for Robotics and Intelligent Machines and a lead researcher on the project.
“This work is about developing a better way to understand people’s activities, and build systems that can recognise people’s activities at a finely-grained level of detail,” added co-author Edison Thomaz, graduate research assistant in the School of Interactive Computing.
“Activity tracking devices like the Fitbit can tell how many steps you take per day, but imagine being able to track all of your activities – not just physical activities like walking and running,” said Thomaz.
“This work is moving toward full activity intelligence. At a technical level, we are showing that it’s becoming possible for computer vision techniques alone to be used for this,” he said.
The group believes they have gathered the largest annotated dataset of first-person images to demonstrate that deep-learning can understand human behaviour and the habits of a specific person.
The ability to literally see and recognise human activities has implications in a number of areas – from developing improved personal assistant applications like Siri to helping researchers explain links between health and behaviour, Thomaz said.
Researchers believe that someday within the next decade we will have ubiquitous devices that can improve our personal choices throughout the day.
“Imagine if a device could learn what I would be doing next – ideally predict it – and recommend an alternative?” said Daniel Casto, a PhD candidate in Computer Science and a lead researcher on the project.
“Once it builds your own schedule by knowing what you are doing, it might tell you there is a traffic delay and you should leave sooner or take a different route,” he said.
PTI