Researchers have developed a smartphone app that captures full-body motion without the need for special cameras or expensive equipment.
Engineers at Northwestern University in Illinois, U.S. have created a system called “MobilePoser” which transforms a user’s smartphone into a movie-quality full-body motion capture system.
Most film fans are familiar with motion-capture techniques, which are often revealed in behind-the-scenes footage.
To create CGI characters — like Gollum in The Lord of the Rings trilogy or the Na’vi in Avatar — actors wear form-fitting suits covered in sensors, as they prowl around specialized rooms. A computer captures the sensor data and then displays the actor’s movements and subtle expressions.
Traditionally, motion capture has been a complex and expensive process, typically requiring actors to wear sensor-laden body suits and perform in studios equipped with specialized, high-tech cameras.
“This is the gold standard of motion capture, but it costs upward of $100,000 to run that setup,” Northwestern University Professor Karan Ahuja, who led the study, says in a statement.
“We wanted to develop an accessible, democratized version that basically anyone can use with equipment they already have.”
However, according to Northwestern University researchers, the MobilePoser system leverages sensors already embedded within consumer mobile devices, including smartphones, smartwatches, and wireless earbuds.
Using a combination of sensor data, machine learning and physics, MobilePoser accurately tracks a person’s full-body pose and global translation in space in real-time.
The app captures and converts this data using a customized AI algorithm, resulting in full-body motion capture without any added cost or equipment. The algorithm estimates joint positions and joint rotations, walking speed and direction, and contact between the user’s feet and the ground.
Although the MobilePoser app doesn’t match the precision of costly motion capture technologies used by major Hollywood studios, its creators suggest that using multiple devices can further reduce tracking errors.
However, in a study published this month, the team also emphasizes that MobilePoser remains reliable enough to deliver high-quality motion capture with just a single device.
“The accuracy is better when a person is wearing more than one device, such as a smartwatch on their wrist plus a smartphone in their pocket,” Ahuja says.
“But a key part of the system is that it’s adaptive. Even if you don’t have your watch one day and only have your phone, it can adapt to figure out your full-body pose.”
The researchers also believe that the technology in MobilePoser could have applications in entertainment — unlocking new possibilities in immersive gaming and fitness.
Image credits: Header photo licensed via Depositphotos.