Ballerinas are stepping into sensor suits so one Christmas you may be able to understand the nuances of The Nutcracker

Throughout the festive season, countless individuals delight in the enchantment of ballet spectacles such as “The Nutcracker.” Though the stories of timeless performances are widely known, general audiences often miss the subtle narratives and emotions dancers seek to convey through body movements—and they miss even more when the narratives are not based on well-known stories.

This prompts the question: how can dance performances become more accessible for people who are not specialists?

Researchers think they have the answer, which involves putting dancers in sensor suits.

Loughborough University academics are working with the English National Ballet and the University of Bremen to develop software that will allow people to understand the deeper meanings of performances by watching annotated CGI videos of different dances.

Leading this endeavor is former professional ballerina Dr. Arianna Maiorani, an expert in “Kinesemiotics”—the study of meaning conveyed through movement—and the creator of the “Functional Grammar of Dance” (FGD), a model that deciphers meaning from dance movements.

Dr. Maiorani believes the FGD—which is informed by linguistics and semiotics (the study of sign-based communication) theories—can help create visualizations of “projections” happening during dance performances to help people understand what dance means.

“Projections are like speech bubbles made by movement,” explains Dr. Maiorani, “They are used by dancers to convey messages and involve extending body parts towards significant areas within the performance space.”

“For example, a dancer is moving towards a lake, painted on the backdrop of a stage. They extend an arm forward toward the lake and a leg backward toward a stage prop representing a shed. The extended arm means they are going to the lake, while the leg means they are coming from the shed.”

“Using the Functional Grammar of Dance, we can annotate dances—filling the projection speech bubbles with meaning that people can understand without having background knowledge of dance.”

Dr. Maiorani and a team of computer science and technology experts—including Loughborough’s Professor Massimiliano Zecca, Dr. Russell Lock, and Dr. Chun Liu—have been creating CGI videos of English National Ballet dancers to use with the FGD.

This involved getting dancers—including First Soloist Junor Souza and First Artist Rebecca Blenkinsop—to perform individual movements and phrases while wearing sensors on their heads, torso, and limbs.

Using the FGD, they decoded the conveyed meanings behind different movements and annotated the CGI videos accordingly.

The researchers are now investigating how these videos can facilitate engagement for audiences with varying levels of dance familiarity, aiming to transform this research into software for the general public eventually.

Of the ultimate goal of the research, Dr. Maiorani said, “We hope that our work will improve our understanding of how we all communicate with our body movement and that this will bring more people closer to the art of ballet.”

The Loughborough team worked with experts from the University of Bremen, including Professor John Bateman and Dayana Markhabayeva, and experts from English National Ballet.

They are also looking at how the FDG can be used in performance and circus studies and analyzing character movements within video games to determine any gender biases.