Taiwanese Dancers Project Memories with Xsens Motion Capture

Blending complex imagery with dance, 3D printing and Xsens motion capture, ‘The Inheritance’ is a performance in which dancers interact with live 3D projections to tell their history. Created by Storynest Director Hsin-ChienHuang, ‘The Inheritance’ describes a time when Taiwan was under martial law. Closed borders and the inability to communicate with people outside of Taiwan left many people stranded abroad. Attempts to help could result in harassment at home or on the street, leading to a state of constant fear in the population.

Hsin-Chien Huang is still concerned by state-sanctioned history and what it may conceal. Through ‘The Inheritance’, he expresses his own family’s struggle, using dance and various forms of new media to turn memories into a form of visual sculpture. Set in front of a giant screen, the dancers perform within a series of nine stereographic projections. In this case however, the projections are led by the dancers and the real-time motion capture data they are transmitting.

Real-time 3D Stereo Projection

“We wanted to give the audience a genuine sense of three dimensions,” said Huang. “This way the dancers’ performance seems to extend to the audience. To blend together the performance with the background, we use active 3D stereographic projection combined with the physically present dancers. This technique creates a greater depth of field for the background. Meanwhile, the audience, wearing 3D glasses, feels that the dancers and setting are reaching out to them.”

Huang considers the fact that people have consistently used the human body as an instrument and unit for measuring space-time. To take advantage of this, the team employs the Xsens MVN inertial motion capture suit during the event. Because the suits can work outside of a studio, and record a 240 FPS capture rate, the team is able to register extremely subtle moves from its dancers. The computer-controlled virtual characters have been mapped to dancer movements, enabling an interactive experience. Premade animation and physics simulations are also loaded into the system, giving Huang the opportunity to use predefined motions when it suits the story.

To manipulate the show in real-time, the performance data is streamed to the crew’s rendering and interactive software. There, the mocap data can be duplicated, mirrored, randomized, delayed, or accelerated in the moment, enabling collaboration between Huang and the dancers, even when communication is impossible.

Re-animation & 3D Printing in Post

Using Xsens’ mocap software package, MVN Studio, Huang can record and edit the data and then attach it to whatever character he needs to. “It allows me to replay a dancer’s mocap data again and again, changing what I need to within my interactive program. It’s an excellent way to collaborate with dancers when I’m not working with them directly,” said Huang.

While some may see shapes changing before their eyes, Huang sees a chance to make something physical out of the projections. “With accurate mocap data, human hands become a 3D scanner,” he said. “Our walking paths record the contours of the terrain we have travelled. When I covert these paths into 3D models and print them out with a 3D printer, these once ambiguous memories become visible. They can be shared and re-touched.” This desire has led Huang to 3D print many of the projections the audience sees in the show. www.xsens.com