Faceware Brings Emotion & Realism to NBA 2K16

Realistic, visually accurate facial animation has become an engaging aspect of many narrative-led, action video games like ‘Grand Theft Auto’ and ‘Halo’, where it helps draw players into the story and holds their interest over time. This technique is also proving useful as a means of adding realism and emphasizing the personalities of athletes in sports games such as ‘NBA 2K16’, the current title in the NBA series. Not only must this kind of game accurately reflect the strategy and technique of the sport itself, but also the intensity and drama experienced by the players.
Faceware-nba-2k16


Performance capture stage manager Anthony Tominia has been working on the NBA series team for six years, and described their efforts to improve the way they sell the emotion of basketball. “While working on 2K16, we wanted everything to be as realistic as possible,” he said. “That means everything from motion capturing the basketball action itself to representing the real life players in digital form.

“This year we’ve been trying to achieve more expressive emotions, such as better looking lip sync in our ‘My Career’ story mode in which players put together their own backstories, and distinctive reactions that mirror what actually happens on the court. We wanted to delve into the narrative movements this time and sell what’s happening on an emotional level.”

Faceware-nba-2k16-4


2K has used Faceware’s hardware and software for facial performance capture for several years, and used them again to bring ‘NBA 2K16’ to life, capturing the facial performances of over 30 different NBA athletes. Actions both on the court and off were recorded by carrying out full performance capture for the in-game animations and also recording dialogue for the story mode.

Once acquisition was complete, Faceware Analyzer and Retargeter applications were used to transfer the motions to the game play. Analyzer is markerless facial motion tracking software, used by following a fairly simple workflow. Through Faceware’s proprietary facial recognition system, Analyzer tracks the face and creates its own facial motion capture data, converting video of an actor’s facial performance into motion files for use in Retargeter.

Faceware-nba-2k16-2 Faceware-nba-2k16-5

Retargeter software was used to apply the facial capture data from Analyzer onto 2K16’s facial rigs through a plug-in to their 3D software. It is compatible with Autodesk products Maya, 3DS Max, Softimage and MotionBuilder. The user gains control from Retargeter’s combination of the captured poses and movement data, and can use this to produce convincing facial animation.

The 2K team could tailor this process to fit their project’s requirements very closely because Faceware’s software has a straightforward scripting interface. Anthony said, “In this case, we wrote a lot of Python scripts that allowed us to automatically solve the capture that little bit faster. There were over 232 lines spoken by each of the 30 players, and we found that we could easily batch, through Analyzer, 232 lines per player without having to touch the data.

Faceware-nba-2k16-6


“We did the same thing with Retargeter. In our pipeline the Python script does the retargeting, creates play blasts and saves those files immediately for review. The process doesn’t have to be manned, so we could capture during the day, and then leave the system to analyze and retarget overnight. We would come in the next morning, quickly identify which files needed the most touch up and pass those on to the team.”

The ease of use and customization of the software made it possible for them to shorten cleanup time on NBA 2K16’s animation down to three or four minutes per second of data. Faceware also helped NBA 2K team customize their hardware by building a custom camera that could capture data at a higher, non-standard frame rate the team prefers for the game.

Faceware-nba-2k16-3

“At the end of the day, this kind of work will always rely on the touch of a human animator to flesh out the capture,” said Anthony. “But in our line of work we want to get as far into the pipeline as we can before a human has to touch that data, and Faceware delivers excellent results in that regard. Four minutes per second wasn't really that much time, because nobody needed to touch the file until that point, a huge advantage in a production like this.”   facewaretech.com