Spencer Cook was one of the first members of the SPI team brought onto the show during pre-production in August 2010, before the shoot, when the script was still underway and the sequences were still being designed. He spent time in New York, the shoot location where he and director Barry Sonnenfeld worked with the previs team. Rick Baker and his crew were also still working on the design and build of the alien prosthetics that were integral to the digital animation team’s work at SPI.

Spencer explained that SPI’s animation responsibilities for this film encompassed three types of characters and creatures. The alien characters were one type, and were either all-digital or a challenging combination of live actor, prosthetics and animated digital extensions. Second, extremely accurate photoreal digital doubles were built and had to be animated to intercut invisibly into live action plates. Finally, in the movie’s signature chase scene the heroes’ monocycles needed to appear as digital elements in certain shots, and animated to match the riders’ actions.

Post Animation
Spencer, as animation supervisor, used preproduction to familiarize himself with sets and sequences in which the film’s characters and creatures – all or partly digital – would be interacting with the live action elements in the shots, although he explained that much of exactly how this interaction would be performed could only be worked out once principle photography was underway, and was developed on-the-fly.

He did not have a chance to remain on set for principle photography. He attended some very early shoots only, but his main role was to prepare the team in LA, make sure the models and rigs were underway, and carry out early animation tests. “I would have preferred to be there during the shoot, but a huge amount of preparation was needed before we could actually start animating any shots.

“Keeping that prep work running in parallel with the shoot was also important. My daily contact with production was through Ken Ralston and Jay Redd.” About 70 per cent of the animators in his team were in Vancouver. Spencer collaborated remotely for the first time using Imageworks’ video telelconferencing system.

The SPI team digitally enhanced and extended prosthetic make-up artist Rick Baker’s physical extensions that turned actors into aliens. Spencer Cook’s and the animators’ task was to animate these digital extensions. Each character had passed through a lengthy, varied design process. Barry Sonnenfeld was especially concerned that the actor’s performance should shine through for the lead alien, the villainous motorbike-riding Boris. Not only was an accomplished comedian, Jemaine Clement, playing this role but Boris also has to walk about with other humans without attracting too much attention.

Sticky Fingers
“Rick Baker had designed all of the prosthetics extremely carefully,” said Spencer. “The tiny ‘fingers’ surrounding Boris’ eyes were specifically made to express his mood and agitation. Toward the end of the film as he grew angrier the fingers would twitch excitedly – this was our animation team’s work, applied to digital replacements. During his calmer moments, when the fingers were folded up and rested around his eyes, the physical prosthetics were left in the shots as captured.

“The distinction seems simple in theory but actually achieving the completed shots wasn’t always straightforward. Decisions about the process were made in post, well after principle photography and evolved as Barry decided just how much he wanted to see the fingers moving at any moment. If I had my way, of course, they would be moving all the time because I loved the way it looked! But Barry was concerned about the effects fitting into the story.”

Therefore the movements were used only at key moments. They had to experiment and exchange ideas about specific moves and patterns, so in the end it was very subjective. The fingers were not meant to be Boris’ primary characteristic – just one that accented his wider performance.

The animators had an exact digital double to work with of the actor in the prosthetic make-up both as Boris in the present day and as he was in 1969. The doubles were based on full digital scans of the actor. “Because they needed to blend precisely into the live action photography, they represented a huge amount of work and thought,” Spencer said of the doubles.

“We were animating a digital model of Boris that was complete in every way, with beautifully jointed fingers and animation controls to grab, pose and apply keyframes to. The teams who create these doubles for us always astonish me – the modellers, and texture, hair and cloth artists. I’ve worked with models like these on several films now and the artistry behind what we animate is amazing.

Playing Straight
“We can use these duplicates not only not only for the prosthetic extension work but also for full digital double performances for the longer, wider shots where we see Boris in extreme stunts that couldn’t be shot practically or with a stunt actor. All together, using the doubles meant constant intercutting shots of the real actor with the digital one. Especially in the Cape Canaveral sequence when the heroes fight Boris on the gantry around the rocket, the shots are a grand mix of lead actors, stunt men and digital doubles.”

Again, sequences like this involved a lot of decision-making in post that affected the animators’ work. While they knew fairly accurately which shots they would be working on and what action had to unfold, even the story might change slightly over the course of the work. Also, shots sometimes ended up with either more or less digital animation than expected.

While Boris remained the principle challenge, because of having to perfectly coordinate the upper digital portions with the lower live action part, the straight digital doubles of the lead actors presented a different matching challenge. “We couldn’t make any of their performances too ‘big’. A piece of animation might look amazing in a single shot but if it failed to fit the context or with surrounding shots the sequence would be spoiled. It demanded close observation of general and specific actor performance.

“The animation team would know in advance what the final cut was going to be. We can animate tests or a different take or version of a shot, and if it is in the correct frame range, we can insert it into the sequence ourselves without waiting for editorial and complete much of the refining ourselves.”

Hole in One
Rick Baker worked as a more or less independent agent on the prosthetics, and the teams at SPI adapted to accommodate his work. An example of his inventiveness was Boris’ weasel-like extension, a terrifying little creature that crawls from a hole in the palm of Boris’ hand to wreak havoc on the story’s heroes with a rain of needle sharp spikes.

SPI received the design from Rick and built the creature digitally, and Spencer’s team took on the animation. He said, “My first steps in the early animation tests were to allow its design to imply the behaviour and style of movement - insect or spider-like with multiple sharp spindly legs. From the start, the script and Barry very clearly indicated a hyperactive, skittering creature.

“Because Boris and his weasel are virtually one and the same, Barry wanted the creature’s action to feel linked to Boris’ performance. Although I wasn’t present at the shoot, once we had the plate at SPI our job was to make the animated creature fit in with that performance. We had to study it frame by frame – Boris’s eyeline, how he held his hand, what he did as the creature jumped in and out of his hand or onto his shoulder. Boris’ friend Lily who helps him break out of prison also looks and turns to watch it run around, which needed more retrofitting.”

Headless
Tony was a comic character whose head is pulled off and, even as he continues to speak, used as a bowling ball in one sequence. The way it was shot afforded the animators a lot of flexibility and it was therefore more straightforward than the sequences with Boris, in which the animation had to line up precisely with any existing movement in the actor’s face. The bowling ball head did have to line up to the body and work technically as a matchmove but much of the performance could be hand animated by watching what the actor was doing.

“No specific facial capture had been done,” Spencer explained. “There were two sets of performance – what the actor did on set and what he did after the shoot. Two cameras recorded the actor simultaneously, one from the front and one from the side, while he performed the same lines.

“They went through the scene in several different ways – under and overplayed, for example, even changing the lines slightly as they went, which meant animating him a little differently to the set performance. Then we mixed and matched Barry’s chosen clips to result in the final performance for the distorted bowling ball face. The ultimate goal was simply for something funny. We tried exaggerated and cartoony looks to see how far we could push the model.”

Slippery Character
An oversized alien fish that attacks the lead character Agent J in a Chinese restaurant was the team’s major all-digital creature. It was still almost completely undefined in terms of design and looks at the time of principle photography. The shoot proceeded without even a physical prop for the actor to work with. Agent J’s performance was shot while a complex system of wires pulled and pushed him into various moves and positions on set for the animators to coordinate with later once the fish model was completed.

Based on an initial set of sketches that VFX supervisor Ken Ralston had made, modelling, texturing and rigging of the fish went ahead while Spencer’s team considered how it would move, how big its flippers should be to be able to balance its weight on land, how thick it should be to have enough weight, strength and presence. These decisions had to be made in parallel with the build in order to make the fish work in animation.

A major component was the face. Because the fish had to partly swallow Agent J, how big should its mouth be? How can it be powerful, dangerous and monster-like but also comic and fun? At the same time it had to be realistic enough to fit into the live action world with the actors, which is why it was important for Spencer to find and use real world references –even for a fantasy creature.

Prehistoric Smile
Walruses and sea elephants gave them the powerful moves and weighty aspects it would need and for the face, Spencer actually studied images of a prehistoric fish that lived 45 million years ago with a strong jaw that added a mean, dangerous look to the fish. To make sure his design ideas would work in three dimensions, he worked alongside the modellers and riggers. Ken Ralston’s sketches showed widely varying ideas from which the director chose his preferences but the final looks continued to evolve over time.

Meanwhile the riggers made a crude rig for Spencer to use on early poses and basic animation tests while he gained a feeling for its personality and body language. A large part of the design process led from this work because the model had to achieve certain, specific actions.

“When the actor uses his hands to pound the fish we animated it to react accordingly. But it had to look spontaneous and natural despite the outlandish nature of the fish. As the audience sees the skin move and jiggle, these effects have to be consciously applied. The character or creature animators and the effects animators, who add the layers of detail that enhance performance, worked hand in hand. Skin reactions, dripping slime – the success of the creature animators’ actions relied on the compositing and rendering of these effects, and vice versa.”

On the Streets
An extreme case of animation and effects coordinating with live action was the monocycles chase sequence through the streets of New York. This chase was another combination of live-action, green screen performances and digital sets. The background plates were shot at night, stabilised and matchmoved. Cars, signage and street props were rotoscoped to further improve the integration of the monocycles into the photography.

Accurate digital models of the monocycles as well as the actors were required, shot for shot, plus Boris, his spooky alien friend Griffin and their motorbike. Digital period cars had to be added plus elements of any collisions that would have been a challenge to coordinate in camera.

Consequently, some shots comprised a live action plate, an actor performing on green screen plus digital parts of the monocycles. When Agent J rides over a car, rotoscoped mattes of the cars were first placed into the matchmoved plate. At this point the animators needed to animate the digital Agent J with his monocycle – as realistically as possible given the situation. At certain moments it was better to use a digital car as well for the interaction. The 3D crash effects were composited in.

The editors continued working on the cut throughout and if necessary, the background plates could be adjusted or changed to suit the cut. If a green screen performance in a shot had to be exaggerated or enhanced, they sometimes selectively replaced the actor with a digital character and keyframed him to blend accurately.

At one point a live action truck and bus crash was staged and captured but later during the cut, the editors wanted some extra camera angles on this event which meant rebuilding the street environment and crash, relying on thorough surveys taken during the shoot. Night time street lighting, water in the shots and building textures were particular challenges. The recreated crash itself used both simulations and keyframe animation to achieve the correct dynamics and interactions, plus FX work like sparks, smoke and flying debris.

Words: Adriene Hurst
Images: Sony Pictures Imageworks