Ted – The Bear Essentials

Production VFX supervisor Blair Clark from Tippett Studio devoted nearly two years to working on ‘Ted’. He had met VFX producer Jenny Fulle, now with Creative Cartel, over 20 years ago when they had worked together at ILM. Just prior to ‘Ted’, Blair and Jenny had worked together again for the first time on the feature film ‘Priest’. As soon as she began gathering teams for ‘Ted’ in 2010, she contacted him - and he jumped at the opportunity.

As the movie’s story opens, the protagonist John is a little boy who finds that his Teddy bear, Ted, has come to life, walking, talking and soon becoming his best friend. The two never part, and when the plot jumps forward to the present day, we find that John, age 35, and Ted are still together. Life is far from peaceful, though. While John struggles to decide if Ted, the foul-mouthed, undisciplined but loveable bear, can really fit into a steady life in Boston with a job and girlfriend, madcap adventures unfold.

Animation Tests
In the film, Ted is a photoreal, animated CG character performing in a live action film. Initial character and animation tests for Ted began at Iloura in Melbourne and Tippett Studio in California a couple of months after Blair came on board. Very soon, the test shots from Iloura stood out for their ability to capture Ted’s looks and bold personality. Those tests set the tone and looks for the rest of the show.

Blair and Jenny Fulle had also decided quite early in pre-production which sequences would go to each studio, working with VFX associate producer Eric Torres. It helped prevent direct cuts from Iloura’s to Tippett’s bear, which would simply have made the task more challenging than it had to be. The sequences were divided approximately in half for each studio, and the production kept quite closely to plan.

Motion Capture
Apart from the skill of the animators, much of the consistency in Ted’s animation across the film was due to the fact that it is based on motion captured data and reference video of the director and lead actor Seth MacFarlane’s own performances. Even before any effects studios had been approached, Seth had carried out some tests to see what motion capture was like to work with, at a studio fitted with a typical, large volume stage, numerous fixed cameras and a Moven suit covered in markers.

A scene was shot and the resulting data sent to each studio with some dialogue. Each team was asked to clean up and review the material within the context of their own animation pipelines and, using a simple Photoshop image supplied by the production, provide a rendered version of the bear’s performance.

Starting with this image, the mocap data and a rough, undeveloped 3D model showing what Ted might look like, the Iloura team began working on their character tests by shooting some background footage on the streets around their studio in Melbourne. They built the bear and carried out technical studies to enable them to composite their model into the shots, walking around and talking, and then used the tests as a template for developing the animation and behaviour further. All of this background work helped sell Iloura’s concept of Ted to the production.

Although Seth’s performances were captured in the motion capture sessions, this data came with inherent limitations. The technique was especially useful on set, in that the suit could quickly be calibrated and ready to go as soon as Seth and the other actors were ready. On the other hand, the team found it wasn’t ideal for capturing walking and certain other, specific types of movement.

“The motion capture served best as a good base for timing and gave us an accurate feeling for Seth’s gestures and style of movement, but it required a lot of cleanup and interpretation,” said Glenn. “The animators would use the data alongside video reference of Seth’s performance to work out the best moves for Ted.”

Further interpretation was needed to overcome the essential differences between Seth’s and Ted’s bodies. Seth had eyes that can look around, while Ted’s eyes were fixed in his head and he would have to turn his head considerably more to achieve similar glances. Thus, while cues were taken from the motion capture, they relied just as much on the video to capture the real intention of the performance.

Keyframe-Mocap Blend
Certain portions of the film were fully keyframed, especially the action sequences involving full body motion, where Ted was actively running and jumping around. “We did a lot of work on the climactic chase scene at Fenway Park in Boston, for example, which was almost entirely keyframed. But the mocap was much better for the subtle, pure acting moments when Seth was sitting or standing, delivering his lines and using distinctive hand gestures and subtle timing.”

The idea of working with motion capture was initially a bit challenging at Tippett as well, where their very talented and creative animators’ previous experience with the technique hadn’t been extensive or entirely positive. Nevertheless, they seized the opportunity to develop a combined keyframe-mocap process for Ted that worked well.

“Having the data was useful in a few respects,” said Blair. “Although I doubt the show could have been finished on time with as good a result by relying on motion capture completely, it did bring us up to speed almost immediately on our scenes, replacing what we might normally have achieved in a blocking pass, and left us with very few ambiguities about Seth’s intentions. Because it was coming to us from the actor playing the bear AND from the director, we had confidence in the data.”

It called for a fair amount of cleanup and lacked facial or hand motions, but the base Ted performance to build from was all there. The VFX production company Creative Cartel had hired two important members of the production. One was motion capture supervisor Darnell Williams, who monitored the capture sessions on set and decided if any re-takes were needed. Then the data would go to the previsualisiation specialist Webster Colcord, who took on a post-vis role in this case.

Bear Control
Certain details needed regular adjusting simply to make shots work. The arms and hands often had to be rotated, for example, because they tended to intersect the body due to Ted’s rotund shape. While the disparity between Seth’s size and proportions and those of the bear were taken into consideration so that it didn’t need more than manual adjustment to the shots, the goal was to avoid cramping Seth’s acting style on set with too many practical barriers. It was better to handle as cleanup, also part of Webster’s preparation to the data before passing it on to either Tippett’s or Iloura’s animators.

A key aspect of Webster’s previs work involved collaborating with Blair, Seth and the DP on camera moves. For example, several weeks ahead of the shoot for the Fenway Park sequence, Webster began work with a 3D model of that environment, blocking out camera moves and building the whole sequence while showing it to Seth for comments and changes.

Blair explained, “This work was especially useful because they only had a couple of nights available at Fenway Park for the shoot. It gave them a chance to sit down and quietly figure it all out first, with much more detailed reference than storyboards. Once the camera moves were chosen Webster prepared crude looking but quite accurate pieces of animation, inside the Fenway Park model. He had a simple bear model and rig with just enough control to convey the intentions of the scene’s action. It was essentially an animatic but highly customised, accurate and detailed. Thus, between Darnell and Webster, little was left to chance, either during the shoot or during animation.”

The hotel room fight scene between Ted and John, another dramatic moment in the story, was also choreographed well in advance of the shoot by stunt coordinator Scott Rogers and John’s stunt double, using the production designer’s blue print of the hotel room as a guide. They shot it on video using their own practical Teddy bear and edited it together, all in advance. Once the cut was approved it was used on set to replace storyboards. Thus the crew were able to shoot this critical sequence in just two nights. All bear action and camera angles, the hardest parts in terms of drama, had been chosen and planned in advance, and very little changed on set.

Ted Evolution
The animation was one side of the Ted project, but the model was another, and both had to coordinate and keep evolving throughout production. The Teddy bear shape – pudgy with tube-like arms and legs – caused on-going challenges to achieve the required range of movement and managing this was a continually evolving process. They also had to figure out a walking style, how far he could swing his arms, and how much his fur would interfere with movement.

Building Ted involved input from the lead of every dept from the start. Tippett Studio’s VFX supervisor Scott Liedtka, whose work on ‘Ted’ started shortly before principle photography once Iloura’s artwork had been approved, spent much pre-production time ensuring his team got Ted looking right and matched that approved artwork. He said, “Modellers worked with animators and riggers from the outset. Fortunately, Seth provided plenty of notes to tell them clearly what he wanted. As an animator himself he was focussed on getting the lip-synch and eyelines right, and capturing the correct emotional intent.

“After a month or so, we were quite confident about understanding what Seth wanted to see. We also watched a lot of footage from ‘Family Guy’, Seth’s TV show, to familiarise themselves with his sense of humour and comic style. Progressing from sequence to sequence became more predictable. Also, as both director and lead character, Seth could take full responsibility for scene changes and updates, re-recording the voice or re-doing the motion capture and delivering it to the team.”

As a talking character, the face rig needed a dedicated effort to create the lip-synch, easily as much or more work than the full character and involving its own round of test shots and collaboration between modellers, animators and riggers to make sure all the shapes were accommodated. Tippett and Iloura continuously touched base regarding this part of the performance as well.

Fur and Cloth
While Ted’s model had to remain quite simple and stubby to be a convincing Teddy bear, techniques like clever camera angles helped in spots that needed a special move. At one point, when he had to reach up with a stapler and clip his ear, the camera moved in tight on the ear to direct the viewer’s attention and conceal how much his small arms had to stretch to reach the top of his large head, but in fact it was possible to keep him on model for most of the film. While they could stretch him, they tried to avoid it.

Two important aspects of Ted’s model were the fur and cloth. Because Ted is actually made of cloth, not just wearing it, a cloth solve was required to give the correct wrinkles and twist in the fabric under his fur. At Iloura, Ted’s hair resulted from the ongoing evolution of hair and fur through many previous projects starting with ‘Charlotte’s Web’, when they wrote a hair system from the ground up. Each subsequent project has progressed their hair rendering.

Iloura also put some R&D into ripping Ted’s cloth. At the end of the story when he climbs up a tower, he is ripped in two, revealing fluffy stuffing. They looked at sample films of materials and fur ripping, which the producers also liked, but they needed to do some of their own cloth solving with pre-scored tears and frayed edges, and give the stuffing the right sort of translucency. They could effectively re-purpose their hair system to handle these looks as well.

The Perfect Match
Although Iloura and Tippett were sharing the bear shots and needed to keep him completely consistent, the software pipelines at the two studios were quite different. They were unable to share hair systems and some other aspects of the bear asset. Iloura had produced the initial tests using their own fur, which Tippett then studied to produce the same clumping and frizzing in their fur. However, because fur and hair are among Tippett’s strong points, Iloura was inclined to adjust their fur’s looks slightly too, after looking at Tippett’s fur, resulting in some back-and-forth between teams until they felt that both had reached a fully consistent look.

Tippett has well-developed tools both for the way any fur looks in light and the way it is groomed. While the synthetic fur was an issue in itself, the main challenge was matching Iloura’s fur in the tests - the eyebrow fur needed one look, the ears another and so on - while using their own proprietary tools and their renderer that were quite different to Iloura’s. It was an exercise in reverse engineering and added an extra layer of technical concern for Tippett as opposed to Iloura’s primarily creative focus.

Scott said, “Ted’s fur employed a fair amount of subsurface scattering and indirect lighting from HDR panoramas. Each of these required render breakouts,” Scott said. “We developed a shader especially for Ted with certain lighting components, giving the compositors a lot of control and a way to fine tune the looks, softer, coarser, with more or less contrast. Even small changes could give very different results.”

They worked by aiming first for shot to shot continuity and then did a final pass for fur quality, adjusting scattering or specular qualities. However, for speed they could import a ‘Ted look’ composite as a start point, comprising a render stack with all the controls exposed and ready to dial up or down.

No Muscles
“For Tippett, with many years of experience in very lifelike, photoreal design, build and animation of creatures and animals, Ted was slightly new kind of character in some ways,” said Scott. “On one hand, Ted was a real, familiar object – a Teddy bear – that needed to look real, fit into the plate and interact with the other actors. But Ted has no bones or muscles driving his movement, so the tools they had developed for creatures had to be re-thought to some extent.”

Actually, though, the ‘conversion’ was mostly mental. Once the artists embraced the idea of a creature simply stuffed with fluff instead of muscles under skin, they had few problems. Ted still walks and talks and move his limbs around like any other creature. They could use the same tools but with a few changes. “For example, we set up a cloth simulation to achieve the required ‘baggy’ feeling for Ted that worked across all shots. We’ve applied simulations to creatures before of course, as a task to complete on selected shots, but this sim was applied at the very end of production to every single Ted shot,” Scott said.

“So, technically, there wasn’t as much difference as expected between a wolf for the ‘Twilight’ movies and a stuffed bear like Ted. An advantage for us was that Seth wanted a subtle, adult performance and model, not a cartoony over-the-top character. This is a type of character we felt comfortable with.”

Render Crossroads
Blair commented that the need to coordinate the Tippett and Iloura toolsets was not as problematic as anticipated, perhaps, and not as much as it could have been a few years ago. Both studios are animating in Maya, for example, and all compositing was done in Nuke, so the pipelines were based on some of the same tools.

“Deviations did arise with the two, very specific fur generating softwares, developed over time and with different rendering packages. Tippett uses Renderman, and Iloura uses V-Ray and 3Delight. So the key issue was getting those final renders to match visually. As we worked toward the development of a production-ready bear, Tippett and Iloura continuously shared rendered images and information to be able to progress at the same speed and remain fully aware of what the other team was producing. It wasn’t a matter of withholding work until it was complete. Instead, the teams compared progress notes and learned from each other.”

Once they were both rendering final bears, they performed a test by picking a background plate with a mocap file of Seth walking in a circle, and worked back and forth to ensure they could get the animated bear from each studio to look the same. The process allowed them pick up the differences more easily. The final piece was also shown to Seth and the producers who, in fact, were astonished at how closely the two bears matched.

Toolsets
Comparing the two pipelines, at Iloura Maya was used for rigging and animating, modelling in 3ds Max or Maya, plus ZBrush for sculpting, whichever suited the artist and achieved the best result. The animation was passed to a Maya cloth solve and then to lighting, RealFlow was used for liquids.

The lighting was set up in Maya using both V-Ray and 3Delight to render. V-Ray handled the basic skin passes and many assets such as liquids, restaurant props and anything Ted touches, to be able to cast shadows properly. The hair, however, was rendered in 3Delight. The team has developed a technique for passing data from V-Ray to 3Delight. V-Ray can generate much of the global illumination but they find 3Delight Is a bit better at manipulating millions of hairs.

Tippett’s animators use Maya, and so the modellers, who also use Mudbox, and riggers focus on making sure the models and rigs all work in Maya. RenderMan is their renderer. Tracking tools have been written in-house to also work in Maya. Effects are created in Houdini and Maya, especially Maya cloth. The lighting TDs write many of their own tools as well. Mari and Photoshop are used for paint and textures.

Compositing and Integration
Much of the smooth, effortless appearance of Ted moving naturally through the live action shots is owed to several factors - compositing, integration and lighting. Making sure the lighters had complete information on the light in every shot was critical, either on location or on stage. Glenn and CG Supervisor Avi Goodman both spent several weeks on set, at slightly different times, working with Blair Clark from Tippett. Creative Cartel had their own team of data wranglers on set, capturing HDRs, measurements, plus camera and grey ball data.

Glenn said, “They used the Civetta system for the lighting, which is similar to HDR imaging but captures the data twice, at different heights on set, and produces a stereo pair panorama for every camera set-up. It first takes one complete 360° survey, and is then lifted to a higher position, where it completes another 360° survey.

“Data is then extracted from the combined data, allowing the team to hand build the sets by triangulation between the two data sets. We could rebuild the interiors accurately and re-project the result back onto the set geometry – in other words produce a texture mapped version of the interior of every set and every location we needed. This helped us carry out such tasks as casting shadows, and reflect the HDR information back onto the Ted character as well.

“Ted would be rendered by himself, followed by shadows and reflections and so on as another pass, plus a full 3D model of the whole environment which would include all light sources and all surfaces that needed to reflect light and cast and receive shadows.”

Absent Friend
With the lighting data so complete, Blair feels that it is also important on any project to make the composite artistically pleasing. “You need to listen to your DP who might say, ‘I’m lighting the scene this way now, but if your character were actually here I might place an eyelight over here as well, or a light to shape the head.’ Those things will be missing and it’s up to the lighters to fill in the blanks for the DP and crew.”

Making objects in the plate interact with Ted convincingly was another important part of the integration job. When he sat on the couch or jumped on the bed for example, the furniture had to respond naturally. When he and John drank shots or smoked a bong together, the glass, liquids, smoke and fire had to be created on their CG simulation pipeline and animated to respond to whatever Ted was doing.

Scott said, “When plates arrived, we matchmoved the camera and any objects Ted contacted – furniture or John’s fingers in the fight scene – creating exact CG versions to use when placing Ted into the shot in 3D space. Then we would put in a rough version of him to make sure it looked right, in which case we would carry through with animation and fine-tuning the looks and progressing to light, rendering and compositing.” An advantage when tracking Ted into scenes was the relatively stable shooting style for most sequences. Shaky or wild moves were not called for.

Scott feels that the quality of the composite was owed mainly to the skills of the team. “No weak links in here! Lighting is key, of course, and it’s important to get a finished, accurate render from the lighters for the compositors to use. There are different approaches to the compositing-light relationship. Some people start with only a rough lighting render and wait to refine it in the comp,” he said.

“But at Tippett, even if we render breakouts to allow a few adjustments, the compositors prefer to get the bear properly lit before they start. That way they can focus on integration - shadows, focal blur, occlusion, bounce – anything missing because the bear hadn’t been in camera. The story involved a lot of smoking, for example, needing a special kind of integration, and getting this to work with the glass bong relied heavily on compositing. The renders were done in several pieces, requiring more work sometimes than the bear.”

Stuffie Pass
Most shots included what Blair called a ‘stuffie pass’, a rehearsal with a practical, stuffed bear to make sure of the moves and positions. Camera angles and lights were also set up with him in place. All aspects of the shot were engineered to make sure he remained the leading character, though his personality and performance were inherently strong.

Blair would puppeteer the stuffie around the set, acting Ted’s part while Seth voiced the lines, with the other cast members. This pass was also useful for integration in post production - as a lighting pass, for example – simply by having something physical in camera. Once the bear was removed for the actual plate, an eyeline tool was placed on set to aid the actors.

The eyelines between Ted and the cast could be challenging. The crew tried as far as possible on set to get this right in camera, but once the bear was placed into the shot later the animators sometimes moved him to a position where the eyelines engaged better. Furthermore, the large team of animators had to coordinate their efforts across the whole sequences to ensure the continuity of Ted’s position from shot to shot.

“As I see it, what makes the movie good is Seth’s writing,” said Scott. “What we can take credit for at Tippett is that Ted feels like a real character. The viewer can forget the technique and just believe that he’s really there.” www.iloura.com.au www.tippett.com

Words: Adriene Hurst
Images: Courtesy of Universal Pictures International and Tippett Studio