THOR - WHEN WORLDS COLLIDE

Design Development
“Scott Edelstein was critically involved on set, and eventually went on to lead the digital environments team. The only set for our sequences that was physically built was the lower area of Laufey’s palace, which is where the characters talk with Laufey prior to the battle itself. This is open at one end, like a horseshoe, so that any shot looking out into that open area had to be a digital environment, and anything above the set required extension. We also added more refined architectural elements to the set where required.”

Many of the original concepts and designs the team worked with came from Marvel’s art department and production design team, which they had to develop in 3D in much greater detail. Creature designer Nick Lloyd and model and texture lead Miguel Ortega finalized the Frost Giant designs. “The approach was collaborative. When we were working on the Frost Beast, for example, we presented dozens of concepts. Marvel combined some of these ideas with their own concepts, and ended up incorporating everything into a hybrid creature that became the final result.”

Dark Light
The lighting across the environments is cool and dark. The main challenge for lighting lead Betsy Mueller was to create not only the mood and tone of the sequence, but also to create and communicate the scope of the heroes’ world, extending into the deep backgrounds. She used depth perspective, layers of light and shadow, and atmospherics like snow and fog to do this.

Kelly said, “All of our snow and atmospheric effects were done out of Houdini. We simulated a variety of turbulent wind conditions and density, saving them out as different libraries. Ideally, we could load a particular library for a given shot and it would work out really well, but often we’d have to adjust the simulation slightly. For any given shot, at a minimum we would generate a background, mid-ground and foreground layer of snow and fog. This approach proved to be especially helpful to conversion company Stereo D when converting the 2D final composite to stereo 3D.”

Camera Moves
Digital Domain’s lead matte painters and conceptual artists, including Matt Gilson and Minoru Sasaki, played a critical role in finessing the final look of many of shots. They not only came up with some impressive concepts, they took their ideas into the final picture by using techniques that used multiple projection cameras within Nuke and Maya.
“Almost all the camera moves with a live-action component were match-moved from what the production camera was doing, although sometimes we would extend the move out higher or alter it to some degree, in order to fit the shot better. The all CG shots typically started out as previs from the Third Floor, who did the production’s previsualisation overall, or Digital Domain’s internal previs team. We would take this data, incorporate it into our world scale and refine it if necessary.”

Shattered World
The Jotunheim battle includes huge amounts of shattering and breaking up of the ground and structures. “Ryo Sakaguchi supervised the effects animation on the sequence. The really big shots for the effects team were those that involved complex rigid body dynamics simulations,” Kelly said. “All of the structures and the terrain had to be modelled according to very precise specifications in order for the fracturing algorithms to work correctly.
“For example, the models couldn’t have any overlapping vertices. Once the models were fractured, we’d run the simulation and after adjusting the parameters over many iterations, we’d arrived at something that looked good. One of the biggest challenges, though, was getting this geometry, now broken-up, back into our lighting pipeline with proper UVs, textures and displacements.”

While the initial modelling and UV mapping were all done out of Maya, all of the fracturing and simulations were done out of Houdini with proprietary tools, then back to Maya for lighting out of RenderMan. The software used on the project overall was primarily Maya, Houdini and Nuke. For developing assets, they also used ZBrush, Mudbox, Mari and Photoshop.

Duelling Giants
Eric Petey supervised animation on the project. Many of the Frost Giants were fully animated CG characters. Motion capture was used extensively for them, using motion clips that were captured at Giant Studios in Los Angeles. “Giant has a great virtual production setup where you could have two 6ft tall performers fighting each other in the mocap volume, for example, but then on the monitor you would see a 6ft 4in Thor fighting a 10ft tall Frost Giant composited and animated in real time,” Kelly said.
“This was very useful when it came down to physical contact. For example, if Fandral stabbed a Frost Giant in the chest, we needed the motion capture performer to aim higher, like around the neck or head, because the CG character would be much taller than its human counterpart performing in the mocap volume. Ultimately the cleaned-up motion capture worked quite well for many of the shots, but in cases where the director wanted to change the animation and we didn’t have the proper clip, we would have to animate those shots by hand.

Cats & Rhinos
“The Frost Beast was a true ‘monster mash’ of different ideas. The dozens of concepts that we presented to Marvel were creature designer Nick Lloyd’s ideas," said Kelly. "Miguel Ortega built the digital model which Christopher Nichols textured using elephant skin as a major reference for its exterior look. The face, resembling a turtle crossed with the rancor from Star Wars, was something that Marvel specifically referenced in their design notes.”

Obviously, nothing like the Frost Beast exists in nature, so Eric Petey’s team animated this creature without the benefit of motion capture. It was all done by hand to mimic the motion of a large cat, but at the same time it had to be as powerful as a rhinoceros, able to burst through anything in its path. That combination created challenges for computer graphics supervisor Eric Fernandes and the effects animation team, which had to shatter ice and rock as the Frost Beast chases after the heroes. It required a coordinated team effort to put such a menacing creature on screen.

Post-Conversion
Digital Domain worked together with production and the post-conversion company Stereo D to devise ways to identify and deal with four types of stereo shots. “Type 1 applied to shots we didn’t have to do anything to, which was equivalent to the procedure for all the non-visual effects shots,” Kelly explained. “Type 2 referred to the all-CG shots, for which we would render a left and right eye, resulting in a ‘true stereo’ shot. The Jotunheim sequence in fact included almost 90 shots that were all CG. We were responsible for both images on all of those, and so they didn’t need dimensionalising.

“For Type 3, we would provide Stereo D with any layers that would help make their job easier such as mattes, z-depth information from the renders, snow and atmospheric layers. In Type 4, Stereo D would post-convert the plate and provide us with a camera, and then we would render out both eyes for true stereo CG elements.

Rainbow Vortex
The Bi-frost concept encompassed a number of different looks in the film. While BUF created the physical, crystal rainbow bridge between the observatory and Asgard City, Fuel VFX handled the environment for the characters flying through space and time within a vortex or ‘wormhole’ effect. BUF also developed the entry and exits into this vortex, and other vendors like Luma and Digital Domain also contributed various related rainbow effects and visual aberrations at certain points in their work. However, the teams didn’t actually collaborate on the effects.

“Unlike other projects where looks are determined quite early, vendors had to design some of these on their own, on the fly,” said Paul Butterworth, VFX supervisor at Fuel VFX. “Occasionally, we got a sample of what had been done by another team through Wes Sewell who might get an exchange going. But the contact was not consistent and the work involved a certain degree of isolation.”

Faster Than Light
All of the artists knew that the Bi-frost was based on Norse mythology in which it literally serves as a bridge between worlds to walk along. But Marvel wanted a new spin on this to base it more firmly in science, so each vendor tried to place this physical reality on their work. Fuel focussed on the idea of people flying through wormholes and put extensive research into this theoretical phenomenon.

“We speculated that if you were moving faster than the speed of light, you wouldn’t see anything, and had to decide how to take it from there,” Paul said. “To visualise a non-visible event, we looked for natural, recognisable phenomena that generated rainbow colours such as refractions through glass and oil, the aurora borealis with wild strands of energy and so on to help draw the audience into the effect and accept it. We were trying to satisfy the viewer’s curiosity about what the Bi-frost was.”

Whatever Fuel created had to cut right next to BUF’s crystal bridge structure, and possibly be followed by a shot from Digital Domain or Luma, for example, with no break in the viewer’s recognition. As they worked they often wouldn’t have a chance to see the edit of these effects side by side until much later, or even in the final film.

Early Development
Paul had in fact been working on ideas for ‘Thor’ ever since Fuel VFX had contributed to Marvel’s previous project ‘Iron Man 2’, when ‘Thor’ had been mentioned as their next film. Fuel had meetings with the VFX supervisor and production designer on this team as well to start on looks before the script was finalised. At that point they knew about the interest in wormhole types of looks and atmospherics, and started on loose, experimental R&D into ideas for the energy streams.

Meanwhile the shoot was completed and Digital Domain started building the ice planet, Whiskey Tree began on Asgard and, once the production had a rough cut, Fuel VFX was asked to return to look at the remaining effects, although there had even been talk that the Bi-frost might not feature in the film after all. “Marvel generally likes to rely on a number of smaller teams and go back to them for special tasks. This can make the projects more fun and provides more opportunities for creative input,” said Paul.

The team cautiously developed a pitch in the form of a title sequence with some of the test work that the VFX supervisor had liked earlier, and the production liked the pitched shots well enough to include them in the film. Fuel VFX were re-engaged to develop the looks for their Bi-frost effects.

Energy Streams
The effects were mainly created with fluid simulations, and started with energy streams twisting around each other, created from a particle system based on a fluid solver. “We tried several ways to create particle branches. Trying to build them by hand often looks artificial but using fluid particle emitters to generate energy streams as organic branches to which we could attach particles was more successful.

“At the same time, they had to be approached artistically because we weren’t creating known objects like a car or a robot with familiar qualities, moves and materials. This made the project exciting for us. We felt we had been called on because the producer and Marvel believed we were capable of interesting looks. We were pleased to have achieved something no one had seen before. This project largely consisted of moving art for all vendors. The effects couldn’t be researched and referenced very easily. Not even the Nordic legends provided images to refer to, and there weren’t any ‘Bi-frost wormholes’ to look at on YouTube.”

Colour Zones
They completed many concept and look development frames to hone the looks before they started to animate, but it was a project in which look development tended to continue alongside actual shot production. To get started they had a green screen shot of a live actor falling through space to create a background for. The rest of the backgrounds were created for digi-doubles that the team built, textured, rigged and animated to match the live actors and their performances. They also designed an array of different cameras to move around in the Bi-frost journey. Once they had composited these elements into their shots, the editors cut them into the footage to check the flow of the story.

The saturated rainbow colour palette was approached by finding groups of two or three colours that worked well together instead of including them all in every frame. As the character travelled through the bridge he would pass through zones of different colour groups. Once they had established these colour fields working around the scene and environment, they could project them back onto the character to integrate them into the background.

From the set, they had photographic reference of the actors and a digital scan of each one in different outfits. Maya is their principle animation, rigging and lighting tool. The doubles were all set up in Mudbox, exported as large files of displacement maps for the textures. All the Bi-frost elements were rendered in mental ray, the doubles were rendered with RenderMan, and everything was composited with Nuke. A number of show-specific tools were also written, mainly to move data between applications.

Stereo FX
A core challenge the team had to deal with was the fact that the movie was originally shot in anamorphic mono format, and needed to be converted to stereo 3D in post. Left and right images were extracted from the footage by projecting the image and reprocessing it into two, slightly offset images. The Fuel VFX team, however, would complete and submit their vfx shots first in mono and then receive them back again as two separate ‘eyes’, to re-do the shot in stereo. The left eye was completed first and then the right eye, in which the effects work and convergence would be adjusted to produce the stereo 3D effect. “The difficulty was that we weren’t working with known values. The work has to be done by eye, and it’s not an especially scientific process,” Paul said.” 

Surface of the Sun
Fuel VFX team of 25 artists also extended their talent to Odin’s chamber, where the work focussed on a story point that Odin needs to retreat occasionally and recharge himself, falling into a sleep beneath a special light-ray or field of energy. “In the original concept frames, it appears as a large light beam shining down, but the production wasn’t enthused by this presentation. It simply looked like a gradient and not especially interesting or compelling,” Paul said.

A further challenge was the need for the actors to hold dialogue across Odin’s bed, and a beam would have interfered with their performances. Instead, a dome-like field of energy was explored as an alternative. They considered a surface like the sun’s would also be effective – fluid, burning and gaseous. Their energy field, developed by CG Supervisor Pawel Olas, incorporated bubbling particles, allowing it a feeling of depth when seen in stereo. It was achieved through a series of particle simulations with fluid properties in shimmering gold colours, filling up the space around him. The helios effects were small fluid simulations to give it a natural organic quality of energy, similar to the concept of the Bi-frost.

Words: Adriene Hurst
Images: Courtesy of Paramount Pictures

MORE FEATURE ARTICLESFeatured in Digital Media World. Subscribe to the print edition of the magazine and receive the full story with all the images delivered to you. Only $79 per year. 
PDF version only FREEsubscribe