Pulling together the many layers of assets, lights, explosions and visual effects that make the fantastic universe of ‘TRON: Legacy’ was no small task. To achieve the director’s vision, Compositing Supervisor Paul Lambert at Digital Domain worked with a team at The Foundry and artists from numerous vendors. One vendor was Prime Focus, who sent the film’s heroes out across the digital grid landscape in the massive Solar Sailer.
The shoot took place from March to August 2009 in Vancouver, where VFX Supervisor Eric Barba from Digital Domain worked on-set with a small integration team. ‘TRON: Legacy’ was Paul’s first stereo project, and one of the first movies made on the Pace rig after ‘Avatar’. The stereo work felt new to everyone at Digital Domain, where they have worked on dimensionalised projects before TRON but not one that was shot in stereo.
This film was significant for Digital Domain because they were working with the production and director from the very outset, carrying out previsualistion and running the Art Department in-house. “Digital Domain has never had that level of access to a production before,” said Paul. “We have always worked as vendor under an on-set VFX Supervisor hired by the production. But on this project, our VFX supervisor Eric Barba took that role, giving us much more contact, and everyone found working this way very enjoyable.”
The Digital Domain team completed 17 of the film’s sequences including major set pieces for the Disc Game, Light Bike Battle, Light Jet Battle and the portal sequence at the film’s climax. They also set up the looks on all sequences outsourced to other vendors, where the teams would develop them further following the rules and guidelines for the show, which were specific and precise. Consistency was crucial for this film and because about half the film’s shots had to be outsourced, maintaining control was a major task.
To use the follow focus system, they would input their camera data, the CG from two different elements and, for example, information from the animation about where that camera should be. From there, they could work out from the plate how to correctly focus their work in stereo space, which helped give the CG an integrated, realistic look.
They handed out plug-ins to do all flare work as well, because the film features a highly stylised flare with scan lines, controlled by angles of light. Such tools were developed on the actual shots, and by the time they sent them out, they were complete pieces of software, fairly intuitive to use. For example, to add a flare to a shot, the plug-in would use the location of the other lights in the shot to produce the appropriate ‘TRON flare’.
Another tool helped produce the correct glow lines on each character’s suit, reflecting the particular colour palette associated with his identity. These lines often had to be enhanced to look brighter and show the correct colour and flicker frequency. “For some things like pulling the mattes to add the glows, there wasn’t a prescribed procedure,” Paul explained. “Sometimes the glow could be keyed, but in other instances it had to be rotoscoped. The suits sometimes failed and the light would go out, which meant it had to be rotoscoped back in.”
“Throughout, we were working out what we could and couldn’t do in stereo compositing and tricks that we could get away with. We needed to gain a clearer understanding of interocular convergence and were learning how to compare one eye with the other to create a better image. Some people have more natural skill at this, while others have to learn and can get headaches at first. Double checking every shot for consistency is time consuming but critical.”
The action in the TRON world was played out over a clearly defined area. When you looked out over a wide area, for example in the Light Bike sequence, you would see a view in which every asset had to be located on the correct coordinates, so that when you rotated the camera around you would still know where you were in that space and what you would see. Likewise, on the Solar Sailer’s flyover sequence, because the characters were travelling so far, they had to actually pass by places such as the stadium or the arena, all positioned in the correct locations on the grid.
“The many optical effects were also a major factor in integrating live action with CG layers. All the optics created for the light bikes – moving into camera, turning, twisting, jumping – were done in the composite, slightly stylised of course but they added tremendously to the real-world feeling we achieved in the shots.
“In the Light Jet sequence, flying in a massive environment with atmosphere everywhere, we were trying to composite everything together, keeping the glows and interactivity in place with bullets flying everywhere as well. We wanted to give a believable look to tracer fire and exhaust systems. All the tiny bits of optics really helped us finesse the CG to look as real as possible.
The climactic Portal sequence involved hundreds of layers and passes. It had the Light Jet environment as the background, plus clouds, the sea and atmospheric passes, then the bright portal itself composed of various elements, and the bridge. Adding the CG Jeff Bridges added another full system of passes – it all added up to around 150 passes all together.
“Across TRON, the environments were top lit with quite flat lighting, which was intentional of course but did make our head integration work more difficult. We had found on ‘Benjamin Button’, where we had similar work, that if the lighting was fairly moody with identifiable point sources of light, it actually made it easier to get everything working together, while the exterior shots took longer because the light was flatter without a direct source,” Paul said.
“However, all of TRON was made this way – no moon, sun or time of day. This was challenging at the beginning of production when we tried to define the Light Jet world, our first sequence. Having no specific source of light made shaping the planes, textures and surfaces in the environment more difficult.”
Back at the studio, before a shot was assigned, they would assemble this HDR and break it down into its specific geometry. The team would also have surveyed the set, taking all measurements, collecting textures and pictures of all the lights to be able to rebuild that scene in Nuke, which they would test for correctness.
Once satisfied, they would publish the information out to the lighters, so they had all the data they needed for the lights cut out from the HDRI and put onto cards with all the associated geometry and could light the head correctly according to the shot for a photoreal result. This pipeline has now been used on several projects at Digital Domain.
In some cases, the team had plenty of concept art. For others, there was next to none and they needed to flesh out their shots according to the established rules. Because Digital Domain had been directly with production from the start, Prime Focus' artists could focus immediately these tasks instead of first undertaking an extensive R&D and concept stage as they normally would.
They were given three pieces of concept art for the Solar Sailer, along with a rough model used for previs. What its design needed was what the production called richness – subtle detailing to not only communicate the practical, technical nature of all elements inside TRON but also its large scale, a quality the base asset lacked.
“While there were several practical set pieces for the Solar Sailer, no single, master set piece was made,” said DFX supervisor Jon Cowley. “As result, portions of the set, for example sections of the cargo pods, were shot live action that we, in turn, had to match in CG on our platform-only set. Other parts of the vessel like the sails only existed as concept art and we had the flexibility and freedom to further develop the look, movement and energy pulses associated with them.” Modelling for the ship was done inside Maya and Mudbox, and at times with ZBrush.
Speed and Distance
The brief stipulated that the ship should move very fast, on an essentially straight line at a constant speed, and at an altitude of 10,000 feet. At this height, of course, objects on the ground scarcely appear to be moving, and they needed a means to create more interest and a sense of motion and speed. They added some flex to the sails, for example, but what was most effective were the quantity of atmospheric elements the artists layered into the surrounding space. In the end, a huge number of these atmospherics was required, and creating and managing them became a major part of their work.
Adding atmospherics meant dealing directly with stereo requirements. The team couldn’t employ some of the 2D techniques typically used for atmospherics, such as putting footage of clouds or smoke on cards. “In a volume of this size, given the amount of depth we had to convey, everything had to be 3D CG and had to be rendered,” said Jon. “It’s amazing what can happen. When something like a cloud is completely out of focus off in the distance, if it isn’t sitting in the in the proper spot in stereo space, you’ll suddenly have a soft fuzzy cloud popping off screen or sitting too far into the background.”
For the terrestrial landscape, they wrote they own procedural landscape generator that obeyed the TRON rules. For example, the rules about angles dictated that all objects would be constructed with 30°, 60° and 90° angles, like the construction of a computer chip. Their procedural toolset was built accordingly and used to construct most of the environment, into which they would drop ‘hero’ sculpted pieces wherever required. “This tool had to be made quite show-specific for TRON,” said VFX Supervisor Chris Harvey, “but no doubt we’ll recycle the code at some stage for use on another project.”
Digital Domain supplied various shot references from other sections of the film plus the concept artwork for the environments. The shots supplied for reference were various hero shots that illustrated the style for the film and at times showed bits of landscape. Apart from that, the team had three rules to follow. First was the direction of travel, a straight line toward the portal, as dictated by the story. Second was the 30°-60°-90°-angle rule. Third was the layout of a V-shaped canyon, composed of 30° angles that the Solar Sailer entered on its way to the portal.
Skies and Cityscapes
Dealing with the fundamental changes required to produce matte paintings for stereo movies was a challenge. Jon explained that nothing can be cheated or approximated, all geometry had to exist and be created, modelled and assembled to occupy 3D space. They used the paintings for a lot of the very large assets because it was easier than trying to texture and render every detail.
Jon said, “Stereo can throw a wrench into a shot that should otherwise be pretty straightforward. For example, in this case, the glass in the elevator needed reflections added to it in post, which normally isn’t so difficult. But these had to be proper reflections, receding in depth in stereo.”
Single Light Kit
However, getting it right in the first place took some effort – it wasn’t simply a matter of matching their lighting to an existing plate, but had to be consciously art-directed like the clouds. Chris said, “An analogy we used early in production to capture the look was that TRON should have some characteristics of a stylish car advertisement. Our goal became creating that feeling of light emanating from the beam of light they were traveling on, and the impression of light from the direction of the portal and city, paying particular attention to the artistic nature of the shot and yet trying to maintain and believable and consistent lighting direction.”
One of the more difficult lighting developments was establishing the set-up for all clouds and environments. They wanted to ensure that they could tweak and control this lighting further along the pipeline in Nuke. Because it would be affecting so many shots, they especially didn’t want to commit themselves to an approach that couldn’t be adjusted in the composite to suit a special request from the director or VFX supervisor. They wanted to avoid going back to extensively relight shots.
So they broke down their set-up into an RGB-type of pass. Jon explained, “The convenient aspect of this uniform lighting was that, across the wide expanse, we could limit the sources we had to deal with to the light they were travelling toward, the light they were coming from, and the generic fill lights used for our atmospherics and clouds, that were being created by the beam. That gave us three basic light sources, just right for an RGB system.”
From that stage, they would work on the left eye and fully render it in CG, and then the right eye, also fully rendered in CG. Jon watched the render process carefully. “Within the 13 minutes of material, a particular render might not come off the farm for three or four days, Jon said. “It may look great but, for some reason, have required 20 hours per frame. The right eye now needs rendering, but that render time is not really acceptable within your workflow. So you have a dilemma – do you stop and figure out where the settings could be improved and render the left eye again or, because it looks good, just retain the settings and render the right eye the same way? You can’t change the settings for the right eye only. They must be identical.”
Data management to keep track of the corresponding passes and layers for the left and right eye of each shot was another substantial undertaking. They wrote special tools to manage these details, freeing up the lighters and compositors so they could concentrate on getting the left eye to look as good as possible. Only a small dedicated team – consisting basically of Jon! - had to worry about the right eye. These tools, or interfaces, allowed him to control the process with out having to open Nuke or Maya or revisit the scenes to track what was generated or being used in the composites, and match the right eye to this.
Another mandated software was Nuke. In this case, Prime Focus had not been using Nuke in their pipeline. They had been a Digital Fusion studio, and found they had to change their entire 3D pipeline for TRON, involving a considerable integration process and porting several tools over to work with it. However, Nuke already has many good stereo tools that made some of their in-house tools redundant, avoiding the porting problem.
Because Digital Domain supplied the different Nuke-specific ‘show-look’ tools, like the glow lines tool mentioned above, as well as stereo tools, everyone was working on common ground. Prime Focus especially liked OCULA’s tools for dealing with stereo anomalies. “A high-contrast show like TRON could have become even more demanding for us, especially regarding light polarization and reflective surfaces that potentially create stereo issues,” said Jon.
Life in Stereo
On the other hand, this was Jon’s first stereo show. “On my first day on the project, I sat in the theatre with Chris looking at shots while he pointed out all the details we would need to address – but I couldn’t see any of them. It was a visual blur to me. So I’d say that the first hurdle on stereo projects is helping the team to become more ‘stereo savvy’ and able to identify issues within the images. You can’t do this alone or by reading books on the subject. You need a mentor to walk you through visually.
“The difficulty of learning to work in stereo can be underestimated – it does change the work of a VFX artist. Stereo also needs an experienced DP on set to shoot the blue screen and other VFX shots for the team to work on. Some aspects of stereo footage can be corrected in post, but others cannot be.”
A major stereo issue in the post production arose because one camera on a mirror rig shoots through a mirror and the other shoots a reflection from a mirror. “Reflected light is naturally polarized, so the effect creates differences between the left and right images,” Paul explained. “When you have a scene with light sheens and reflections, these will be noticeably less evident in one eye than the other and when viewed together in stereo, the result is disturbing and unnatural to the viewer.
“Polarization issues emerged in every shot containing on-set reflections. Unfortunately, not only is the world of TRON a very shiny place, but it also features characters wearing shiny black suits. The ColourMatcher plug-in for Nuke from The Foundry’s OCULA package enabled us to take the colour information from one eye, analyse the same information in the other eye and correct the pair by making the colours consistent across both images.
“We worked with The Foundry over a few months to help refine ColourMatcher, and it worked out well for TRON. It’s not a ‘plug-and-play’ kind of tool but we were able to successfully hand the set-up that we had developed from the original software to the outsourced teams on the project. When they ran it, it would produce four different outputs, each tuned for different tasks – one for details, one for adjusting focus, one for large gradients and so on, so there wasn’t a single fix that addressed everything. They would generate these outputs and make a collage of the best parts of each, creating the other eye and overcoming the polarisation issues. Once it was running it was fairly straightforward.”
For v2.1 of OCULA , The Foundry incorporated their set-up and consequently the plug-in now consists of only a few nodes instead of hundreds, making it quicker and easier to get a correction. Paul still feels it’s necessary for users to go through the process of generating the various outputs and collaging them together themselves, instead of going for a one-click procedure.
|Words: Adriene Hurst
Images: Courtesy of Walt Disney Pictures
|MORE FEATURE ARTICLES|