Francois Dumoulin talks about recreating 19th-century Nantucket for
‘In the Heart of the Sea’ with concept art, large scale set extensions,
CG water and matte paintings for 260 VFX shots.

Rodeo FX Sets Sail ‘In the Heart of the Sea’

Rodeo FX recreated 19th-century Nantucket for ‘In the Heart of the Sea’, including concept art, large scale set extensions, CG ocean water and evocative matte paintings for over 260 VFX shots. ‘In the Heart of the Sea’ portrays 19th-century Nantucket island, off the coast of Massachusetts, and the whaling industry that led to rapid growth there from the early 1800s through to the middle of the century. In the story, writer Herman Melville goes looking for a survivor of a whaling ship that had been sunk by a massive whale, as background for his novel, ‘Moby Dick’.

The film is based on the sinking of the Essex, a real ship, by a sperm whale in 1820 and the hardships suffered by the survivors at sea. Rodeo FX worked with the director Ron Howard and the production to reveal the island as it was during the whaling era through an invisible blend of practical assets, CG set extensions and digital matte paintings.

Visual Journey

Concept artist at Rodeo FX Olivier Martin produced successful preliminary sketches for the project in mid-2013. Once the team joined the project, VFX supervisor at Rodeo FX François Dumoulin met the production on a trip to the set in London in September that year around the time the set was under construction, to capture reference images for set extensions.

The production’s art department built an extensive set depicting Nantucket at the Warner Bros. studio in Leavesden near London, complete with streetscapes lined with building exteriors and interiors. Olivier created 2D paintings in Photoshop over Francois’ reference images to illustrate potential looks for the extensions.

“It was a detailed set and was an excellent starting point for constructing the rest of the town,” said François. “Our task was to expand the physical set, filling the space with hundreds of realistic houses and industrial buildings that lead down to the ships on the wharves, including all the pollution of the whaling industry, and continue the view out to sea. We wanted to create a visual journey that gives viewers a sense of time and place.”

The period covered in the story, 1819 to 1850, was a period of expansion and development in Nantucket. The production wasn’t necessarily looking for an exact replica of the real Nantucket from a particular historical period, but for painterly looks that would evoke an impression of the town in different eras. When creating the concept art for Nantucket, Olivier had looked at 19th-century artists such as Turner, Sargent and John Atkinson Grimshaw for inspiration. “Studying these painters gave us ideas of how to depict the era,” he said. “The mood, light and composition of their paintings guided the visual style of our work.”

Procedural Flexibility

Once they had captured those looks, they had to be able generate views in any direction and at any time of day. Beyond history and environmental conditions, they also had to be able to vary scenes to reflect economic, social and commercial differences and change.

This level of flexibility demanded a considered approach for the set extensions, and started with a full CG build of Nantucket incorporating scans of the set obtained from the production, plus Francois’ on-set documentation that included textures and potential camera angles around the local area. Most of this 3D model could be reused for each version of Nantucket as it transformed from a village to a city. Variations were handled procedurally, and would be applied to shots mainly at render time.

They built a library of 15 to 20 basic buildings with designs determined by researching old maps and paintings from those times, as well as the reference photography, and studying the information on architectural detail and materials such as windows, doors, roofs and chimneys, supplied from the production designer Mark Tildesley.

They also plotted out a map of Nantucket and selected different areas in which to specialise in certain building types and neighbourhood styles, replacing houses with industrial buildings in some areas, adding more ships, piers and whaling boats. Chimney smoke, mist, seagulls and city movement including people walking on the streets were added as practical or CG elements. Because the shooting scenarios were diverse from sequence to sequence, they wanted to be ready for anything.

Francois said, “This was really the biggest challenge of our work on this project. Each sequence needed to have a specific quality applied to it, including colours. So, once we were prepared for the renders with different set dressings of rendered props, materials, dust, levels of garbage and so on, a procedural approach meant we could make the necessary variations as they were called for.

A Town with Character

“Then we worked Ron’s feedback into our concepts to accent and augment the historical representation of the town in order to support his vision of the story. An entity, such as a town, is like a character in a story so the qualitative details, such as scale, perspective, mood, all contribute to that character, and the story itself.”

Their approach to skies came from the fact that the production was shooting in London in the autumn, when the sky tends to change continuously from bright and clear to dark and cloudy. Francois was able to collect imagery for very realistic sky domes based on those skies. When it came to atmospherics, the DP Anthony Dod Mantle was inclined to use camera filters to build up atmosphere. This created a challenge for Rodeo when they wanted to apply their own atmospheric effects to the plates, and meant they had to balance the elements they added in post with the in-camera look.

For the geometry underpinning Rodeo FX’s digital Nantucket, they adopted a rolling landscape taken from images of an unpopulated island in the Canary Islands. The team combined plates of the location with their set extensions, and augmented the composite with matte paintings designed to recreate the atmosphere a village devoted to whale oil production.

Olivier noted, “As an on-going theme in the film, Ron and Mark Tildesley wanted to show the contrast between the beautiful untouched coast of Maine and the pollution generated by the whaling industry in the city itself. We designed our Nantucket to look dirty, from the chimney smoke and the extraction of whale oil on the beaches.” For the island terrain in the backgrounds, Rodeo FX could prepare fairly low resolution meshes from information found on Google Earth, as these shots usually involved only limited parallax.

Out to Sea

The whaling ship, the Essex, is central to this story and had to be handled with care. While the original Essex itself was destroyed, a very similar ship of the same era, the Phoenix, still exists and has been restored into an early-19th-century tall ship. The art department revamped the Phoenix to closely resemble the Essex, and also built a slightly scaled up replica of it to shoot in a water tank at Leavesden studios. The ship rested on a gimbal rig that allowed it to move, mimicing a ship’s rolling on the waves for footage needed when the story follows the ship out on the open sea.

After some weeks of photography in the water tank with the replica, a crew sailed the Phoenix down to the Canary Islands, where the production captured further material for the more expansive ocean shots. Because the tank water wasn't a good match for the real ocean shots around the Islands, Rodeo FX ensured continuity between these two types of shots by combining practical footage from the production, and their own CG water assets, creating waves and the look of wind on water.

“For wider views around Nantucket and shots of and from the ship, we had to extend the water all the way to the horizon,” said Mikaël Damant-Sirois, CG supervisor at Rodeo FX. “We made a variety of CG water so that we could match the different types of shots, and made the tank water darker and more dramatic to simulate the roiling sea.” They created their own generic ocean shader, which especially helped with scenes of the ship leaving the pier, for example, which would start out with shots captured in the tank, but then cut to views of the real sea from the bridge of the ship, necessitating a water transition between sets.

Different frequencies could be applied for natural variations in the surface. They also needed interactive water for about 20 shots, which required many hours of customisation in Houdini. The water development for this project has proved to be an important step in improving the CG pipeline at Rodeo FX. Until recently, the team has earned its reputation mainly through expertise in 2D, 2.5D and 3D digital matte paintings and environments, but now they are gaining recognition for more diverse CG skills as well.

Scanline VFX, another vendor on the film, went to Leavesden to source reference material for a full CG model of the ship that their team would need for the storm shots surrounding the Essex. Rodeo FX borrowed this model for their views of the ship from the town. Views from the ship were created as matte paintings, handled as cycloramas

360° Camera Move

A shot that highlights the combination of practical assets with CG set extensions and matte painting occurs as the Essex leaves port for the open sea. The camera hovers above the ship and swoops around, executing a 360° reveal of the sea and the bay of Nantucket. Rodeo FX replicated this same camera move in CG, then extended the sequence to reveal more of the city. Once the client was happy with the shot composition, Rodeo FX added lighting to the 3D model, adjusted details such as matching the sky to the plates, and then completed a CG render.

Francois Dumoulin, VFX Supervisor, Rodeo FX

Francois said, “Originally, the final shot was not intended to be a view of Nantucket but of the island. In the end, however, only the boat and camera move were preserved from the photography. Rodeo FX’s familiarity with the whole area, from near and far and most camera angles, meant we already had what we needed to cover the move and retain the original parallax.

“We also understood the depth within the view, allowing us to assign depth to different assets and identify layers for the render, building the scene with matte paintings on separate layers. The layers were then re-projected for the composite. We were working at a very high resolution, up to about 20K, and successfully re-used a number of our CG assets.”  www.rodeofx.com

Words: Adriene Hurst
Images: Warner Bros. Pictures