Les Misérables – REBEL SONGS

Les Misérables – Rebel Songs

Sean Mathieson took the role of overall VFX supervisor, working on set at the live action shoot and various element shoots, and carrying out previsualisation, while Richard Bain led the team at Double Negative, the main VFX vendor handling over 250 shots. Principle photography lasted 12 weeks from February to May 2012.

Revolutionary Paris

In some early conversations with Richard Bain, the director and DP explained their intentions to build sets for many street scenes in Paris inside a studio at Pinewood including sequences showing the student rebels defending their barricade against the French army. Unsure at first about how well these extended outdoor sequences could be captured in a studio, Richard anticipated that the director might be aiming for a stage-like feeling, but as the production progressed Tom indicated that he also wanted to open up views of Paris for the audience and include wider shots.

Therefore, a primary focus for Richard early on was deciding the most effective, practical way to build Paris with the resources they had, and identifying the locations as soon as possible. Tom insisted on historical accuracy, and was especially keen to show Paris architecture from before the Hausmann era of the 1850s, when the existing neighbourhoods were levelled and central Paris was re-built in the contemporary style.

Victor Hugo’s story takes place before this era, which meant researching buildings that scarcely exist anywhere now, especially not in Paris. What did Paris look like at that time, especially at Place de La Bastille and around its famous Elephant sculpture? While the prison and weapons store had been destroyed during the Revolution, how did their remains relate to the rest of Paris? Tom also wanted to shift the story between four different time periods in the life of the hero, Jean Valjean, and include time transition shots that needed previsualising.

Production designer Eve Stewart shared research with Double Negative’s  team that she had done on the architecture for the Paris set pieces, and suggested they visit the town of La Rochelle in France, where pre-Hausmann architecture can still be found. There, they were able to shoot enough imagery to create buildings for Paris.

Wide Angle & Transition Shots

Instead of trying to complete a comprehensive 3D build of the city, Double Negative put their 3D effort mainly into the street extensions. “But even for shots from the top of the police station, a high vantage point that required very wide views,” said Richard, “we could successfully project 2D imagery onto geometry and still work with the various crane moves the DP used to capture singing performances. Although Tom never expected us to create flyovers of Paris or zoom in from long distances, transition shots could still be challenging.

“For example, in order to bring the camera down from the police station to the Place de la Bastille, crossing a lighting change as well, a special camera move was developed that first looks up into the sky before returning to the ground. For one thing, we had no Paris asset to bridge locations and second, Tom wanted to use that up-turned camera move as a motif to refer to God’s presence in the story. Because it allowed the VFX team to avoid building out Paris flyovers, saving the budget, it was consequently was used in several sequences.”

For that particular shot, the camera had to transition from the rooftop to the young boy Gavroche inside the Elephant at Place de la Bastille. On set they only had a very short crane move onto Gavroache that needed extending. Lead TD Jenni Enyon and Compositor Julia Reinhard worked with CG supervisor Alison Wortman, using 2D projections of the elephant’s textures onto 3D modelling. Starting from a locked off crane position pushing into Gavroche, they could re-project the textures and, using a LIDAR scan of the elephant, move onto it from a 30° angle. Fortunately, all the action of the extras in the environment was captured in camera in the plate.

“Creating big CG camera moves would not have matched the DP’s style for the film in any case, which often relied on hand held camera work and avoided sweeping crane moves. Locked off establishers – pulling straight back and up and then transitioning to techniques using projected textures - were used to open the set and show what was happening across Paris,” said Richard.

Shooting Style

An example is a shot over Paris that Double Negative started and passed to Rushes featuring Valjean singing with Marius, who has been injured at the barricade. The camera pulls back on a 50ft crane but because the crew was working on a stage, the cameraman quickly ran out of space under the lights. However, Tom’s vision for the shot included revealing that the isolated, battered student barricade is surrounded on all sides by the army. By pulling back further still, the audience would be able to see that life across Paris carries on regardless of their struggle, and that their prospects seem hopeless.

Sean Mathiesen explained, “To make the situation evident, the camera on the crane pulled back into the lights as far as possible. From there Rushes carried out a digital take-over employing a LIDAR scan of the barricade and entire stage, plus the scans of buildings, to build up enough 3D environment to cover the shot as far as it had to go. The transition point between the techno crane move and this larger digital asset was achieved with projections – the artists went out to the widest frame, took textures from the digital model and extended these out as a matte painting, projected this back onto the model, and then applied the matchmove and camera move.”

The Mill handled similar shots. For most of the movie, 3D modelling of buildings could mainly be confined to set extensions near the action and the camera, wider shots relied on 2D projections, and the widest views employed digital matte paintings. The big exception to that scenario was the title sequence, leading into the opening shots, which demanded 3D builds of an entire French navy ship and surrounding wharf environment, plus water simulations for a heavy storm at sea.

French Navy

It may be surprising that this film, essentially live action, should start with sequences that rely so heavily on VFX, but Tom had a very clear idea of what he wanted to see. Sean described the previs period as the production explored concepts of the camera emerging from the storm, helicopter shots of the ocean and a view of the French Armada. Double Negative contributed concept art.

He said, “One of Tom’s primary ideas was showing the dark below and light above – God’s presence. The camera would start underwater, and move up following a rope to reveal a French flag floating in the water. We would then continue following the rope up, break surface and see the massive stormy sea, populated with the formidable French Navy. From there, the camera was to follow some rigging to the broken mast and ropes of a damaged ship and follow one of these ropes to the shot of Valjean singing with the other convicts, the beginning of the film.

“This progression between environments was a story in itself and took a great deal of previs. Perfecting the CG-to-practical set transitions took virtually the entire production time. But it was a chance to create something greater than the stage production was capable of.

“The crew set up at U Stage at Pinewood with an underwater camera in a tank full of murky water loaded with live action particulates and ink. The camera followed a rope up to an enormous French flag – a practical element about 20ft long – with wave machines and practical rain at the surface. We enhanced these shots with practical water elements shot at Artem physical FX company in London.”

Water Simulation

From that point of course, they had to provide a CG ocean, and add rain elements and smoke to fill the environment with storm effects. While relying mainly on CG water simulation using Double Negative’s in-house tools dnOcean and squirt, Sean also travelled with a crew to Cornwall, shooting waves and water action as material for sprites and elements that would help create their stormy environment, with sympathetic shapes along the coastline. “It’s often easier to create general undulating surfaces for oceans than convincing foam and drips, he said. “We used the piers as vantage points for looking back at waves breaking on the beach, or the tall cliffs to look down at the ocean for realistic textures,” Sean said.

These shots move into the opening sequence introducing Valjean in 1815 as a convict, labouring in a contingent of prisoners as they haul a French war ship into a dry dock. The boat is entirely computer generated, based on digital scans and textures from the HMS Victory docked at Portsmouth. Double Negative managed the numerous ship textures for ropes, decks, winches and pullies to manage, and adjusted the colour scheme of the boat to display period French naval colours. They even shot the nautical detail with crane moves and drips because, from this digital space, they transitioned directly to a real rope that Valjean is holding onto in a practical set.

 Double Negative also needed to build the wharf detail around the ship within a full 3D CG view of Toulon harbour. Later, after Valjean is released on parole by prison guard Javert, he mounts a set of steps as he leaves Toulon to begin his new life, and looks back at the scene, requiring another full 3D build.

Toulon Environment

Their concept team, led by Simon Gustaffson, had started straightaway on the sequence. Richard said, “It passed through several iterations in the script – including a camera move approaching Toulon from above - all of which were going to be the most expensive shots in the movie and require a lot of design and build time. While the director finalized the look, the team worked on concepts and previs, continuing well after the shoot got underway. Tom referred us frequently to period marine oil paintings and favoured Simon’s earliest concepts from the start, which referenced one of these paintings.”

The Toulon environment was based partly on historical research of the area but also on a pencil sketch that Tom supplied showing how he saw the layout of the harbour and drydocks. He wanted to reveal the convicts and prison through the storm as the camera craned over the top of the ship. The build of Toulon was mainly a matte painting for the distance but the prison was modelled and textured with reference to the actual Toulon prison of the period. The dry dock area was based on LIDAR and texture information that scanner Craig Crane and CG supervisor Alison Wortman acquired in Portsmouth.

Adding to the pressure for high quality work and a complete model was the fact that Tom’s filmmaking style involved several cameras, affording choices in the edit until quite late into production. This made visual effects turnover less straightforward for the artists, who were aware that the shots may change at anytime without notice. As a result, this opening sequence was finessed and re-worked to within a few weeks of delivery.

Performance Capture

While ‘Les Misérables’ is above all a musical, emphasising the music placed other demands on the VFX teams. Tom hesitated in some cases to sign off on camera moves, wanting to do the music justice or deciding to lengthen a shot to match the beat of the sound track. But the later the decision was made, the less time they would have to complete a shot.

Live singing performances on set made detailed cleanup essential because each singer wore a fair-sized earpiece and at least one microphone. Double Negative in Singapore carried out a large portion of this work, which varied from scene to scene. As their raw materials, they used head and body scans of the principle actors and were able to re-build ears and hair, which could be very complex when the actors performed on green screen.

Valjean’s hair changed several times through the story, requiring several sets of reference photography for his ear. If the earpiece bent this ear they had to produce a 3D asset because Tom almost always used a moving camera. The costumes often failed to hide the mics, which had to be removed.

Drama in the Sky

The sequence at student barricade was shot partly at an outdoor location at Greenwich but also on T Stage at Pinewood. The team at The Mill added the skies as matte paintings to replace the light rigging overhead. Skies were another important element that Tom described from early meetings as “where the drama is”. He wanted to establish an omnipresent God character throughout the film, using the skies as a frequent reminder of that presence.

Emphasising the sky may seem at odds with shooting in a studio, but the DP was concerned about controlling the lighting, especially on T Stage where they had so many shots on the same set, at all times of day and night. Maintaining continuity at an outdoor location would have been a challenge. As well as providing the skies, The Mill’s team ensured the shots had both an outdoor feeling and a religious quality to suggest God watching events unfold and controlling each character’s destiny.

VFX supervisor Sara Bennett explained that to begin, they created a mood board for each scene of various skies to choose from, from intensely dramatic to very simple. “Initially we went for a dramatic look right across the shots, which Tom also liked at first, full of clouds with a Godly presence. But later, concerned that the skies might distract from performances in some cases, changed his approach and reserved the extra drama for certain scenes only,” she said.

“For most skies our matte painting department used a large database of photographed sky elements we’ve gathered over many years, but some of the more scene-specific, dramatic skies were matte painted using Photoshop. When placing light sources, moon or sunlight, DP Danny Cohen’s lighting was a clear guide for us. Light was often flaring around characters and buildings, effects that we aimed to copy wherever it affected our work.”

Integrated Skies & Lighting

They worked with Danny from the start on the look for the skies, which had to be closely tied to the set lighting. The shoot was mainly on the one large exterior street set that the crew had to light for day, night, dusk and dawn. The use of several handheld cameras onset, constantly capturing everything from different camera angles, left few spots to hide the lights, which meant placing much of the lighting up on an overhead grid. They used a Light by Numbers lighting control system to create various programs for different times of day.

“This was a good starting point for our team,” Sara said. “Before we got into any major sky work, we would pick a few hero shots from the shoot, create two or three versions and then speak with him about how he had lit it and the time of day he was aiming for. This really helped us get started and prevented wasting time.”

For continuity, Double Negative shared their La Rochelle building assets with The Mill to help extend out the streets around the barricade on the stage. They built up the street geometry themselves to add the depth and scale, and projected Double Negative’s high resolution stills and reference material onto it.

The camera work caused a few other challenges. The use of wide angle lenses resulted in distortions which required tracking shots and ensuring they had camera, lens and set data to line up their work accurately with the photography. The handheld work, shot on film, produced small, quick movements and motion blur that sometimes made tracking difficult. The lack of green screen on set demanded quite a bit of rotoscoping for the skies, and the set lights often cast flares across the set that needed cleanup.

Unghostly Ghost

When Tom Hooper approached The Mill about the character Fantine’s return to the story as a ghost when Valjean is dying, long after her own tragic death, the team immediately began thinking of traditional, transparent ghostly looks. But Tom had very different ideas and wanted to avoid visual clichés. He wanted instead to create something spiritual, ethereal and beautiful purely by using elements existing in the plate or on set.

For example, the church interior where the sequence was shot featured stained glass windows, the effect of which they enhanced by introducing more colour into the images. The set also contained numerous candles. Noticing that when the actors were captured against these candles, they produced a beautiful, naturally ethereal effect, the artists decided to add extra candles behind Fantine in several shots. They could use that same natural effect to show that she had returned from death without making her look like a traditional ghost.

Pulling Crowds

The story of ‘Les Misérables’ involves several crowd scenes but the production had a limited number of extras, up to 300 and sometimes only 80 or 90, especially when they needed to be wearing elaborate costumes. Two primary crowd techniques were employed. One, used at Fantine’s funeral sequence shot at Greenwich, was especially useful in wider shots of the procession, meant to be a mile long. The usual method is to capture groups of actors as elements from the same height and angle as the camera on the main shoot, but because Tom was using six cameras across the action, Sean and Richard realised that the crowd top-up shots could not be captured on location.

“Double Negative’s CG supervisor Alison Wortman prepared a breakdown of all camera heights and angles from the shoot, which we took with a film crew to a back lot with a 50ft techno crane and about 100 extras, to carry out a shoot using approximated, matching camera moves,” said Sean. “We shot small groups which the compositing team used on cards, while doing a matchmove on the original crane move. As the actors weren’t on green screen, this also demanded a lot of rotoscoping, but the small groups could be effectively interspersed with larger ones to avoid repetition. Flag-waving introduced further variation in the shots, and of course another important task was hiding or removing the six camera crews within the crowds!”

Optical Flow

Another crowd technique was employed down in the dry dock at Toulon in the opening sequence, where the convicts are hauling in the war ship. As the camera descends into the dock, considerably more convicts were needed than they had extras for. In order to shoot the extras for that camera move, they set up an array of cameras, three on tripods and three positioned above on a deck. These actors were shot on green screen, moving in sync with the main shoot actors, and using a global shutter reaching to each camera. The result was an optical flow, that is, apparent motion caused by the relative motion between the cameras and each ‘agent’ – in this case, a moving, textured 3D model of an extra.

In this way, the green screen extras could be used in among those captured with the crane move on set. It could be used to some extent for the funeral sequence although the artists had to work within the limits of the green screen area, only about 12m square. “The technique is a combined development between Jeff Clifford and Ted Waine from Double Negative’s R&D and the one-cam camera system that One of Us, also in London, has developed,” said Richard. “With these six cameras and the One of Us crew, we could successfully capture about five extras at a time, resulting in a 2.5D sprite that held 30° to 40° of camera motion, either up and over them or around them depending on the set up of the array.”

Their team in Singapore again proved their skill for the scene down on the wharf among the prostitutes, which had been shot within a confined studio where the prows of three ships had been built as practical set pieces. The artists digitally extended the rear views of each ship and provided sky replacements. A painted backdrop had also been placed in the background instead of green screen, with the idea that close-ups on faces could be shot with the background out of focus instead of inserting another full digital replacement. Nevertheless, many shots looking up into the lights needed refining and adjusting, such as compositing fine hair detail onto the sky replacements.

“Both the cleanup and extensions from the Singapore artists were impressive, especially since they didn’t have the benefit of talking with Tom face-to-face. But communications between London and Singpore were crucial. We recorded some of the dailies sessions, and to monitor all work in progress, we used Shotgun, where our VFX producer Kate Phillips lodged all of the associated notes,” Richard said.

Helicopter Plates

As Valjean makes his way across the French countryside after his release, he tries to find shelter in a village, where eventually the church takes him in. While the village scenes were shot at Gourdon, the church was a location in England. An important camera move was planned that would pull back from this church as Valjean walks out the front door, showing it situated on a precipice surrounded by the rocky Gourdon terrain.

Sean explained that to be able to accommodate this move with flexibility, they captured helicopter plates at Gourdon, marking out on the ground the spot where they planned to place the church asset later. “We made the most of the helicopter shoot, hovering in various spots to capture 180° of detail from the environment as potential projection material and photogrammetry, as well as textures,” he said. “With all of this, Tom could re-time the aerial camera move to ensure a good line up with the church element shot containing the live action, which he and Danny had been able to closely control.”

The helicopter imagery became their master plate for any CG environmental work needed for this setting, re-projecting textures onto geometry from Google Earth and reworking the photography. Having that real photography as a basis for the sequence gave them confidence and a vision of how the final scene should really look. The cost of the helicopter was justified because not only would recreating some 75 per cent of most frames with CGI be expensive, but the production also faced a tight deadline for it.

Javert’s Suicide

After the pull-back, the helicopter shoot provided a beginning for their transition to the next sequence as well by panning across into the sun. This translated through cloud into a digital environment, dropping down to a projected matte painting, to a crane move into a live action plate of Javert and his lieutenant riding into the city. Taken all together, this sequence required a lot of previs and planning.

Double Negative’s Paris assets and imagery proved unexpectedly useful for Javert’s suicide, a fall from a bridge over a weir. This was a night shot, complicated by the studio’s concern that the fall from the bridge as originally filmed was not from a great enough height to make it clear that Javert had died. Therefore, the team not only had to raise the height of the bridge but also make some major environmental adjustments. “We were also effectively changing the geography of Paris because the bridge depicted had in fact never had a weir under it and the water was shown flowing in the wrong direction,” Richard said.

“The weir itself was shot in a single evening in Bath as a pick up shot, with a stunt actor and two dummies thrown onto the weir, and used three cranes – one for the camera, one for the dummy and for the moon light. This shot served as a plate, around which we built our Paris environment.”   www.themill.com     www.dneg.com

Words: Adriene Hurst
Images: Courtesy of Universal Pictures International