Dian Fossey Gorilla Fund’s new permanent home in Rwanda now has a 360° theatre and VR and AR experiences created by studio Habitat XR to bring people closer to endangered gorillas.
Immersive studio Habitat XR has opened the Irmelin DiCaprio Theater and launched two immersive new experiences at the Dian Fossey Gorilla Fund’s new permanent location in Rwanda. The goal of these projects is to bring visitors closer – albeit digitally – to Africa’s endangered mountain gorillas than has been possible before, without impacting their natural environment.
With that special intention in mind, the Habitat XR team, who are based in Johannesburg and led by CEO Ulrico Grech-Cumbo, has developed the theatre from the ground up. It can display 360° films in 25K resolution and is controlled entirely through a custom-designed app. It has been built to screen ultrawide 16:1 footage – in effect, ‘superpanoramic’ films – upscaled using AI techniques. The theatre has eight daisy-chained 4K laser projectors, displaying 50 million pixels onto a 43 x 2.5m canvas, constructed of high-contrast film stretched over a custom-made 3D printed frame.
Eight Projectors
These eight 4K projectors are what make it possible to support both formats – 360° at 25K and ultrawide 16:1 – at the same time. Once the required overlaps between the projectors’ displays have been factored in, the full screen achieves 25K, installed on the walls of the oval room in a full 360 strip. As viewers enter, they are invited to either stand or sit roughly at the centre of the oval.
The visual display set-up is matched to a 10.2 matrix surround sound system that features 37 individual speakers and two amplifiers as well as more than 186sqm of acoustic sound panelling designed and fabricated by the team to absorb reverberation and allow spatially recorded sounds of the forest to engulf audiences.
At launch, the theatre is screening two original film experiences. The first, titled ‘A Day in the Life of Gorillas’, was produced by Habitat XR and the Dian Fossey Gorilla Fund, following four-year-old mountain gorilla Gasizi and giving remarkable insight into his family, known as Pablo’s Group. The second film, ‘Saving Mountain Gorillas: The Dian Fossey Legacy’, is a production from the National Geographic Society's Impact Media Lab. Created and produced in collaboration with Habitat XR, it explores the history and future of gorilla conservation at the Dian Fossey Gorilla Fund.
Exploring the history and future of gorilla conservation at the Dian Fossey Gorilla Fund.
Capturing the Gorilla’s World
Ulrico talked about what it takes to capture and produce films capable of taking advantage of the Dian Fossey theatre. “We used 360-degree cameras for both films,” he said. “‘A Day in the Life of Gorillas’ was shot mostly on the Kandao Obsidian R camera in 2019. Originally, it hadn’t actually been filmed for the theatre, but rather for a fundraiser that helped raise $5M for the campus before construction began. But we had captured so much incredible footage on that shoot that we were able to compile an entire new story as an opening piece for the theatre.
“The National Geographic Society film, on the other hand, was captured in 2021 more specifically for the theatre. We used the Insta360 Titan and filmed in monoscopic 11K, knowing that it would be a challenge for us to capture footage at a resolution anywhere close to the theatre’s resolution.”
Cameras and resolutions notwithstanding, with experience, Habitat XR’s expertise in filming wildlife has grown appreciably since the company got underway in 2016. “What we've learned is that it's all about predicting behaviour,” Ulrico said. “Since we don't have zoom lenses, the free-standing camera has to be in very close proximity to the action.
“As we also don't want to affect natural behaviour, we need to place cameras where we think the gorillas are just about to do something interesting. The crew were often diving into stinging nettles to hide from the camera. Under cover, they would monitor the feed on the Intsa360 Titan’s FarSight remote viewing and control system.”
Camera Strategy
Eventually, alongside the highly skilled Rwanda Development Board trackers and guides as well as the Dian Fossey Gorilla Fund researchers, the crew learned to strategise shot placements and make lighting-fast decisions about camera placements. What made the work even trickier is that the crew’s time with the gorillas was limited each day by park regulations preventing the possible transmission of human diseases to them. They had to trek up the mountain to where the gorillas lived each time they wanted to spend an hour with them.
However, not all the scenes in National Geographic Society's ‘Saving Gorillas: the Dian Fossey Legacy’ were of gorillas, and by the time filming for the project began, Ulrico’s team had already designed the theatre's screen, which helped them frame the shots better. “The least high tech device on the shoot was also the most important – a cardboard cutout we placed on top of our live preview monitor to approximate the crop of the 16:1 screen!” he said.
Post Production
Regarding post processing, in order to produce the format and image quality needed for the theatre’s screens, the first consideration is that the theatre features a 16:1 aspect ratio ‘ribbon’ of 360° video, which means spending a tremendous amount of time stitching the different cameras together and adapting the horizon to fit all of the scene's action within this band.
Ulrico said, “We have worked in cinematic VR for a long time and, historically, displays like VR headsets have undercut the potential of the camera resolution. But in this case, the theatre display's resolution is higher than what the cameras can achieve right now, so we experimented intensively with AI upscaling software, which doesn't just divide pixels to create a higher theoretical resolution on paper, but rather analyses texture and attempts to more accurately approximate what the footage would look like if captured on an even higher resolution sensor.
“We use a piece of 3rd party software, now in beta, for this job. Understandably, as the technology is still in its infancy, the results can be fairly hit-and-miss, requiring many experiments and iterations, but it has made a noticeable difference in quality on the theatre's screen and was adopted for both films. Although currently both are playing only at 16K, they do cover the full 16:1 area of the 360° canvas.
Habitat XR also created the theatre’s wireless projection control app, mentioned above, that allows operators to manage the screen’s content or mode through an Android tablet. They can create a playlist of different content the app sees available on the server and schedule play times, and then trigger specific videos manually. A holding image mode controls ambient backdrops for events not requiring films and, finally, a presenter mode allows guest lecturers at the campus to plug a laptop into the system, which then projects two instances of the 16:9 screen onto the flattest section of the theatre's curved screen.
Gorilla Interactive
Outside the theatre, Habitat XR has developed controller-free VR kiosks and augmented reality applications that let audiences immerse themselves in gorilla behaviours using a touch tablet.
The gardens of the campus have been built to closely match the gorillas’ natural habitat. Walking along the trails, visitors can take a virtual gorilla trek with Habitat XR’s GorillAR iPad app in which a photoreal CGI silverback gorilla leads the way on an interactive foraging trip.
There are seven different experiences along the trail -- Meet and Greet, Chest Beat, Poop, Bamboo Feeding, Stinging Nettle Feeding, Wild Celery Feeding, Nest Building. Each experience is triggered by scanning a small icon that the iPad app recognizes, which also helps orientate the gorilla given that the ground plane is not flat.
To further improve the integration of the virtual gorilla with the surrounding environment, the app has a front-facing camera that uses ARKit’s ARLightEstimate to analyse real shadows on the viewer’s face to approximate the lighting direction, intensity and colour temperature of the ambient lighting. The information gathered from ARLightEstimate is useful in their shading algorithms to make the graphics from the real-time renderer match the scene captured in-camera.
Empathy and Inspiration
“The team at Habitat XR believes immersive experiences can go far beyond entertaining audiences,” said Ulrico. “Our dream is for the theatre and our other XR experiences on the campus to bring people closer to gorillas than they have been able to go before, to fill them with empathy and understanding and to inspire them to take action.”
Along with conservation, a key goal of the project for Habitat XR was to challenge the conventiona assumption that technological innovation doesn’t happen in Africa. “We’ve created one of the world’s most advanced projection theatres and some of the most subtle and engaging AR experiences, and all of these are adjacent to a rainforest in the heart of Africa,” Ulrico said. www.habitatxr.com
