Corgan MediaLab Fuels Visualisation with OpenDrives Collaborative Storage

Corgan MediaLab is the media department of Corgan architectural and design firm, operating out of offices across the US. The MediaLab takes care of video production and post-production for the company, including animations and motion graphics plus VFX and 3D architectural renderings. Their customers range from education, aviation, health care and data centre facilities, to interior construction and commercial projects.   

To support Corgan's wide customer base, the MediaLab comprises artists with the various skills and expertise needed to visualise and develop video narratives for all of these diverse industries in a collaborative environment. Their pipeline consists of Adobe Premiere and After Effects, Nuke, 3ds Max and V-Ray.

Upgrading the Workflow Infrastructure

A short time after VFX Creative Director Ludovick Michaud joined Corgan about five years ago, MediaLab began experiencing a slow-down across their workflows and identified the storage infrastructure as the main source of problems. The team needs to work within non-linear workflows, but versioning was non-existent at that time, and the load time for small videos and playback would often lock up the network.

The team worked with OpenDrives as consultants as well as a supplier to support their production workflows, which are varied and intensive in terms of compute power, handling work including 3D rendering, VFX and film services. The overall goal was to replace a tape-based workflow and to help facilitate reuse of footage. The OpenDrives production system they chose, Summit, is augmented with the company's backup and archiving system Ridgeview.

Summit is a hybrid NVMe and HDD storage system that gives immediate access to large files and projects. As a single storage system, dividing large projects among multiple volumes or moving files between systems in order to make room when working on files is not necessary. It handles multiple concurrent workflows including video editing, VFX and colour grading in all of the file formats, resolutions and codecs that Corgan works with.

Cache Management

In particular, its cache management and NVMe acceleration are designed to optimise high-throughput workflows. “It means the files we access most often are always there,” Ludovick said. “That allows fast feedback for the team especially when we’re all working on the same project. After a file has not been used for a pre-determined period, the cache clears that file and automatically syncs back with the 'real' file on the hard drive, making sure we don’t lose data.

“Because most of our files are pretty big – ranging from 30 or 40MB up to 2 to 4GB – this functionality is very useful for us. Files of that size can take longer the first time they are accessed but once loaded into the cache, they become much faster to access, to the point where all we need to do is manage the network, not interfere with the server.”

Ridgeview is a high-density all HDD storage array for nearline and active archive applications with 5 GB/s aggregate throughput. It scales to multiple petabyte capacity where necessary, so that it can form the basis of a disaster recovery plan, and natively uses NFS and SMB NAS protocols to connect to the file server. Its data resiliency and reliability suits long-term storage scenarios but it has the speed and performance needed to maintain accessibility for users, making it a viable alternative to LTO libraries, even for raw camera ingest.

Building a Real-time Workflow

Using Summit and Ridgeview, Corgan MediaLab has been able to build a real-time workflow for their shared, mixed media environment that gives the team members enough interactivity for collaboration across multiple applications and workstations. It also integrates with all of their associated 3rd party equipment.

“It has three aspects – a production system giving each person direct access through Nuke and 3dsMax, an automated archiving system and a footage library,” said Ludovick. “We spent the first two to three months figuring out the system while OpenDrives' support team  worked consistently alongside our team to make it work as we needed it to. Since then, we have only had to make a regular yearly call for updates.”

One of the biggest changes OpenDrive has made at Corgan is the level of control over how they use their files and data. “For example, technically, versioning is a workflow issue more than anything. However the system we had access to before OpenDrives offered no space to allow for versioning and consequently the team didn't see a real need for it. In contrast, Opendrives has allowed us to implement a versioning workflow by giving us enough space and making scaling up much easier. Also the flexibility to add drives as we go, without stopping our work, was very attractive to us and is done using a form of ZFS protocol.”

ZFS uses storage pools to manage physical storage without relying on volumes and their associated problems of partitions, provisioning or isolated storage. Thousands of file systems are able to draw from a common storage pool, using only as much space as is actually needed at one time.

Working Over the Network

The re-use of footage has been supported in a similar way. Their current Opendrives system is big enough and can scale up to five times what they currently have before they will need a new system. Ludovick said, “The Omni cache system for Premiere Pro works especially well. When all editors, usually three or four, are using the same cache, one editor creates the cache while the others have free, fast access, even for 4K footage. We don't need to load any of the footage locally. Everything is edited by the editors directly from their drive, over the network.

“The fact that we are able to create our own footage library with access to as much space as necessary when we need it makes us very flexible. We don’t have to buy all of the capacity now, just when we are ready."

New Workflows

Over the last few years, the large scale of Corrigan's projects is making virtual and augmented reality a more and more important part of communicating information and details about projects to clients. The huge amounts of data involved in this kind of work have been a source of concern for Ludovick, but the OpenDrives system has made it possible to keep projects progressing smoothly without constantly checking that enough space is available to carry the datasets, or that the drive will be fast enough for the team to keep working at a consistent, comfortable speed.

He also doesn't have to limit the number of people working on the server before access speed becomes an issue. “In the past we often experienced uncertainty about whether data had been lost or was syncing properly with the cache. We also had to remember to create snapshots of datasets manually, which took varying amounts of time to complete,” he said.

Automation

“Now we are confident of having space available for a project, and if not, I can obtain and install new drives, run a quick program and have the space ready in a very short time without holding up the team. When doing 4K dailies over the network, for example, we can move 2GB movie files without noticing speed issues. Typically, we have numbers of people working at regular intervals, simultaneously with the render farm, and no one has complained or lost work or data since we moved to the OpenDrives system.

“We also have an automated snapshot system that works overnight while everyone is out of the office. As a result I'm not sure how long it takes, except that the job is always done in the morning, and we have set up a smart system that only keeps a certain number of snapshots as we go so that the older ones don't take up too much space.”   opendrives.com