Notre Dame Customises Hybrid Cloud Storage from Spectra Logic

The University of Notre Dame has a very large data collection that needs to be managed over time and capacity. The data is looked after by the university’s Office of Information Technologies (OIT), and because the office must serve a diverse group of clients on campus, finding enterprise storage that would work for all of them means meeting certain specific requirements.

First, Notre Dame’s Centre for Research Computing needed to protect their research data from the organisation's hundreds of servers. Also, Notre Dame Studios, a multimedia systems and production resource for people on campus, needed a way to manage their video content, audio podcasts and the recording of live events and classroom lectures.  

Other clients at the Hesburgh Libraries wanted to set up long term archival storage for two critical sections of their holdings. The University Archives needed large-scale storage for digitised material, such as print photographs and recordings, and official University records, such as educational transcripts, publications and annual catalogues. The Libraries also needed a storage system for CurateND, their institutional repository that preserves the University’s research data and original academic material.

Apart from client requests, the OIT also wanted to explore the potential of incorporating a cloud tier into their storage architecture and determine the best way to use the cloud as part of their workflow.

Notre Dame Studios

Notre Dame Studios’ storage requirements are somewhat more complex than Hesburgh Libraries or the Centre for Research Computing. The facilities operate from the Fighting Irish Media (FIM) Centre and the Rex and Alice A Martin Media Centre, and handle multimedia production services, live streaming, event AV support and video asset management to a number of university groups. Among the groups is Fighting Irish Media itself, which focusses on the university’s sports teams’ events.  

FIM alone annually generates a growing volume of 150TB of athletic video footage and media assets. Notre Dame Studios must manage two main levels of assets – raw media streams referred to as library originals, and working copies in mezzanine and proxy formats. To achieve a balance of access to different parts of the system, these files must follow different storage policies – moving between repositories as content is worked on, shared and protected.

Notre Dame Studios content is managed between primary storage, Spectra Logic Black Pearl converged NAS/object storage and the AWS public cloud; Hesburgh Libraries data moves between Fujifilm Dternity tape media and the Fujifilm cloud; and the university's HPC research data uses Teradactyl backup system infrastructure components. All three repositories are finally archived in the Spectra T950 Tape Library.

Therefore, Notre Dame Studios needed a system that would help them to organise quick access to content for end users, and meanwhile automatically control primary disk costs, and back up and store assets long-term in a reliable, affordable manner.

Starting with Archive

To first address the need for archive, Notre Dame had consolidated its storage environment in 2014 by deploying a Spectra T950 Tape Library to back up and archive data from several university departments. The T950’s design allows OIT to divide the library into different partitions for research data and media assets. Furthermore, the library supports nearly every software package written for open systems tape, serving as a network backbone that interconnects the partitions and allows the exchange of information between them to meet their numerous user requirements.

Storage Convergence

More recently, the university deployed a Spectra BlackPearl Converged Storage System, an object-based platform, accessed through CatDV media asset management software, to upgrade their system and address their on-going media management challenges. Earlier on, managing assets at Notre Dame Studios’ had involved manually migrating files from spinning disk to backup and archive tiers, which took too much time, was prone to error and it did not allow users to restore content to online storage as and when they needed it. 

However, by using BlackPearl, the studios can now automate the number of copies made on each storage target and specify how long the data will reside there.

According to the studio’s custom storage policies, the library originals mentioned above are automatically moved off of the more expensive primary disk after a predetermined period of time, with BlackPearl sending one copy to tape and another to the cloud for disaster recovery, and leaving behind a mezzanine version as the live working copy. Once the mezzanine files are moved to tape for long-term storage, a proxy version is left behind for producers and editors who may need to search, review and recall assets.

BlackPearl and CatDV MAM work together, automating workflows and maximizing the overall efficiency of varying operations. From day to day, BlackPearl also enables the studio’s users to search and restore files on demand, regardless of location, from their CatDV interface.

Hybrid Cloud

Notre Dame decided to implement a hybrid cloud storage strategy, managing both on-premises and cloud storage via Spectra’s BlackPearl. They use cloud storage for disaster recovery only, moving to the cloud just the data that does not need to be accessed in a timely manner. Any data moved to the cloud through BlackPearl will also be stored on a local tape to make local restoration faster.

The BlackPearl cloud platform puts OIT in a position to serve a wider range of campus customers, and they intend to scale the solution and replicate their media management approach for other university users.

Ready to Grow

BlackPearl helps make storage less expensive and simpler by combining NAS interfaces and object storage RESTful API interfaces with a number of storage targets that work for various workflows. Data can move directly between online/nearline/object storage disk, tape and private and public cloud storage so users can optimise their tiers of storage for cost efficiency and ease of use.

Designed for growth, Spectra Logic’s Black Pearl and tape libraries have been by accommodating the storage requirements of organisations at each stage of their lifecycle. They are not supplied through licensing models, and when data growth requires more slots than the library’s current capacity, Spectra transfers the components of the existing library to a larger chassis.  spectralogic.com