FIFA Women’s World Cup Remote Production Leaps Ahead with Aspera
After their successful delivery of the 2018 FIFA World Cup from Moscow, FOX Sports has taken their experience in remote production to the 2019 FIFA Women’s World Cup in France, running from 7 June to 7 July. The goal in Moscow was to produce coverage and highlights for the 64 World Cup matches in Russia, while working from their production facilities in Los Angeles. In 2019, they have once again put systems in place for low-latency, high resolution production – this time between the Paris IBC and LA.
In Moscow, FOX Sports deployed an integrated system consisting of IBM Aspera FASPStream high speed data transfer, Telestream Lightspeed Live and Vantage platform, and the Reach Engine MAM system from Levels Beyond. With Lightspeed Live, a scalable, multi-channel capture system that ingests live media directly into production and broadcast workflows, the system supported high-quality streaming to remote production teams in near real time, over unmanaged IP networks.
Improving on Success
Digital Media World spoke to IBM Aspera’s CTO Mike Flathers, recently returned from Paris some days before the tournament ends, about how his company has handled the project in France. “While our results in Moscow were not so perfect that we couldn’t make some improvements, the delivery was a success overall. All live streams were reliable, and the essential workflow was good. The Women’s event will reveal just how much better we can make it,” he said.
After Russia, confident that they had a system that works, they also knew that they would have more bandwidth from Paris to LA than they had from Moscow. Mike said, “The goal for FOX in Moscow was to keep the delay from camera to the editors’ Adobe Premiere Pro suites down to under 10 seconds. The system actually achieved an average delay of only about 7 seconds.
“The team was using a 1GB line from Moscow, capped at 750Mbps because it had to be shared with the event’s communications traffic. This meant that if the time slots of two matches overlapped in the schedule, a short, automatic delay would be introduced between the two camera stream transmissions until the overlap ended.
Aspera Streaming for Video
Innovative Integration
“For Moscow, the most innovative aspect of the project was the integration between Aspera FASPStream, the Telestream Lightspeed hardware server and the Levels Beyond MAM system. We had formed a very tight integration that included a feedback loop used to monitor and adapt the system to conditions and avoid interruptions to the stream. During the initial trials of the system, Aspera constantly monitored the stream and continuously tweaked the bitrate, speed and other parameters to optimise its performance.”
The integration combines video processing acceleration from the Lightspeed server with the Aspera FASP protocol’s high-speed transfer capabilities, using the FASPStream file-based APIs to initiate, manage and control transfers from one file system to another.
Instead of starting a transfer by reading a file from disk and ending the transfer by writing to disk, the FASPStream API supports transfers to or from a custom process running in memory with another custom process running, in this case, on a transfer server in a stream-to-file scenario.
The API’s use of FASP maintains speed over varying distances, and allows users to control security and bandwidth. Transfers aren’t limited to a file or file set – they can accommodate a stream of bytes as they are being created or captured, and allow access to portions of a file in memory during the transfer process rather than waiting for the entire file to be written to disk.
Auto Archive
Mike quantified and compared the systems and performance in the two locations. In Moscow, FOX ingested 20 HD-SDI feeds and two UHD-HDR feeds per match for a total of 22 Lightspeed streams, which created over 2 petabytes of content over the course of the tournament. Automatic archiving functionality transferred this content via FASP to object storage in the cloud on Amazon S3. The archiving was delayed until the files were complete, but in Paris the upload can begin on growing files. The chosen codec is AVC Intra, which LightSpeed supports and the editors can edit directly on Premiere Pro.
“Because fewer games are being played at the Women's World Cup, we expect that a little less than 2 petabytes will be created and transferred to object cloud storage. But at both events, only 1 petabyte of local storage is available at the IBCs,” Mike said. “Given that we are creating more than that amount, it is important to be able to quickly and reliably archive data remotely in the cloud and continuously free up local storage to make room for new content as it is ingested.”
FOX has deployed more than the three simultaneous live camera streams per match that the Moscow crew delivered to the editors in LA. “For the Women's World Cup, we have already supplied several times as many - through the Round of 16, we have already sent over 350 live edit streams to the editors,” he said.
Low Bit-Rate Clip Engine
Apart from the auto archiving of growing files, Mike described some of the other new innovations in use at the Paric IBC. For example, a clip engine for low bit-rate streams stores the editors’ in-out points for social media or rough cuts. These markers can be used to produce clips in a similar way to an EDL leaving the footage in place in cloud storage and extracting only the clips.
“At the same time we are writing the high-bitrate live-edit streams to LA and archiving those same high-bitrate streams to object storage in the cloud, we are writing lower bit rate HLS proxies to the cloud that can be played back in real time. Wherever they happen to be, the editors can playback these low-res proxies and set in/out points that will be used to generate clips. Those in/out points are then sent to Telestream Vantage at the IBC in Paris, or to the associated Telestream Cloud where the actual clips are generated from the high-res source content.”
Aspera Connection Manager
A recent development is an application with a new UI called Connection Manager, used to direct streams from one location to another while accelerating those streams with FASPStream. As those streams are being directed, thumbnails are generated for users to review as well as video and transfer detail metadata. The directed streams can be dispersed to multiple locations concurrently, as well as written to storage at the remote locations.
An important aspect of this poject is that while some of the post production work was done on site in Moscow, in Paris, ALL of the post is being handled in Los Angeles. Mike said, “It’s really a testament to the confidence of FOX Sports and saves costs substantially. FOX does not need to comply with local labour regulations and their staff prefer working in familiar surroundings as well.” asperasoft.com
Words by: Adriene Hurst, Editor
