Fraunhofer Digital Cinema Develops IMF, Light-field & MPEG-H Audio

The Fraunhofer IIS at theFraunhofer Digital Cinema Alliancehas developed a new light-field software plug-in forAvid Media Composerthat uses intelligent algorithms and multi-camera systems to capture multiple simultaneous views supporting new options in post-production, in particular, avoiding the time and expense of re-shoots. The new software plug-in allows integration of this functionality into existing post-production software, where data processing and creative work with light-field data can be done through an intuitive UI.

Because various mishaps can occur on set to prevent capturing the intended shot, Fraunhofer has been working on a system of filming using multi-camera systems, which they calllight-fieldtechnology, that captures several different views in a single recording, which are then used in post-production to recover creative opportunities.

Until recently, the processing and testing of this way of working with light-field, or multi-camera, data was only possible with complex proprietary programs which are difficult for most users to handle. Fraunhofer IIS has introduced a software plug-in for Avid Media Composer that allows working with this type of data and selecting and adjusting the parameters for different effects over a GUI.
 
Its auto-calibration technique can correct geometric distortions in the camera position without test charts, calibration patterns or special markers. Depth maps of high pixel density are calculated based on the scene. These depth maps can be used to generate new high-quality views for 2D, 3D or multi-view displays.

Through intelligent processing of different views, change in sharpness, change of the perspective, 3D effects or camera movement in all spatial directions can be calculated. This allows as many extra views as necessary to be generated from the existing views. Changes or modifications of the parameters are operated by algorithms in such a way that the user sees a preview of a scene with the adjusted effect at any time. The adjustment of the effects or camera movement is carried out over a menu with timeline and slider functionality.

The advantage of the integration into Avid Media Composer as a first step is that many users will not have to switch between different software programs. Further plug-in integrations are planned. The processing software is currently in the state of first implementations and testing, but has potential for the design of new, cost-efficient working processes in recording and production.

easyDCP 3.0 now Supports IMF

easyDCP 3.0 from the Fraunhofer Digital Cinema Alliance now includes support for theInteroperable Master Format, IMF, and improved processes for creating and checking subtitles and drag and drop functionality.

The easyDCP Software Suite for post-production handles creation, playback and encryption of digital cinema packages. It is used as a stand-alone software and also been incorporated into products fromBlackmagic Design, QuantelandDrasticas a means of supplying DCP generation via a convenient plug-in. An API from the Fraunhofer development team allows direct integration of DCPs into existing working environments. The IMF is intended particularly for data exchange between various post-production companies and for optimization of internal exchange.

Once the new IMF format has passed SMPTE standardization, it will also be added to the scope of easyDCP´s functions as an I/O format within easyDCP 3.0. IMF is a file-based system for storing high-quality image and audio data, plus subtitles, in a uniform format – the Interoperable Master Package, or IMP. IMP is suitable for exchanging film material independently of manufacturers and devices and is intended to save time and money purely by by using file-based work steps only. IMP is used as a master/source package for generating a wide range of different distribution formats.

Real-Time MPEG-H Audio Encoder System for Broadcast

The Audio and Multimedia Division of Fraunhofer IIS has been progressing interactive 3D sound transmission for broadcasters. This new TV audio system is based on theMPEG-H audiostandard, and will allow TV broadcasters to offer live broadcasts with object-based 3D audio across all devices. Viewers will have the ability to tailor the audio to suit personal listening preferences.

According to Fraunhofer, the system will encode elements of the audio as interactive objects so viewers at home may adjust the sound to their preference. This new hardware will give broadcasters the ability to encode true 3D sound, improving surround sound broadcasts to create a more realistic audio experience.

With the MPEG-H standard now completed, Fraunhofer is preparing for field tests and eventual deployment of the system. Based on the AAC codec family and other developments, this system aims to allow broadcasters to upgrade their existing system when adopting MPEG-H without having to go through major hardware changes.

“Our work with the new MPEG-H TV audio system so far has been done by capturing audio from a live event, and then encoding it with software on a computer. We have now produced a prototype of a real-time encoder for interactive and immersive TV audio, integrating MPEG-H into a broadcaster’s plant for live trials and tests,” saidRobert Bleidt, General Manager, Audio and Multimedia Division, Fraunhofer USA Digital Media Technologies. “The system will encode elements of the audio as interactive objects so viewers at home may adjust the sound to their preference. The hardware will give broadcasters the ability to encode true 3D sound.”

Their real-time hardware prototype, which was demonstrated at IBC 2014 withQualcommmobile chipsets and software supplier, andTechnicolorproduction services., is able to encode audio for live broadcasts from stereo up to 3D sound in 7.1+4 H format with further tracks for interactive objects including commentary in several languages, ambient sound or sound effects.

Fraunhofer’s system consists of the real-time encoder for contribution from outside broadcasts to the studio where a professional decoder recovers the uncompressed audio for further editing and mixing, and a real-time encoder for emission to consumers - over the Internet for new media use or for trials of upcoming over-the-air broadcast systems such as ATSC 3.0. It also includes a professional decoder used to monitor the emission encoder's output. www.iis.fraunhofer.de