Elemental Builds Software Infrastructure for a HDR Future

Elemental’s senior director of solutions marketingMike Callahantalks here about the potential and challenges of software approaches to HDR video delivery. He notes that while techniques and standards for encoding, transmitting and decoding HDR metadata are still in development, these factors are what will determine how TV providers, content owners and other participants in the industry will take advantage of HDR video. The company feels that in the meantime video producers, users and consumers can best support HDR through software development.

Standards Still on the Horizon

In particular, companies involved in encoding content are likelier to be able to control costs and future-proof themselves with a software-based HDR infrastructure. New colour and luminance standards can be adopted and adapted more quickly - without forcing expensive decisions or the need to run isolated hardware HDR systems alongside existing SDR equipment.

 “From a technical perspective, one of the biggest challenges associated with HDR is that the consumer device market is way ahead of the companies generating and delivering content,” said Mike Callahan. “For example, while you can walk into a mainstream retailer or shop online and see numerous TV sets promoting HDR, the reality is that there are no standards for bringing HDR content to TVs nor are there any specifications that TV sets have to meet before claiming HDR readiness.

“By the end of 2016 the standards situation is expected to be much clearer and the Consumer Electronics Association should have some marketing rules in place. It should be noted, however, that multiple pay TV services will launch HDR before the standardization processes are completed.”

As part of their interest in supporting HDR delivery in the short term, Elemental is seeking practical approaches to makelive HDR videopossible, particularly because live sports will be a key demonstrator and driver of HDR. An example might be a soccer match played on a pitch in which one area is brightly lit and others are in shadow. HDR makes it easier to capture all areas of the pitch effectively, while the use of SDR first requires a decision about which areas to focus on to ensure a satisfactory viewing experience.

Mike said, “Similar to the transition from SD to HD, and HD to 4K, HDR demands the reconsideration of all aspects of live production. Camera operators, editors, production and post professionals will have to adapt their skills to a slightly different workflow and learn how to use new tools.”  

Encoding for Modern Monitors

In 2012, ITURec 2020was published to define and improve aspects of UHDTV such as display resolution, frame rate, chroma subsampling, bit depth – and colour space. [Diagram above.] Increasing image bit-depth adds shades and gradations to images, creating smoother transitions within colour families and in effect more closely resembling the colour range of human vision.

High dynamic range aims to put into effect the expanded Rec 2020 colour space. As a standard, HDR describes and protects the content creator’s intentions using metadata to instruct the decoder. HDR sends metadata about how content was created to a display device, organized in a way that the device can use to take advantage of its own capabilities.

For theCRT displaysandLCDsused earlier on,gamma curveswere used to carry display instructions from an artist, through the encoder/decoder. The gamma curve, however, is not relevant to modernLEDsandOLEDsmanufactured with regional or uniformly dispersed backlighting. As display manufacturing improves, including HDR data in video will continue to allow existing devices to always make a best effort in rendering images instead of encountering limits.

For example, the formula called theelectro-optical transfer function, EOTF - sometimes called perceptual quality or PQ - was introduced to replace the CRT gamma curve as a more precise, specific way of describing luminance mapping according to directions from a content creator. EOTF is, in fact, a part of theHEVC standard, and is a useful way to make sure video quality doesn’t degrade during the transformations and use of other standards involved in a typical distribution workflow.

Backwards Compatible

Meanwhile, content creators and owners want to show their images under the best possible conditions, and manufacturers need content to encourage consumers to buy their products. Therefore, Elemental sees the next major step for video delivery as the development of systems - encoders, televisions, set-top boxes and so on - that can reliably use the science behind HDR to enhance the consumer experience and create ameasurable advantagefor providers.

But in practice, the current HDR-capable devices still vary a lot. Elemental anticipates considerable development time before traditional and OTT delivery will be standardised for HDR workflows, streaming or devices. The company suggests that at this stage the key considerations concernbackwards compatibility– specifically, the ability to use existing encoding and decoding hardware, and the transmission of backwards compatible metadata for HDR and SDR.

In a backwards-compatible approach, when an SDR television receives a video signal, the set-top boxes or display are able to ignore any HDR metadata that may be present. Otherwise, if supporting HDR meant replacing encoders, decoders and cable set-top boxes and maintaining two sets of content, SDR and HDR, then non backwards-compatible approaches could be expensive for some providers, creators and manufacturers.

Mike Callahan said, “OTTandBlu-ray2distributors deliver video on demand, not real-time broadcasts, and so may not be as concerned about backwards compatibility. Their issues are associated with maintaining two sets of file deliverables. Furthermore, OTT vendors require customers to purchase and maintain their own equipment, which they only need to support for two to three years. Blu-ray2 players won’t be launched before 2016 and since they are all new devices and systems, at least at the outset there won’t be concerns about backwards equipment compatibility.

“In contrast,pay TV operatorstypically provide set top boxes and installation services to customers, and run dedicated networks delivering content continuously - even when no one is watching. Doubling the number of channels in order to support both an SDR and HDR version directly affects their network infrastructure, especially since providers may be supporting the devices they rent for five to 10 years.”

HDR Metadata – One or Two Layers

CarryingHDR metadatathrough workflows is another structural choice, in this case between a dual and single layer approach. In adual layerapproach, SDR and HDR video streams are carried separately through the workflow, giving high quality viewing without constraining data size. In asingle layerapproach, the SDR video is integrated into a single layer with the extra metadata it needs to produce HDR images, plus side-car metadata.

A drawback of the dual layer approach is that legacy systems expect a single video stream, and running two streams makes critical support operations like emergency messaging, ad splicing and insertion, and on-screen displays more difficult. A single layer approach is better suited to work inside an existing video workflow. The companies and industry bodies putting forward recommendations regarding these approaches includeTechnicolor, Dolby, Philips, BBC/NHK, SMPTE, MPEGand many others. Elemental sees a divided future.

Mike said, “I can’t speak with certainty but, as a shrewd guess, I believe that physical media manufacturers will use dual-layer HDR as a way to promote the highest quality, and that backwards compatibility issues will never be completely overcome for broadcasters. Conversely, while some OTT and pay TV services might provide VOD systems based on dual-layer, I believe the single-layer approach will be the chosen route for all live broadcasts and the majority of OTT systems.

Quality, Cost – and Other Trade-offs

“Furthermore, most proposed systems can provide a no-compromises solution, or one with some compromises, depending on the requirements of their customers - that is, content creators or distributors. For example, a customer could include or leave out wide colour gamut, provide smaller or larger amounts of metadata or use more or fewer bits of data to represent the HDR data. With single or dual layers, they can achieve parity in thequalityof the solution, and control thecostthrough the size of the metadata.

“Generally,dual layersare considered to have better HDR capabilities - but also significant downsides in terms of implementation. Most providers present a dual-layer solution for demonstrations but are more likely to go to production with asingle-layersolution that makes some compromises requested by the delivery network.

“When it comes to implementation, the content distributors may be forced to make certain trade-offs affecting quality. For example, basic HDR solutions will get by with less than 1% additional bitrate required for metadata, and the most advanced ones will use up to 30% of additional bitrate for HDR metadata. For the most part, however, the end consumer will have a better picture and never notice the difference between different services because it is simply not easy to compare them.”

The lack of a guarantee that a single approach to HDR will be agreed upon is especially relevant for content owners and creators. If a content owner was asked to deliver HDR content in two different formats for two different distributors, the content creator could encounter issues later on when trying to market its library for reuse and rebroadcast.

Elemental maintains that in an environment where demands change and evolve regularly, keeping as many components insoftwareas possible may reduce capital and operational expenses. Software can be developed forflexibilityandscalability, and to take advantage of increases inprocessing powerand growth of thecloud, to extend the types of video processing and delivery tasks that can be performed without major hardware investment. www.elementaltechnologies.com