Understanding the Evolving Video Codec Landscape

Video compression is critical to delivering content to audiences and, as demand for high-resolution video content increases, the broadcast industry has been pressured into innovation with new compression techniques. Broadcasters and pay-TV operators have begun to explore new systems that take compression further, with better results. According to Chief Strategy Officer at ATEME, Remi Beaudouin, “Codecs are now being developed at an accelerated pace with new ones set to launch later this year. So, to better understand the future of codecs, we must first look at the current codec landscape.”

There are currently two mainstream codecs, MPEG-2, the historic codec used for SD and the first digital deployment, and H.264, otherwise known as Advanced Video Coding (AVC) which was established for the transition to HD. These standards-based codecs were both primarily formulated for the broadcast market and have proven to be the two most successful so far. “Initially, International Telecommunication Union (ITU) standards like AVC were designed with TV in mind and have since been extended to OTT. Although neither codec lends itself to use for streaming, they could be the answer for converged services,” Remi said.

Today’s New Codecs

Remi said, “While the increasing popularity of streaming has highlighted the need for new codecs, it’s important to remember that a huge number of people are still consuming HD. As future codecs are developed, new systems could therefore be used to improve these existing codecs. This will mean that those broadcasters still using older codecs will be able enhance their viewing experience for all audiences. High Efficiency Video Coding (HEVC) was originally seen as the ITU successor of AVC, however, its pace has been slowing down owing to limited 4K penetration and royalty issues.”

Like H.264, HEVC still only predicts square video blocks.

Compared to AVC, HEVC delivers high-quality 4K video that is at least 50% smaller than before, but goes against the trend of trading storage for computation – it requires less storage space than H.264, yet requires more CPU power. As a result, the networks already used to deliver HD SDR Rec 709 video will be able to deliver more data into video, such as higher dynamic range, wider colour gamuts and larger resolutions. The infrastructure won’t immediately require more bandwidth and the more powerful CPUs in new smart TVs, tablets and phones can decode more data out of the networks without requiring more bandwidth.

“A different option has emerged in the form of AV1, which is a royalty-free OTT-centric alternative to HEVC, supported by giant technology groups including FAANG (Facebook, Amazon, Apple, Netflix and Google) through the Alliance for Open Media. AV1 has been designed primarily with OTT, and therefore the progressive scan format of delivering video to screens in mind, in which all lines of each frame are drawn in sequence, which makes it the optimized choice for this type of service," Remi said.

Codecs in Development

“With similarities between the two, the use of AV1 or HEVC primarily depends on the broadcaster or pay-TV operator’s needs and existing infrastructure. For example, both provide similar bandwidth efficiency compared to AVC - in the range of 30 to 50% depending on the resolution. The advantage of AV1 lies in its main value proposition of being royalty-free. Currently, AV1 is seen as the ‘codec of choice’ for streaming media distribution and is supported by organisations including Amazon, Google and Netflix.”

A diagram from Bitmovin describes the upcoming codec AV1.

Looking to the future, Remi expects codecs to change even further as the broadcast industry develops standards to handle new resolutions and more sophisticated content types. He said, “Versatile Video Coding (VVC) is currently in development by the Joint Video Experts Team (JVET) and is anticipated to be finalized this year. VVC capabilities are expected to support immersive content, resolutions from 4K to 16K and 360˚ videos.” Considered to result from a continuation of the research devoted to HEVC, VVC can also use Intra prediction to predict a video block – that is, using what has already been decoded in the neighbouring blocks (including rectangular ones) of the same frame.

“Meanwhile, MPEG has made the development of Essential Video Coding (EVC) a priority as a licensing-friendly, standardised video coding method to address business needs, such as video streaming,” Remi said. This codec is structured for now in a baseline profile, including tools with expired patents or royalty free, with a main profile that adds some new tools. To ensure flexibility, the EVC standard will be able to selectively activate or deactivate each tool beyond the baseline profile as a means of removing tools that cannot be licensed under reasonable terms after it is finalised," he said.

A Few Predictions

“Other than the usual commitment of reducing bitrate by 30 to 50% compared to previous standards (in this case HEVC), it’s too soon to determine the exact benefits and advantages of VVC and EVC. Nevertheless, these new codecs are likely to shake up the market over the course of this year and beyond, bringing with them a degree of complexity. As each is due to be set in stone over the course of this year, it may take until next year or even 2022 before it’s possible to ascertain if they will find their market.”

Remi Beaudouin, CSO at ATEME

Remi also made some of his own predictions. “At this stage, it looks unlikely that there will be a clear ‘winner’, due to each codec having different properties, some more suited to streaming and others to traditional broadcast, and every broadcaster or pay-TV operator having different infrastructure and legacy systems. Instead, there will be several dominant players per niche with AV1 potentially becoming the choice, as predicted, for streaming media distribution due to its development still to come, its foundation and its supporters. At the same time, it’s likely that H.264 and HEVC will also be among the most popular codecs in use.”

He feels certain, however, that the continued development of new and upcoming codecs will have a positive impact on all areas of broadcasting as the knowledge gained and systems generated come into use to improve old and new codecs. Therefore, in the future viewers will have an enhanced experience, whether they are watching content in HD or 8K.   www.ateme.com