H.266 Codec: Versatile Video Coding (VVC) Explained

 

This article provides a quick overview of the VVC codec: what it is, how it performs, what it costs, and what it’s good for. As you will learn, VVC (also H.266) is one successor to the standard-based codec HEVC. VVC is designed to deliver a 30% to 50% bitrate reduction as compared to HEVC at the same perceptual quality with an encoding complexity of 10x HEVC and a decoding complexity of around 1.5x. The VVC specification was finalized in July 2020.

Since VVC is a standard-based codec (like HEVC) it probably won’t be supported in browsers owned by members of the Alliance for Open Media who act to promote their own AV1 codec. As an example, though HEVC was finalized in 2013, playback still isn’t supported on Google Chrome, Mozilla Firefox, and Microsoft Edge, so it only plays on 18% of available browsers according to Can I Use. It seems unlikely that VVC will fare any better than HEVC in these browsers.

From a decoding perspective, in my tests of the Fraunhofer VVC decoder, detailed below, VVC playback was much more CPU-intensive than HEVC, so even if there was browser support for VVC, few publishers would deploy it for software playback on computers or notebooks. For this reason, VVC will have to wait for hardware support (and browser support) for success on computers, which will take at least two years from the July 2020 finalization date, plus the time necessary for a critical mass of VVC-capable devices to accumulate. Of course, VVC playback on mobile and living room devices will require hardware support with the same delays for hardware development and market growth. 

VVC’s royalty situation is best described as “complicated” and this may affect both encoding performance and product availability. Though hardware developers need to start considering VVC support immediately, most content publishers can likely wait until well into 2022 or much later to start considering this new codec.

At the time of this writing, VVC isn’t supported in any of Wowza’s products or services. 

 

About VVC

According to MPEG documentation, Versatile Video Coding (VVC) is a codec “drafted by a joint collaborative team of ITU-T and ISO/IEC experts known as the Joint Video Experts Team (JVET), which is a partnership of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG).” As with AVC/H.264, and HEVC/H.265, this new codec took a name from each parent, VVC from MPEG, and H.266 from the ITU. As explained in this article, the codec was called Versatile Video Coding because it is “meant to be very versatile and address all of the video needs from low resolution and low bitrates to high resolution and high bitrates, HDR, 360 omnidirectional, and so on.”

Like AVC and HEVC, VVC enjoyed a long development cycle which began in 2015. In this regard, VVC is the classic codec successor to HEVC. However, unlike AVC and HEVC, which were the only codecs released by the two standards bodies for several years before and after, VVC was launched proximate to the release of two other codecs, Essential Video Coding (EVC), which was finalized in May 2020, and Low Complexity Enhanced Video Coding (LCEVC), which was finalized in October 2020.

 

VVC Royalty Status

As explained here, HEVC’s royalty policy has been widely criticized as confusing, excessive, and opaque. To recount, there were three pools, one launched about four years after the specification was finalized. Known royalties exceed $60 million annually compared to $9.75 for H.264. Two of the three pools clearly don’t charge royalties on streamed content, the third is still studying the market and deciding — a full eight years after the specification was initially released. So, publishers encoding with HEVC still don’t know if royalties may apply.

This dysfunctionality clearly hindered HEVC adoption by software publishers and likely provided the primary motivation for the formation of the Alliance for Open Media. It may also have a profound effect on VVC adoption by chip vendors, as well as computer and other device manufacturers.

To explain, when HEVC was finalized in 2013, most chip and device manufacturers immediately began deployments because H.264 was an affordable standard with near-universal acceptance by publishers and players, and they assumed that HEVC would be the same. That assumption is now gone.

Back in 2013, satellite and cable broadcast dominated streaming markets and there were no serious codec competitors. In 2021, streaming is the dog and broadcast the tail, and AV1 is very relevant competition for both VVC and HEVC. So are EVC and LCEVC.

As you would expect, many of the technology contributors to VVC are also in one of the HEVC pools. So how will VVC avoid a similar outcome?

 

Interoperability Signaling and Patent Pool Fostering

The first technique is to provide interoperability signaling that would allow for different VVC encoding profiles that could exclude certain encoding “tools.” One potential result of this would be to exclude a certain patented tool because the owner wants to charge excessive rates. While this could help avoid excessive rates, it also introduces uncertainty as to encoding and decoding performance.

That is, what happens if the most powerful encoding tools are excluded from most commercial profiles because their patent owners wouldn’t agree to commercially reasonable terms? While it feels unlikely that VVC performance will be diluted in this fashion, we won’t know until royalty policies from every contributing company are set. 

The second technique was to use patent pool fostering to encourage IP owners to form a single patent pool and accelerate the release of royalty terms. These efforts led to the formation of two pools, one administered by Access Advance (formerly HEVC Advance), the other MPEG LA. Both administrators also manage HEVC patent pools.

Access Advance, which launched its pool before the fostering efforts began, has already released its terms, which boosts the annual royalty cap from around $40 million for HEVC to over $60 million. Companies that license both HEVC and VVC from Access Advance pay a higher rate for VVC, but HEVC royalties are waived on products that include both. MPEG LA, which didn’t launch until after the fostering efforts, has yet to announce terms and it’s unclear whether a third pool will form.

It’s anyone’s guess where the combined VVC royalty rate will end up. At the same time, AV1, which may not be free but will certainly have a lower cost per unit royalty, is already being shipped on many 2021 model smart TVs and plays efficiently on about 75% of the installed browsers. This allows publishers to spread the encoding costs over many more viewers and maximize bandwidth savings.

 

Performance

As with all standard-based codecs, VVC has a test model, which implements all available encoding tools to deliver maximum compression efficiency. However, the test model is not designed for optimal performance and is typically never used for commercial encoding; it’s just a vehicle for testing maximum codec efficiency. In a presentation available here, version 11 of the VVC test model delivered 41% greater efficiency than HEVC but took 7.4x longer to encode and required 1.6x the CPU for decode.

   

Different commercial implementations of VVC will have different performance characteristics. In December 2020, I tested a prerelease version of Fraunhofer’s VVenC encoder and VVdeC decoder. In these tests, VVC encoding was about 10x slower than the x265 HEVC codec, though, in a recent email, Fraunhofer claimed that encoding times were now much faster. As a point of reference, encoding times for the Alliance for Open Media’s AV1 implementation, aomenc, was about 7x HEVC and about .7x VVC.

You can see the quality comparisons with the Alliance for Open Media’s AV1 implementation, aomenc, x265, and the x264 H.264 codec below. If you read the VVenC line across, you see that VVenC delivered a 39% efficiency gain over x265, which is in line with the test model and impressive, but was only 11% more efficient than AV1.

   

I tested decode on an HP ZBook Studio G3 notebook with an 8-core Intel Xeon E3-1505M CPU running Windows 10 on 32GB of RAM. Fraunhofer’s decoder, VVdeC decoded the 1080p 30 test file at 37 fps, compared to 116 fps for AV1, 138 fps for H.HEVC, and 277 fps for H.264, all using the default FFmpeg decoder for the respective codecs without hardware co-processing. Like the other codecs, VVdeC codecs consumed between 70-100% of CPU during decode, making it too CPU-intensive for software-based playback on a fairly powerful notebook.

 

Looking Forward

Beyond competition from AV1, LCEVC, and EVC, the traditional codec market, which is based upon a standard set of tools that date back to 2003 and before (see this article), is ripe for disruption. As an example, many companies, including NVIDIA, Broadcom, and WaveOne have announced programs involving AI-based codecs. As more and more video is consumed by machines like cars and robots, AI-driven codecs, with different tuning for human and machine viewing, feel inevitable. Whether this happens within the relevant timeframe of VVC’s success or failure is unknown.

The earliest we could expect to see initial VVC player deployments would be in mid-2022, and then it will take time to accumulate a critical mass of VVC-capable players. Once these players appear, content publishers should start analyzing whether to integrate VVC into their encoding plans. As discussed in this article, dropping bandwidth costs from content delivery networks (CDNs) are making it harder and harder to financially justify adopting a new codec, particularly if it’s expensive to encode.

Overall, the individual VVC codec developers have made great strides in delivering the promised bandwidth savings, though final performance won’t be known until the royalty policies are set and we know which tools are in which profiles. Beyond that, given the diverse range of other factors in play, it’s impossible to know at this time whether VVC will ever reach critical mass.