H.265 Codec: High Efficiency Video Coding (HEVC) Explained
This article provides a quick overview of the HEVC codec: what it is, how it performs, what it costs, and what it’s good for. We’ll conclude with a section on what you need to know to effectively deploy the HEVC codec using the Wowza Streaming Engine software.
What Is the HEVC Codec?
H.265/High Efficiency Video Coding (HEVC) is the successor codec to H.264, which, like H.264, was jointly developed by the ISO/IEC Moving Picture Experts Group and ITU-T Video Coding Experts Group (VCEG). Also like H.264, HEVC has two interchangeable names, HEVC from MPEG and H.265 from the ITU. The ITU-T started work on H.265 in 2004, MPEG in 2007, and the standard was approved by both in January 2013.
HEVC deploys a number of technical advancements over H.264 as discussed here. According to the ITU approval document, H.265 “was developed in response to the growing need for higher compression of moving pictures for various applications such as internet streaming, communication, video conferencing, digital storage media, and television broadcasting.”
As a practical matter, HEVC has enjoyed its most successful adoption in the living room, where it’s seen as an enabler for 4K video, particularly 4K delivered with high dynamic range (HDR). HEVC has also achieved significant success in contribution and newsgathering markets. In December 2019, LiveU, which manufactures encoders and transcoders for news production, reported a 65% growth in HEVC unit sales in 2019. As a result, 50% of field units used to stream sports were HEVC-based and LiveU reported a 400% growth in HEVC-based sessions.
HEVC is a royalty-bearing codec. In contrast with H.264, which had one patent pool with reasonable pricing (like an annual cap of ~$9.75 million), HEVC has three patent pools launched as late as four years after HEVC was approved with known royalties exceeding $65 million per year. The second pool, from HEVC Advance (now Access Advance), was launched without a cap and with a content royalty. This is attributed as motivation for Microsoft and Google to form the Alliance for Open Media, which ultimately released the competitive AV1 codec. More on HEVC licensing below.
Compatibility
| Codec Compatibility | Browser | Mobile | Smart TV/OTT |
| H.264 | Virtually all | All | All |
| VP9 | Virtually all | Android, iOS | Most |
| HEVC | Very little | Android, iOS | All |
| AV1 | Edge, Firefox, Chrome, Opera | Android | Nascent |
The website CanIUse lists HEVC browser compatibility for both desktop and mobile at 18.08% as compared to H.264’s 98.23%, VP9’s 96.26%, and AV1’s 74.25%. This comes as no surprise given that market leaders Chrome, Firefox, and Edge are all owned by members of the Alliance for Open Media. Apple is also an Alliance member but started supporting HEVC on iOS, Mac, and Apple TV before joining the Alliance. Given the low penetration in the browser market, it’s no surprise that few publishers are deploying HEVC for browser-based playback.
In the mobile space, Apple added HEVC playback to iOS, MacOS, and tvOS and debuted hardware support on newer Apple devices in June 2017. HEVC support is bifurcated on the Android front, with Google supporting HEVC in software but only up to main profile, level 3 for mobile devices, which means support for playback of up to 960×540@30 fps. To compete with Apple devices, however, many Android mobile devices also have HEVC hardware decoding; in August 2018, Scientiamobile reported that 57% of Android devices were equipped with hardware decoding compared to 78% of iOS devices.
HEVC is almost universally supported on Smart TVs and premium OTT devices and is the preferred format for 4K and high dynamic range videos.
Performance
When gauging codec performance, you consider two aspects: (1) encoding complexity or speed, and (2) encoding quality. With most codecs, including the four discussed in this series, there’s an inverse relationship between the two. That is, the better the encoding quality the longer it takes to encode, which translates to higher encoding and transcoding costs. Since no codec can completely replace H.264 at this point, these encoding or transcoding costs are additive to existing H.264 encoding-related costs.
| Performance Compared to H.264 | Encoding Complexity (Speed) | Encoding Quality |
| H.264 | Baseline | Baseline |
| VP9 | 2–15x | ~35% |
| HEVC | 2–15x | ~35% |
| AV1 | 15–30x | ~50% |
When analyzing codec performance, remember that there are multiple HEVC implementations, each with its own performance characteristics. Most HEVC implementations also have presets that let you trade off encoding time for performance. As an example, with x265, the HEVC encoder included in FFMpeg, the medium preset encodes in about 11% of the time of the very slow preset, and only delivers about 1% lower quality than very slow, which is irrelevant for most producers. Of course, H.264, VP9, and AV1 also have multiple implementations with multiple presets, which complicates codec comparisons and is why the encoding speed column in table 2 shows such an extensive range.
When considering encoding quality, note that the figure represents the bitrate reduction you should expect when comparing a single full-resolution encode from HEVC, VP9, or AV1, as compared to H.264. Obviously, this will vary based upon the codec and preset used.
In addition, any actual estimate of real-world savings from adopting a new codec must consider before and after encoding ladders and the actual distribution of the streams to your viewers. If distribution is top-heavy, as most are in the U.S. and Europe, you should be able to harvest a good bit of the number shown in Table 2. If you distribute mostly in the middle range, bandwidth savings will be much lower, though video quality should be significantly improved.
Suitability: HEVC vs. H.264, VP9, and AV1
Now let’s examine HEVC’s suitability for a range of specific encoding applications. HEVC is very well suited for live origination because the lower bitrate drops the outbound bandwidth requirements and because there are many affordable HEVC hardware-based encoders. There are also many HEVC hardware transcoders leading to the excellent score for that task.
While most low-latency implementations use the H.264 codec, HEVC is compatible with Low-Latency HLS and is an available format for Low Latency HLS using the Wowza Streaming Engine. On the other hand, HEVC is not compatible with WebRTC and is not available in Wowza’s WebRTC implementation.
| Codec Suitability | Live Origination | Live Transcode | Low Latency | 4K | HDR |
| H.264 | Excellent | Excellent | Excellent | Poor | Poor |
| VP9 | Poor | Poor | WebRTC | Excellent | Poor |
| HEVC | Good | Good | Nascent | Excellent | Excellent |
| AV1 | Nascent | Nascent | WebRTC | Excellent | Nascent |
HEVC’s sweet spot is the living room because of encoding efficiency and compatibility and predominant usage in all high dynamic range specifications, including Dolby Vision, HDR 10/10+, and HLG.
Royalty Status: HEVC vs. H.264, VP9, and AV1
HEVC’s royalty status has been wildly and widely criticized, including by MPEG’s founder (and then director) Leonardo Chiariglione, who in a 2018 blog post entitled A crisis, the causes and a solution, reviewed HEVC’s fractured royalty policy and stated “At long last everybody realizes that the old MPEG business model is now broke, all the investments (collectively hundreds of millions USD) made by the industry for the new video codec will go up in smoke and AOM’s royalty-free model will spread to other business segments as well.” About 18 months later, Chiariglione was gone from MPEG, an organization he had led since its founding in 1988.
Here’s the picture for content producers, who likely comprise the vast bulk of readers of this article. There are three patent pools, MPEG LA, Access Advance, and Velos. MPEG LA charges no royalties for content. Access Advance charges a royalty on commercial content distribution on physical digital media, so streaming does not trigger a royalty. Velos Media runs the third pool, and over 8 years after the finalization of HEVC, still states, “As it relates to content, we will take our time to fully understand the dynamics of the ecosystem and ensure that our model best supports the advancement and adoption of HEVC technology.” While it’s unlikely that Velos will come hunting smaller producers for royalties on HEVC-encoded content, if you’re a very large producer you certainly can’t rule it out.
| Codec Royalty Status | Encoder | Decoder | Paid Content | Free Internet Content |
| H.264 | Yes | Yes | Yes | No |
| VP9 | No | Consumer device | No | No |
| HEVC | Yes | Yes | Some | Unclear |
| AV1 | No | Consumer Device | No | No |
What You Need to Know to Produce HEVC
Version 1 of the HEVC codec is the predominant version for streaming delivery, and it has two video profiles, main and main 10, which both use the 4:2:0 chroma sampling. As the name suggests, the latter is ten-bit, which allows for greater color depth, while the former is 8-bit. If you’re creating HDR content with HEVC, main 10 is preferred, though some producers prefer main 10 even for 8-bit content. If you decide to deliver 8-bit content via main 10, be sure that your decoder is compatible with main 10 streams.
Both Wowza Streaming Engine and Wowza Streaming Cloud can ingest live HEVC streams in either profile, though main is generally preferred. The Wowza Streaming Cloud can ingest HEVC and pass the stream through but can’t transcode other rungs to HEVC.
Wowza Streaming Engine can transcode to HEVC ladder rungs, with both the main and main 10 profiles available, though again, main is preferred. Wowza is in the process of switching from the MainConcept to the Beamr HEVC codec and you can read about how to configure your ladder rungs here.
Like most HEVC codecs, the Beamr codec uses presets to define different encoding speed/quality tradeoffs. The supported values are 98 (slowest, highest video quality and CPU usage) to 108 (fastest, lowest video quality and CPU usage). As the Wowza article suggests, “you may have to experiment to find the right value. If you set this parameter to a value lower than your available CPU resources can handle, real-time encoding will lag and eventually stop. When this happens, skip frame messages appear in the Wowza Streaming Engine logs.” In my experience, you should expect HEVC transcoding to be about 4-8 times more CPU intensive than H.264, though this obviously will relate to the presets that you use for each.
Overall, HEVC should be considered for streaming to mobile and living room devices, but not for general browser consumption. In the next article, we’ll discuss the AV1 codec.