Update: What is Low-Latency HLS and How Does It Relate to CMAF

 

Note: Since posting this blog, the Low-Latency HLS specification has been incorporated into the overarching HLS standard. To learn more, check out our article on the update.

 

At last year’s Worldwide Developer Conference (WWDC), Roger Pantos announced the spec for a brand-new extension of the HTTP Live Streaming (HLS) protocol: Low-Latency HLS. The spec promised to achieve sub-two-second latencies at scale — while also offering backwards compatibility to existing clients.

The industry took notice for a number of reasons. As an extension of HLS, wide adoption of Low-Latency HLS was assumed. After all, HLS is the most common protocol in use today (with more that 45% of the broadcasters in our 2019 Video Streaming Latency Report employing it).

But what remained unclear was the speed at which vendors could implement support for Low-Latency HLS. For one thing, Apple’s announcement interrupted an industry-wide effort to reduce latency via chunked transfer encoding for CMAF. It also required the use of HTTP/2 PUSH, which many content delivery networks (CDN) couldn’t yet offer.

And the rumblings didn’t stop there. Some had doubts as to whether or not this extension would work at scale. What’s more, the overwhelming consensus was that Apple had some kinks to iron out before the technology could actually be implemented.

 

Recent Updates to Low-Latency HLS

Low-Latency HLS (or LL-HLS) has already evolved drastically since inception. Apple removed the HTTP/2 PUSH requirement a few weeks back, as communicated via their HLS mailing list. Instead, the extension now uses shorter media chunks and the new tag #EXT-X-PRELOAD-HINT. This change reflects just how fluid the spec still is.

Our friends at THEOplayer explain:

“With this tag, a server publishing a Low-Latency HLS stream is required to announce the most likely location of the next media data which will become available. This allows a player client to perform a request, allowing the data to flow in as soon as the next part of a segment becomes available. This process can then be repeated, allowing the removal of additional round-trip time when loading new media data (which was the main reason to use HTTP/2 push).”

Another huge concern when Apple first outlined Low-Latency HLS was its lack of interoperability with low-latency CMAF for DASH. This update has removed several stumbling blocks by allowing LL-HLS to mimic the chunked-transfer approach. Improved compatibility between Low-Latency HLS and low-latency CMAF for DASH should help curb complexity and improve CDN efficiency.

To further unpack the impact of this development, let’s start at the beginning.

 

CMAF and the Race to Low Latency

It happened like this: Once upon a time, any content distributor wanting to reach users on both Apple and Microsoft devices had to encode and store the same audio and video data twice. That’s because Apple’s HTTP Live Streaming (HLS) protocol specified use of the .ts format, whereas Dynamic Adaptive Streaming of HTTP (DASH) almost uniformly used .mp4 containers.

Think of it like chargers for phones and computers. Androids and iPhones don’t use the same chargers, leading to more cords than necessary in an already cord-ridden world. Because Apple uses proprietary cables for proprietary ports, users are stuck going to the Apple store every time they lose a charger. The HLS protocol was like a proprietary charger for streaming to devices that supported Apple’s specification. MPEG-DASH, on the other hand, worked for delivery to everywhere else.

Miraculously, Microsoft and Apple simplified everything by announcing a new standard called the Common Media Application Format (CMAF). This specification would use fragmented .mp4 containers that could be referenced by both HLS and DASH.

This was great news. Content distributors no longer had to encode and store the same data twice. The single format would save broadcasters money and streamline video delivery.

Beyond just solving complexity, vendors across the industry started working together to reduce the latency of HTTP-based streaming with chunked encoding and chunked transfer encoding. By leveraging this technology, CMAF could be used to enable sub-three-second video delivery.

 

Industry-Wide Efforts to Decrease Latency

At Wowza, we got to work supporting low-latency CMAF. Teams at Akamai, THEOPlayer, Fastly, and more had their noses to the grindstone with the same goal. This was important, because in order to work, vendors across the streaming ecosystem needed to optimize their products accordingly. We invited customers to sign up for early access to low-latency CMAF, and got excited about the prospect of delivering streams to both Apple and Microsoft devices using chunked transfer encoding.

While Apple hadn’t committed to these efforts, the community was gung-ho. Periscope had written an article about an open-source Low-Latency HLS (LHLS) standard that they’d developed with this technology, Akamai’s Will Law had dedicated a ton of his time to chunked-encoded and chunked-transferred CMAF, and our own engineers had sat down with him to discuss how it would impact the streaming industry.

Then, Apple introduced Low-Latency HLS.

 

Low-Latency CMAF vs. Apple Low-Latency HLS

Up until WWDC19, the streaming community had been driving toward low-latency CMAF using chunked transfer encoding. Apple’s announcement clarified that they would be supporting a different low-latency technology for HLS. One of the key differences between their own specification was the use of HTTP/2 PUSH.

To help assist with this process, we pivoted to add support for Low-Latency HLS as defined by Apple. By November of last year, our customers were able to start building their own Low-Latency HLS workflows with the Wowza Streaming Engine 4.7.8 release.

However, speedy video delivery at scale doesn’t happen in a vacuum. Vendors across the entire workflow needed to support this spec for it to work. And, as mentioned above, HTTP/2 PUSH remained a challenge for most CDNs.

 

Removing HTTP/2 PUSH Requirements

Apple’s recent revision to the Low-Latency HLS specification will require the organizations that were spearheading implementation to adapt. That said, the removal of HTTP/2 PUSH should simplify adoption.

We remain committed to supporting large-scale deployments of Low-Latency HLS through integration with CDNs and players. And we’ll continue to develop against the evolving spec.

Until we learn more, here’s a snapshot of Low-Latency HLS as it stands today.

 

Apple Low-Latency HLS: A Snapshot

Apple designed this extension to decrease latency at scale. While the protocol originally relied on HTTP/2 PUSH delivery, this requirement has since been removed. Instead, the extension uses shorter media chunks and the new tag #EXT-X-PRELOAD-HINT.

• Playback Compatibility: Any players that aren’t optimized for Low-Latency HLS can fall back to standard (higher-latency) HLS behavior. HLS-compatible devices include: MacOS, Microsoft, Android, and Linux devices; all Google Chrome browsers; several set-top boxes, smart TVs, and other players.
• Benefits: Low latency at scale and backward compatibility.
• Drawbacks: As an emerging spec, vendors are still implementing support and requirements remain in flux.
• Latency: 2 seconds or less.

Note: Since posting this blog, the Low-Latency HLS specification has been incorporated into the overarching HLS standard. To learn more, check out our article on the update.