HTML5 and Media Source Extensions

There has been considerable effort to standardize audio and video in the browser through HTML5. My colleague, Jamie Sherry, summarized this in a recent blog post. HTML5 has come a long way to standardizing audio and video in the browser; however, HTML5 audio and video tags still have limitations, including: 

• Streaming is not supported
• No Digital Rights Management (DRM) or encryption support
• Difficult to customize controls, especially consistently across browsers
• Codec and container support is different between browsers

The streaming limitation is the most critical to Wowza users, and the online video world. Without streaming, features like live streaming, time-shifting, and adaptive bitrate (ABR) are not supported. Some browsers allow streaming using HLS with the HTML5 media tags, but this is non-standard and won’t work in all browsers. Most people use Flash or Silverlight-based players, but as these technologies are deprecated and browsers move away from plug-in models, an alternative is needed.

The Media Source Extensions (MSE) specification is being drafted to bridge the streaming problem in HTML5. While this won’t allow streaming directly to the media tags, it will provide core technology to build browser-independent player technology. This allows the players, through a JavaScript API, to push audio and video into the media tags. At a high level, MSE provides:

• A JavaScript API to build media streams
• A splicing and buffering model.
• Knowledge of several byte stream types:
    • WebM
    • ISO Base Media File Format
    • MPEG-2 Transport Streams

This will allow player technology to be developed to work across all browsers. It will allow VOD, live and time-shifted (DVR) streams to be played back. And it will facilitate advanced streaming use cases such as adaptive streaming, ad-insertion, and even video editing.

So what’s the state of MSE? 

The specification is a working draft, with the intent to become a W3C recommendation. So it’s still changing. Since implementations of MSE will appear long before the specification gets approved, what really counts is what the browser vendors are doing. Chrome and Internet Explorer 11 have good MSE support. Unfortunately, IE11 is only supporting it on Windows 8.1. The Safari and Firefox developers are actively working on it as well.

YouTube and Netflix have rolled out players based on MSE. If you watch Netflix on IE11/Windows 8.1 or Chrome/ChromeOS, you are viewing using an HTML5/MSE based player. On these same platforms, you can try out the experimental YouTube HTML5/MSE player.

At Wowza, we are actively experimenting with MSE and MSE-based players. We expect to see wider adoption of MSE in the browsers in 2014. Although it’s early days for MSE, I believe it has traction to bring cross-browser streaming to HTML5.