360° Immersive Video and Extended Reality (XR) in 2021

 

There’s no denying that virtual reality (VR)-powered immersive video hasn’t lived up to its hype. Several factors contributed to this, including the lacking quality and processing power of early VR headsets, the overall costliness for both consumers and content distributors, and bandwidth limitations.

But that doesn’t mean VR is dead in the water. Danny Glover’s suitcase cell phone in Lethal Weapon was a very early, unappealing glimpse into what a mobile phone could be, and the same goes for VR over the past decade.

Innovators are now looking beyond gaming and entertainment to discover much more interesting ways to use this technology. What’s more, VR has ushered in a whole new slew of reality-bending technologies that will dictate where we go next.

Read on for a full rundown of AR/VR/MR — as well as some of the exciting immersive video projects Wowza’s engineers have brought to life.

 

The Four Rs of Computer-Altered Content

In the video streaming world, acronyms and buzzwords abound. Virtual reality (VR), augmented reality (AR), and mixed reality (MR) are all different flavors of futuristic digital media — thus giving rise to extended reality (XR), a catchall term for the bunch. 360° and holographic content also fall within the realm of XR.

 

Extended Reality (XR)

More specifically, XR describes any computer-generated experiences that merge the viewer’s natural environment with a superficial one or create a new environment entirely. XR covers the full spectrum of computer-altered realities, which I continue to break down below.

   

Virtual Reality (VR)

VR describes a wholly digital environment, where the user is immersed in a simulated reality — independent of their physical one — via a VR headset or similar device. VR environments can be created using real-world video content (such as 360° immersive video), computer-generated synthetic content (such as the interactive 3D simulation powered by Google Tilt Brush), or a combination of both.

   

Augmented Reality (AR)

AR fuses the virtual and actual by superimposing digital objects onto the viewer’s real-life environment. Pokémon Go, Google Sky, and Snapchat filters are all examples of AR. While those applications only require a smartphone, other implementations of this technology leverage AR glasses or heads-up displays (HUD).

 

Ready for the next level of displays? The augmented reality head-up display (AR-HUD) of the new #SClass provides loads of #augmentedreality content for #drivingassistance systems and navigation information. #MercedesBenz #MBUX pic.twitter.com/LwGqGJvRjB

— Daimler AG (@Daimler) July 10, 2020

 

Mixed Reality (MR)

MR blends AR with VR by enabling virtual objects placed in the real world to interact with their physical environment. A key aspect of MR is occlusion, wherein computer-generated objects will be blocked from view if a physical object is placed in the forefront. The Microsoft HoloLens is one of the leading mixed reality headsets around today, and Apple is working on a lighter-weight alternative to compete with it in 2022.

 

Immersive Video and 360° Streaming

Also referred to as immersive video, a 360° video stream is a form of VR that gives users the ability to control the content by looking around in different directions while wearing a VR headset. When 360° video is viewed on a computer or smartphone, it’s no longer considered virtual reality.

 

Holograms

Holograms are three-dimensional digital images placed within the physical world. These can be found across the many forms of XR. When used for MR, holograms interact with their physical environment in real time to create a hybrid reality.

   

The XR Landscape in 2021

So how do they all compare in 2021? Put simply, XR and MR are much newer concepts than VR. The same can be said for the technology behind these applications. VR-powered immersive video has had the most time to mature and is therefore finally poised for large-scale adoption.

 

Terminology Trends According to Google Books Ngram Viewer

 

At Wowza, we’ve seen a lot of traction with 360-degree live streaming over the past year. Restrictions brought about by the pandemic obviously played into this, but so too have advancements in the industry.

For one, the quality of VR hardware has finally caught up. VR streams used to be pixelated and laggy, resulting in what’s called the screen-door effect (which is what it sounds like: it looks as though you’re viewing the content through a screen door). Users also complained of motion sickness and discomfort when wearing a headset. And without a compelling user experience or solid enough hardware, there wasn’t much impetus for companies to create great content for VR consumption.

Today, though, high-resolution immersive video content delivers a much better experience at an affordable cost. We’re also starting to see the next wave of 5G-enabled live VR with 8K capability (which Wowza supports). Beyond this, a realization that the actual value of VR lies beyond gaming and entertainment has opened up new opportunities. I predict that cameras and headsets will continue to improve as immersive video finds its way into every industry.

 

Shifting From B2C to B2B: Industries Embracing Immersive Video and Extended Reality

Wowza’s Professional Services team has worked with several organizations looking to combine XR with live streaming to better engage viewers. From healthcare to e-commerce, the opportunities are limitless. Here are some of the places where we’re seeing adoption in 2021.

 

Real Estate and Construction

Say you’re moving across the country and need to purchase a home ASAP. Instead of waiting until you arrive at your destination, why not get started touring houses via a VR headset? A 360° live stream would allow you to walk through the property and ask questions, just like you would at the actual location.

Our team worked with one organization to power this exact capability for commercial properties, and it shows promise for a host of other needs. Inspections, quality control, walk-throughs, and more could leverage 360° VR technology in a similar fashion. AR technologies could also enable computer-aided drawing (CAD) overlays for those on a construction site, thus improving efficiency and collaboration.

   

Live Commerce

Live commerce, a term used to describe the combination of streaming video and e-commerce, promises to revolutionize the retail industry and consumer shopping habits. XR technologies will allow for additional innovation in the space by enabling remote users to try before they buy — without ever leaving their house.

IKEA has already launched an AR app that lets buyers virtually “place” products within their homes. By simply pointing your camera at the empty space, the app will overlay an image of a couch, table, or rug. They’re even giving users the ability to capture 3D room plans and design entire rooms from scratch within the app.

Immersive 360° VR experiences will also help engage remote buyers in the fashion space by allowing them to study an outfit from every angle and experience the excitement of being a spectator at a runway show.

 

Education and Training

Powering education with 360° immersive video makes a lot of sense — which is why our Professional Services team is quite familiar with these workflows. For one thing, it gives everyone a front-row seat, making specialized training that usually takes place in operating rooms or remote locations available to a much larger audience. Participants can control the angle and move around the room, ensuring an individualized experience for each viewer.

   

Healthcare

Our Professional Services team recently created a live VR training solution to provide medical experts with the information needed to make real-time decisions by viewing live content in and around medical facilities. This helps reduce travel costs and the risk of exposure to harmful environments. But low-latency technologies are key.

Some of the capabilities we helped enable include near real-time delivery using WebRTC, recording and archiving streams for VOD playback, and scaling playback using HLS. Security was also a major requirement to satisfy HIPPA compliance, so we worked to configure multiple layers of content protection using encryption and authentication.

AR smart glasses could also find their way into diagnostics and treatment. Google Glass Enterprise Edition 2, for instance, embeds streaming technology into a pair of glasses. Wearers can consult remote experts across the globe — while continuing with their work. The voice-controlled device uses AR to display additional data in the periphery of sight and integrates with existing Google applications.

   

This type of device could enable surgeons to view patients’ vital statistics during procedures or quickly locate tumors and organs before making an incision. AR could even allow a surgeon to make virtual labels about noteworthy anatomical features, which remain in the correct location as tissue is moved and stretched during surgery. And did we mention that the hands-free technology prevents the spread of germs?

XR capabilities also lend well to memory care for seniors, therapy for individuals with autism, and pain management for children.

 

Challenges and Enabling Technologies

So what’s it going to take for the many promises of XR to go mainstream? The right hardware, connectivity, and infrastructure are all key to delivering high-quality experiences at scale.

• Hardware: VR cameras and headsets have improved, but there’s still room to go. As the equipment improves and costs decrease, we should see an uptick in adoption.
• Connectivity: XR requires a decent amount of bandwidth for content creators and end users alike. This also plays into latency and interactivity, which is why the rollout of 5G shows a lot of promise.
• Infrastructure: Device compatibility and how content behaves across devices continues to be an issue. What works on an Oculus in Chrome might not work on a Pimax in Firefox, so you’ll want to consider the streaming protocols, codecs, and player technology of your specific workflow — which is where Wowza enters the picture.

 

XR Streaming and Immersive Video With Wowza

Whether latency is your biggest concern or quality is a must-have, Wowza’s streaming technology can be architected to achieve your goals while ensuring that the content is optimized for end-user devices. Our engineers are also experts on what’s needed for successful 360° immersive video live streaming deployments — making Wowza Professional Services an excellent option for anyone getting started.

Benefits of Wowza’s Immersive Video Solutions:

• Low Latency: Achieve sub-second latency by architecting your workflow with WebRTC.
• Quality: Deliver 8K immersive video with passthrough streaming.
• Adaptive Bitrate: Leverage real-time transcoding to deliver the best viewer experience possible.
• Multi-Device Delivery: Ensure that your stream plays back across a range of VR-enabled devices with protocol flexibility.