3D Audio is the Hottest Opportunity in VR, Here’s Why

2017.05.02ㆍ by Gaudio Lab

3D Audio is the Hottest Opportunity in VR, Here’s Why


Increased Consumer Interest and A Higher Standard for Immersion

When people watch content via a traditional medium like a TV or a theater, the screen in front of them represents a virtual world. There’s a separation between the real and virtual world, which makes it difficult for some viewers to remain engaged with the content. Through VR, however, people can isolate themselves completely from the real world, and when that happens, the entire adventure becomes incredibly visceral. “Being there” is the whole allure of VR, so sounds must be 3D to support the immersive nature of the experience. Sound spatialization consists of placing positional data to sound sources and gives the listener the impression of a sound source within a 3D environment. When the team at Gaudio Lab, a leading spatial audio company, showcases two different demos — one with spatialized audio and one without — they notice a common pattern of reactions. After taking off the headset, people profess “The spatialized one felt so real” or “I didn’t know that audio made such a big difference.” The hunger for spatial audio becomes insatiable once consumers experience it at least once. There’s no going back.


Powerful New Tools and Revenue Streams for Content Creators

One exciting opportunity for 3D audio centers around live concerts and sporting events. With VR, content creators can overcome physical limitations to serve the audience by reproducing and redistributing live events. But the true seduction of VR is the power to create experiences that could never exist in the real world. Why sit in the nosebleeds of a concert when you can be on-stage in VR, inches away from your favorite artist? As more fans engage with performances that way, it’s not crazy to think that 3D audio could lead to entirely new musical styles, potentially disrupting the music industry. When music is delivered in stereo formats, the sonic experience is shackled by the finite quantity of loudspeaker placement. 3D audio can be delivered via an unlimited number of virtual speakers if the music sources are in an object-based format, or structured as individual mono signals. This allows content creators to experiment with new methods of sound design and produce projects that are totally different from the typical music consumption experience. Music is about to leap past traditional techniques, and performances can now take place all around you, not just in front of you.


Binaural Rendering Requires Talent and Technology

The combination of consumer interest and creator need has given birth to a thriving 3D audio market with high demand, but there are plenty of challenges when it comes to delivering the technology on a practical level. A personal and mobile VR experience inherently requires a pair of headphones for audio output, and delivering 3D audio through that two-channel format instead of layers of loudspeakers is no easy task. That’s why binaural rendering plays such a critical role in the VR listening experience. This signal processing technology synthesizes 3D sound scenes comprised of object, channel and/or Ambisonics into two-channel outputs.The corresponding signals should then be rendered in a way that reflects where the sound is actually coming from, taking into account the relative direction and distance between the sound source and the listener.

Gaudio Lab in the media
Gaudio Intros Free “Works” 3D Sound Plugin For VR And 360° Video

Gaudio Intros Free “Works” 3D Sound Plugin For VR And 360° Video   Gaudio has released Works, a freeware 3D sound utility (AAX plugin format for Mac OS only) for processing audio material that will be used in VR and 360° video.   I never would have guessed that there would be a free imaging software for VR and 360° video but nevertheless, the G’Audio Works proves me wrong. This AAX plug-in for ProTools is designed to be added at the end of each track (mono, stereo, or 5.1) and then the master plug-in should be added to end of the mastering chain in ProTools. The user should then assign the tracks to objects within the G’Audio Works software so that they can then be manipulated as 3D audio.   Gaudio Works has an edit-as-you-watch feature to ensure proper sync with the video before bouncing. In some cases, this would simply mean that the software produces a stereo audio file that encodes immersive 3D audio that matches the video source. In VR and 360° video situations, the software exports a GAO file allowing the audio objects in your session to be placed in real time as the end user moves their head or mobile device around. The audio will match the video being rendered.   The bouncing process is fairly quick for the amount of processing going on behind-the-scenes. Offline bounce is 30 times faster than real-time.   “Works” operates in tandem with “Sol”, which is the company’s custom player for 3D audio and video. After creating a GAO file, the user can utilize Sol to test their mix and ensure the audio is accurately represented in 3D. Sol can also be integrated into mobile devices and is available in SDK format. There are currently several tutorials and demonstrations to watch on YouTube.

Futuresound: Will Accuracy Overcome Familiarity For VR Audio?

Futuresound: Will Accuracy Overcome Familiarity For VR Audio?   John Dewey, who is known for exploring the philosophy of pragmatism, proposed that experience is based on two principles. The first concept of continuity explains that all experiences affect future experiences, for better or for worse. Experiences, therefore, are not independent of each other nor are they one-time events — each one has a relationship to something that happened in the past or will affect something in the future. For example, the fact that you are finally watching a VR piece is related to the fact that you recently got a membership for a VR content subscription service like Wevr’s Transport VR. Or maybe your enjoyable experience of The Mummy VR Zero Gravity Stunt Experience inspired you to go watch The Mummy movie in theaters.   Within this continuity framework, the VR listening experience has the potential to be very confusing. The way you hear sounds in VR might be totally different from the way you have been listening in the real world or while consuming more traditional content. At a concert or a conference, you see where the sound is originally generated from — the musician or the lecturer. However, unless you’re a VIP every time, you are actually hearing the sound from loudspeaker locations and not from the actual sound sources. This discrepancy also occurs when you watch something at the theater. The screen, filled with things that are responsible for the sound, is placed at the front while the sound is actually projected from speakers in various locations. Fundamentally, however, people are familiar with this listening experience — visuals are in front of them and sounds don’t necessarily match what they’re seeing.   Discrepancy in sound perception   We build new knowledge through experiences and all of our current knowledge is based on the experiences that came before. Even our imagination, a realm that is not physically accessible in the real world, is influenced by previous experiences. The more experiences we have, the more creative we can try to be.   Consuming different styles and concepts of VR content will be the key to building our experience base moving forward. We might learn that positioning ambient sound to the sides is more comfortable than spreading it throughout the entire scene. We might learn that volume manipulation could be effectively used to help people adjust more easily to this relatively strange listening experience. Or the decision might still fall down to familiar but false or strange but true, and even then, only after thousands and thousands of experiences have been accumulated.