The Spatial Audio Decode: Part 1

2018.03.06 by Gaudio Lab

The Spatial Audio Decode: Part 1

As immersive media and technologies continue to expand, there’s no doubt that spatial audio has gained popularity from a  cross section of industries. Whether you’re a developer, content producer, or perhaps looking to implement some cutting-edge audio technology into an app or enterprise solution you’re developing, you might have spent a little time trying to figure out what spatial audio is exactly. If you’re new to this world, perhaps after a few google searches on spatial audio, you might find yourself in the trenches of dense research papers and overwhelming terminology. Fortunately, Gaudio Lab is dedicated to boiling down and simplifying this information to help us better understand 360 audio technology and production, as well as how it can enrich the quality of your projects.


Before digging in, let’s address the most important question: “What is spatial audio and what does it actually do?” Although the answer may seem apparent, it’s still something that might yield a different response from every person you speak to. A big reason for this is that spatial audio can be utilized across so many different mediums. Is it intended for virtual reality? Video games? Enhanced music listening? Hollywood movies? The list grows faster than our ability to catch up with spatial audio’s potential. Regardless of its uses, all aspects of spatial audio converge on one basic idea: utilizing technology to recreate our natural auditory perception that can be heard on any listening device. Through some fascinating technology (which we will cover in this blog series), we can trick our senses into believing we are in the middle of any auditory world whether it is real or designed. Although traditional stereo and multichannel audio recordings have aimed to do this over the last half century, spatial audio brings new perceptive dimensions to the party that have not been fully realized until recently.


While “immersing” yourself into this world, keep in mind that there are a few top-level concepts that are useful to know and don’t require a Ph.D. in applied physics to understand. That being said, the core of what makes spatial audio so intriguing is how it’s largely based on our perception.


The Listening Position

The most common question we get about spatial audio from newcomers is “Oh yeah, isn’t that like surround?” And in some ways, it is. Spatial audio is all about the ability to “localize” a sound, where if you had your eyes closed you could imagine certain sounds emanating from a particular direction and distance.


For years, mixing engineers have tried to create this depth of field in traditional stereo and surround recordings through different techniques. At its most basic, this can be achieved by panning, where the amplitude of a particular sound in a mix is either louder or quieter in that specific audio channel. For example, if a guitar in a stereo song was “panned” to the left, it would increase its amplitude to the left channel while decreasing its amplitude to the right, making the guitar appear to be coming from the left speaker. In 5.1 surround, there are even more loudspeakers to distribute a sound to, enabling the listener to localize sounds coming from both the front and rear horizontal axes. Although this gives an approximate location of a sound source with greater detail than stereo, it still doesn’t provide the very important vertical axis of height.


It’s good to think of sound in your listening position as a representation of spatial axes like X, Y and Z. Typically speaking, X represents sounds coming from the front and back, Y represents the left and right while the Z axis represents sounds from above and below. Though it may look like a trip down a blurred memory lane of mostly forgotten math classes, it is very helpful to visualize the relationship between sound and the listener this way to understand how spatial audio works.


Another complication for using surround configurations as an accurate representation of 360 degrees of sound is that it is dependent on so many variables for the end user.


First of all, not everyone has surround in their home, and even for stereo, many people either don’t have the speakers placed correctly or aren’t listening in the coveted “sweet spot” (which is the optimal position to hear sound the way the audio engineer intended during content creation). The accuracy of audio through surround speakers is largely dependent on the position of the speakers in a room, as well as layout of the room. The idea that more elaborate loudspeaker configurations will solve these problems certainly exists, although this isn’t always the most practical approach for the average consumer and still bounds the listener to a particular room.


Applications of Spatial Audio

This leads us to another broad and basic question for spatial audio: “How do we consume it?” The potential applications are virtually limitless, but what has brought spatial audio into the spotlight is its uses in virtual and augmented reality. Whether using a head-mounted display or just viewing 360 videos on your desktop, the most common way to experience spatial audio in VR is through headphones.


Although it seems impossible to be able to hear sounds coming from behind, above or below you out of two tiny speakers wrapped around your ears, the core technologies in spatial audio makes this a reality. To be able to have these immersive experiences over something as ubiquitous as headphones is great news for any consumer, as the vast majority of people already have access to them.


Spatial audio technology is, of course, not limited to headphones. Keeping the issues with loudspeakers stated above in mind, there are very effective applications for spatial audio in sound bars. Compact, affordable and effortless to position, sound bars provide a great way for spatial audio to enter the realm of Hollywood films, music and games.


What makes this such an exciting time for spatial audio is that its uses are growing in such a wide array, even positioning itself beyond the scope of entertainment. Teleconferencing for instance can be improved by having auditory directionality. Before building a new music stage or recording studio, virtualized acoustic spaces can utilize spatial audio to hear the sound of an instrument in a room before its construction. There is even research on the benefits of spatial audio as a navigation aid for pilots. As you can see, the possibilities for spatial audio are as endless as the real world that we are trying to emulate and we at Gaudio Lab are excited to be at the forefront of the industry.


Stay tuned for Part 2 of my series that takes a deeper look into how spatial audio works and the technology that makes it all possible.

CES at a Glance

CES at a Glance We started off the new year in full swing exhibiting at CES, the largest consumer tech show held in Las Vegas. This year’s show brought nearly 200,000 attendees from across the globe to the city’s infamous strip, along with the industry’s biggest innovators, influencers and trendsetters that set the stage for this year’s emerging technology. From corrective eye lenses for dogs to a self-driving suitcase, the number of entertaining and out-of-the-box gears and gadgets were as endless as the traffic lines of Ubers and buses shuttling people to and from the show’s nine locations. However, G’Audio spotted numerous exciting trends in the spatial audio field to keep an eye on this year.   Here’s a list of trends and emerging tech in 2018: Soundbars Soundbars seem to be taking the spatial audio industry by storm for its boasting capability to produce 3D sounds without using multiple speakers, like 5.1, 7.1, or even 9.1 surround sound. German audio company, Sennheiser, showcased its soundbar prototype for home theater 3D audio. The Ambeo 3D Soundbar includes 13 speakers, nine across the front and two angled on the top. The prototype doesn’t have a release date just yet, but is expected to hit the market by the end of the year. Fraunhoufer’s upHear’s spatial audio microphone processing technology is now able to playback on soundbars, enhancing the immersive audio experience (please note this is solely an algorithm and the company does not produce hardware for soundbars). Other audio companies that showed soundbar technology include Qualcomm, Creative Labs and SotonAudio Labs.   Holographic Displays Holograms used as visual displays made multiple appearances this year, making us feel like we had stepped into a futuristic 3018 instead of the present 2018. British startup, Kino-mo, had holographic adds floating above Eureka Park, the show’s startup sanctuary (where we were also exhibiting in the AV section). Other holographic tech companies included Merge VR’s Holo Cube that lets users interact with holograms, and Hologruf, which showcased 3D holographic displays.   Augmented Reality AR has seen a spike in the past few months, where industry bigwigs, such as Microsoft, Amazon, Dell, and even StubHub have invested in the technology’s future promise of popularity among consumers. The G’Audio team checked out some interesting AR booths at Eureka Park that included both hardware and software, such messenger apps.   Virtual Reality Although VR has experienced many ups and downs this past year, CES proved the industry is still advancing, offering better quality content, new technology and more affordable hardware.   Here are some exciting VR tech updates: Tobbii Eye Tracking is pretty self-explanatory. The company was integrated into HTC Vive and can immensely improve reaction times in VR. NextVR, a fellow SoCal company, has improved its livestreaming video solution that’s compatible with most HMDs and is working towards incorporating 6DoF into the new resolution. Contact CI Haptic Gloves simulate the sensation of touching by recreating the way in which human hand muscles move.   What’s New in the World of VR Headsets There were many new HMDs showcased throughout the week. The biggest advancement for HMDs in VR was HTC Vive’s announcement of the HTC Vive Pro and Vive Wireless Adapter is more user-friendly, offering higher-resolution and lighter weight. Kopin Elf Reference Design displayed its super slim reference VR headset that includes 2K OLED. Huawei announced its VR headset, VR 2, which will support an IMAX virtual giant screen experience and will debut in China this January. Pimax released its ultra-wide 8K resolution HMD, as well as iQIYI Intelligence’s QIYU VR II that also supports up to 8K resolution and is able to recognize VR content in various code formats. Additionally, Facebook teamed up with Xiaomi and Qualcomm for the new Oculus Go headset.   Going Beyond Gaming and Cinematic VR VR is a multifaceted medium that goes beyond the entertainment and gaming sectors, and can be used for a multitude of purposes, including education, healthcare and even training. Many of these companies exhibited at CES this year. In particular, G’Audio enjoyed visiting Looxid Lab, which provides sensors to detect brain waves, and eye tracking to analyze the emotions of users, as well as what direction the user is looking in the VR content. Additionally, an analytics service is offered to advertisers or research companies for marketing purposes. We’ve also seen an increase in VR as a tool for training employees, as seen at a UK startup that specializes in creating industrial training programs developed in Unity.   G’Audio at CES We announced some exciting news of our own at CES. We’ve launched our livestreaming audio renderer! Sol Livestreaming audio solution for 360-degree video transcends the limitations of current livestreaming formats. Now, users can fully experience the sound of live entertainment and sports events without physically being present and as if they have the best seat in the house. Specifically designed to serve up Ambisonics audio signals, Sol Livestreaming provides an accurate sense of the entire soundscape. Furthermore, we’ve adapted our GA5 format for livestreaming and squeezed B-format Ambisonics into the popular AAC codec. Utilizing this ubiquitous codec allows Sol Livestreaming and its renderer to be easily adopted across multiple platforms, bringing truly immersive audio to content creators and consumers alike. Overall, CES was an action-packed week full of new and exciting technology, but most importantly, we saw numerous booths in the spatial audio realm, showing the industry is continuing to grow and advance. We can’t wait to see how this year will unfold!