Have you ever turned off the sound from a horror movie? 👀

Or have you ever watched the video of the glorious Throne Room scene of Star Wars without any music? Awkward.

If you can even turn Bert & Ernie into real gangsters just by changing the music, we should agree that there’s enough proof that sound and music contribute to any experience. It’s not a coincidence that brands are integrating sound into their branding strategies more and more often.

These examples are pretty old school though. Technology is creating endless ways and possibilities that go beyond your average YouTube ad, making you interact with your audience whilst even being able to support them emotionally if needed.

So, how does music move along with this? Can it create even more engaging experiences? (Spoiler alert: yes it can)

A fairly unknown umbrella term for this is INTERACTIVE AUDIO, which means:

• music that moves with people
• music that reacts based on people’s choices

⏩ But what’s the current status of interactive audio? ⏪

We talked about it with Job Poels and Roscoe Williamson, respectively Digital Producer at MassiveMusic Amsterdam and Head of Branding at MassiveMusic London.

Before we go into your findings, could you first elaborate what interactive audio is?

When we talk about interactive audio, we talk about audio or music that can be influenced by the listener and moves along with their experience. This is in contrast to linear audio which simply plays from A to B.

Take a website for instance: depending on where the user clicks, we can choose the perfect music for each situation and transition seamlessly whenever the user decides so.

In more complex forms, you could create music that adapts to a complete dance performance, or to the heartbeat or mood of the listeners as they progress through their day.

If I say ‘interactive audio’ what’s the first project that pops into your head?

We’d love to give credit to artists who use interactive audio to promote their music. A great example is Helios, who made his album completely interactive. You can literally discover his album by getting lost (and clicking around) in a fictional world.

Another great installation that was set up a few years ago was called Laser Forest. You could walk through the forest and make music whilst touching the lasers, creating a surreal sensory experience.

What did you find out during the research on interactive audio?

How challenging it is to create it and how, often, it can be done better.

Back in 2016, we initially took a look at audio on websites. In most of the cases, the audio was either annoying, not working, or maybe even worse, redundant – as it didn’t really add anything to the experience. If you’re asking people to put on headphones and pause their favourite Spotify playlist for your website, the sound better be good. A track that plays quietly in the background and one ‘bleep-bloop’ sound when you click an ‘OK’ button is going to leave a bitter aftertaste.

This is not because people are lazy but because, both from a technical and a musical perspective, it’s really complex to get audio properly synced to the visitors’ hectic, non-rhythmic clicking.

To try it out ourselves, we created an experiment called ‘mmorph’, a website where we give visitors full control of a track. By moving their phone, they can add drums, delays, switch to the next part of a track or play melodies that are always in key and on time, regardless of the musical experience of the user.

As a result, we created 50+ variations of a track and switched the instruments and pacing of the music.

Guess it also helps to foster collaborations?

Absolutely. When it comes to collaborating with our network of composers, we’ve always had a flexible back and forth process with hundreds of revisions to iterate. Our composers eat, breathe and sleep music. However, they’re not always experienced in working together with coders, and vice versa.

This is why we set up a bridge between these two different worlds. It allows us to create the craziest, most technically innovating and challenging projects, whilst still delivering the musical quality people are used to. A creative ‘breeding place’ for interactive projects.

People might think: Ok but, why would we need interactive audio? Isn’t it something for audio nerds?

We’ve been spreading the message about the emotional value music can bring and how this can be used to create engagement amongst people. Sound logos have been effective for years, people fall in love with music through commercials, and brands are finally acknowledging that musical activations are a great way to connect with their audience.

It all works well, but when an experience gets interactive and audio doesn’t move along, it often falls flat to something that feels either repetitive or non-responsive. Even non-musical people will notice this. It’s like owning a very cool and expensive suit to impress someone, but forgetting to put your pants on.

When done correctly, audio elevates the experience to a whole new level. When done wrong, people will either turn off the sound or walk away.

This doesn’t mean it has to be complex or crazily futuristic. It could be as simple as the ‘piano stairs’ installation that went viral years ago.

Then why isn’t everybody doing it?

Actually, more and more people are doing crazy stuff with interactive audio. I think it used to be a technical limitation for a long time, especially in web environments. Luckily, this limitation is not that much of an impediment anymore, so anyone with a cool concept that involves interactive audio is most likely to be able to develop it.

Whether you’re working as a Brand Manager for a fashion brand, or as a Producer at a digital production company, there are loads of opportunities ready for the taking. Obviously, people with amazing and out-of-the-box ideas are always welcome to blow our minds.

What is possible now that wasn’t before?

We can take this as far as our imagination goes. We could take a complex classical piece and completely map it to a website. Or detect movement of a human body and let the music play along during a dance performance.

We could also take an art installation where all the artworks have their own composition. Walking through the exhibition would be both aesthetically as well as musically appealing.

Some of these are artsy projects, sure, but if correctly implemented in a branding strategy, they have the potential to create an immense impact on the target audience.

Well, we’re not giving out all our ideas for free but, you know, a cup of coffee will get you a long way 😉

What has the future in store?

We think there’s still a lot of unexplored terrain to discover as even the craziest ideas are now becoming more and more realistic.

We’re also interested in seeing how certain developments in interactive music can positively affect mental health and wellbeing.

We already gave it a try with Mindscape, the world’s first ever voice-activated anxiety app, in collaboration with Cult LDN, where we composed 5 tracks aimed at alleviating anxiety, lifting low mood and aiding sleep. We also produced sound design underpinning the breathing exercises and also a short mnemonic for the invocation of the app.

There is a very interesting cross-pollination of music, mind research and interactive technology at the moment which makes it a wildly exciting time to be (a)live.