Features I by I 30.11.16

Is VR technology about to revolutionize the way we experience music?

With affordable technology from Sony, Google and Samsung making its way into millions of homes, 2016 has been the year of virtual reality. Artists including Björk and D∆WN have been at the forefront of using the technology for videos, but are we on the edge of a VR revolution in musical expression? Chris Kelly investigates.

Compared to the cutting-edge robotic self-love of ‘All Is Full of Love’ and the reality-bending, nesting doll musical of ‘Bachelorette’, Björk’s ‘Stonemilker’ is – at first glance – relatively simple: Björk sings directly to the camera as waves gently lap the beach on which she wrote the song’s lyrics. But Björk’s art is never as simple as it seems, and ‘Stonemilker’ is no standard-issue video: it was shot in 360 degrees and built for virtual reality, creating the illusion of Björk singing directly to the viewer, with 30 string players sitting in a circle around you.

Björk and Andrew Thomas Huang saw a “potential for intimacy” with the technique, and they delivered. But unless you went to MoMA PS1’s VW Dome or were among the super-early adopters who owned a VR headset, the intimacy of ‘Stonemilker’ was probably lost. For most people, the 360 degrees of the video would be experienced by clicking and dragging on YouTube, in 2D. And while this was novel, it wasn’t particularly interesting, as the rash of 360-degree videos that followed – often heavy on gimmick but light on concept – would prove.

“Virtual reality isn’t just “coming soon” to the world of music: it’s already here.”

Since the release of ‘Stonemilker’, however, the virtual reality landscape has shifted drastically. As 2016 winds down, the promise of VR is about to be realized for a much larger audience as several major players enter the market. Samsung’s Gear VR launched in late 2015, followed by the wide release of the Oculus Rift, HTC Vive and – just in the last couple of months – PlayStation VR and Google Daydream View. With electronics, competition often benefits consumers, and a near future where VR headsets move beyond early adopters to a market share more akin to that of gaming systems or tablets seems within reach.

As she had done with music, video and multimedia experiences (e.g. 2011’s Biophilia, the first “app album”), Björk’s entry into VR broke barriers for subsequent artists. Musicians as diverse as The Weeknd, Squarepusher and Run the Jewels have released VR music videos, while Paul McCartney and Beck have used VR to share live concert experiences. Virtual reality isn’t just “coming soon” to the world of music: it’s already here. Still, there are questions about where music industry VR goes next, and what both artists and audiences can expect in the future.

This past summer, Dawn Richard took the plunge into VR with ‘Not Above That’, a music video experience that turned the viewer into the pilot of a spaceship on an intergalactic journey and recreated Richard and her dancers as digital sprites on the ship’s interface. The video was the next step in a process that began a year earlier, when Richard released the ‘Titans x James Dean’ video, which used motion-capture choreography to turn the human form into diamond and black onyx creatures. Not one to repeat herself, Richard wanted ‘Not Above That’ to take her presentation of the body and how it moves to the next level.

“When you close your eyes, ‘Not Above That’ feels visual, it feels like going to another place”Dawn Richard

After attending workshops during New Frontier programming at Sundance, it became clear to Richard that virtual reality was the “next logical step” for her next music video. “When you close your eyes, ‘Not Above That’ feels visual, it feels like going to another place,” she says. “I wanted to make sure I gave [the audience] the same idea, visually.” The song is about the transition from a vulnerable, human state to another, otherworldly dimension, and VR allowed Richard to better translate the song’s sonics and symbolism than a traditional video would. But it wasn’t an easy sell.

For Richard, an independent artist without major label backing, it took a lot of work to find a company that believed in her vision for ‘Not Above That’. “People haven’t figured out how to make [VR] accessible,” she says. “Unless you’re a big brand or company, [they] don’t know how this will work for you.” Eventually, boutique studio VR Playhouse signed on. “They thought what I was doing was groundbreaking,” Richard says, “[especially] as an independent black woman getting into 3D, gaming and VR.”

‘Not Above That’ was the beginning of Richard’s VR journey – not the endpoint. Her new album, Redemption, is being released on a USB necklace that includes VR “prefaces” for each song on the album. She also performed in YouTube’s first-ever 360-degree concert, an opportunity she says she was honored to have, but one that came with its own challenges and setbacks. “I went there with really high expectations and big ideas, and they didn’t know if they could pull it off,” she says.

Richard wanted to take full advantage of VR, creating a starscape that would make the entire site look like another dimension, while YouTube expected her to perform a regular live show, similar to Coachella’s VR experience. This push and pull resulted in something in the middle: Richard and her dancers performed on a circular stage, working to capture all of the choreography without losing dancers to the “seams” that result from the “stitching” of VR images. Emboldened by the YouTube experience, Richard hopes to use VR at future concerts, as a way to perform for audiences that wouldn’t be able to see her otherwise, due to tour logistics, disabilities or other accessibility issues. “VR brings the world a bit closer.”

“Virtual reality is a parallel [to clubs], a new hybrid between gaming and Second Life for people who like to sit and go into a fantasy world.”Ash Koosha

Perhaps no musician has further explored how virtual reality can affect the live music experience than Ash Koosha. The Iranian-born, London-based producer has worked with TheWaveVR to perform live virtual reality shows at both London’s Institute of Contemporary Arts and on Boiler Room. Equipped with HTC’s Vive headset and its wireless controllers, Koosha uses TheWaveVR’s custom app to control sound in a virtual space, manipulating stems, seeing waveforms and triggering effects in a spatial manner. Meanwhile, his audience uses VR headsets to enter that environment, without having to be in the same venue.

While some may see this virtual experience as a threat to physical clubs, Koosha does not. “I’m a firm believer in the physicality of sound, and how it affects body,” noting one of the facets of physical clubs that VR cannot replicate. “Virtual reality is a parallel [to clubs], a new hybrid between gaming and Second Life for people who like to sit and go into a fantasy world.” He compares the distinction between live and virtual experiences to the differences between mobile technologies. “It’s not like, ‘are you into iPads or are you into phones?’ It’s just a different form of experiencing music.”

To that end, Koosha has begun working on “virtual reality albums” that unify sound and visual components, building a new experience that is unlike anything that currently exists in the realms of music and video. Two years ago, he started translating electronic music into sound objects. “I was making music that would make sense in a VR environment,” he explains. “Stems, sounds, movements: each part would translate visually, to different colors, point clouds or particles.” Then, he collaborated with Iranian computer artist Ali Eslami, who designed an environment for Koosha’s objects. The result, Snow VR, was exhibited at Mutek this year; the pair are currently working to bring it to other venues.

As Richard found out with her YouTube experience, Koosha has also bumped up against some of the limits of virtual reality technology. The software and hardware is not yet at the place where Koosha can fully realize his VR dreams; there are limits to resources and processing power when you’re an artist and not a gaming company. And until hardware makers figure out how to put all the device’s computational power in the headset, users will be handcuffed to their computers. (There’s also the issue of headset bulkiness, a concern raised by everyone I interviewed.)

But Koosha also sees virtual reality being limited not just by technology, but by creativity. For him, too many VR experiences – he estimates about 60 to 70 percent – aren’t fully embracing the “virtual” in virtual reality. “What is the purpose of VR? It’s to build reality that exists on its own,” he maintains. “Why replicate reality like cars and ships when we can make a virtual reality that doesn’t exist, and push the boundaries of imagination? We can do way better than this.”

“VR can expand on what the music video and the fanclub or fanzine were in the past.”Doug Buffone

The creative shortcomings of musical VR experiences is a concern shared by Sarah Boardman, the head of music at Pulse Films, a production company and media studio that has produced everything from Beyoncé’s Lemonade to the short-form teasers for Radiohead’s A Moon Shaped Pool. “At this stage, VR is very much still a gimmick,” she says. For Boardman, VR videos often lack the story, concept or narrative thrust that propels a traditional video. “To this point, I haven’t really seen a video that has both the technique and the idea.” Even those with an interesting concept – like Björk’s Jesse Kanda-directed ‘Mouth Mantra’ – fail to sustain viewer interest: how much time do you really want to spend in Björk’s mouth? “You get a little bored of it.”

She also proposes an interesting thought exercise: take your favorite music videos of the last 20 to 30 years, and think about which would be made better by virtual reality. She couldn’t come up with one (I suggested Prodigy’s first-person ‘Smack My Bitch Up’, but struggled to think of any others). That’s not to say that music videos won’t be able to creatively use VR, just that it might take a new type of talent and director that doesn’t come from the narrative-driven, commercial video space. “In my experience, most of my directors have struggled with it,” she says. “They don’t know how to apply their way of traditional thinking to the format.” And it’s not just veteran directors that are hesitant to use VR. She finds that video creators in their late teens and early twenties are not at all interested in VR: they’re into the short-form content that fuels social video platforms.

Some in the industry are more bullish on the applications of VR in the music industry. “VR can expand on what the music video and the fanclub or fanzine were in the past,” says Doug Buffone of VAR Consultants and Renaissance Media. “Musicians can share their story and their musical vision directly with fans.” But even as a self-described “evangelist” of the medium, he “tries not to spread the hyperbole too thick.” “I don’t believe in all this nonsense that it’s the year or the summer of VR. It’s a life choice that people have to make about getting into it, and understand that they’re at the vanguard.

It seems that both the artists and audiences at this virtual reality vanguard are excited about its promise but aware of its limitations, whether in the technology itself or the applications of it. But Buffone maintains that the rules of VR are not set yet, and warns creatives to not get dogmatic about “where you put the camera, where action is oriented, what pieces look like.” With that in mind, perhaps the VR of the near-future will be nothing like what we’ve seen in its infancy, especially because of the literally-limitless possibilities of what virtual reality can be. Even Björk would have to admit that being on the beach with her is just at the water’s edge of virtual reality, and that being in her mouth is just the first taste of it.

Chris Kelly is on Twitter

Read next: Iranian futurist Ash Koosha is pushing electronic music into a virtual reality



Share Tweet