The Best VR Experience at Tribeca Film Festival Was All About Sound

Usually when we talk about virtual reality’s role as the evolution of film, it’s because of what the medium allows us to see, be it a far-away place we couldn’t travel to otherwise or a fantastical world our mind is fooled into thinking we’re a part of. So it’s rather ironic that the best VR experience I saw at the recent Tribeca Film Festival was a depiction of what it’s like to not be able to see anything at all.

The film, Notes on Blindness: Into Darkness, is a 14-minute, six-chapter story about what it’s like to go blind. It was written by writer and academic John Hull, who lost his sight at age 43 and documented the experience in an audio diary that narrates the piece.

When I sat down to “watch” the piece, I was skeptical; it seemed counterintuitive that I should have to strap a screen on my eyeballs in order to experience darkness. But there was something I wasn’t accounting for: sound.

Once you put the headset on—I saw it in a Gear VR—the world goes mostly dark. You’re looking at a black background with little blue and white pixels floating around, sometimes forming a vague outline of nearby objects which a sighted person would see. The film then makes brilliant use of 3D sound to reveal the hidden images of the objects around you in your mind, the same way a blind person’s heightened sense of hearing helps them visualize and navigate their environment.

3D sound mimics the way we hear in real life, and is a crucial part of what makes a virtual environment realistic. Most recordings we listen to today are in stereo, panning between the left and right ear. But adding the aural dimension of depth is how the brain locates things in space, so you can hear sounds that appear to be coming from off in the distance, right up close, behind you, and in every direction in a 360-degree sphere.

Notes on Blindness was recorded with binaural audio, using a dummy head to capture sound that mimics the way our ears hear naturally. The effect, combined with beautiful graphic imagery, creates a forceful sensory experience. It’s one of the only VR pieces so mesmerizing I was able to forget about the inherent discomfort of having an heavy headset strapped to my face.

The dummy head used to capture binaural audio was on display at Tribeca Film Festival. Photo: Meg Neal

In the first chapter, you’re staring into the darkness while the narrator starts describing the sounds around you. When he points out that the person sitting to the left is turning the pages of a newspaper, that sound is brought to the fore and a faint silhouette of the newspaper-reader is starts to take shape. Now that object is part of your awareness. Next it’s the hum of a highway in the distance to the left, cars pulling in and out of the parking lot behind you to the right, footsteps walking by on the path in front of you: different sounds for flip flops, women’s heels, kids running.

By the end of the chapter the entire soundscape is revealed, and you can perfectly picture the scene in your imagination. You become aware that you’re sitting on a bench in a public park.

In my favorite chapter, you’re inside a house and can hear the rain pouring down outside. Now imagine if it could rain inside, the narrator prompts. The rain acts like an acoustic clue: The way it hits and bounces off the contours of different objects reveals their shape, size and location, so you can start to tell what and where they are. In the goggles, as you focus your gaze on certain prompts, it triggers the sound of the rain falling on that object: a teacup, a large pot, a jar with a lid. One by one the objects’ acoustic shapes are revealed, and you notice you’re standing in a kitchen.

The experience—which is actually a companion piece to a feature length documentary of the same name—was easily one of the most moving pieces of cinematic VR at the film festival, which wrapped up on April 24 in New York City. There were a few other crowd pleasers that successfully transported you to another world, but those superior experiences used advanced headsets like Oculus Rift and HTC Vive that include positional tracking tech that lets you move your head around within the scene, instead of a static 360 view. Notes of Blindness was demoed in Samsung Gear, without this capability, making it that much more impressive that it could evoke the same level of emotion.

VR storytelling is at it’s best when it’s used to give the viewer a heightened sensory awareness. Magic Leap founder Rony Abovitz recently described alternative reality as “a journey into inner space,” and that’s exactly what Notes on Blindness does—at it’s core, it’s about cognition. There’s no way a 2D “flat” film could achieve that level of sonic perception, and without it you’d be hard-pressed to really portray what it’s like to be blind.