How Traditional Storytelling Is Ruining Virtual Reality Film

I spent much of the past two weeks with a screen strapped to my face, watching filmmakers’ best stabs at telling a story in virtual reality.

The cinematic VR showcased at the recent Tribeca Film Festival ran the gamut from live-action documentary to animated science fiction. Very few of these experiences were any good. But the real crowd-pleasers, animated fiction like Allumette and Invasion, racked up waitlists as long as eight hours.

Videos by VICE

Looking at the difference between the good and bad VR, I started to glean some insight into the nascent artform. No one has even come close to mastering the medium, but it’s clear that holding on to the traditional rules of storytelling is a surefire way to make disappointing VR.

Even borrowing the term “film” to describe immersive storytelling isn’t an entirely accurate description, and we don’t yet have a word to replace it with.

Tribeca dubbed its virtual exhibit “Storyscapes,” which is actually a pretty good descriptor: The best VR at the event were the pieces where the filmmaker created a world, and you experience the story from within it. The emotion evoked from the landscape and the characters in the world is the story. It’s not about watching a series of events; it’s about viscerally responding to the energy, the vibe, the spirit of a space.

Yes, everyone recognizes that this sounds like some trippy shit.

“Sometimes I think we sound like lunatics, but it’s just the way we talk about it when we’re talking about the medium,” said Félix Lajeunesse, one half of the VR studio Felix & Paul, during a conversation in the festival’s green room.

“It’s an experiential medium,” said Lajeunesse. “It’s feelings. It’s impressions. It’s immediate. It’s visceral. It’s not intellectual. It’s very trippy because that’s the nature of an experiential medium. It’s what it is.”

***

Cinematic VR necessitates breaking the traditional construct of a linear story that hits certain beats: this happens, then this, then this. Good immersive content doesn’t just deviate from this pattern, it flips it on its head. Instead of plotting moments on a timeline, it builds a world for the viewer to experience on their own. Instead of trying to control what the viewer sees or feels, the filmmaker is sculpting a time in space, where everything that happens in between the beats is part of the story too.

“It’s a spatial kind of storytelling, rather than a temporal,” said Paul Raphaël, the other half of Felix & Paul studios, which has created some of the best live-action VR I’ve seen to date, including Strangers and Wild.

If a VR film is trying to get people to look where they’re supposed to, it’s already asking the wrong question, Jessica Brillhart, principal VR filmmaker at Google, told me over the phone. She spoke at the Tribeca Film Institute’s TFI Interactive forum about the future of storytelling, and showcased her DeepDream VR project at the festival’s Interactive Playground. Brillhart has the enviable job of filming a ton of stuff with Google’s stereoscopic rig, Jump, to see what works and what doesn’t in VR filmmaking, and try to figure out what the language for this emerging artform will be.

“It’s more, how do we craft an entirety of a world to be able to harness the agency of the viewer being able to look wherever they want to look,” Brillhart said. “To see it as world-building, instead of trying to put things in a box.”

Storytelling through virtual world-building, especially as a way to portray genuine human connection, is still mostly uncharted territory. In no small part because the tools and standards for creating specifically for VR in mind don’t even exist yet.

Cinematic VR is marooned in a catch-22: The content isn’t good enough yet to justify paying $500 to $800 (plus the necessary computer hardware) for a high-end headset. But without mass adoption of the gear, we won’t have the diversity of perspectives and experimentation needed to make quality content.

That’s where Google comes in. Cardboard viewers opened up 360 video to the average consumer, but not true VR. More recently, Google’s Jump platform is designed to streamline the capture, stitching, and editing process to put VR in the hands of more creators. (Jump will still run you many thousands of dollars, so we’re not talking consumer-ready quite yet.)

“I can only imagine how much we’re going to learn from opening this up a lot of different voices, a lot of different minds, to see what they bring back,” Brillhart told me. “The medium won’t survive if we don’t have the kind of diversity of thought we have with film now. It just won’t.”

* * *

If you’ve watched any 360-degree video you’ll notice that one persistent challenge of spherical content is that you don’t know what direction you should be looking to avoid missing any crucial plot point.

A lot of today’s VR tries to solve that problem by incorporating cues to direct the viewer. One of the experiences at Tribeca, SENS, the first graphic novel adapted to virtual reality, literally has you follow a series of arrows directing you through a white snowy CG maze. But SENS only solved the problem halfway. I knew where to go, yes, but I failed to see the point: It wasn’t a game, but it wasn’t a story either.

The VR experiences that worked, on the other hand, were the ones that created a new world you felt a part of, that in itself evoked emotion. In other words, the ones that got closer to achieving the holy grail of virtual reality: “presence.”

In the VR gaming world, presence refers to a threshold for fidelity where the brain is fooled to thinking its synthetic environment is real. In VR storytelling, the concept is a bit more nebulous. It’s about figuring out how to take that powerful feeling that you’re in another space and use it to tell a story.

“Presence is a pretty esoteric word. But it’s fine. It’s ok. We accept that,” Lajeunesse said. “Sometimes you can feel like it’s almost… like there’s some spiritual quality. We want to nurture that presence. We try to make sure that storytelling doesn’t destroy it.”

* * *

The best VR storyscapes at Tribeca were the ones that got closest to achieving that kind of presence. The biggest hit was a lovely little film called Allumette, which Wired went so far as to call “the first VR film masterpiece.” The 20-minute fictional feature is a heartbreaking story set in a gorgeous 3D sky world that’s animated with such brilliant detail and texture it looks sculpted, not CG.

Allumette had the benefit of an added level of interactivity. While most of the films at the festival were watched on a Samsung Gear VR—where you can look around in 360 degrees but don’t have the dimension of depth—Allumette was made for high-end headsets with positional tracking. So you, the viewer, have more agency to move your head forward to get a closer look at the character’s face, or look around corners to explore the fantastical city.

The story of the little girl in Allumette was the first VR film that made me tear up in my headset.

Needless to say this advanced technology (the standard in VR games, which have had a four year headstart on films) goes a very long way to achieving presence and in turn, a more authentic emotional experience—a heightened awareness of your surroundings. The story of the little girl in Allumette was the first VR film that made me tear up in my headset.

On the nonfiction side was a chilling experience called 6×9, a piece of journalism produced by the Guardian, which drops you in a tiny cell in solitary confinement. A narrator is feeding you facts about the criminal justice system, but the most impactful aspect of the nine long minutes is just looking around your new reality: the bed, the desk, the door, the bed, the desk, the door, the bed.

There’s no positional tracking (this isn’t possible for live-action film yet, though it’s in the works) so I couldn’t indulge the instinct to smush my face right up against the little square window in the door. But even without any interactivity, the creator told me that people found the experience so emotionally disturbing, nine minutes was about all they could handle.

Even more poignant was a piece called Notes on Blindness, created based on the audio diary of writer and academic John Hull, who lost his sight halfway through his life. The scene is a dark abyss with pixelated blue and white bits outlining where the objects around you would be if you could see them.

It uses three-dimensional binaural audio to reveal the objects in the scene one by one until the whole soundscape comes together, and you can imagine what the scene would look like. It’s a beautifully crafted world—the world inside his mind—that makes you understand the reality of blindness in a way a traditional documentary couldn’t do.

The VR films at Tribeca let you ride a dragon, swim with dolphins, and float through a meditative psychedelic seascape. But what was absent at the event was any live-action cinematic VR that really harnessed presence as a mechanism for storytelling.

“There’s something powerful about what VR can do that a lot of people are missing,” said Brillhart. “I think a lot of the poor content comes from stuff that just shouldn’t be in VR at all.”

Many of the storyscapes at the festival fell victim to this. A journalistic piece, The Ark, transported the viewer to Africa to hang out with the last remaining endangered white rhinos. The spherical view showed the animals if you looked left and an interview with a stem cell scientist in her office if you looked right. While the subject matter was compelling, the platform added nothing to the story that wouldn’t have been accomplished in a 2D video—except maybe confusion.

But the biggest hurdle to creating quality VR film isn’t the tech, said Brillhart. It’s wrapping your head around the new mindset of the creator as world-builder and the viewer as storyteller. What will that mean for the next leap in the evolution of cinema?

“That question about the language of VR is kind of the question that everyone is thinking about, right?” Brilhart said. “Like, once the gimmick wears off, what’s left?”