Video & Filmmaking

Photojournalism Meets Virtual Reality

July 20, 2016

By Jon Blistein

Frontline’s “On the Brink of Famine” follows an emergency food drop in South Sudan. As journalists and documentarians embrace virtual reality, they are exploring how to use it to tell authentic, unmediated stories. 

Now that a virtual reality (VR) experience is available to anyone with a smartphone and $15 to spend on a Google Cardboard headset, VR has become a viable new medium that has attracted filmmakers, game designers and, increasingly, journalists, documentarians and news juggernauts like The Guardian, The New York Times and PBS’s Frontline.

The journalistic possibilities virtual reality presents are monumental: not just to show viewers a story, but to place them in it. In 2015, both The Times and Frontline rolled out their first VR projects. The Times brought viewers into the neighborhoods of children affected by the global refugee crisis (“The Displaced”) and to the fence that divides the adjacent towns of Nogales, Arizona, and Nogales, Mexico, where an unarmed Mexican teenager was shot by a U.S. Border Patrol agent (“10 Shots Across the Border”). Frontline, meanwhile, took viewers to the foot of the tree where the most recent Ebola virus is said to have originated (“Ebola Outbreak”), and into a field full of villagers in South Sudan awaiting air drops of food (“On the Brink of Famine”).

“Being able to go to a place and be present with a person is going to affect your understanding of the story,” says Jenna Pirog, the Times’ first VR editor. “As the technology evolves, the implication that you’ll eventually be able to go to a place in real time and see a news event occurring is huge.”

But for now, Pirog and others are focused on a more immediate goal: Figuring out the best way to tell a journalistic story in VR. Traditional filmmaking techniques like montage, quick cuts and talking head interviews don’t translate well in VR, so practitioners are still parsing this new medium’s narrative vernacular. Even the medium’s quintessential quality—immersion—raises thorny questions about the role of the journalist, and journalism itself.

Fergus Pitt, a senior research fellow for Columbia University’s Tow Center for Digital Journalism, notes, “The whole conceit of VR, and its strength, is that the viewer believes that they are in a real place and they are getting an unvarnished and unmediated experience. But like every media it’s an incredibly composed and contrived experience.” All visual journalists grapple with authenticity, representation and storytelling, but Pitt suggests these issues are especially crucial in VR, a medium that promises the viewer greater presence than linear documentary or photography.

It starts with the 360-degree camera. Shooting in 360 forces filmmakers to stop thinking in terms of a traditional 16:9 frame that acts as a window into another world, says Tyson Sadler, a director for the AOL-owned VR company Ryot. VR, by contrast, is about finding a space that brings the viewer fully into the scene, though a photographer’s instinct can help when establishing what Pitt calls the “natural forward”—the first shot the viewer sees when they put the headset on.

As for the cameras themselves, single cameras that fuse multiple sensors and lenses into one body have entered the marketplace recently, but the rigs that offer the best quality and immersion are essentially tripods outfitted with between four and ten GoPros. While these rigs get the job done, GoPros can overheat and their batteries don’t last long (Sadler suggests carrying extra cameras, batteries and a DeWalt USB field charger); Ben C. Solomon, a multimedia journalist and video director for The Times, adds it’s not easy to access media on GoPros or monitor them, especially in remote locations with little Wi-Fi and few power sources.

Of course Solomon notes that in six to 12 months a new camera could change all of that. But it won’t change the unique, new question VR journalists have to ask: Where should I stand? Pirog points out, filmmakers and photojournalists stand behind the camera, close to their subjects; but in VR, most filmmakers set up their shots and hide off-camera while the action unfolds. Neither The Times nor Frontline forbid journalists from appearing on camera, but the current belief is that their presence would disrupt the immersion.

Some of Pirog’s filmmakers have said this set-up makes them feel disconnected from their subjects, but Sadler, who co-directed “10 Shots Across the Border” with Ryot’s Ben Roffee, says conversations with the family of the slain teenager, José Rodríguez, helped them flesh out the shot list and storyboard they had going into the production. Sadler adds that while they followed the Rodríguez family, there was no specific direction beyond, “Do what you normally do,” and he notes being forced to leave the scene while filming has an upside: “There’s an old saying that when the camera is present, the authentic departs. But with VR, when the camera person leaves, a lot more natural action can take place.”

While visuals remain key, VR immersion is also heavily dependent on sound, which can guide the viewer like it does in real life. The Times recently added 360-audio to its VR app, meaning the onus is on the filmmakers to know what to listen for while placing mics in the field.

“You’re looking for things that represent that environment,” says Francesca Panetta, an audio specialist who led The Guardian’s first VR piece, “6×9: A Virtual Experience of Solitary Confinement.” The idea is to paint a mental picture of a place using only sounds. “For something like solitary confinement, it’s a wide range of sounds: Prison guards, the inmates next door, the food slot opening—all of that is part of the story.”

For “6×9,” Panetta used 25 hours of background audio provided by Frontline (recorded for their non-VR doc, “Solitary Nation”). Panetta combined the myriad tracks into two mono streams which play positionally throughout the piece; coupled with interviews, “6×9” features 18 audio streams, though not all play at once and some are even interactive.

While lush field recordings add to the immersion of VR, audio narration and interviews are crucial for creating narrative while viewers explore scenes over the course of a 15- to 45-second shot. Omniscient narrators give the filmmaker more control over the story (Will Lyman, the actor who does voiceover for all Frontline’s documentaries, narrates their VR projects), while gathering audio on the spot allows the subjects to tell their own story (most pieces, including “10 Shots” and “On the Brink,” mix both). In “The Displaced,” however, The Times used subtitles effectively, if not sparingly, so that characters’ voices could be heard in their native tongues, adding more to the immersion. Frontline’s “6×9” also uses second-person narration provided by former inmates and psychologists who’ve studied the effects of solitary confinement to make the user feel like they’re the subject of the piece.

The “6×9” project is also unique in that it was created in CGI with Unity, a software framework primarily used to design video games. Time constraints and access to a solitary cell made Unity the most sensible choice for “6×9,” and the animation does not detract from its journalistic value. But it does highlight the delicate balance?—between authenticity and representation, immersion and narrative—that VR filmmakers and editors will continue to grapple with as they define the parameters of this medium. Still, Frontline executive producer Raney Aronson-Rath says certain journalistic ethics remain in place.

“The biggest thing for us is veracity and truth,” says Aronson-Rath. “We don’t like staged situations and sometimes in VR, situations are easier to stage because it helps the filmmaking. But it’s really important that our work is documentary in a journalistic sense, that we’re using these same ideas and the same strictness that we have in the linear space. And that’s hard sometimes, but in VR it’s important—it’s crucial and it’s mandatory.”

See the story in the digital edition (for PDN subscribers; login required). 

Related: Should Photographers Jump on the Virtual Reality Bandwagon?

Ivan Sigal on Bringing New Ideas and Underrepresented Voices to Storytelling