VR’s Killer App: Film
Judging from the first wave of VR hysteria in the ’90s, one might presume that the most captivating virtual reality experiences would come in the form of video games or some kind of immersive simulation program, like the Holodeck from Star Trek: The Next Generation.
Indeed, virtual reality gaming and augmented reality—wherein computer generated imagery is projected upon real-life objects—may one day be all the rage. However, at the moment VR gaming is a complex proposition, requiring a great deal of computing power. At least one demo of a Holodeck-like program exists, but it requires the user to wear a heavy belt and remain tethered to a high-powered computer by a cable.
“In order to make (VR gaming) feel realistic, you need to have a really high frame rate, so you need to have a pretty beefy computer,” says Robert Hamilton, who recently earned his PhD from the Center for Computer Research in Music and Acoustics (CCRMA, or “karma”) at Stanford. While there, Hamilton worked on creating musical games—research he is continuing in the virtual reality space at his current job, as a professor at the Rensselaer Polytechnic Institute in New York. Hamilton is currently building a VR musical instrument at RPI. “We’re a little ways away from having the VR gaming experience everyone would want on a commodity laptop.”
For his part, Broock is betting that virtual reality films, which place the viewer in the center of some kind of action they wouldn’t otherwise be able to experience, are likely to be the first thing to take off.
“Cinematic VR is something completely different, because it’s the world encompassing you,” Broock says. “For someone who’s never tried it, it’s extremely difficult to describe. Your eyes are the eyes of the camera. I call it an emotional punch to the gut. It’s so completely different.”
Viewers with a cell phone and Google Cardboard can get an idea of what Broock is talking about right now. Download one of the many demos available in the Google Play or iOS App Store, strap the goggles to your head and put on some headphones. No matter where you are in the world, you’ll be swept away—perhaps to a scenic vista, a red carpet reception, the sidelines of a sporting event, or to a lonely grove of trees on the Pacific Crest Trail for a chat with Reese Witherspoon.
The Witherspoon clip actually exists. It was created as a tie-in to her 2014 film, Wild. In the video, Witherspoon plays Cheryl Strayed, the woman who wrote the book upon which the Academy Award-nominated film was based. She sits alone on a rock, just a few feet away, staring the viewer in the eyes—or so it seems at first. However, should the viewer follow Witherspoon’s physical cues and turn around at the right time, he will see that Witherspoon’s character is, in fact, talking to the specter of her dead mother.
The three-minute clip, titled Wild—The Experience, received favorable reviews in Gizmodo and The New York Times. Part of the reason media like this works in VR is that it is brief and meant to augment another product.
“This is not replacing television,” Broock says. “This is not replacing film. It’s its own thing.” Virtual reality films offer viewers a “new kind of experience,” he says. For example, having an intimate encounter with a celebrity, standing on stage at a concert or traveling somewhere far away—say, the Sistine Chapel in Vatican City, or the Hagia Sophia in Istanbul.
Sound Guides The Eyes
Of course, visiting a foreign land is about more than sight. Strapping on a headset and sitting in a swivel chair isn’t going to give you the smells, the taste, or tactile feel of a place. However, sound—an invisible, but critical component of any virtual reality experience—can be replicated with exacting precision.
“The ears point the eyes,” says Jonathan Abel, a consulting professor at CCRMA, before launching into an explanation of the importance of sound in creating a realistic virtual experience. “Sound will situate you in space. The sound of the space will convey a bit about its architecture—its geometry, its materials, that kind of thing. It’s an important part of existing in an environment and it is an important part of creating a virtual environment.”
While the virtual reality experiences of today have surpassed those depicted in Lawnmower Man and Johnny Mnemonic by leaps and bounds, they still wouldn’t feel right without realistic sound. A carpeted room with wood paneling and thick drapes should feature a dull sonic fingerprint, just as a large hall with hard surfaces should be reverberant.
“All these things are super useful when you’re creating a virtual environment,” Abel says. “If you get the sound right, it makes the visuals look better. It makes it a more compelling environment and it makes it the kind of thing that you’re more likely to get immersed in.”
Abel and his colleagues have been working for years on sound modeling and have made great strides in creating sonic maps of spaces both large and small. These maps can be turned into algorithms, and those algorithms, in turn, can be used to make any sound take on the characteristics of the room in which the aural map was created.
Which brings us to the Hagia Sophia. Abel and his team captured the treasured architectural relic’s sonic fingerprint. And although no accompanying visual VR experience has been created for the Hagia Sophia, if one were, the technology Abel and his team have developed could be paired with the film to produce a virtual tour of the space that would allow viewers to hear their voices echoing through the ancient basilica.
These are the kinds of experiences that Broock and his company are betting will draw the interest of the public.
“Travel is going to be huge,” Broock says, speaking about the near future of virtual reality, which he envisions offering “brief bursts of raw experience.”-
A Ways To Go
As it stands presently, virtual reality technology isn’t perfect.
All of the VR goggles I’ve worn have made my face sweat, which causes the display to steam up and become uncomfortable. As I understand it, this is a problem for anyone who runs hot. Additionally, some of the videos I watched made me nauseous. That’s a problem Broock insists can be remedied with better algorithms.
But after experiencing the Oculus Rift and Gear VR—along with a number of videos produced by Jaunt VR and Mind VR—the potential is clear. And the experts I spoke with for this story certainly agree.”
“I think eventually this is going to revolutionize the world,” Guy Coggins says. “I think in the relatively near future, people are going to be spending a great deal of time with virtual reality.” For him, it’s just a matter of time.
“We’re still in that feeling out phase, where every developer wants to show off,” says Hamilton, referring to the glut of VR demos available and the accompanying dearth of anything incredibly earth-shattering. “But I think it will shake out.”
One thing Hamilton is particularly amped for is the future of augmented reality. He describes a computer rendering of what an augmented reality version Minecraft might look like—where “the world of Minecraft is on your table and spilling onto your floor. That is insanely compelling to me.”
He also envisions using virtual reality to conduct training for surgeons and to allow geneticists to examine a patient’s actual DNA as if they were the size of a single fleck of RNA—Magic Schoolbus-style. “The more you start mixing realities and the more that we can be transported into these other spaces, the more we will start using them,” he says, “because it is a really compelling experience.”
