The a16z Show - a16z Podcast: Writing a New Language of Storytelling with Virtual Reality

Episode Date: December 18, 2015

Chris Milk calls virtual reality the “ultimate empathy machine.” The filmmaker and founder of VR shop Vrse talks with a16z’s Chris Dixon about how virtual reality can connect with people in ways no other medium can. Milk describes the ways virtual reality production veers from the traditional techniques of filmmaking, and why the results can transport people to places and feelings that we’ve never experienced -- except in the real world. The discussion happened as part of a16z’s 2015 Academic Roundtable. The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments and certain publicly traded cryptocurrencies/ digital assets for which the issuer has not provided permission for a16z to disclose publicly) is available at https://a16z.com/investments/. Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information. Stay Updated:Find a16z on YouTube: YouTubeFind a16z on XFind a16z on LinkedInListen to the a16z Show on SpotifyListen to the a16z Show on Apple PodcastsFollow our host: https://twitter.com/eriktorenberg Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures. Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

The content here is for informational purposes only, should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security and is not directed at any investors or potential investors in any A16Z fund. For more details, please see A16Z.com slash disclosures. Welcome to the A16Z podcast. I'm Michael Copeland. Chris Milk calls virtual reality the ultimate empathy machine. The filmmaker and founder of VR shop verse, that's V-R-S-E, talks with A-16Z's Chris Dixon about how virtual reality can connect with people in ways no other medium can. Milk describes the way as virtual reality production veers from the traditional techniques

of filmmaking and why the results can transport people to places and feelings that we've never experienced except in the real world. Chris Dixon starts things off. So now we have Chris Milk. Maybe if you could tell people what you're working on. We're sort of a media technology company that's building the tech to serve the evolving language of storytelling in virtual reality. I can go deeper into that. You've written about it and talked about the sort of idea that there's a new grammar to virtual reality

and how the default state is belief and not disbelief. Can you talk about that and how? And maybe some of the lessons you've learned so far. I think we're still figuring out how to make things. VR, but... Yeah. I mean, grammar is a good word, because I talk about...

I talk about, like, how we're writing this new language of storytelling in what is a new medium. And I would say, like, at this point, we're just, like, figuring out how to sound out words. Like, it's not even grammar at this point. I think, like, in my take, just using a lot of VR myself, like, everyone originally thought it would be first-pring.

person games and those turn out to be actually very bad in VR. Yeah. And they actually, you know, because if you think about it, you're now in the body of that person and all of these things we've built like people shooting at you, which in the world of rectangles, you needed this intensity because you were sitting far away from it. And like, but now once you're inside of it, it's like, whoa, someone's shooting at me. And it turns out, at least in my experience, much calmer, third person or steadier and less violent experiences work a lot better.

Yeah. It's a trivial example, I guess. Yeah. So the stuff that seems to work, like, the stuff that I see in its kind of raw, unmolded form, having that spark inside of people when they feel it that goes beyond just the wow factor of like, oh my gosh, it's virtual reality, is feeling transported to a place and feeling connected to the people inside of that place that you find yourself in. And there's a real, there's a really interesting human.

connection that's happening with the audience and the subject matter and the characters inside of the virtual reality world that you're building. And there's a lot of, and I talk about this, and I did a pet talk about this whole topic. I'm saying virtual reality

is like it is the ultimate empathy machine. You see people feel, feeling, resonating with these people in these worlds that you never ever would see in five minutes. So can you talk about some of the examples like the, I think the refugee camp Yeah, so we did, we started really quickly.

I got connected with the United Nations and started making virtual reality films with them. We are doing all kinds of things. We're making horror films. We're doing comedies. But I really wanted to do something with like a higher purpose right off the bat. So we got with the United Nations. And what we're doing is we're making these series of films. where we essentially bringing you to a place of people in need somewhere in the world

and letting you understand their situation and their daily existence. And we're not just showing it to people on the streets, raising money, which we are doing, but we're also specifically targeting world leaders, both in the United Nations and places like Davos and World Economic Forum, the people that can actually affect change from the top down, and showing them these films where we might not find themselves in a Syrian refugee camp, which was the first film that we shot in Jordan. And that one, we tell the story of a 12-year-old girl named Sidra, who's been living there a year and a half.

She fled through the desert with her family and ended up in this camp. And then we just did another one in Liberia where we tell the story of this woman that survived Ebola, contracted it, survived it, and now is immune to it and is helping the community, educating the community and helping others in the community because there's a real stigma for orphans of those that have died of Ebola and they're sort of outcast and she's taking care of them. But they're very, they're very transportive.

So we see those sort of nonfiction things working incredibly well. That's going to work well in narrative, but narrative is, you know, where, when it's nonfiction, you sort of understand as an audience, instantly, I am the perspective of a camera. This is something that's happened in time that I'm existing within. Whereas when you start telling narratives that are more like movies,

there's what your perspective represents. When you're inside a bedroom and there's a couple talking in bed, it's different than when it's on the rectangle, because now you're not just a witness to it through a window, you're actually in the bedroom with them. So what does that mean? and can audience become comfortable with that? There's also, I mean, the things that we're seeing don't work.

There's a lot of things with camera movements and the way that you treat the camera that can make you nauseous. So the way that, the big reason that you get nauseous in virtual reality is your brain is constantly checking what your eyes are telling it and what your intervistibular system is telling it. and if they don't match up, you get nauseous.

The reason that is is because your caveman ancestor, when he ate the bad woolly mammoth or the poisonous mushrooms, and he saw the world moving like this, and his body made him throw up so that he didn't die, he survived. That's your ancestor. So we have that now when you're in virtual reality, and the world moves in a way that your physical head is not moving, you get nauseous.

So the first, like, best rule of practices out from Oculus was the camera never moves. So the first thing that we did was move the camera. And what we found was that you can move the camera if you keep it at a constant motion. It's acceleration, not velocity. Acceleration and de-aceleration. And because you're interface. This is what the glove of the games, like, try, like Lucky's Tale and things. Like, they have this sort of, like, steady camera following a character.

Right. Right, right. That seems to be what people are converging on. Right. So your inner vestibular system doesn't sense constant motion like it does, acceleration to acceleration. That's why when you're on a plane, you don't feel like you're flying at 100 miles per hour.

So we can do linear. Linear is better, too, than not curving, but we're working with curving, and we think there's ways to do that. But what's interesting is, so now we're, so we figured out, okay, that works, that doesn't work. Now what we're working on is, how do you break the rule just like a little bit to actually four creative results, right? So if you look at cinema, you go to film school, what they'll tell you on day one is you have to learn the rules.

You have to go through four years of film school. And just for those of us who aren't film, like we're all used to now, like, so for example, an establishing shot, right? You show the outside of a building and then you show the scene inside of the building. But that took a long time for filmmakers to understand, right? Isn't it if you go back, like the old Charlie Chaplin movies, they show like five minutes of people just going in and out of Grand Central Station. Yeah. And they don't, I mean, or close-ups or I don't know.

You tell me, right? But didn't it take a long time to figure these things out? So, yes. I mean, it took a while to figure that out. Editing was the first big thing where they realized, like, you could cut from a wide shot to a close-up. And that you would, and that there was a language to understand there. And people, you know, at the beginning of cinema,

it was just a bunch of newsreels, and they were shooting plays from the back of the theater, they didn't understand what the construction of a feature film was, and we're in the same position now in virtual reality. We don't know it's not going to be about making a movie in virtual reality. It's about figuring out what the equivalent of a movie is, what a movie is to cinema or film, what is the equivalent of that in virtual reality?

And that will take years to figure out, but we have to also, if you look, if you just look back at other mediums, they had to figure out the pieces of the language before they figured out what the encapsulated storytelling vessel was. And in film editing was a big one, being able to move the camera. So they tell you these things in film school,

like you need to learn the rules before you can break the rules. And great filmmakers break the rules for creative reasons. And what we've started to do now, and it's early days, but we've started breaking the rules for creative reasons of virtual reality. So there's a film that we have, the film It's called Evolutionverse where the train comes out of you. And there's a moment of that where you rise up off the ground.

And this is a film that I made as a filmmaker. And I can make the camera go from zero to 60 in one frame, and you won't feel a sensation in your body. You'll just see yourself rising. I mean, you'll see yourself now traveling up. And what I did was I built an acceleration into the move at the beginning because I actually want you, I want. I want you to feel it in the pit of your stomach when you start traveling up.

The same way that you feel it in the pit of your stomach when you start traveling up in an elevator. It's actually the same, it's the same thing. So that's a tool. That's 100% of creative tool that we, and it's a tool that we've never had in any other medium because I'm actually getting inside of you and tweaking your physical biology and making you feel like something on an emotional standpoint. From technology to physiology to emotion, that's incredibly exciting. Did you, you know the Michael Abrecht stuff?

He did it, Val, the studies they did. Did you see this stuff? So they did these studies where people would look down into virtual, like they're standing in a room, obviously, but they look down as if they're seeing a cliff. And they've done it 100 times. And they know it's not a cliff. And yet they are unable, their legs just buckle.

Like they're unable, like the, when you get what they call in the business presence, like this feeling of truly tricking your brain, it happens, like the computer analogies, it happens sort of a low-level system beneath your conscious brain, and you literally cannot, you know, your lizard brain has decided there's a cliff there and it will not let your legs move. Regardless of the fact that you've done that 100 times, and you're trained, like this is, this is one of the really astonishing, and to me at least was one of the really astonishing results. Yeah. Sort of how deeply it does sort of trick your, um,

Yeah, well, that's why, I mean, what's happening there, so to speak to the, like, the question of like, is this a new medium? And I talk about how this is, this is the last medium. And the reason that is because it's the first, it's the first medium where the technology actually allows the jump from it being like an observable technology to being a human interface where the, where the technology is not observable, and that's because essentially what it's doing is it's mirroring, currently it's mirroring two of your senses so closely that you experience it as your lizard brain experiences the real world. And what's incredible about it is that even in its rawest first iteration form, or not first iteration, but first consumer iteration, which is like a cell phone on a piece of plastic with some lenses

or a piece of cardboard and some lenses and you feel it on your face and like you see the pixels and it's not full field of view, you still have that feeling where you can't take a step forward off a clip that doesn't look anything like a realistic clip. We know it's be dramatically better in the next couple of years too. Yeah. Any questions?

One of the things that we wind up giving up, I guess if we're putting these glasses on and enjoying a movie or music video is the kind of shared experience that we have when we go to a movie theater. And it's not just the shared seeing exactly the same thing as people sitting next to you, but also the social experiment of like smelling the popcorn and, you know, going on in a specific time. Is there an analog to how that may happen almost like in a same common place where people go to what was once a movie thing? theater and now it's a bunch of, you know, pods that give you access to the best technology of the day. Yeah. Okay.

So I'm going to answer your question one way and then I'm going to contradict myself. Okay. So the first thing is that you come to, I hear this question a lot. And you come to that thought process by comparing it to cinema, which is a natural thing to do. So we all sat in the darkened theater, and this doesn't work that way. And it doesn't work that way because fundamentally it's taking over the senses that give you that shared experience that you have in the movie theater right now. I think that the first thing I realize is that it is a completely different medium, and it's going to have its own set of rules, its own set of pluses and minuses. And people don't talk about books not being a worthwhile, a worthwhile, valid medium, because,

you can't sit in a room with 100 other people and read them together and have that shared experience. Every medium is different. Okay. Now I'll contradict myself. So having said that, I don't want to make a world where everybody is sitting in the dark or sitting in their own living room by themselves having this experience. I think there is great power and virtual reality for the viewer to connect to the person in

of the film, or if we call it a film, and I think that there is also potentially the great power for the audience to connect to others to have a shared collective experience. I don't think it's converted movie theaters. I don't think it's pods. And we're actually working on a number of different ways to accomplish this where, I mean, what you want it to be is you don't want, I mean, this is a, this is a technological, virtual world that shouldn't, you shouldn't need to

be, to feel like you have to go to a place and drive your car and parking, a parking garage, and pay $20 and sit in this room and then drive all the way home. Have you tried toy box? No, I'm trying it. Okay, so this is a great demo where, which, okay, it's, and I think they're talking about it

publicly, but it's one of the Oculus demos they've just done, and it's you're with, you're basically interacting in this virtual world with another person who obviously can be like, in this case it was another room in the building, but they could be just anywhere on the internet. And it's interesting because it's sort of a symbolic representation

of their face, like it's sort of a, I don't know how you describe it, and then like their hands. Like an avatar. But the sense of presence is very, very strong. Like I felt like it was a very, that's why it was a pretty, it was one of the best. To me it was like as a big leap forward

as, you know, and it also by the way, they have full hand motion basically now with this new thing, Oculus Touch, which made another big leap. And this case you're grabbing what you're doing is you're in this virtual world's other person and you have these toys all over and you can grab these like guns and play ping pong and shoot each other and smash a ball and you're in outer space and shoot the guy and he shrinks and his voice sounds different and you're chatting and you're high-fiving and you just feel like you're there and you're and the

hand thing you just lose I mean it's not perfect but it's good enough that your brain is you're there I don't know yeah can I just sort of ask I think it will be very social in the end I mean I think not initially. I think the, and look, I think it's one of these things, my own view is that, you know, it's the delta between kind of how the images look, you know, there's like tumblers of everyone like staring and, you know, men staring at oculus or whatever, you know, joking about how antisocial looks, the delta between how it looks and how it experiences is wider than any other technology in my view. Like, you experience something completely different than how silly you look, right? So it's helpful to hear the kind of views on the social part of it,

but I'm also interested in sort of the technological part of it, in a sense that, you know, when movies first started, someone ever started, people would go there because they didn't have TVs. And now, then people started going to movies because, you know, that was the place you could do 3D and I could do it in your house. Is there a moment in time now where to do something you really want to do as a filmmaker would require very expensive technology to be the viewer? That's not the kind of level of technology that individuals would want to buy, but they would actually need to go to a shared, place so you could leverage you know i'll make something and something cost a hundred thousand dollars to have one of these viewing experiences yeah uh which people aren't going to be buying for a long time in their house um so there's something uh there's a thing in utah called the void which is it's a sort of a large scale virtual reality um um interactive um kind of like

it's all i don't know how you describe it do you know i'm talking yeah yeah there's a couple there's a couple of people doing like arcades, experiences, amusement parks, whatever you want to call them. There's the void. There's a few others that have come to us. You see you're on a spaceship and you walk up to the screen and then you press the things on the screen and there's an actual piece of glass that you're physically touching that's been placed there that's tracked into this world that you're in. That's something conceivably could do. You could add other senses. Basically what you're doing is adding up.

Have you tried a Game of Thrones one? Yeah. Okay. Yeah. So the big thing is like eyes and ears. Yes. Eyes and ears will be, like Oculus is going to be the state of the art eyes and ears. But the other sense is what they'll do. And I have to say, like I was very skeptical of the other sense. I thought they sounded gimmicky. And now I've tried all the demos and actually does add something to it because... Very affecting. Yeah. Your brain and ears are already tricked. And suddenly you have wind. And in one case there was like water splashing and like mist and wind.

And like one of the demos is you're in... I have watched Game in Thrones. but the elevator that takes you up the great wall or whatever, and it's like shaking and you can hear the noise and you feel the wind. And I actually thought it was going to, I was not expecting much, and I was actually pretty impressed by how, like, the whole thing worked for me.

I don't know. Yeah, we had a funny experience at Sundance. We were showing this Vice News piece that we did where we went to one of the protests in New York, one of the big marches around police brutality, and we had a steady cam going through with the marchers and a Vice News reporter reporting. and we were showing it at the New Frontier section

and Sundance, which is a gallery space, and there's lots of people crowded, and everyone has headsets on, and people are bumping into you. And we put people in, and it's like six minutes long, and they'd be in there, and we'd see people go, oh, oh, oh, and they'd come out, and they'd be like, it was, how did you know, like, to hit me when that guy with the orange hat was right next to me

like that with his elbow out, and your brain automatically, just starts connecting the senses together if it believes two of them. So you don't even have to get it that close to make it really feel something. Do you ever feel like with the current camera technology, you're limited in what you can do as a filmmaker? Or like how much better do you think it needs to get for you to artistically accomplish maybe what you want to do?

Or is that just not a limitation? It's, I mean, you... You build your own cameras. Yeah, we build our own cameras only because we have to. I mean, we think that's not a business that we're trying to get into. I mean, there's going to... There will be off-the-shelf solutions eventually, shortly from... We had Google here yesterday, like Steve, who runs the jump project as an example.

I think that's correct. And the main thing is camera, a camera looking out, looking in. looking out. So it's a ball looking out, right? Which means when you're inside of the experience, you're essentially inhabiting the place where the ball of cameras was. And we can move the camera around and bring you on rails to different experiences. And we're doing a lot of that. And we're having great results. All right. Last question. Actually, I had two quick questions. First of all, just could you share with us your guest for the timeline? Like, when do you think a major movie, a major Hollywood studio will start producing these movies? They'll be available to the

public. First quick question. Second question is more technical on how big are these files? Steve from Google Jump was saying that it's like gigabytes of data per minute. How big are your movies? Where do you host them? Do you see Netflix actually

storing these things? I mean, they're going to have a major explosion in data warehousing that they're going to need to solve. So how do you see the technical problem being solved? Okay, so first question, when major studios getting in? I mean, So we are working in partnership with all of the major studios right now.

The main thing that they're doing is marketing for existing properties that are coming out. So they're creating VR sort of marketing experience, you know, like two-minute things to... Right. Not so much trailers, but standalone experiences. Like this Game at Thrones thing I described, which, you know, and Ready Player 1, they're doing some stuff, right? because ready player one's a very famous VR book, which is now Steve Spielberg's making it a movie, and I think they're going to do,

I don't know, I've heard, I'm not getting it from you, I've heard just rumors they're doing a lot of promos around that with VR, obviously. Pacific Rim, they had one, there's like a tornado movie that they have one, they'll show it at Comic-Con, and they'll put it out, I think Godzilla maybe has one, they'll release it to the public as well.

So, that's the thing, is like, it's not about making a movie in virtual reality. Like, how, you can't, like having a two-hour movie it's like that's like when is the first play going to come out in cinema when is the first book going to come out in radio so you don't think it'll be

a two-hour thing it'll be I think some I think people will try to make a movie because that's what we always do is when we find a new medium we try to copy the previous medium into the new medium so there will be movies made I can't tell you whether they'll be good right now if you told me that there was a movie in virtual reality

that was two hours long and had to sit there with a thing on my face, I'd watch it, but I'd know that I probably wouldn't be that comfortable in it by the end. And we don't have the language, like, just even telling that the three-extructor two-hour thing in virtual reality, where does a camera, what does a camera do? Like, where, you can't, you're not going to cut, like, the way that you cut a movie is you have a wide shot and then you cut to a wide shot. Maybe another way of ask when Dan says is when will this be a mass medium? Is that fair? And like, when will just, you know, it'll be just like millions of people every week go and use VR.

Yeah. I mean, that is, that is a, that's the big question is how fast. Somewhere between one in 10 years. Yeah. How fast adoption? Three and 10 years. No one knows. There's a lot of different factors. There's a, we now have, the thing that's moving the fastest is actually the hardware technology. We have all the major manufacturers building the consumer available headsets.

There seems to be a lot of excitement and demand. The thing that's actually, the. least develop is the content side. You will see there's a sort of like wait and see from the like you're not going to make a hundred million dollar movie right now because you're never going to make a hundred

million dollars back from your VR movie because there's not a hundred there's not there's not a marketplace to make it. It's really going to be this sort of what happens right is these these these hopefully these virtuous cycles kick in where you get enough headsets out there and then developers come and then that makes it better

and then more people come and those, in my experience, those flywheels are the hardest thing to predict in terms of the timing. Like, we think it's going to happen, and it feels very much like it's going to happen, and it certainly helps that companies like Facebook are investing a lot of money in it. But it's sort of like, will it be, you know, like I think you could imagine next year half a million people buying headsets. You could imagine mini-million buying. I don't know.

So they're going to be expensive initially. They're going to have some issues. There's going to be not a ton of content. So it's going to be, you know, But the trajectory seems pretty clear. I don't know. I mean, eventually it's happening. It's going to happen.

It's just a matter of how quickly it happens. I think it's like mobile. Like, you know, 95, you would have thought, you know, if you were telling it really, it really, my view as mobile didn't start until the iPhone, for real, like, you know, 2007-8 or something, 2006, 7-8, whatever. You know, in 1993, you probably would have thought it would have happened sooner. You know, you wouldn't have expected it was 13 years.

later. On the flip side, when it does happen, it really happens. Like, look what happened with the smartphone stuff. I mean, this is just everything now, in our world at least. Like, every website, every product, everything's focused on this now. It's just replaced computers for the most part. I mean, desktop computers. So, on the Internet, there's another good one where it kind of like it kind of went along, went along, it was academia, and then suddenly boom, right? I don't know. So on the file size of the tech part of it, yeah. I mean, we use Amazon AWS. They aren't that huge right now, right?

Because I've downloaded your movies on my iPhone. They're not that huge right now. They're not that huge. And we're about to come out with a streaming solution. Being a sort of like, we are a creative-driven technology company. So we build the technology to serve the creative. And in that process, it's like I can, you know, a month ago, we could bring our file sizes down.

to make it load very quickly, but then the quality sucks. So we want to have, like, the premium experience that you're getting the best quality. The filmmakers want to come to us to have their experiences on our platform because they know it will look great. But what's important to understand is we're dealing with version one of cinematic VR right now, which is a spherical video player with stereo vision and directionally changing binaural audio. That's version one, but there's going to be many, many versions after that. And version one basically is under a gigabyte for pieces that are like five to seven minutes long, generally. That's like sort of at the maximum compression that I'm willing to do with what we can do currently.

Okay. Great. Thank you. Thanks. Thanks. Thanks.