The a16z Show - a16z Podcast: VR, AR, and Beyond: The New Medium of Human Experience

Starting point is 00:00:00 The content here is for informational purposes only, should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security and is not directed at any investors or potential investors in any A16Z fund. For more details, please see A16Z.com slash disclosures. Hi, everyone. Welcome to the A6 and Z podcast. I'm Sonal. Today we're kicking off our new series on new medium storytelling with a conversation about VR, AR, that's virtual reality and augmented reality and beyond. And what does it mean when the celluloid that we're now working with is actually the human experience? Joining us to have this conversation, we have within co-founder and filmmaker Chris Milk, big screen founder and CEO Darshan Shanker. And we have Lytra CEO Jason Rosenthal in conversation

Starting point is 00:00:51 with A16 and Z's Kyle Russell in a conversation that was recorded at our inaugural A6 and Z summit. In this conversation, they cover everything from the challenges and potential of these new technologies to the emotional power and new medium affords. Why are we excited about putting big goggle style computers on our faces? To understand A16Z's perspective, I think it's valuable to define terms. I don't want to assume anyone in the room has played with every gadget like I have. So the big buckets to think about in terms of the emerging computing paradigms we're going to be talking about today are augmented reality, which lets you see the world differently, and virtual reality, which lets you see a different world.

Starting point is 00:01:29 The former gives you superpowers. You're never going to go to an event like this and meet someone where you're pretty sure you've seen them before, but you don't remember their name. You're never going to have that awkward exchange because their LinkedIn profile is going to be floating above their head once you've got glasses-sized AR headsets. Virtual reality, on the other hand, gives you true telepresence. You're going to be able to work and play from what feels like the same environment, but from any distance. It also lets you put yourself into more directly so than any other previous medium into the shoes of someone else. You get to truly see the world from someone else's perspective. And so I'm joined today by three amazing entrepreneurs who are helping to make those things reality.

Starting point is 00:02:13 To start, though, I kind of want to do a little bit of level setting. Dershahn, you're a technical founder. You spent a lot of last year building big screen yourself. Why did now feel like the appropriate time to jump into VR as an ecosystem? Well, in 2015, it actually wasn't certain that now was the right time. That was a big question. The question was how good were consumer headsets going to be? How many people would buy them?

Starting point is 00:02:35 Would there be a real market for it? And now is the right time because finally, after years of development, both on the hardware and software side, there's actually high-quality consumer VR headsets that are out there, headsets that don't actually make people sick. At this point, it's the software that's making people sick, not the actual hardware itself. So a lot of those like low kind of low level blocks are just good enough that we can start

Starting point is 00:03:01 building great products, great companies that just weren't really possible before just a year or two ago. Right. And so Chris, within is already one of the main distribution platforms that our colleagues down in Hollywood are using to reach people with the VR content that they're producing. How is Hollywood thinking about the space? How are they thinking about the investments that they're making their expectations around what success looks like, given where the ecosystem is today.

Starting point is 00:03:27 How are they thinking about it? Well, down in Hollywood, they, we're starting to see a shift from VR pieces that are exclusively funded by marketing budgets, because that's all there was to fund them with in the last year, to more of the forward-thinking studios actually funding content for the purpose of monetizing it.

Starting point is 00:03:52 You see just, I think, last week, really the first example of this, which is Fox's The Martian VR experience, just sort of cinematic VR is sort of a game. And it's priced at, I think, 1995, right? And we'll see how it does. But that, you know, these are businesses that are used to making content that cost a lot of money. and then selling it and making their money back. That's not entirely possible with the current addressable market, I think, in virtual realities when you're spending millions of dollars on a piece of content. But the studios realize that they need to start exploring to be in a position that they can make

Starting point is 00:04:43 really refined high-quality content when the addressable market is actually there. To follow up a little bit, when I think about my media diet, it's heavily kind of bifurcated between like the blockbusters I go to every couple months in theaters and binging via Netflix, HBO Now, Hulu. You talked about a $20 kind of something that's associated with a blockbuster movie, kind of investment in that app in terms of like my buying behavior. The thing is you could have all the money in the world. You could not start Netflix of the art today because there's no back catalog to license.

Starting point is 00:05:16 Every, you know, Netflix starts and they license 100 something. years of cinema and television and and they and they have a catalog got it and you pay for it access that catalog that catalog doesn't exist in virtual reality everything that you're watching or experiencing is being built currently speaking of which jason at lightro you're building kind of the next generation of cameras that are going to be used to produce this high-end cinematic storytelling based content how is the experience of VR as a cinematic medium going to evolve over the next few years? Well, I think nobody knows.

Starting point is 00:05:52 I think that's one of the most exciting things about it. I think that if you look at, one of the things that's super interesting about VR is it's kind of the first medium of media where the formats aren't defined and locked in from the very beginning. There's lots and lots of experimentation because it's kind of the first way that people can interact sort of natively with the world around them that they know, but in a completely virtual environment. But if you had to commit to a most probable future path, how are you kind of envisioning where things are going to go?

Starting point is 00:06:24 Yes, I think it's going to look like a lot of other forms that we've seen, right? There's going to be narrative forms of VR. There's going to be inherently game-like experiences. There's going to be live events. And where we are now, and I think Chris and Darshang are doing amazing jobs of running different experiments in this. Nobody yet quite knows what's going to work. everybody is trying to iterate to the first version of product market fit for the experiences. And so, you know, in order to make VR headsets, you know, provide a killer app,

Starting point is 00:06:59 we're excited for gaming, we're excited for Hollywood to produce content, where the entertainment value is just so high compared to what you get from classic cinema that, you know, kind of draws people to the platforms. I would question the premise about it. So I think if you look at any new medium, right, be it the internet or mobile games or what have you, it's never been the companies that were dominant in the last form of entertainment or media that have become the leaders in the next form. And I think for sure that's going to happen here.

Starting point is 00:07:27 I think Hollywood has some interesting assets and that they've got, you know, known IP, which people love. And so I think there will be great VR experiences created around Star Wars and Harry Potter and other things, but it's not going to be those companies that own that IP necessarily doing that. Got it. And where I was going with that, though, was, you know, this is all around kind of industries producing content. However, you know, you look at the web, you look at mobile, kind of previous computing platforms. Social media became dominant very quickly.

Starting point is 00:08:00 You know, power of network effects. People like to communicate with the friends. They like to share things about their lives. Darsh, how do you think self-expression is going to work in VR? Like, are we going to be just sharing in the same way that? we share photos and videos captured from our phones today, you know, flat media? Are we just going to be doing that with 360 videos as our phones get that capability? Or are there kind of other avenues of self-expression that VR opens up?

Starting point is 00:08:25 It's hard to say, probably all of the above, in the sense that 360 content, once it becomes extremely easy to film, extremely easy to capture for every consumer, the way we capture with our phones, any sort of photos, that becomes really easy to share. But first, you have to make it really easy to capture, make it really easy to distinguish. You have to solve some of those low-line problems first. But that whole range, like for big screen, it's a lot more about sharing your 2D content. So you might be working on something and you might be able to share it with a friend. It doesn't require 360 capture technology to catch up and go mainstream.

Starting point is 00:08:59 But in terms of like user-generated content, allowing people to create, kind of like what Minecraft has accomplished in the past decade, allowing people to create their own virtual worlds and explore it with their friends, we might be a little early for that. that. In terms of user-generated social media type functionality, we again might be too early for that. It's hard to predict when it'll be ready for that. But if you look at the history of the internet, the web, and mobile, a lot of bundling was occurring in the early days where it wasn't completely user-generated. It wasn't GeoCities super early on. It was a lot of, it was AOL. It was kind of bundled news before everybody could make their own website. It was few, it was like major news websites that you would visit first before you started making your

Starting point is 00:09:43 own before WordPress or something. So hard to say, it might be a little early, but even today, you can create some pretty cool stuff and share it with your friends. Got it. And so one of the common concerns that I hear a lot when I'm trying to, you know, talk about how amazing VR is going to be as it takes off is, yeah, but isn't it really isolating? You know, you're wearing something that's literally, in the case of VR, AR, maybe not so much, but with VR in particular, you're writing something that's literally blocking your view of the rest of the world around you, you know? And so people have, you know, understandable concerns, like, are we all going to be in our own kind of escapist fantasies, not interacting with each other?

Starting point is 00:10:21 From a storyteller's perspective, you know, cinema, we go to the movie theater together. We sit on the couch and watch Netflix or catch up on Game of Thrones with our friends. Jason, how do you think about cinematic? and social kind of coalescing in VR. I mean, to me, I think that the, I think the bold case is that VR has the potential to be the inherently most social medium kind of of any form of technology that we've seen so far.

Starting point is 00:10:50 Even with the goggles? Because I think imagine, you know, if you had the power or two, instead of just going to the movies with your friends, like what if you could actually be in the movies with your friends? And right, you could each give each other, you know, unique superpowers and interact with an virtual environment. And so I think that if you look at where we're,

Starting point is 00:11:06 we are, I mean, we sort of think that media is social today, right, because we can watch an NBA game or a football game and, like, have a tweet stream either, you know, on the side of it or on our phones. But imagine if you could actually be immersed in the whole experience, and rather than, you know, needing a text-based metadata set of social interactions, if you could be physically immersed in the environment. And I think actually, you know, you look at what Darsham's dealing with big screen, right? I mean, you can explain your product, but it is like, it is the most fun way to play video games with your friends, doing it in a VR environment versus the way we do it today. Yeah, so really quickly, because I want to keep on the social aspect a little bit.

Starting point is 00:11:46 Darchan, could you maybe explain what Big Screen does for its users? Sure. So Big Screen puts you into a virtual world. You can invite your friends or your colleagues into it. You'll see your computer in front of you, all of your existing apps, whether it's video games or Netflix or whatever you have on your computer screen. And when you look over, you'll see your friends. They can be thousands of miles away, but you'll see them, you'll see an avatar of them. It's like being in a living room or a conference room together with people that aren't there physically with you. That's the social collaborative power of VR being applied to our day-to-day lives that we already do today. And so kind of expanding on that,

Starting point is 00:12:23 I have a recommendation for everyone in the audience of a video you should check out. On YouTube, later, find time to search for Microsoft Research Hall Portation. There's a demo from Microsoft of research where basically they surround a room with cameras and death sensors and scan people in and are able to then kind of teleport their holograms to different environments. And so you get this kind of social experience, but it looks like they're in the room with you rather than making you put on the headset that completely blocks out the real world. And so I mentioned that to transition a little bit to AR because, again, something that we hear a lot, even from people like Tim Cook at Apple is, you know, they're more excited about augmented reality.

Starting point is 00:13:05 than virtual reality because it doesn't have that blocking out the real world. You actually get to make eye contact with people. With that said, AR is a couple of years out even relative to VR in terms of difficulties around optics, in terms of computer vision. So as you're building virtual reality products and experiences today, how are you thinking about AR? What lessons that you're learning from virtual reality are going to apply to augmented reality as that kind of emerges, maybe becomes more dominant for certain use cases? is what transfers over and where do you have to start with fresh eyes?

Starting point is 00:13:37 I think fundamentally what you're talking about is a medium that is is human experience. Like that is that is the cellulite. It is actual human experience that you perceive as human experience. So AR is human experience in the place that you are. VR is just human experience in any place that you can be transported. And I think what you're going to see in the long term, especially with virtual reality, is the sort of democratization of human experience in the same way that the internet led to the democratization of information. So you can, I mean, it sounds like hype, but we're in the early stages of it

Starting point is 00:14:14 right now where you can feel like you are anywhere. And because the tech industry, we love to come up with new acronyms and more jargon to throw at people. There's a couple of other terms that people are kind of talking about a lot in the space. Mixed reality, the idea of kind of merging virtual objects and augmented experiences together, kind of a nebulous blend. Intel talks about, yeah, merged reality. So virtual reality experiences where it's scanning in the room. And so it, you know, puts, you know, if I was wearing a headset, it would project virtual avatars onto all of you so that I still feel like I'm kind of present here, even though I'm in a virtual world. Do you spend much time thinking about these kinds of blended experiences? Or, you know,

Starting point is 00:15:00 how do you even think about the relative mix that makes sense to go with as you're building things? I mean, I think your point is right, right, which is that the way it'll likely play out is that there's going to be a broad spectrum, right, from a purely synthetic virtual world like the Matrix all the way up to, you know, the point that you open with of us all having our Facebook and LinkedIn profiles floating above our heads in meetings. And I think that the, I mean, the good thing is that we're sort of in the state. of the medium now where everybody's working on the underlying infrastructure technologies and kind of the first experiences around those.

Starting point is 00:15:39 And it doesn't matter if it's AR or VR. The investments that you need to make right now and the opportunities that that creates I think are exactly the same. And what will happen, I think, is that as both mediums mature, we'll start to see them diverge and see new applications that may work better than one versus the other and then others that work well across the... Right, and incorporating even more mediums. Today, virtual reality is kind of growing out of gaming

Starting point is 00:16:08 with things like what Within is working on. You know, film is contributing to a lot of the content that's available. In terms of, I guess the correct term would maybe be asymmetric experiences. So people who are in virtual reality, you know, some of the people involved in the experience with other people are on their phones, also engaging with them. when does it make sense to build something asymmetric and when does it make sense to build something

Starting point is 00:16:33 where it's VR AR from the outset and you don't compromise on that. I think that's like kind of a philosophical debate that we're all having. I was just asking you. Yeah, I know. And so I think that was a fun discussion. So I'd love to kind of rehash on the fit maybe. I mean, personally, I think you should build things that are the most inclusive

Starting point is 00:16:53 to the most amount of people that they can have compelling experiences with that. And when you get into the finer details, it really depends on the experiences that you're building. I think that there's, people are looking at this medium right now as it's either an evolution of cinema or it's an evolution of video games. At first, it was an evolution of video games. And I think that the reason that was was because the people making it were all video gamers.

Starting point is 00:17:15 It was like what Mark was talking about at the opening yesterday with the Edison thought that the phonograph would be used for sermon because that was his world. That's what he, that was his reality. And the people that were building virtual reality were gamers. So everything is going to be about games. And then slowly we started to see cinematic VR bubble up in the last few years. And then it's like, okay, well, maybe it's the evolution of movies. And it's actually, I don't think it's either of those because I think it's its own new medium and its own right.

Starting point is 00:17:43 It will take things from each of them and it will evolve, like what Jason was saying, that the format is undefined. That's a really important thing to understand about this medium. It makes it actually different than other mediums is if you look at cinema, for instance, you have the format is birth, at the birth of the technology. You have a sequence of rectangles played one after another. That's birth with the motion picture camera, the motion picture projector. That format does not change over the lifespan of the medium. The language of storytelling in the form of a feature film or television show or digital

Starting point is 00:18:15 video or UGC, that does evolve. That does change, but the format always stays the same. What's different about virtual reality is the format is fluid. And you see it changing on a bi-monthly basis. basis where you're now able to tell a new kind of story because the actual format has changed. To ground that a little bit, what kinds of specific changes unlock new types of experiences? Like what kinds of improvements to the underlying platforms kind of drive those new capabilities? Well, it's really, it's about levels of immersion and it's about your interactivity with the world around you.

Starting point is 00:18:51 So right now you have a real-time rendered world that does not look real that you can interact with or you have a a photographed experience that is photorealistic and that people look human. And you can, at the moment, you can't walk around the room. What Jason is building, you will be able to walk around the room. But you can't interact with the person that's been shot. Brad Pitt was there. He was photographed. You can't walk up to Brad Pitt and start talking to him and he'll talk back to you.

Starting point is 00:19:22 You imagine on the real-time render side, that's going to get faster. you're going to have real-time rendered humans and environments that are driven by AI that you can interact with. And at each stage of this, the storytelling structure also has to completely change. Eventually, it just all culminates to Westworld. Pretty much. And so those of us in the room are privileged in that getting to try virtual and even some AR virtual reality and even some AR experiences isn't completely out of reach just based on cost alone. Most of us can probably get access to one of these headsets if we really tried. And so I'd love to kind of just go through one by one.

Starting point is 00:20:01 What's one experience that you would recommend the device they would need and the specific experience that you think everyone should try to understand maybe a specific facet of where these spaces are going? I know that's a tricky one. So whoever kind of maybe has an answer, top of mind can jump in first. I'd actually probably say, So Oculus made a film called Henry. And that was the first time where they have you like sit down on the floor.

Starting point is 00:20:32 And they have this experience that's like happening all around you. And it was like an interesting storytelling experience. And that was my personal first time where I had tried a what film in VR could really be like native to VR, taking advantage of VR. High-end VR headsets like the Oculus Rift. That's where that's, if you want to try Henry, try it on the Oculus Rift. That took advantage of being able to actually stand up, walk around a little bit, and to really get a sense of being in a different place with a character that actually, the animated character makes eye contact with you.

Starting point is 00:21:05 And I've heard anecdotally, people would take off the headset and they'd have a tear because it was like, oh, that was a cute story. It was kind of a sad story. That was my first kind of, oh, wow, experience. Got it. If no one has one prepared yet, I'll hop in with my recommendation. So I think an experience definitely worth trying is Destinations, an app from Valve on the HTC Vive. And so what Destinations does is it lets you basically upload entire environments that other people can explore.

Starting point is 00:21:42 And you can add things like stand at these specific points and an overlay will show up giving context about the senior end. There's one experience where they actually took thousands of photos from a Mars rover and used a technique called photogrammetry, which extracts the geometry from those photos to make a rendered scene. And so you actually get to stand on Mars. And so you are kind of walking around, and it gives you little blips of information about, oh, this is the rover, this is how long its mission was,

Starting point is 00:22:11 this is how long it took to collect all these photos. And all I can think is this is the future of museums in the nearer term and the further term, this is the future of education. Like, why, you know, why read a textbook about, you know, significant moments in history when you could just kind of actually get to live it? I think that that's absolutely worth a try if you can get your hands on it. Chris is too modest, but, like, if you haven't seen his film, Clouds Over Sidro, which is about the Syrian refugee crisis, I think you can watch here, right, in the

Starting point is 00:22:40 Inveagers' Pavilion. Like, I mean, it's a, it's the most human way to connect someone to what's going on in another part of the world and you watch it and you feel like you're there and you have a level of empathy for what's going on that you just don't get from reading the newspaper or watching YouTube or anything like that. I love Pearl, which is you should see on a vibe, which is Google, what are they called, Story Studios? That's Oculus. Google Daydream Labs? No. Anyway, no. Story. It's story something. Pearl. It's just a beautiful little animated story that you can lean and move around in. Actually,

Starting point is 00:23:24 not move, but you're in the front seat of a car. And the whole story takes place within the framing of this car between this father and daughter and it's music driven. And it's just, it's really utilizing the limitations of what's possible now to, to form something really beautiful and human. I'll tell you just a personal story that's interesting that I haven't really shared publicly, which is it's an experience that you can't actually have. but I would imagine you will be able to have in the coming years the technology gets better. And it's very hard to capture video and VR right now. Jason's working on a super robust professional version. In the last three years, we've had to build our own camera systems

Starting point is 00:24:03 to film the things that we want to film because there's no cameras. The cameras didn't exist. And I filmed something. This was like two and a half years ago. I shot this soccer game in Los Angeles. and we were doing a test, and I was there with our camera tech, and I was there with my girlfriend at the time. And it went onto a drive somewhere, and we looked at some other footage,

Starting point is 00:24:27 and at the time, we were testing how close you could get and where you could put camera positions. And about a year later, someone found some of the footage on the drive that I had not seen and stitched it. And post that moment in time, I'd gone through a really difficult breakup with my girlfriend, and said, hey, we found this thing on the drive. You should watch it. I put this headset on.

Starting point is 00:24:49 And I was standing next to myself holding the hand of the girl that I was no longer speaking to. And I have never in my life experienced a more powerful piece of media than that moment. No movie, no anything. More powerful than any voicemail than any photograph than any video. So it was, I was with, it was like I was, you know, Ebonyger Scrooge and Christmas past. Like I was standing there with my old self. And I felt the presence of my old self. And I felt the emotions of myself with my girlfriend at the time.

Starting point is 00:25:27 And UGC is very difficult at the moment. But it will be able, it will be, it will be, it will transform an industry of consumers capturing memories. Right. And replaying them. Well, well, thank you for sharing that. that anecdote. I know that that might have just brought that those memories up again. And thank you all for joining. To get a taste of the kinds of experiences we were talking about, there's a within booth with a couple gear VR headsets up at the innovators

Starting point is 00:25:54 pavilion. Thank you all for your time.

The a16z Show - a16z Podcast: VR, AR, and Beyond: The New Medium of Human Experience

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.