a16z Podcast - a16z Podcast: VR, AR, and Beyond: The New Medium of Human Experience

Episode Date: January 24, 2017

The building blocks for VR and AR are finally here -- but the content is just beginning. So everything you'll actually experience and consume in these new mediums over the next few years is being built right now. Formats aren't yet defined or locked down, and the field is bubbling up with experiments in forms, formats and genres, from narrative to games to live events. As we begin to have real time rendered characters and AI-driven environments that you can interact with, the storytelling structure will also need to completely change. Are these mediums inherently social -- or just the opposite? What will self expression look like? What experiences are being built? Because fundamentally, that is the "celluloid" we are now working with -- human experience, says Within cofounder and filmmaker Chris Milk. Bigscreen founder and CEO Darshan Shankar, Lytro CEO Jason Rosenthal, and Milk join a16z's Kyle Russell in conversation about the challenges, potential, and emotional power of these new technologies -- on this episode of the a16z Podcast, recorded at the inaugural a16z summit. The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments and certain publicly traded cryptocurrencies/ digital assets for which the issuer has not provided permission for a16z to disclose publicly) is available at https://a16z.com/investments/. Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.

The content here is for informational purposes only, should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security and is not directed at any investors or potential investors in any A16Z fund. For more details, please see A16Z.com slash disclosures. Hi, everyone. Welcome to the A6NZ podcast. I'm Sonal. Today we're kicking off our new series on new medium storytelling with a conversation about VR-A-R, that's virtual reality and augmented reality, and beyond. And what does it mean when the celluloid that we're now working with is actually the human experience? Joining us to have this conversation, we have within co-founder and filmmaker Chris Milk, Big Screen founder and CEO Darshan Shanker. And we have Lytra CEO Jason Rosenthal in

conversation with A16 and Z's Kyle Russell in a conversation that was recorded at our inaugural A6 and Z summit. In this conversation, they cover everything from the challenges and potential of these new technologies to the emotional power and new medium affords. Why are we excited about putting big goggle style computers on our faces? To understand A16C's perspective, I think it's valuable to define terms. I don't want to assume anyone in the room has played with every gadget like I have. So the big buckets to think about in terms of the emerging computing paradigms we're going to be talking about today are augmented reality, which lets you see the world differently, and virtual reality, which lets you see a different world. The former gives you superpowers. You're never

going to go to an event like this and meet someone where you're pretty sure you've seen them before, but you don't remember their name. You're never going to have that awkward exchange because their LinkedIn profile is going to be floating above their head once you've got, you know, glasses-sized AR headsets. Virtual reality, on the other hand, gives you true telepresence. You're going to be able to work and play from what feels like the same environment, but from any distance. It also lets you put yourself into more directly so than any other previous medium into the shoes of someone else. You get to truly see the world from someone else's perspective. And so I'm joined today by three amazing entrepreneurs who are helping to make those things reality.

To start, though, I kind of want to do a little bit of level setting. Darshan, you're a technical founder. You spent a lot of last year building big screen yourself. Why did now feel like the appropriate time to jump into VR as an ecosystem? Well, in 2015, it actually wasn't certain that now was the right time. That was a big question. The question was how good were consumer headsets going to be? How many people would buy them? Would there be a real market for it? And now is the right time because finally, after years of development, both on the hardware and software side, there's actually high-quality consumer VR headsets that are out there, headsets that don't actually make people sick. At this point, it's the software that's making people sick, not the actual hardware itself. So a lot of those, like, low, kind of low-level blocks are just good enough that we can start building great products, great companies that just weren't really possible before just a year or two ago. Right.

And so, Chris, within is already one of the main distribution platforms that our colleagues down in Hollywood are using to reach people with the VR content that they're producing. How is Hollywood thinking about the space? How are they thinking about the investments that they're making their expectations around what success looks like given where the ecosystem is today? Yeah, how are they thinking about it? Well, down in Hollywood, they, we're starting to see a shift from VR pieces that are exclusively funded by marketing budgets because that's all there was to fund them with in the last year to more of the forward thinking studios actually funding content for the purpose of monetizing it. You see just, I think, last week, really the first, the first example of this, which is the Fox is the Martian VR experience, just sort of cinematic VR is sort of a game.

And it's priced at, I think, 1995, right? And we'll see how it does. But that, you know, these are businesses that are used to making content that cost a lot of money. and then selling it and making their money back. That's not entirely possible with the current addressable market, I think, in virtual realities when you're spending millions of dollars on a piece of content. But the studios realize that they need to start exploring to be in a position that they can make really refined high-quality content when the addressable market is actually there.

To follow up a little bit, when I think about my media diet, it's heavily kind of bifurcated between like the blockbusters I go to every couple months in theaters and binging via Netflix, HBO Now, Hulu. You talked about a $20 kind of something that's associated with a blockbuster movie, kind of investment in that app in terms of like my buying behavior. The thing is you could have all the money in the world. You could not start Netflix of the art today because there's no back catalog to license. Every, you know, Netflix starts and they license 100,

something years of cinema and television and, and they have a catalog. Got it. And you pay for it. Access that catalog. That catalog doesn't exist in virtual reality. Everything that you're watching or experiencing is being built currently. Speaking of which, Jason, at Lightro, you're building kind of the next generation of cameras that are going to be used to produce this high-end cinematic storytelling-based content.

How is the experience of VR as a cinematic medium going to evolve over the next? few years? Well, I think nobody knows. I think that's one of the most exciting things about it. I think that if you look at, one of the things that's super interesting about VR is it's kind of the first medium of media where the formats aren't defined and locked in from the very beginning. There's lots and lots of experimentation because it's kind of the first way that people can

interact sort of natively with the world around them that they know, but in a completely virtual environment. But if you had to commit to a most probable future path, how are you kind of envisioning where things are going to go? Yes, I think it's going to look like a lot of other forms that we've seen, right? There's going to be narrative forms of VR. There's going to be inherently game-like experiences. There's going to be live events. And where we are now, and I think Chris and Darshang are doing amazing jobs of running different experiments in this.

nobody yet quite knows what's going to work, so everybody is trying to iterate to the first version of product market fit in those experiences. And so, you know, in order to make VR headsets, you know, provide a killer app, we're excited for gaming, we're excited for Hollywood to produce content, where the entertainment value is just so high compared to what you get from classic cinema that, you know, kind of draws people to the platforms. I would question the premise about it. So I think if you look at any new medium, right, be at the Internet or

mobile games or what have you. It's never been the companies that were dominant in the last form of entertainment or media that have become the leaders in the next form. And I think for sure that's going to happen here. I think Hollywood has some interesting assets and that they've got, you know, known IP, which people love. And so I think there will be great VR experiences created around Star Wars and Harry Potter and other things, but it's not going to be those companies that own that IP necessarily doing that.

Got it. And where I was going with that, though, was, you know, this is all around kind of industries producing content. However, you know, you look at the web, you look at mobile, kind of previous computing platforms, social media became dominant very quickly. You know, power of network effects. People like to communicate with the friends. They like to share things about their lives. Darshan, how do you think self-expression is going to work in VR? Like, are we going to be just sharing in the same way that we share photos and videos?

captured from our phones today, you know, flat media? Are we just going to be doing that with 360 videos as our phones get that capability? Or are there kind of other avenues of self-expression that VR opens up? It's hard to say, probably all of the above, in the sense that 360 content, once it becomes extremely easy to film, extremely easy to capture for every consumer, the way we capture with our phones, any sort of photos, that becomes really easy to share. But first, you have to make it really easy to capture, make it really easy to distribute. you have to solve some of those low-line problems first.

But that whole range, like for big screen, it's a lot more about sharing your 2D content. So you might be working on something and you might be able to share it with a friend. It doesn't require 360 capture technology to catch up and go mainstream. But in terms of user-generated content, allowing people to create,

kind of like what Minecraft has accomplished in the past decade, allowing people to create their own virtual worlds and explore it with their friends, we might be a little early for that. In terms of user-generated social media-type functionality, We, again, might be too early for that. It's hard to predict when it'll be ready for that.

But if you look at the history of the Internet, the web, and mobile, a lot of bundling was occurring in the early days where it wasn't completely user generated. It wasn't GeoCities super early on. It was a lot of, it was AOL. It was kind of bundled news before everybody could make their own website. It was a few, it was like major news websites that you would visit first before you started making your own. before WordPress or something. It's hard to say. It might be a little early,

but even today, you can create some pretty cool stuff and share it with your friends. Got it. And so one of the common concerns that I hear a lot when I'm trying to talk about how amazing VR is going to be

as it takes off is, yeah, but isn't it really isolating? You know, you're wearing something that's literally, in the case of VR, AR, maybe not so much, but with VR in particular, you're wearing something that's literally blocking your view

of the rest of the world around you, you know? And so people have, you know, understandable concerns, like, are we all going to be in our own kind of escapist fantasies, not interacting with each other? From a storyteller's perspective, you know, cinema, we go to the movie theater together. We sit on the couch and watch Netflix or catch up on Game of Thrones with our friends. Jason, how do you think about cinematic and social kind of coalescing in VR? I mean, to me, I think that the, I think the bowl case is that VR. has the potential to be the inherently most social medium

kind of of any form of technology that we've seen so far. Even with the goggles? Because I think imagine if you had the power or two, instead of just going to the movies with your friends, like what if you could actually be in the movies with your friends? And right, you could each give each other unique superpowers and interact with an virtual environment. And so I think that if you look at where we are,

I mean, we sort of think that media is social today, right? Because we can watch an NBA game or a football game and have a tweet stream either on the side of it or on our phones. But imagine if you could actually be immersed in the whole experience and rather than needing a text-based metadata set of social interactions if you could be physically immersed in the environment. And I think actually if you look at what Darshan's doing with big screen, right? I mean, you can explain your product rather than me,

but it is like it is the most fun way to play video games with your friends doing it in a VR environment versus the way we do it today. So really quickly, because I want to keep on the social aspect a little bit, Darchant, could you maybe explain what Big Screen does for its users? Sure. So Big Screen puts you into a virtual world. You can invite your friends or your colleagues into it. You'll see your computer in front of you, all of your existing apps, whether it's video games or Netflix or whatever you have on your computer screen. And when you look over, you'll see your friends. They can be thousands of miles away, but you'll see them, you'll see an avatar of them. It's like being in a living room or a conference room together with people that aren't there physically. with you. That's the social collaborative power of VR being applied to our day-to-day lives that

we already do today. And so kind of expanding on that, I have a recommendation for everyone in the audience of a video you should check out. On YouTube later, find time to search for Microsoft Research Hallportation. There's a demo from Microsoft Research where basically they surround a room with cameras and death sensors and scan people in and are able to then kind of to teleport their holograms to different environments. And so you get this kind of social experience, but it looks like they're in the room with you rather than making you put on the headset that completely blocks out the real world. And so I mentioned that to transition a little bit to AR because, again, something that we hear a lot, even from people like Tim Cook at Apple is, you know,

they're more excited about augmented reality than virtual reality because it doesn't have that blocking out the real world. You actually get to make eye contact with people. With that said, AR is a couple of years out, even relative to VR in terms of difficulties around optics, in terms of computer vision. So as you're building virtual reality products and experiences today, how are you thinking about AR? What lessons that you're learning from virtual reality are going to apply to augmented reality as that kind of emerges, maybe becomes more dominant for certain use cases?

What transfers over and where do you have to start with fresh eyes? I think fundamentally what you're talking about is a medium that is human experience. Like that, that, that is the celluloid. It is actual human experience that you perceive as human experience. So, AR is human experience in the place that you are. VR is just human experience and any, any place that you could be transported. And I think what you're going to see in the long term, especially with virtual reality, is the sort of democratization of human experience in the same way that the Internet led to the democratization of information. So you can, I mean, it sounds like hype, but.

We're in the early stages of right now where you can feel like you are anywhere. And because the tech industry, we love to come up with new acronyms and more jargon to throw at people. There's a couple of other terms that people are kind of talking about a lot in the space. Mixed reality, the idea of kind of merging virtual objects and augmented experiences together, kind of a nebulous blend. Intel talks about merged reality. So virtual reality experiences where it's scanning in the room, and so it puts, you know, if I was wearing a headset, it would project virtual avatars onto all of you so that I still feel like I'm kind of present here, even though I'm in a virtual world.

Do you spend much time thinking about these kinds of blended experiences? Or, you know, how do you even think about, like, the relative mix that makes sense to go with as you're building things? I mean, I think your point is right, right, which is that the way it'll likely play out is that there's going to be a broad spectrum, right, from a purely synthetic virtual world like The Matrix all the way up to, you know, the point that you open with of us all having our Facebook and LinkedIn profiles floating above our heads in meetings. And I think that the, I mean, the good thing is that we're sort of in the stage of the medium now where everybody's working on the underlying infrastructure technologies and, and kind of the first experiences around those. And it doesn't matter, you know, if it's AR or VR, the investments that you need to make right now and the opportunities that that creates, I think, are exactly the same. And what will happen, I think, is that as both mediums mature,

we'll start to see them diverge and see new applications that may work better in one versus the other and then others that work well across both. Right. And incorporating even more mediums, you know, today, virtual reality is kind of growing out of gaming with things like what Within is working on. You know, film is contributing to a lot of the content that's available.

In terms of, I guess the correct term would maybe be asymmetric experiences. So people who are in virtual reality, you know, some of the people involved in the experience, but other people are on their phones also engaging with them. When does it make sense to build something asymmetric and when does it make sense to build

something where, you know, it's VR-A-R from the outset and you don't compromise on that. You know, I think that's like kind of a philosophical debate. There's a question I was just asking you. Yeah, I know. And so I think that was a fun discussion. So I'd love to kind of rehash it on the fit maybe.

I mean, personally, I think you should build things that are the most inclusive to the most amount of people that they can have compelling experiences with it. And when you get into the finer details, it really depends on the experiences that you're building. I think that there's people are looking at this medium right now as it's either an evolution of cinema or it's an evolution of video games. At first, it was an evolution of video games. And I think that the reason that was is because the people making it were all video gamers. It was like what Mark was talking about at the opening yesterday with the

Edison thought that the phonograph would be used for sermon because that was his world. That's what he that, like that was his reality. And the people that were building virtual reality were gamers. So everything was, it's going to be about games. And then slowly we started to see cinematic VR bubble up in the last few years. And then, is like, okay, well, maybe it's the evolution of movies. And it's actually, I don't think it's either of those because I think it's its own new medium in its own right.

It will take things from each of them and it will evolve, like what Jason was saying, that the format is undefined. That's a really important thing to understand about this medium. It makes you actually different than other mediums is if you look at cinema, for instance, you have the format is birthed at the

birth of the technology. You have a sequence of rectangles, played one after another. That's birth with the motion picture camera, the motion picture projector. That format does not change over the lifespan of the media. The language of storytelling in the form of a feature film or television show or digital video or UGC, that does evolve. That does change. But the format always stays the same. What's different about virtual reality is the format is fluid. And you see it changing on a bi-monthly basis where you're now able to tell a new kind of story because the actual format has

change. To ground that a little bit, what kinds of specific changes unlock new types of experiences? Like what kinds of improvements to the underlying platforms kind of drive those new capabilities? Well, it's really, it's about levels of immersion and it's about your interactivity with the world around you. So right now, you have a real-time rendered world that does not look real that you can interact with or you have a photographed experience that is photorealistic. and that people look human, and you can, at the moment, you can't walk around the room. What Jason is building, you will be able to walk around the room. But you can't interact with the person that's been shot.

Brad Pitt was there. He was photographed. You can't walk up to Brad Pitt and start talking to him, and he'll talk back to you. You imagine, on the real-time rendered side, that's going to get faster. You're going to have real-time rendered humans and environments that are driven by AI that you can interact with. And at each stage of this, the storytelling structure also has to completely change. Eventually, it just all culminates to Westworld. Pretty much.

And so those of us in the room are privileged in that getting to try virtual and even some AR virtual reality and even some AR experiences isn't completely out of reach just based on cost alone. Most of us can probably get access to one of these headsets if we really tried. And so I'd love to kind of just go through one by one. What's one experience that you would recommend the device they would need and the specific experience that you think everyone should try to understand maybe a specific facet of where these spaces are going? I know that's a tricky one. So whoever kind of maybe has an answer, top of mind can jump in first. I'd actually probably say, so Oculus made a film called Henry, and that was the first time where they have you like sit down on the floor.

and they have this experience that's like happening all around you. And it was like an interesting storytelling experience. And that was my personal first time where I had tried a what film and VR could really be like native to VR, taking advantage of VR. High-end VR headsets like the Oculus Rift. That's where that's, if you want to try Henry, try it on the Oculus Rift. That took advantage of being able to actually stand up, walk around a little bit and to really get a sense of being in a different place with a character. that actually the animated character makes eye contact with you. And I've heard anecdotally, people would take off the headset and they'd have a tear

because it was like, oh, that was a cute story. It was kind of a sad story. That was my first kind of a wow experience. Got it. If no one has one prepared yet, I'll hop in with my recommendation. So I think an experience definitely worth trying is destinations. an app from Valve on the HTC Vive. And so what Destinations does is it lets you basically upload entire environments that other

people can explore. And you can add things like stand at these specific points and overlay will show up giving context about the scene you're in. There's one experience where they actually took thousands of photos from a Mars rover and used a technique called photogrammetry, which extracts the geometry from those photos to make a rendered scene. So you actually get to stand on Mars. And so you are kind of walking around and it gives you little blips of information about, oh, this is the rover, this is how long its mission was,

this is how long it took to collect all these photos. And all I can think is this is the future of museums in the near term and the further term. This is the future of education. Like why, you know, why read a textbook about, you know, significant moments in history when you could just kind of actually get to live it? I think that that's absolutely worth the tribe can get your hands on it. Chris is too modest, but like if you haven't seen his film Clouds Over Sidra, which is about the Syrian refugee crisis, I think you can watch here, right, in the Innovators Billion. Like, I mean, it's a, it's the most human way to connect someone to what's going on in

another part of the world, and you watch it and you feel like you're there and you have a level of empathy for what's going on that you just don't get from reading the newspaper or watching YouTube or anything like that. I love Pearl, which is, you should see on a Vibe, which is Google, what are they called, story studios? That's Oculus. Google Daydream Labs? No. Anyway, no.

Story, it's story something. Pearl. It's just a beautiful little animated story that you can lean and move around in. Actually, not move, but you're in the front seat of a car, and the whole story takes place with then the framing of this car between this father and daughter and it's music driven. And it's just, it's really utilizing the limitations of what's possible now to, to form something really beautiful and human. I'll tell you just a personal story that's interesting that I haven't really shared publicly,

which is, it's an experience that you can't actually have, but I would imagine you will be able to have in the coming years of the technology gets better. And it's very hard to capture video in VR right now. Jason's working on, you know, a super robust. professional version. In the last three years, we've had to build our own camera systems to film the things that we want to film because there's no cameras. The cameras didn't exist. And I filmed something. We were, this was like two and a half years ago. I shot this soccer game in Los Angeles and we're doing a test and I was there with our camera tech. And I

was there with my girlfriend at the time. And it went onto a drive somewhere. We looked at some other footage. And at the time, we were testing how close you could get and where you could put camera positions. And about a year later, someone found some of the footage on the drive that I had not seen and stitched it. And post that moment in time, I'd gone through a really difficult breakup with my girlfriend and said, hey, we found this thing on the drive. You should watch it. I put this headset on. And I was standing next to myself holding the hand of the girl that I was no longer speaking to. And I have never in my life experienced a more powerful piece of media than that moment.

No movie, no anything, more powerful than any voicemail than any photograph than any video. It was, I was with, it was like I was, you know, Ebeneger Scrooge and Christmas past. Like I was standing there with my old self and I felt the presence of my old self and I felt the emotions of myself with my girlfriend at the time. And UGC is very difficult at the moment, but it will be, it will be, it will be, it will transform an industry of consumers capturing memories and replaying them. Well, thank you for sharing that anecdote. I know that that might have just brought those memories up again. And thank you all for joining. To get a taste of the kinds of experiences we were talking about, there's a within booth with a couple gear VR headsets up at the Innovators Pavilion.

Thank you all for your time. Thank you.