This Week in Startups - Building the future of VFX and video editing with Runway CEO Cristóbal Valenzuela | E1724

Episode Date: April 19, 2023

Runway CEO Cristóbal Valenzuela joins Jason to discuss their suite of generative AI video editing tools (3:03), Runway's AI Film Festival and the rapid evolution of filmmaking (22:21), Runway's ability to lower the barrier to entry for aspiring filmmakers (36:05), and more! (0:00) Jason kicks off the show (3:03) What is Runway?  (6:39) Runway's text-to-video demo (11:08) Clumio - Start a free backup, or sign up for a demo at https://clumio.com/twist (12:08) Latent Diffusion paper (16:30) The Late Show With Stephen Colbert (20:51) Squarespace - Use offer code TWIST to save 10% off your first purchase of a website or domain at https://Squarespace.com/TWIST (22:21) Runway's AI Film Festival (29:08) The rapid evolution of filmmaking (31:59) Favorite directors and films  (36:05) Lowering the barrier of entry to filmmaking (41:20) Microsoft for Startups Founders Hub - Apply in 5 minutes for six figures in discounts at http://aka.ms/thisweekinstartups (42:51) Driving costs of filmmaking to zero  (44:57) Training Runway's data set & Fan Fiction  (52:07) Runway's generative capabilities  (54:30) Safety concerns  (58:20) The challenges of operating Runway FOLLOW Cristóbal: https://twitter.com/c_valenzuelab FOLLOW Jason: https://linktr.ee/calacanis Subscribe to our YouTube to watch all full episodes: https://www.youtube.com/channel/UCkkhmBWfS7pILYIk0izkc3A?sub_confirmation=1 FOUNDERS! Subscribe to the Founder University podcast: https://podcasts.apple.com/au/podcast/founder-university/id1648407190

Hey, everybody. We're back with another awesome AI interview as part of our Innovators in AI series. That's what we're calling it here. We've never seen technology move as fast as artificial intelligence is currently moving. This is not hyperbolic. This is reality. Every week, every 48 hours, somebody builds on the work of an open source project or another company or by doing prompts or by connecting things together. So we're going to dedicate a large percentage of the show to AI this year or until this thing slows down. So today I'm joined by the co-founder of a company called Runway. Runway did some of the earliest research that eventually led to the creation of stable diffusion. That's the project that takes text and turns it into an image, right? You type in, I'd like to make a painting in a Renaissance style of Donald Trump in a courtroom and then all of a sudden you start getting really amazing output. So the language that we're going to use to program going forward is going to be called English.

So the CEO of Runway is going to walk us through. And that's really what I'm trying to focus here on the show in 2023, actually using the tools and seeing them in action. So you don't just get to hear the opinion of some podcaster who's not using the tools. I'm using the tools. My producers are using the tools. And the people on the podcast are building the tools. You're going to actually see them at work. This is why you come here to this week in startups just to get inside information.

And what runway is doing is merging generative AI with visual effects, VFX. So the visual effects team, that won best picture for everything, everywhere all at once, a really dope film. They used runway to help build the images that you saw in that best picture winner. Also, Leachaudet Show with Stephen Colbert. They're also using this tool runway. And many other brands are doing that. So it's an amazing interview. The founder is really brilliant.

Shares a ton of practical examples in demos. You're going to love this one. Stick with us. This week in startups is brought to you by Clumio. When you're building a company, don't let backups and compliance requirements distract you. Let the data protection experts at Clumio help with immutable air gap backups

that put compliance on autopilot. Visit them at clumio.com slash twist to start a free backup or sign up for a demo. Squarespace. Turn your idea into a new website. Go to Squarespace.com slash twist for a free try. When you're ready to launch, use OfferCode Twist to save 10% off your first purchase of a website or domain.

And the Microsoft for Startups Founders Hub helps all founders build a better startup at a lower cost from day one. Startups get up to $150,000 in Azure credits, access to free Open AI credits, free dev tools like GitHub, technical advisory, access to mentors and experts, and so much more. much more. There is no funding requirement, and it only takes minutes to join. Sign up today at a.k.a.m.m.S. This week in startups. All right, everybody, this AI stuff is moving faster and faster. So here at this week in startups, we're going to have somebody on once or twice a week who's building something interesting in AI. So consider this week in startups, this week in AI for the next couple of months or until this crazy, crazy pace slows down. Today I'm joined by Chris Valenzuela. He is the CEO and co-founder of a company called Runway. It's an AI browser-based creative suite.

You can check it out at Runwayml.com. It's just 30 different tools on the website, photo editing, video editing, auto editing, genera of AI, instant visual effects like removing the background on a photo, text to color grading, audio silence removers, all that kind of stuff. He started runway back in 2018 before Jenner of AI even had a name. Chris, welcome to the program. Thank you for having me here. All right.

Right after graduating NYU in New York City, my hometown, you started building this. And then things have gotten pretty busy. It's a SaaS-based company. it's a free tier, but after that you pay 12 to 28 bucks a user per month. Not to similar than I think the pricing of Canva or the Adobe Creative Suite. And you've got all kinds of people using this, including the late show with Stephen Colbert, likes to use it, I guess, to make interesting video graphics. Explain to us what you've built and how it's going.

Yeah, happy too. So runway is, as you were saying, a company that's built a lot of products and offerings for the creative economy. But at the core, we're actually a research-driven company. We got started around 2018, and since then we've been publishing and really pushing the space on the genetic AI landscape for both image and video. And so very proud of being pioneers in research in that space with things like, stable diffusion and latent diffusion and Gen 1 and Gen 2, which are, we consider very important models that have driven a lot of the excitement that we see nowadays. And so we do that research, we do the fundamental research for those models to be built,

those foundational models, and then we build products on top of those models ourselves. So it's been quite a journey since we started. How many people are using this product now? Because you've raised a ton of money. I noticed you recently raised, wow, 50 million at a $500 million valuation. So how many people are using this too? How many people are paying for it? We have millions of customers all over the world that range from professional post-production

films and award-winning directors to art ad agencies and post-production teams and small creators as well. It's everyone really who wants to tell a story with video can be and should be using runway. So it's millions of users. Most of those would be free, I would assume, because that's a large number. You've got tens of thousands of people paying for this, something in that range? We have a large amount of paying customers.

We have enterprise deals and subscription-based offerings. You can also pay for usage of the platform, so you buy the credits to, like, render video as well. So, yeah. So maybe you could show us, and I'll spoil. Sportscast it a little bit. So maybe you can show me. I know you have some new features coming out,

but maybe show us what it does here. And then sports cast it a bit because most people listen to podcasts, as you know, and describe what they're seeing on the screen. I will. So here's a research work that we've been working on for the last couple of months. It's a state-of-the-art video generation model.

Gen 1 and Gen 2 represent our family of video. generation models. This is a way of rendering and creating video in completely new ways in ways that haven't really been possible before. So let me walk you through what Gen 2 really does. There are eight different modes of using Gen 2 and Gen 1. One of the most perhaps interesting ones and different ones is that you can generate video using a text prompt. So you might come across or I have seen like image generator models and kind of like models that are able to create images out of pure text, this is very similar, but instead of generating one single image, generate a continuous video frame, right? This can be of anything. You can basically type anything

you want, and you'll get a consistent video in that style. So I'm just showing here an aerial footage of a mountain range, and we basically see that shot a mountainine with some kind of like lights. There's the sun on the back. You can see the consistency of the frames kind of apply here. And then you can- So that was generated just with that one sentence, or did it take more prompts? No, just that one sentence. You just input a sentence and you get a video. And so you can use-

How long would the video be? Is it like limited to a five-second or ten-second video? No, right now we have 15-second video clips support. So we're working towards allowing our customers to do more than that. But yeah, 15 seconds will be available very soon. in the product. And so you can use a sort of different input mechanism. So the one I was telling you here is like text to video.

It's another one. So I'm just using the prompt of late afternoon sun peeking through the window of New York City loft. You basically get that representation on the right. But then here's interesting. Here's an interesting combination of things. You can drive the video generation process with a combination of a text prompt and an image. So in this case, where you're seeing here is we call this character mode.

You can input a video of an image of yourself or anything at all and then describe how you want to animate or create a video out of that single image using a language description and basically you create that video based of that. There's also a variation of like... So in this one it said, just so we read the text here, you uploaded an image of a dapper-looking gentleman with a very nice mustache and a wispy hair and it says... That was also generated, by the way. Oh, so that was a generated image.

And then you said, you upload that generated image and say, a low angle shot of a man walking down a street illuminated by the neon signs of the bars around him. And we get what looks like, you know, a very diffuse background of lights hanging in the air, almost like he's in Hong Kong or something on an alleyway. And he's walking around. it's a little bit weird, a little jerky, but it's pretty impressive for, I guess, is this a 1.0 of text to video generation? Yeah, this is the first iteration that we made public. Next iterations are going to come over the next couple of months.

So this wouldn't replace a movie that were a TV show, but it would certainly be an amazing storyboard or a piece of collateral to start with. I think right now it definitely can be a collateral or a B-roll or some sort of a storyboarding tool. But photorealism and consistency in quality of outputs, it's only a matter of time and we're working towards getting that really out of the door as soon as possible.

Again, it's think about this as like early versions of GPT 1 or 2, right? You're seeing earlier iterations and early models that are able to render this and do this. These models didn't exist just a couple of months ago. Are you trying to build the next Uber or Robin Hood? I hope so. Well, here are three important tips of you. First, make sure your data is protected. Obviously, remember, just because your data is in the cloud doesn't mean it's automatically backed up.

Your data is your responsibility. You've got to make sure it's backed up. Second, don't let backups and compliance requirements distract you. Let your engineers focus on product innovation, not writing scripts for compliance audits. And when it comes to backups, the major players have major limitations like snapshots. If your account is compromised, your backups are compromised. But Clumio stores your backups on a different server. So if your main account is compromised, your backups are always safe the way it should be.

And third, take control of your cloud costs. You'd be surprised how much of your storage costs come from backups, snapshots, versioning, and replication. Or maybe you wouldn't be surprised because you've seen these jaw-dropping bills. Well, Clumio can help you with all three of these important points. They provide turnkey data protection, that is air, gas, act, immutable and cost optimized. Clumio has saved customers over 30% on backup costs while putting security and compliance on autopilot. Visit them at c-l-u-m-i-o.com slash twist to start a free backup or sign up for a

demo. That's clumio.com slash twist. How long have you been working on this text to video just to give people an idea? And then how many people have been working on this? This is based on an open source project that you're building on top of? No, this is entirely our own research. We've been pioneering research in the space from a long time. So we are the authors and the creators of a model called Latent Diffusion that was then turned into Stable Diffusion. We've createders of a few audio-based models as well.

And so really we've been pioneering... Is this Stable Diffusion a open-source project? Yeah, an open-source project. We open-source and released with the University of LNBU Minich in Germany. Oh, so you were part of that original project? Yes. So here are the authors. Alimum Eminich, High Library University and Runway.

Those are the three main organizations behind. When did stable diffusion? And you said there was a name for stable diffusion before it was called that? So this is the paper. So this is, you can consider this the origins of that innovation, right? High resolution image synthesis with latent diffusion models. And so latent diffusion was actually the shirt to describe this approach, which is the technique, the research behind this.

What is latent diffusion? fusion models. Explain to us technically for non-technical people what this actually means. And when did this research paper, this was written in when 2020 or something? 2020. Oh, okay. Last year. Yes. It's, I mean, the published, how fast this is moving.

Yeah. The paper was released and published in April 13 of 2022. Subsequent iterations of a model have been released since then. and the model has been open source and, as you know, has taken a life of its own. Latent diffusion introduces a different technique to generating images using a deficient-based technique in the latent space. That's basically, like, in short, what we've kind of like improved and introduced here. And it's a very generalizable approach that can be used for different techniques,

not just image synthesis, but in-painting and using semantic maps for driving those image generation processes. it was pretty much like a very, very unique approach towards tackling the speed and the consistency of image generators out there. And I'm sure you've seen the kind of wave of creativity that this model and then stable diffusion has created. So yeah, I'm very proud to be part of that transformation. So did you help write that paper or you started building the open source project on top of what was learned in that paper? No, so the paper, again, is the collaboration between two research organizations, Runway and the University of LMU Munich. Our research scientists, together with the research scientist at LMU, kind of like pioneer their approach, that was driven by runway, which was, I guess, the company we founded for years ago exactly to drive this kind of like research outcomes out.

Got it. And so in 2018, I came out to NYU, did research there, started the company right out of school, and then started building the company, raised money, started building the research efforts here. One of the many publications we've done was this, that it's, I guess, one of the most well-known. So you have this latent diffusion, then you have stability, AI. That's another company that kind of leveraged this open source project to build generative video and AI products? So once this diffusion model was released, multiple, I guess, the model took a life of its own, and we've seen all sort of different companies building open source solutions on top of it, which is great.

I think that drives a lot of the field forward in very positive ways. But as the kind of like original authors of the paper and the work and the research and the engineering here, we've always kind of like focus on improving and continue to push the boundary of research, and that kind of like manifests itself in our family of Gen X models now. So this is for us the next frontier in video generation that takes us a lot of the insights from our image generation process from before. And this stuff is starting to make its way. I had mentioned Stephen Colbert before, but maybe we could pull up the image you used runway to make a music video, sorry, commemorating the anniversary of the American Dream ice cream by Stephen Colbert.

and I actually have the URL of it here. I don't know if you have that. Oh, you probably have it in your customer stories. So how does one get a customer like Stephen Colbert? How did that happen? And then what is the reception you got from Hollywood? Because it does seem like this is a very clever beachhead market. People who are doing late-night TV shows, they love doing skits.

but they don't have a huge budget. They're building this stuff in real time. So it's not like they're going to, you know, be able to afford to go to industrial light and magic and make something. So did you do this for them or they did it themselves or did you send some people over to their offices to teach them how to do it? They did it themselves. If you could see it, that'd be great.

Yeah, it's, they did it themselves, which is funny. The story about how the team behind the little show started to use runway was basically, we don't have marketing ourselves. So everyone who comes to runway comes mostly via worth of mouth. Someone inside the team here started to use one of our tools. We have around 35 different tools. So the tools I mean telling you and showing you, Gen 1 and June 2 are one of the many tools that we have. We have one tool for erotoscoping, which is a traditional frame-by-frame task that editors do.

And it takes them like a lot of time, like hours, if not days of work. In runway, you can do it in like, I'm not kidding, like a second, like literally one second. One of the editors started to use it for the show and eventually pick up the entire team inside the Colbert show. And now they be using it for a bunch of different edits. And they've taken, and this is true in the interview that I'm showing here, six hours of work combined into six minutes. And that is the power of like this kind of like technology when you can automate and go ahead. And what we're seeing here is a dancing, a pint of ice cream with a singer in front. of it who looks like, um, is that a real singer?

This is Katie Perry, yeah. Oh, it is, in fact, Katie Perry. And then some people in shark outfits. So what's real here and what's fake? So the ice cream, uh, it's, it's fake that's been, that's been added in post. That's the visual effects component. Are the sharks real? The sharks are real, yes.

But the, this is what's mind blowing about this. I'm looking at it with you. And I'm like, that looks like Katie Perry, but because I knew this is a computer generate. I'm like, is that a computer generated? Kiti Pera? I assume the sharks were real and the ice cream was real and then everything behind them was fake, but the sharks are not real. Dancing pint of Beninjuries isn't real and then obviously the pineapple

trees behind it. That's crazy. So just to be clear, just to be clear, everything here is like real. The only thing that the magic here is you combine all of those real elements into one singular shot that you believe is happening in real, right? But the Ben and Jerry shot is shot somewhere else. Katie Perry shot somewhere else. Oh, okay. So those are elements.

And then you said mix them together. Right. But mixing them together takes you a lot of work because you have to go frame by frame matching. And this is the work that editors, this is the work that BFX studios are spending hours of work doing, right? This is why green screen success. You shot in a green screen and then you take the actor out of the green screen, right? Amazing. If you don't have a green screen, then you're bounded to like hours of work.

work to recreate that effect. And that's where, like, our tools come in and just help you do that, right? Got it. This is, this is from the Super Bowl. It was a Super Bowl performance. She did something like this years ago. So this is all stitching that together. Exactly.

So this, this is amazing that, like, you are getting to the point where the elements are just hard to determine what's real and what's not. And you said this takes them a couple of hours to do this when they have those elements already. Yes. So they went from six hours to six minutes of work, which, which, which for any filmmaker, editor really, really makes a lot of difference. If you can tell from the podcast lately, we've been doubling and tripling down on Founding University of launch.

In fact, it's basically the future of our venture capital firm, and that's awesome because I'm working with a couple of hundred early stage founders really early and getting to see what tools they use. You know what tool they show up with most? They show up with Squarespace. They put up their first website instantly, quickly with Squarespace. and it's beautiful and it makes them look like a million bucks. The thing you may not be aware of is that Squarespace,

beyond the beautiful templates that make your company look like a million bucks, and that work on mobile. It's not just a pretty website. It is a powerful e-commerce platform now, and they have member areas. What's a member area? You know people like to sell content now and premium content? It's a big business.

Well, they have that built in to Squarespace, and they don't take, you know, double-digit percentages of your revenue you like those other platforms do. And those have appointment scheduling. So, you know, if you're doing a business where you're a consultant, you want to charge for your time, well, you have scheduling built into it as well. And this is the brilliance of Squarespace. It's going to look beautiful as you know. So here's what I want you to do. Just head to Squarespace.com slash twist for a free trial. When you're ready to launch, use the offer code Twist. You save an extra 10% off your first purchase of a website or a domain. We love your Squarespace. You know how it is.

When you're a technologist, everybody in your family, your friends, your circle, your network come to you and say, hey, I got to get a website up. Can you find me a developer, a designer, a product manager? And you just say, you know what? Yes, I can find you all that and more at Squarespace.com slash twist. You mentioned film and Hollywood, which brings me to, like, actually our film festival that we hosted in New York and in San Francisco just a couple of weeks ago. This was a celebration of films that are already being using or have incorporated AI

techniques into the process. And we're not talking about, like, prototypes. talking about like real filmmakers who've been working on film for for years. We had Darren Aronovsky coming to the panel to chat about. Darren Aronovsky came. Yeah. Wow. I hosted in the 90s,

I hosted a screening when he was broke for a film called Pi, which I think he shot on, he shot it on eight, um, eight millimeter, I think. And he was broke and I met him at a party. And I said,

wow, I really want to see your movie. And he said, uh, oh yeah, I love your magazine. magazine, J-Cal. And I said, I'd love to host a screen. He said, would you host a screening for it with, like, Silicon Alley Tech people? I was like, sure. How do you do that? He's like, you rent at the area, and you invite people. And I was like, all right, I'll do that for you. And I spent like $1,500, renting of the air and having drinks. And he was so kind, Darren Aronofsky. He came and he hung out with us and the star of the film. And we just had drinks. And we did that over on Avenue B and like 12th, whatever that movie there over there was. It was quite fun. So he came. So he's obviously. into this. These are short films that were made.

These are short films. We premiered 10 short films that were made in a combination of either entirely generated AI techniques or analog we generated. These are films made by either professional filmmakers, new filmmakers, and really like just showing the quality, really like, it's just mind-blowing here. The films are already accessible in our website. So if you go to runo-mail.com, you might get a sense and a chance to see them all, which is definitely I would encourage you all to. Which one won? Did one win the AI Palm Door?

Yes, the Grand Prixx. That's how we call it. There was a Grand Prix, a gold, a silver, honorary, and then merits. The Grand Prix was an incredible film by Ricardo Fusetti, who's been working on visual effects for some time now called Generation. The first time I saw the film, it just like, it goes back. It's on the website. Can we just hit the play button? Let's hit the play button.

Let's watch it. Okay. So we're watching this. I'll sports cast it. Okay. It's got a little warning. that could be flashing images. We're watching this.

You don't have the audio, though. That's, yeah, I don't need to have the audio. It's good for us to sort of describe what's happening here. But it was a close-up of a human. Now there's a human dancing on the floor in, you know, essentially some sports bra and shorts, and then their body is turning all kinds of different crazy colors. I'm assuming this is a human that's actually been shot on video,

or is that a computer-generating human? That's a real human. Then now you're seeing, yes, and now you're seeing there's a voiceover that's kind of like telling a story about evolution behind the scenes. So you can hear that right now. But there's a music that's accompanying this. And now you're seeing this human being turned into some sort of animation and like graphics are merging with the body of the dancer. Wild. Those are all generated.

And so the beings, the aspects and the effects here were all driven by AI generation, kind of like techniques. Got it. So they found a modern dancer. interpretive dancer, or just a person who's really high dancing like a maniac, and then put all these incredible effects on it, which are truly stunning and it's hard to look away. And if this was made in the 80s or 90s,

or let's say the 90s for MTV, this would have cost a million dollars in graphic effects. It's like this outdoes any Peter Gabriel computer effect, you know, sledgehammer or whatever. That was claymation, but, you know, all these different things that were done in the 80s and 90s for music videos, this would be the number one video of the year.

Yeah, definitely, definitely. And this is only the beginning. I think this is, this is technologies and tools that are only accessible for the last couple of years, to be honest. But to get to our point on Hollywood, these are tools that the industry has already been embracing and we're actually working really closely with filmmakers themselves to incorporate more of this AI revolution into their processes. We have a division of runway called runway studios that hard.

partners with filmmakers and with film studios and with filmmakers to really help them understand the power of Gen.D.B.I. for film or for video generation. And so we work with a few films already, with the Berlin Film Festival, with a few films in Barcelona and a few other, like, places. This wasn't really possible two years ago. Oh, this is really impossible. Like, yeah, actually, the tackling of our companies make the impossible. We're making things that were impossible just a couple of years ago. Uh, so, yeah. I just say, like, two years ago, literally, this wasn't on the table, maybe even 18 months ago. This all started in the last 18 months.

We've been working on this for the last four years, I think, but more importantly, I would say the last six, eight months, has we've seen an explosion, the quality, consistency, and a real understanding of the technology. And I think that really helps a lot. So definitely it's been a lot of work behind the scenes, but something like Gen 2, which is a tax two video model was, yeah, literally not possible a couple of months ago. All right. So some questions here, I think, are very important.

What does it cost to make a music video like that in terms of what you have to pay your company? So what did runway make in fees, in software fees, to make that dance video we just saw, which was super trippy, and would have won video of the year at the MTV Music Video Awards in 94? So there's a cost of training a model. And so we have a large research cluster that we use to basically, train these models from scratch. First of all, invent them and then train them. Then there's the inference cost. Not your cost. The cost to me as the director. Oh, the cost really depends. It depends on like the team that you have and the scale of the

ambition and the idea that you want to like execute. But let me give you an example of like. I'm just saying the software cost. If I had five people working on this and we used your software for me. It's 15 bucks. So it's just 15 bucks a month. You get. So this is what I'm sort of getting at. There's all the amount of money you spent. I'm assuming you spent millions of dollars building this model with dozens of people? Yes, we're 40, 40 people in the team. So 40 people and millions of dollars invested over four years

or tens of millions, let's say, low tens of millions, to now make a tool that can make music videos that in 1994 would have won video over the year and the person who made it probably gave your company 20 bucks a month or if they had 10 people on it, 200 bucks a month. That's the power of disruptive technology. to just make anything, everything more convenient

and easy to use. All right. So at the pace this is going, you happen to know a lot more than myself and the people in the audience because you're actually building this. At the pace this is going, certainly you're aware of what John Favreau has been doing

with the Mandalorian, with those 360-degree screens that they use as the background. You know this technology? I don't know the name of it. Virtual production. Okay, virtual production. So describe what virtual production. is and what has led to, you know, Disney with Star Wars being able to make without sets

incredibly compelling products like The Mandalorian in a very short period of time at a much smaller budget and then compare where you're at. And when would these two things be indecipherable for myself and my daughters? How many seasons out, we're on season three or four of the Mandelorea now, how many more seasons, how many more years until these two things are basically indecipherable? I'd like the exact number of years, please, but you don't have to go to months. With days or seconds as well? I can't just just years.

I think it's always good to look at like the pace of research in a scale. So if you see Gen 1 and Gen 2, video generation was really. hard to imagine in the quality of results that we see today just eight months ago, right? So eight months ago, we went from like, hey, you can do this to, hey, now you can render photo realistic video. And you can type anything. You can just give me a problem that I'll generate it for you. I think we're still a lot of ways to improve these models, and we'll see that over the next

couple of months. I think the next 12 to 18 months will see a big step up in quality of and control ability of these models over time. And so we're not far away from getting an entire Mandalorian made with JourneyDVI. Five years? Three years? Oh, definitely less. Less than five?

Less than three? Yeah, for sure. No. Really? Indecipherable? So here's a hymn. You're going to be able to generate, you're going to be the main Mandalorian character, right? The fact that you can render and generate films on demand,

will allow you to generate any film that you ever wanted to generate. So that's an interesting also position to be in where the type of films that you're consuming right now are going to be very different from the films that are going to start watching in a couple of years because it's going to be made very differently. You are going to be the main actor in every film, video scene that you ever imagined. And also, you can generate every film, technically. You will be able to generate entire films. And so it's a very different, I guess, position to be in.

because, you know, there's this incredible thing that occurred. I don't know if you're aware of it. You seem to be, you're a fan of the film? You're a fan of cinema? Of course, huge fun, the film. Huge fan of cinema. You got a favorite, favorite director or two? What's your favorite?

A huge fan of Darren Ariphanowski. I think that's, yeah, first. You got a favorite was a Requiem for you? What's your favorite? I think Reckham was definitely a highlight. The Fountain was also like a big word for me. Fountain is just... It's so good.

And then more recently... You know who was supposed to star in the fountain? Yeah, I think it was... Yes, and then he decided to go somewhere else. That movie's just beautiful. And then the whale more recently, I don't know if you've seen the whale, but... Oh, the well was incredible. It's so good.

Yeah. Yeah, I mean, if you... What's your favorite? Well, you know, Requiem for a Dream is one of the most... powerful films ever made. Yes. The fighter. I'm sorry, the wrestler, rather.

The wrestler, yeah, yeah. Is just extraordinary. So, you know, it really can go two ways with Darren. There is the surreal and then there's the character performance, right? And these, if you look at something like the whale and the wrestler, these are incredible performances in each one of those films. And then if we look like something, you know, Pye, Requiem and the fact, These are very high concepts.

They also have great performances. Don't get me wrong. But conceptually, Requiem for a dream, how the storytelling occurred, the fountain, how the storytelling occurs were pie. They're very big think kind of films, right? And they rely not primarily just on the performance of one exceptional person where you connect with them, which is really what happened in the well with Brendan Fraser and the wrestler with Mickey Roark.

But yeah, he's just an extraordinary filmmaker. Who's number two on your list? You got a number two. It's hard to get favorites, to be honest. I've tried to watch as much. We'll work backwards from a film. What are the films that if you had to watch a film for the 10th time, you'd have no problem watching it.

Be excited to watch it at the 10th time. Yeah, I mean, a lot. I watched Melancholia too many times. Oh, really? A lot of Frantir. Yeah, it's a beautiful film as well. I have too many. I mean,

Melanchion. What a pull. That's a good. Of course, yes. That's a, that is another one of these like intellectually mesmerizing, interesting films if you haven't seen it.

It's, yeah. And then the square, I don't know if you watched the square. No. And Forge Major movies as well. Yes.

Yeah, yeah, yeah. Fascinating. At least I got a movie review out of this. I'm a big Ridley Scott fan, gladiator, Blade Runner, etc. I like a big Ridley Scott. What's the favorite, Britney Scott? Oh, yeah,

Kirkersaw, I guess. Huge. You know, for Kurosawa, I go both ways. I love the film noir and I love the samurai genre. I love high and low and stray dog. If you haven't seen the you haven't seen those two, on the film noir side,

and then obviously seven samurai thrown to blood. I mean, it's just all amazing. Yes. It's just, It's hard for people to watch those films because they're a little bit too slow and methodical. And people want to look at their fucking phones and they can't just sit. It's like somebody who goes to a museum and you want to take in a picture or the Venus de Milo. It requires you to quiet your mind and absorb it for, you know, but five minutes of your life, which for in today's modern era to let your mind absorb something for five minutes straight,

let alone for 120 minutes and give yourself over your attention over to something is very hard for humans is really something's been lost I agree I agree I do think still that

a lot of the best films are yet to be made I think we're one of the things that are going to be really interesting about your DVI is that you're going to start seeing stories from filmmakers

that never had access to the tools that only a handful of filmmakers you have right so that from a from a democratization and storytelling perspective it's interesting to see from that from that side which is we've loved cinema with all of these directors of filmmakers who really pioneer really interesting like narratives and films yet i don't think we've we've come like even close to the level of creativity will start to see once you can put bfx studio kind of like

tools in the hands of everyone and that that is like a really interesting um to have in mind as we continue to think about just journey right in general. I think this is the key observation, and this is something I watched happen at Sundance in the late 90s in the early 2000s when I would go there, you know, and I grew up in New York and I was in New York during the 90s. A digital camera, the VX 100 or 1,000 came out the Sony. You could record on digital cameras, but blow it up to be on actual film. You'd blow up your digital to film. This is before they had digital projectors in movie theaters. So there was this path

where a guy named Bennett Miller created a film called The Cruise shot on this VX 1000 or whatever it was and Darren Aeronovsky shot 8 and then blew it up and then, you know,

many other people, Wayne Wang did Center of the World and a bunch of films were made on digital cameras and it really did change who could make a film and then at Sundance you started to see

all these digital films come out because you could shoot an unlimited number of takes and you can do it in more of a guerrilla fashion and you didn't have to develop film. Just at the end, when the film was finished, you would then send it to get printed up to film, and now that's not even necessary.

So here you're going to have people who don't know how to use a camera and don't have the ability to access $1,000 or $2,000 camera be able to make something. And that is just next level fascinating. Yes, yes. And that for me is like the most interesting thing that will start to see coming out of these models.

story, you know, and that was really the basis of Sundance was trying to find new voices to tell stories. People who hadn't had access and, you know, documentary, whether it was documentaries or Napoleon Dynamite or riding giants or supersized me. These films were just a different generation of filmmakers. If it hadn't been for the digital camera, it could never have happened. No. Yeah, it's just a different. It's just a different set of affordances and creative possibilities.

And so you have filmmakers who really, I think a lot of filmmakers are just technical innovators. They're the ones driving the innovation forward. I mean, think about Darren. He's pioneering a lot of things in the space. And I think that combination of art and science is where we start to see more specifically in Jurydivai as more people start to get into the field and understanding the power of these tools. This is a new camera. That's the best analogy to think about this.

These models would allow you to capture a reality and a set of expressive tools. in the same way that the camera did for us 150 years ago. The level of output from a creative standpoint is it's like unprecedented and yet to be seen because we haven't had this technology before. But really thinking about it as a tool to do those kind of storytelling, I always go back. I don't have you watched this beautiful documentary on Disney Plus about the story of ILM. I think that.

I know if I've seen that, yeah. I definitely recommend that. And it's kind of like taking the position. and the perspective on really understanding how the feel of visual effects was born. It wasn't a thing until like a bunch of hackers and thinkers and artists just came together to make a film about Star Wars, right? And they're flying like starships in the space, right?

Imagining that was and festival was impossible until someone had to create it. So and then from there on like a whole set of like domains and films were possible. I think we're early stages of something very similar here. Yeah, I mean, it's so funny. I have a list of the films from 99 to like 2000 in that era. And it's wild to see like Harmony, Corinne doing Julian Donkey Boy. That was a digital film. And right next to it, Star Wars Episode 1, The Phantom Menace, one of the first like major blockbuster films, Blair Witch Project was another one. Lars von Trir, the idiots in 98, and the center of the world by Wayne Wang in 2001. tape by Richard Linkletter. And you're starting to see like some of these very Steven Soderberg did a full frontal.

I mean, really like famous David Lynch did rabbits. These were films that could not have gotten funding. And so that's, and 28 days later was also that Danny Boyle's film. Wow. A lot of these films would not have been given the budget. You know,

if they had to use actual film stock. And I think this is going to be the very interesting thing that you're going to do. All right, everybody, our friends from Microsoft are here. Tom Davis, a senior director at Microsoft for startups, and you're a former founder. You are here today to talk to us about the giant leaps that Microsoft has made in the AI space. You've been giving Azure credits to startups, and that's delightful and amazing. But people really want access to the Open AI API.

Yeah, absolutely. So there's two things. First of all, we've got a benefit that we offer our startups. They can get two and a half thousand. dollars worth of OpenAI credits so they can get access to the latest and greatest models that Open AI are delivering. But then they get access through the hunt up to 150,000 dollars worth of credits that we offer through founders hub to leverage the Azure open AI service, which has a full SLA around it. So when they want to go into production and really have that reliability that we

provide with with the Azure SLA, they can leverage the Azure open AI service APIs. And they can do things like with the GPT models with codecs for coding, and also for the Dali models as well for images. So it's a full service. It's not just the great APIs that you get and access to the LLMs. They can build out their own LLMs using open source, and then they can manage those with our AI tooling services as well. Amazing.

Well done. And if anybody wants to sign up for that, do it now. While you are in front of your computer, aka.org.m.S. slash this week in startups. AKA. dot MS slash this week and startups. Well done Microsoft and well done, Tom.

Tell me about, you know, who is most interested in this today? Is it independent voices and people tinkering? Or is Hollywood really sweating what you're doing? And who in Hollywood is starting to tinker? Who's tinkering? Hollywood is particularly excited about what we're doing, to be honest. I think we are working really closely with both filmmakers themselves, but also production houses and studios. I mean, from a cost perspective, we really think about these are tools that will drive cost down.

So the big promise here is that you're going to take the cost of content down to zero. So making professional films, the cost of making a film will continuously go down to zero. From a producer perspective, from a studio perspective, that's great because you can make more stuff, right? And filmmakers are also understanding and embracing that. It also means that you can iterate more in your ideas. If you're a filmmaker and you want to like execute in a particular idea, instead of like spending too much time and too much money on one singular shot, imagine having a world where you can try a bunch of those and go with the one you like the most.

And so from Hollywood, as you were selling before, like with the late show with Colbert, translating hours of work into minutes of work, that's the value. They don't care about the AI and the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the. and the models and everything all. It's like, this is a, these are tools that are about time and cost. And so, of course, you're going to embrace it because it's just so liberating. It's so creatively inspiring to be able to do this so fast. And so Hollywood on the one end, I think our conversations with folks in the space and our

collaborations are already driving with companies and professionals there have been really positive in the most little sense. And then of course, you have other spectrums of folks who never had the chance of having those tools at their disposal. And that for us is going back to our conversation around like new stories where we'll start to see more and more

over time, those new stories that will start to emerge. What was this all trained on? What data set was this trained on? Because it understood a New York loft, it understood mountains, I get that, but it also understood like a low angle shot. I'm assuming

there's some amount of training that you're doing inside your company to take what are industry terms, a tracking shot, a wide shot, a close up, you're training it on that? Or, and you're saying, hey, this is what those are here are examples of it? Or has it just scooped up every single movie ever made? How did you train this? It's, it's, it's, there are a lot of different like systems and models behind the scenes of seeing something like this been possible. And so models are trained. on different datasets depending what you're trying to do and accomplish.

And so we have our own internal data sets, but we also partner with media companies and industry experts to train models own their own data sets. So think about you're a filmmaker, you're working on a new animated film, you have all these reference shots about a character you want to animate. You can give us those data, those images, the references, and we'll train a model just for you. So you basically now have a system, an AI system that you can prompt with images, with tax, wherever you want, and you can generate infinite amount of videos out of it. The more input data that you have, the better you will understand consistency,

coherence of characters, like temporal consistency of objects moving across the shot. And so it's really about customization, if you think about it. You start with a baseline model, and then from there on, you can just keep on improving to have the model learn more about it. So if you were to work with, say, the Simpsons, you could say give us all 30 seasons of the Simpsons. And if I tried to use your product right now and said, make me Marge Simpson, you know, as a Jedi Knight, but who also has the superpower, has an Iron Man suit, would it be able to do that? Or because those are very highly protected IP sources, you can't train on that data. Therefore, we're going to see some maybe the Simpsons make their own AI. And you could have Simpsons GPT or Simpsons AI where you take your software, their content library, and they say, hey, you can go to this website and you can make your own Simpsons, and we're going to have our Simpsons Film Festival, just type into the box

what you want to happen, and then we'll have our own contest, but it has to live on Simpsons AI TV. And here you're exactly right. And the interesting thing is that I truly believe that creation and distribution will become more intertwined, right? So think about video, think about a movie. You create the movie, you render the video. The movie or the video file is baked, right?

It's pre-made. Everyone is watching the same film, right? That's someone directed. The moment you have the systems running in real time, you're going to be able to visualize and render and view a video, but that video is being generated the moment you're watching it, almost in real time, right? And so the notion of like taking a baked video

and streaming it and streaming it in real time might change as well. And that for me is like a key change. in the market in terms of like understanding how distribution and creation will start to change radically, specifically again on the video side of things. Yeah, so this fan fiction stuff is going to get really interesting. There's

a website on YouTube, or so a channel on YouTube, Star Wars Theory, and he was like a super fan and he made this fan fiction called Vader. and I'll just play a little clip for it here. And it's obviously real actors, but I think a lot of the backgrounds were created.

He does fundraisings for, I don't know, $20, $30, $40,000 to do these. And his fans give him the money and he goes and does it. And Star Wars is pretty cool about if you're doing non-commercial, they'll be okay with it. But, like, you know, it's pretty impressive. And I think that this is going to be the future is you'll be able to go on Disney Plus,

and then there'll be Disney Plus fans. and they'll let anybody create stuff and then maybe they'll only let certain ones onto the platform the best of. But if you go on the web, maybe you'll be able to see other stuff. But this is really going to be a fascinating new world. Has anybody, I know you can't talk about private conversations and stuff like that, but is there somebody on the IP level of a Marvel Star Wars,

Simpson's South Park, who is currently working on this right now? I think there are many companies paying attention to it for sure. They know, and it's same as you kind of like notice now and make sense of it. I think a lot of companies are making an understanding that that's an option they can take. And from a market perspective, it's just a great opportunity to get your customer, your IP and your characters and your stories out to more people and also engage them in different ways. By the way, I don't have you seen Star Wars uncut. It's a similar project, but every shot is made by someone else,

someone different. But Star Wars movie, that's like, that's really nice. Yeah. Yeah, that was a crazy project. So they, and I think they're doing it also. Yeah, Star Wars Uncut was they just told everybody do 10 seconds, but do it in your own fun style. You could do claymation, you do animation, you could do stop motion. You could do live action.

You could make it whatever. Yes. And yeah, those things have been out for a while. Yeah. I like that level of great. There's also the Johnny Cash project that was very similar. It's like just just one frame. Everyone, like all the fans design one frame.

Just design one frame and just teach them all together. Only one frame? Oh, I thought it was like a couple of seconds. No, I think the Johnny Frame, the Johnny Cash project is just one frame. I may be wrong, but that was my understanding. That's wild. Maybe that was the Raiders of the Lost Ark one. Then there were the kids.

I don't know if you ever saw this, but there were a group of kids who started making a Raiders of the Lost Ark film. and they just worked on it for like 10 years as they got older and older. This is Empire Strikes Back uncut. And yeah, this is something that got the blessing. Yeah, that's Star Wars. But here is the take the Empire Strikes Back when I just sent you. Because what's really creative about that one is you'll see everybody does a different style of Empire Strikes Back.

and they get a couple of seconds each. And this thing is, and this is from 2012, this is 10 years old. But this Empire Strikes Back one. So I put in the slack. Here we go. You're a puppets. So somebody decided they would do puppets.

I mean, it's crazy. And if you put this with ChatGPT, the interesting thing, and has this come up yet inside of your, you know, I mean, it's just crazy watching this, isn't it? It's like so creative. And this is 10 years ago. Here's one that I think, like, if you think about a project like that we have now,

it's so interesting to see. I'm actually excited to see if someone want to take this further here. I'll share with you a project. I saw that in TikTok two days ago. So if you want to open that, that's driving video on the top to someone recording himself and just like his house. It's a piece of paper and like a bunch of other shots. Wow.

And then you can translate that with runway into any. style that you want. That's wild. So is this using your software or just some other A-I? Yes. No, this is runway, yes. This is runway. So basically he takes a shot and then tells runway, hey, here's some text prompts.

So make this in the style of a cartoon, make this in the style of an impressionistic painting, whatever. Wow. So it's giving the starting prompt and then going to the next thing. That's just demented. That's like a lot It's just on a wall with like

floor and it's just like you turn into a beach Or like that one I like a lot as well Like a paper boat And just like a real boat But here's the interesting thing You know I don't know if you know about this auto GPT where people are making like agents That will operate in the background and make stuff

You could literally have somebody say Take today's current events In turn you know Take chat GPT for read the top headlines, translate them into a story that is similar to the Marvel characters and have Marvel characters intercede in these world events and make a movie about it. And so it's like, oh, Ukraine and Russia invaded Ukraine. And then there's this tension in Taiwan and all of a sudden Captain Marvel is going to Taiwan

and, you know, Tony Stark's going to Ukraine. but what would happen is chat GPT would be set on a mission to take the top five criminal stories in the United States or you know dramatic events match it with a character and make a short film make the dialogue and there would be no human involved it would just publish this stuff to YouTube in real time yeah I think that's technically possible everything to describe you could just literally just yeah you can do it you're worried about any of this stuff? I mean, and there's a whole group of people who are worried. I mean, you're making fun images. Is there any downside here that you can, that you, do you have a group of people in

your company worrying about safety? Or is it just like, well, this is entertainment purposes? Or do you worry about some, you know, buddy creating people saying something that would get them canceled and then publishing it on social media and that kind of stuff, impersonations, etc. We take safety really seriously. We have a quality and alignment team inside runway that make sure the model outputs are safe and compliant with our terms of use. And of course, the field is evolving really rapidly and we've been pioneering all of things in the space. And so it's always been a kind of like starting point for us to think about the best possible outcomes or the best possible approaches to use this technology in a safe, safe way. I do think about it.

in perhaps, you know, we're talking about the analogy of the camera, you can point the camera to all sort of like things you want to point the camera too, right? It creates, it's an incredible tool, but it's also a tool that needs to be used in the right way, right? From our district perspective as well, just having the camera won't make you a film the director. We just need to understand where to shoot, what to shoot. This is no difference. Like, you have a tool that allows you to generate something, an idea. It's about iterating on that idea and understanding what works and what's the way. doesn't and what's safe and what's not.

And all of those things are going to be things that we're going to figure out as an industry in the same way that figure out systems and safety procedures for sharing videos and cameras, right? I think we're still very early on all of it. And you might see this with language models being discussed more. It's still very early for the industry at large. But yeah, we take it. We take it very seriously.

Not like nudity, adult content and that kind of stuff. Obviously, there's a place for adult. content in the world. I'm not talking about pornography necessarily, but there could be a scene that is a risque scene from an R-rated movie, or there could be a violent scene in a movie. How do you manage that in your terms of service? Quentin Tarantino, doing inglorious bastards and literally scalping Nazis, would you allow that on your platform or not allowed on your platform? I think we're taking a more pause approach

towards helping people understand how to best use the technology and so right now we don't allow nudity on the platform for example but we're working with filmmakers themselves to understand what that does mean and like what limits can we change

but again I'll go back to this is still very early got it I mean if Darren Aronofsky's came and said listen I'm making Requiem for a Dream it has some nudity in it has some violence it has drug use in it you could make an exception for him, and then just make sure that they're not abusing somebody's likeness.

I mean, I think that is the really dark part of this, is taking people who do, even if you're a public figure, and listen, I'm a public figure. People are making me into, you know, a superhero as we speak on Twitter or they were making Star Wars characters out of the all-in cast. And it's like, okay, I'm cool with it, but like, let's not get too crazy folks. But you can see with these tools that people will get crazy at some point. So I think it's probably like you've probably got a pretty good concept here, which is we don't allow stuff that would be risque right now or could hurt people, but we could have a conversation about it if you want to submit what you want to do,

and we would have the right maybe to veto or prove it since you're using our tool to do it. That might be reasonable. Yeah, I think that's right. I think working with the communities that you're trying to serve is really, really a key component here. So what's the hardest part of your job right now as an AI company? Well, I mean, AI is becoming this insane, you know, people have been working on this for decades. And then in the last six months, all of a sudden it's like somebody lit the fuse and now the rocket is hurtling towards the edge of space.

And so are you just inundated with too much inbound? too many people want to throw money at you, too much distraction or too much competition, too much change in the technology because it seems like this stuff is moving at a pace that is incomprehensible, even to people in the industry. No, yeah, things are moving fast.

I would say, I guess, exactly what you mentioned, that there's been decades of work behind the scenes to get to this point. And for someone and for our team who've been working on this for a couple of years now, it's really important that that consistency and the level of like outputs of our research team and our product teams continue to be the same. There's of course more noise around. More people are paying attention to it, which is in some way great because you don't have to convince people that this is worth paying attention to.

Everyone can just say it. I think we've, I keep saying that we cross the most important chasm that there is in technology, which is the mom threshold, which is my mom now uses runway, understands what I do. Which means that the technology has gotten mainstream enough. it's more accessible or more understandable for more people. That still means that we haven't got into its final form. There's so many things we need to. It's so early. I think that for me is one of the key challenges is really emphasizing

where we are with the technology at large, what needs to be improved, and also exposing more people to how things work. There's a lot of misconceptions around how image generation works and how video generation works and how language models works. And really understanding the ins and outs will help drive a better conversation around the space in a more impactful, positive way.

And I think that's a key challenge for me because things will continue to accelerate really, really quickly because of the glory itself has proven to be extremely, extremely useful. And congratulations also on your work. I understand the team for everything, everywhere, all at once use the product a bit. Yes, they use our roto tool, our rotoscoping tool,

to edit some shots in the movie. And I think that's the power of really like filmmaking They won seven Academy Awards, including best editing. The fascinating thing about that movie is it was the visual effects team. You watch it. I'm assuming you're watching it. You watch the movie, right? The visual effects team behind that movie, which is, remember, a very intense visual effects

movie was made by seven people, right? So seven people. Wow. Yeah, seven people made the entire visual effects shots there. And I think that's a representation of where you start to see more of, which is like very highly creative, incredibly talented folks. using tools that weren't accessible before creating movies that can win seven Academy Awards. So that's a great, I guess, sneak peek into probably what we'll start to see more and more

into in the filmmaking process. Amazing to see what happened in Star Wars and Blade Runner, people using miniatures, people having cameras and all kinds of weird positions, blowing up little models to now this. Absolutely incredible. is Runway. You can visit them Runwayml.com. Chris, thank you so much and continued success. And let's keep trading some movie recommendations, some good recommendations we got today. Yeah, good, brother. We'll see you here. Yeah, and we'll see everybody next time on this week in startups.