No Priors: Artificial Intelligence | Technology | Startups - Open AI leadership shuffle, new diffusion models, and starting the cult of Q*

Episode Date: November 30, 2023

OpenAI’s leadership has taken us all on a rollercoaster so it’s great timing for another host-only episode. This week Sarah and Elad get into what has been going on at OpenAI and what the turbulent leadership changes tell us about the importance of good intent and good incentives when building these influential companies. They also talk about innovative products coming out of Pika Labs, why people are moving away from diffusion models to LLMs, and how, in AI investing, the ASP is the opportunity.  Sign up for new podcasts every week. Email feedback to [e-mail] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil Show Notes:  (0:00) Recapping the OpenAI saga (9:56) AI video products (16:14) Moving from Diffusion Models to LLMs (19:47) The beneficial margins of AI investing

Hi, No Pryor's listeners, time for a host-only episode. This week, Elad and I are going to talk about what's going on at OpenAI, of course, video, Q-Star, what might be next in research and some predictions. Okay, Al-Aid, we have to start with the saga from this past week. What is your take on the outcome and the second-order effects? From a second order effect perspective, this seems like overall positive news for everybody involved. So it looks like on the opening eye side, they're back to being in a really positive, stable situation. I think they still have like the leading model in GPT4. They've reworked the board, which seems like a positive thing.

So imagine if this had happened two years from now, three years from now, et cetera. So it seems like it would net increase the stability of the company and governance and a few things like that or the nonprofit and the company in governance. So as an external viewers seem like a painful thing to go through, but the flip side of it is it seems like they're moving forward and moving ahead in a positive way. And then in parallel, I think it may have ramifications to other areas that we can talk about if useful.

Like what are the second order aspects of this, but it'd be great to hear what you think. Yeah, I think the first obvious lesson is that governance matters, right? And this isn't an area where I think most companies are that, experimental, but I think a lot of entrepreneurs are likely to think twice about placing their destiny in the hands of groups with explicitly mixed incentives now. I'd say generally non-profit governance is not every organization, but as a class known to be kind of abysmal, right? Because performance is hard to measure objectively, and so it often ends up more about politics and specific

relationships and status games than outcomes. The clarity around how much, you know, any board matters was kind of a wake-up call for people. The second lesson that a lot of people will take away or I think should from this whole saga is that money matters, right? The factors of production are labor and capital and compute is the AI-specific form of capital. Microsoft holds the compute here, and that clearly matter. This is amazingly well managed and supported by Sai and Kevin. And then the class of like really special labor here, the team rebelled and the board obviously underestimated the level of support that Sam and Greg had from them. And then I think one thing that is often unsaid because it's a little bit less idealistic is that a lot of opening eye people were very upset this last week about not just the destruction of the mission, which I think was absolutely like genuine, but also destruction of the value they built. and been promised a piece of with the $86 billion tender offer.

It's just a reminder that labor and capital are like leveraged. They're stakeholders. And there's no free lunch or control without skin in the game. And I think they're likely shouldn't be from the view of many of the people involved in this. Yeah, I think you're raising an important meta point, which is basically what are the incentives that different organizations have in place? And ignoring open AI, I think there's a lot of boards, which added board members, for reasons that were politically motivated or motivated by different regulations getting passed

that for certain board changes, et cetera. And you also see that in executive teams. And I think it's really important for people to go back and rethink, okay, who should be on my board and why? What are they representing relative to the board? What expertise do they provide? What insights are they bringing?

Or what strategic views are they bringing? Same with your executive team. And also, what are their incentives? And, you know, there's this view that's kind of been moving around Silicon Valley in terms of the professional managerial class, right? People who have alliance not to the organizations they work at, but external incentives. And those external incentives could be speaking at TED or going to Davos or getting kudos or an award from a specific organization

versus doing what's actually their duty as a representative of the very shareholders of a company in the context of the corporation. And so there are these fiduciary duties that may be being breached by other incentives for different actors who've been added over the last, you know, five, ten years to boards, to executive teams, etc. And I think it's really worth rethinking, like, who do I want on board and why? And it also comes back to some of the companies that have been resetting things relative to politics. I think Shopify did a great job, for example, of saying we're performance-based culture, we're focused on, you know, a very specific mission. We don't want that mission to creep. We're about, we're not a family. You know, like if your uncle shows up drunk and does something bad, you forgive him.

If a board member shows up and does that, then, you know, you don't want them on your board, right? They're being irresponsible. There's also that broader context of, like, how do you want to think about alignment, incentives, culture, motivations? And, you know, is this a good moment in time to sort of pause and rethink some of those things relative to your own company? Yeah. One friend at OpenAI who, I guess, publicly declared that this reignited their belief in clear incentives and good intent in capitalist structures that has actually seemed somewhat radical in many Silicon Valley companies over the last few years. But I think that is going to get rethought when you see what happens when there are unclear or misaligned incentives. Yeah, there's two actually really good quotes to that.

There's one, which is something which I'm going to get wrong, which is something along the line. of like, capitalism is the best way to take care of people that you don't know. You know, it's the means of actually growing the pie in many cases and providing for others through those sort of incentive of markets. But the other one is a Charlie Mungerism. And unfortunately, Charlie Munger passed away earlier today. And obviously, he was sort of a giant of industry. And he had this great saying, which is anytime I think I understand the importance of

incentives, I realize that I'm underestimating the importance of incentives. If we just think about what the other second order, like more commercial effects are, I do think that there is an interest in owning models more in open source models and in at least understanding, like, reliance on a single vendor. What do you think here? Yeah, I think there's a couple people who have built solutions during the last week that, for example, Brain Trust now has a AI proxy where you can use the Open AI SDK to effectively query multiple different models, including Mistral. and Lama through perplexity as well as a variety of other things. GPT for GPD 3.5, I think potentially anthropic. And so it just allows you to be able to both load balance your queries or prompts,

but also interchange models more easily so you can actually look at performance across them. I think Chima similarly has done something over the last week that they've released that helps with some of the proxying and other things. And so I think there are solutions like that that have started to be accelerated in a market that would have happened inevitably. I think everybody, the journey that I see people often take is they'll prototype on GPT4, they'll look at how good it is, and then they'll either keep it on GPT4, particularly if they need like advanced channel logic or other things, or if they need very high throughput and performance

and low costs, then sometimes they'll either switch to GPT3.5 or they'll see if they can fine tune something, right? And that's the only people with the normist scale. Like I don't see very many fine tunes happening in general unless, you know, somebody has an enormous scale and or proprietary data. They just don't want to get out, right? So they'll fine tune this. stroll or Lama or something. So already, I think people were thinking about that, and then that means you need to build an orchestration layer. You may need the proxy.

You may need a variety of things. The dimensions that people were evaluating an LM provider on or whether or not they wanted to control or host or fine-tune themselves just became more clear, right, where, like, reliability became more important. but the reliability, latency, cost control, capability questions were sort of naturally there. And to be clear, like Open AI leads on capability in many areas, in unique capability in some, right? Like code generation, GPT4V, right?

You can do amazing things with that, and people should go build on those tools. I think the ecosystem will mature and opening eye is a great partner. But I think the questions are just much more obvious for anybody reliance. on these models now. Yeah, and I think honestly, a lot of the bigger enterprises I knew always wanted to make sure that if they really needed to, that they could second source something. So I don't think this is a new thing. In other words, one could argue that no matter what opening I does, there'll always be at least one or two other suppliers or vendors or partners for advanced LLM simply because the market always wants an alternative, even just for negotiation

leverage. And so if you look at other markets, for example, in the router world, one of the main reasons Juniper exists is because everybody wants to make sure that they can push on Cisco for pricing. And so they always want to have a second source. That's why Juniper is always 10 to 20 percent, the size of Cisco, right? It's just second sourcing or AMD versus Intel for a very long period of time. So I think often markets will end up with other players just because big enterprises always want to have that option if they need it, even if it isn't as good. And if anything, I think Open AI kind of emerges more stable through this in ways that people didn't expect simply because there's going to be more stability at the board level in a way

that people didn't understand, perhaps, that there could have been instability, right? This is a strengthening event and a focusing event for the company, at least from what I can tell Xter. Do you want to talk about PICA and video? Yeah. Yeah. There are a couple really amazing launches happening in the video space. What's the cause for this?

Like we suddenly have text to video generation and avatar cloning in different ways. What do you think is going to happen in this space? Interesting shift has been happening because basically if you go back a year and a half, Mid Journey launched, Staple Defusion came out, Dolly 2 came out, and there's a whole wave of people saying that they were going to go build on diffusion models. And the image gen was like the thing that everybody was going to go do for like two months. And then Chat 2PT came out. And then everybody was like, oh my God, I need to go work on LLMs and language and natural

language and NLP and all the stuff. And so the entrepreneurial ecosystem went through this sort of zigzag where every was going to do image gen and a bunch of companies started going down that direction. And then the LLM stuff really kind of was substantiated through chat TPT. And then most people went that way. And a handful of founders stuck around on the diffusion model side. And diffusion models, you know, are really popping up. And obviously there was like image transformer and a bunch of other stuff. But they're mainly being used for image gen, for video and for audio, actually. And so there's a wave of people

who've continued to work and crank on this. And they're starting to come out with really interesting products. For example, PICA is a great example where it was two Stanford PhD students who'd been working on diffusion models for some time.

And they made this really amazing creative text-to-video engine. There are other companies like a Hagen or Cinescetia or others that are doing, you know, let me use these diffusion models to clone an avatar or to generate an avatar of a person so that they can either go into

the metaverse al-Azac or alternatively they can use it for marketing purposes they can use it for internal training they can use it for all sorts of applications and then there's some really cool like audio-based things coming out too which i think are starting off more sort of tools to create music or to simulate voice in the context of a soundtrack or you like make edm and you want to add voice to it right and you can just now kind of do some really interesting things there so it seems like there's this really interesting renaissance that's happening in part due to diffusion model work. And in part due to a handful of founders not getting distracted by LLMs, which are super exciting, obviously, but wanting to do things in video. It's a really exciting

trend. And I'm guessing the success of some of these companies and their traction and growth is going to pull more people over to work in this area again. I think it was just an area of less emphasis for the last year relative to language. One of my favorite dynamics that happens in sort of technology ecosystems is that once people show that something is possible, like a lot of talent floods in, right? And you kind of get, oh, you get a lot of competition, but you also get innovation coming in waves. That could be with Mistral developing open source models that are actually interesting from a reasoning perspective at relatively small parameter size. Or it could be Demi and Chenling and the PICA team creating text to video generation models that are really interesting quite efficiently

from a training perspective. And I know we're both investors here, but I've seen a huge wave of people interested, as you said, in media diffusion of different kinds. Now that they know it's possible. And it has real benefits, right? Because it's very cheap to do. And from a data set perspective, the data is reasonably straightforward to get. I mean, it's hard to get, but it's not as hard as, you know, the entire internet and transcribing voice from videos and all the rest of it. The original stable diffusion model supposedly was trained on like 600K of GPU. And these models cost in the millions to train, not tens of millions, at least initially, right? And so that's another big difference relative to the really big foundation models and

language models and all the rest, right? And so you can actually imagine that in the language world, you're going to have a lot more platforms that people build on in the diffusion model world, image, video, et cetera, you're going to have more people kind of grow their own, right? And people should still potentially try things on stable diffusion first, just to test it out. It's back to the, you know, no GP before product market fit, but they can still train their own model in a very economic way relative to like the amount of money a startup would raise. So I think it is a more accessible thing in some sense, unless you just get one build on somebody else's LLM, which people should do for most things initially as well.

Yeah, one of the things I think is really interesting about this space as well is we've actually had leading researchers like say, you know, we're still very, very early in video. Video generation is so hard. It's data intensive. The data is like, as you said, it's not the whole internet, but it's problematic in that a lot of the training has happened on short clips. People aren't sure how to caption. It's expensive to generate. You have like complex like sliding window approaches and others to try to deal with the like temporal coherence problem. There are many more unknowns about how to progress this type of product. technically? Do you mean video specifically? For video specifically? Yeah, because the teams for all

these things are actually quite small, right? The PICA team is reasonably small. The mid-journey team for a long time was tiny. And so I actually think this is a good example where you can do a lot with very small teams. And to your point, there's all sorts of technical challenges. But the reality is you can get to the cutting edge with like a handful of people in these fields, which isn't necessarily true as much for, you know, other types of models or certain types of models at least. So I do think it's striking how few people you actually need to do something really interesting here. And to your point, there's other challenges coming. Three, four years, maybe it becomes harder. I agree with you. You don't need more than a handful of people.

Like, there's empirical evidence now. Maybe a slightly different point, which is instead of like having to have a certain size of team to go deliver an LLM at scale, there are more degrees of freedom in how you would innovate technically in this area. And there's more disagreement on like how to progress. And I think that's actually just interesting for startups. Yeah. I think they should just use QSTAR. So I think that's the main solution to most problems, I feel, in AI today. Okay. Well, do you want to give me some investment advice given QSTAR? Should I go home? Yeah, you should do mainly QSTAR-centric companies. And so, you know, if they're doing QSTAR, you do it. If they're not doing Q-star, you don't do it. So that's one big piece of advice.

I think one of the things I would like to go back to and talk about is your point of view on, like, hey, a bunch of people moved away from diffusion models to LMs. One of the reasons that people moved away from diffusion models to LMs is because there's a lot more sort of text and code in enterprises that is obvious, right? Working with images and video and audio felt more verticalized, like where the B2B use cases. And I think what we're seeing increasingly is the creative fields are our, our, actually pretty commercial. Right. So one of the things I'm most inspired by, and I think there's a lot of money in, is if you look at Mid Journey, one of the biggest, like, biggest knocks on them from investors or naysayers early on was, well, like, how many people want to make images? That's not a

hobby. That's not a social network. Like, what percent of the population are artists? And this was clearly wrong just in terms of the scale that mid-Journey has already reached. There's probably two pieces here, right? Like, one, these tools like PICA and Hagen and audio generation and mid-Journey, they make the pie bigger for creative fields, especially since if you look at Pika or H-Jen, they're really focused on all creators rather than just like the film industry professional. And like if you go look at the mid-Journey use cases, And then I suspect that PICA and Hagen use cases over times, they're very commercial, right?

Like a lot of the things that you named or that people are experimenting with are really about communication, marketing, and advertising. Yeah, I think if you just look at it as market cap of incumbent, right? Adopi's a $280, $300 billion company. Like, that's huge, right? And so I don't think the creative world is small, right? I think a lot of creator economy companies have failed in the past, which is a different thing. depending on I think of the creator economy, right, in terms of celebrity-based marketing or whatever.

But if you actually look at enterprises, obviously they use enormous amounts of imagery and video and other things to reach with and interact and brand and associate with their customers. And you look internally, you need to create imagery for slides or for other things in communication. And so to some extent, you're kind of looking at different proxies and you say, okay, well, what are some of the proxies on the text-based side?

And you can say, well, you add up Microsoft and, you know, a few other companies like that and you're kind of getting a rough proxy for some form of text. Not really, but, you know, I'm just simplifying things dramatically. And then you add up Adobe and a few other companies and that's your proxy for image gen, right? And so I think both are big. And then the question, I think, always with the diffusion model-based companies was, where are the biggest application areas? And the application areas also were a little bit driven by where will incumbents play a role? Where you get blocked by other companies in the ecosystem versus, you know, it's a natural new greenfield thing. Most things aren't truly new

capabilities. Most things are just like making certain things dramatically easier. There's the duality of that. There's the market expansion and more people can do this thing. And then there's a value contraction. Hey, you can do this at a tenth or a hundredth of cost. And so often in markets like this, you see both of those things happen at the same time. You're simultaneously growing the market and shrinking it. Yeah, I mean, you have the, um, the, um, famous, uh, strategy. Like your margin is my opportunity, right? I was going to say, I mean, well, these are actually higher margin things.

I think really what you're doing, if you look at the sort of, um, my understanding, I need to look up these numbers again, but I think it's something like software spend is like, I don't know, making it up half a trillion dollars, $500 billion a year. And then services spend is like three to five trillion a year, right? And so really what you're doing is you're taking services revenue, which is very people intensive and low margin, and you're converting it into higher margin software revenue, but less of it. So maybe you take that $5 trillion and you turn it into $2 trillion, but it's 80% margin

margin, $2 trillion versus 30% margin, right? The margin dollars actually expand. And you see that in other industries. That's kind of what Andrew is doing in defense, right? They're taking a cost plus model, right? You buy a drone from Lockheed Martin for a million dollars. and you get paid by the government 5% cost plus, which means you get 5% as your margin,

so you make 50K off of it. And Andorale will sell the same drone or a better drone for $100,000 with 50% margin. I'm making up the margin, right? But that's 50K. And so you're making the same margin on a tenth of the price. And so I think one of the ways I think about Anderil as a company is they're taking very bad, low margin revenue from other defense companies

and turning it into higher margin. healthier revenue, right? Yeah, I'm going to edit the quote and just say, like, your ASP is my opportunity, right? There's a, like, a democratization that happened in the, like, latter half of the SaaS revolution, or really most of the SaaS revolution, which is instead of there's a hundred very large enterprises that have some sort of CRM because it costs X dollars to deploy and implement Siebel, then you have, you know, tens of thousands of companies who can buy Salesforce and then companies that figure out how to efficiently distribute S&B SaaS on the internet, even though that

is still hard. I think one of the things that is interesting here is, let's just take video generation as the example, the ASP of, you know, hundreds to thousands of dollars an hour, make it single-digit dollars to generate per minute and expand the audience to many more people, right? So that's the democratization that is happening. Somebody told me a joke the other day, like somebody really negative on AI investing. There's only five businesses in AI that have breakout traction right now, foundation models, wifoos, mid-journey, co-pilot and inference platforms. I think there's both truth in it and, like, all.

Also, it's not that funny of a joke because you're, like, it's true. Like, there is a set of things that people are figuring out that are really early, many of which feel pretty different than, like, yes, it's useful to look at the incumbent vendors like Adobe, but you're not fighting really the video editing software spend. You're eating into the production spend. The early internet version of this, by the way, is there's only five things you do on the internet. You go to Yahoo to look for links. You buy Pez dispensers an auction on eBay. You buy some books on Amazon. And then there's probably like two bullshitty companies that you would have quoted as like hybrid things, right?

So if you were looking at the internet circa 96, 97, whatever, you probably would have had a pretty short list of real use cases and then a bunch of stuff you just thought was kind of dumb, right? And you'd be like, look, you're not changing anything. Like, you're still using Microsoft Office and you're still using whatever shrimp grab software. You're still watching TV, right? And so I feel like we're kind of in that era of AI. The stuff that's going to happen right now is the really easy, low-hanging fruit and then a bunch of dumb things are going to get built that aren't going to work. And dumb is not meant in a pejorative way. It just means, like, it's very hard to tell what's actually a good idea in a new market like this.

And that was true of the Internet and that was true of mobile and that was true of cloud. There's a lot of these, like, waves where there's a bunch of stuff that gets. felt, right? So I think it's the same thing, right? It's a very positive sign. This has been such massive traction in such a short period of time for so many companies if you think about it. It's actually kind of amazing. So I'd actually take the other side of that, but I totally get the point. Yeah, well, with the AI focus fund, I agree with you. I didn't change the name of your fund. You should call it like Conviction Star or something. Conviction Star, yeah. But spelled with a Q, like conviction.

Yeah. Conviction star. Okay, L.P. You heard it here first. Conviction Star. Very exciting. I should send you a t-shirt. Thank you, please do. That'll actually be the swag for No Prior Season 1. If you were a guest, you're going to get the No Prior's Tequila and then a Conviction Star t-shirt. Yeah, that's very good. I'm very excited about the tequila. Anybody with the podcast has to have a tequila. We could actually call the tequila conviction star with a cue. Okay.

That'd be pretty amazing. I'm serious. Okay. It could be like a Q-shaped bottle. You know how they have like the really cool bottles for different things? A lot, Gil, guys, are, uh, no prior's branding. Brand marketer, yeah. I think I'm moving to L.A. and starting the brand. If you haven't yet been a guest, please write into the show and, you know, for the low, low price of one GPU, we'll ship you a bottle.

Yeah, we're looking for brand marketer to join the team, too. We're not, actually. If you work for Mr. Beast, just call me. The lot's going to do it. Okay, a lot. Thank you for joining me on No Pryors. Thank you for joining me. And I look forward to getting my Conviction Star T-shirt and Tequila. Exciting.

Find us on Twitter at No Pryors Pod. Subscribe to our YouTube channel if you want to see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priars.com. Thank you.