No Priors: Artificial Intelligence | Technology | Startups - Gaming, Nobel Prizes and At-Risk Businesses in the AI Era

Episode Date: October 17, 2024

In this episode of No Priors, Sarah and Elad explore how AI is transforming consumer apps and entertainment, with a focus on potential integrations in gaming and dating that could shift traditional societal incentives. They reflect on AI researchers winning Nobel Prizes in Science and Chemistry for the first time, discussing what this trend means for scientific discovery. The episode also covers recent AI releases, including their thoughts on OpenAI’s O1 model and Google’s NotebookLM, and examines which companies and job functions are most at risk—or resilient—in the face of AI advancements. Sign up for new podcasts every week. Email feedback to [e-mail] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil  Show Notes:  (0:00) Introduction (0:47) Google releases NotebookLM  (5:20) Integrating AI into consumer apps and gaming (9:11) Future of AI companionship and procreation (14:45) OpenAI o1 model improves on iterative reasoning (18:06) Sarah and Elad reflect on Nobel Prizes going to AI researchers (21:23) Jobs and businesses at risk of disruption (27:18) AI-durable companies

Sarah, welcome to No Pryors. It's such a pleasure to have you here today with me as a guest. How are you doing? Oh, my God. I'm so happy to be here. Thanks a lot. I think you are now joining the number one podcast in, actually I don't know what. The number one podcast amongst her friends.

I'll take it. Yeah, the number one podcast in our family. It's the number one podcast. My mother watches regularly. So it's very exciting for all. of our mothers everywhere. Actually, I'm pretty sure my mom is not listening to this. But if you are, hi, Mom, thanks for your support.

I feel happier for her. So I think one thing that I've been really fascinated with that came out recently is Notebook L.M from Google. And, you know, it's funny because about a year ago or so, me and David, who was on my team, decided to do this cohort at Stanford where we would just sponsor compute and other things for Stanford students to build interesting. consumer apps, and then we'd meet every week and do like office hours and just talk through other building, and there was no financial transaction or ownership or anything.

It was just, we just wanted to do cool stuff, particularly in consumer AI. And what one of the people prototype was a journaling tool, where as you wrote, the sidebar would start to interpret what you were writing in your journal, and you could choose the lens through which you would interpret it. So it could interpret it as a counselor, it could interpret it as a friend, it could, you know, it could interpret it different ways, and you're getting real-time feedback of that journal entry in the context of all the prior ones through that lens and it was really fascinating consumer behavior like it really felt like something special and notebook lm kind of feels like something

special to me from the perspective of you can upload different documents or information and it'll basically turn it into two AI posts effectively two AI bots using voice discussing it as if it's a podcast. That's one sort of application area. It just has this really interesting feel of how can you automate really interesting aspects of information discovery, how do you integrate audio in an interesting multimodal way? It just kind of pulls together a lot of really interesting pieces of behavior that really resonate. It's interesting that I think you love the voice podcast generation feature. I think other people do too. I think other people do too. No, it's not, I, I, notebook LM is my favorite new AI product of late, but that's not the way

I use it. I just use it as like a really nice little pre-built rag gooey attached to all my data. Oh, yeah, no, I agree. It's not just the podcast adding in of it. It's just, I think the last YC batch or two batches ago, they actually had two companies that would allow you to do that. So you could upload a document and effectively you'd end up with a rag index against it. And then you could interrogate it, interact with it, et cetera.

So I guess I've seen a earlier version of that. I don't think it's been as elegant as what notebook album is doing, but I felt that that plus the voice piece really stood out as an interesting step forward. And I agree with you, like the rag piece alone and the ability to interact with documents and data and information is, you know, super interesting. And one could argue to some extent, Lean provides a little bit of that answer provides a little bit. You know, there's a couple companies that kind of have been doing pieces of that in interesting ways. but I agree, we're hitting the era where any piece of content can suddenly become something that you can interact with.

Also, it really helps with a cold start problem with consumers, right? If you insert it into an existing workflow, you use existing artifacts, you connect to existing data. Like, I just think end users, like, you shouldn't rely on them to be that creative about how to engage with these capabilities. And as builders, even Google builders, great job, Google, like enable more of these interfaces. we're just going to get a step function increase in engagement on them. Yeah, it's kind of interesting.

I feel like the main consumer apps were either unintentional or kind of prosumer-ish, right? So chat GPT was kind of an experiment that was launched and became a major consumer app. Perplexity is kind of this prosumer consumer thing. You're using it for research. You're using it for search or using it for a couple different things. Mid-Journey kind of started off consumer, but I think it actually has a lot of use cases that people pay a subscription. for that are a little bit more professional in nature. You know, if you're building out graphics for a pitch deck or whatever it may be.

So it's interesting to see that a lot of these things are starting off more as utilities, while in the consumer social era, obviously there's a lot of just social apps, right, that were the main consumer wave. It was basically commerce, search, and social. And this time we're seeing a lot more around information utility, information and content generation. And, you know, this wave of technologies enabling different consumer behaviors. Are you excited about anything that is like entertainment, you know, games, social media? I'm excited about a lot of it.

I haven't seen anything yet that I think is fully there yet. And there's a lot of concepts that people have been talking about for a couple of years. You know, I remember when NMZVT 2 or 3 came out, people were already talking about, well, what if you can create NPCs that are intelligent, interact with you in a way that makes them feel like people. So if you're in the context of a video game, you should be able to interact with all the characters even if they're machine generated as if they're people, right? And so eventually you blur the line, you know, you create a guild or you create a group that goes and does something, who's real and who isn't in that group, right? What's a bot and what isn't? And at what point do most of your social interactions for a subset of people just become interacting with machines versus interacting with humans? Yeah, I saw two demos, one public that were like kind of interesting or triggering a thought here.

One was the first actual like AI integration into gaming that I thought was pretty good, right? Like between coach and NPC, but more importantly, the ability to trigger actions, like agentic actions in context with the latency that makes sense in a game. Like, I think that's finally coming in like high-scale games. And so that's pretty exciting. Yeah, that's the most interesting stuff I feel is the character is taking on attributes because obviously people can use these things to generate, you know, in-game, you know, images or backgrounds or, you know, there's actually in this group that we had at Stanford, one of the teams did a generative approach to different levels of games. So you could create infinitely many levels of games. of a game using this and it's similar to you know you look at the angry bird franchise and there wasn't really that much difference between angry birds one and angry bird 17 or whatever angry bird star

wars although that was my favorite one you just said there wasn't a difference it makes a big difference to you but the core mechanics are the same that the core mechanics are the same is just the levels are different and the um the art is a little bit you know there's tweaks right but the mechanic is the same and this basically automated that right so as you develop the game you could describe a layout and it would generate the layout for you. You could ask it to randomly generate thing. And you had the same game mechanics, but then the gameplay was a little bit different because of the setup and what you had to do. And so I think there's a lot of that coming too where you should be able to create a mechanic once and then have it infinitely scale it in terms of levels.

Yeah. And yeah, maybe you're only doing mechanic design. But I think that also really will tap into a large existing behavior in gaming, like modding, their entire platforms for this. try, like create levels, create different expansions, visual changes. And so, you know, the ability to give users, like, these powerful, like, AI-based expansions to worlds or mechanisms they love, even beyond the publisher, I think will be a big deal. That's those exciting stuff coming there. And then again, I think there's things like journaling or other types of things that happen But every generation, right, there's a product like that.

There was, in some sense, GeoCities and personal websites, and there was Tumblr and there was live journal. And there's no modern equivalent I know of right now. And so I do think AI will also modernize some of these things that are generational, where every 10 years, there's a new one of these, and it becomes really trendy for the kids. And then you take advantage of the latest technology to do it. I'm still a kid a lot. I'm going to be part of it.

I think of you that way. I think I'd be that way. Shio City's V8, guys. Actually, I've one test for whether or not you're a real AI kid. I was talking to, you know, I had this dinner at my house. They didn't invite me. You're invited. You just ignore your texts.

A good friend of mine who is a researcher at one of the labs. He just had a kid, so he's not actually like 20. But I don't know how we got on this topic. We got on the question of, I think he brought it up casually. He's like, well, like, you know, my, my son is probably going to have an AI partner instead of a, you know, like a human romantic partner, as one might believe in 2023. And he's like, I was like, oh, how do you and your, how do you and your wife feel about that? And like, no grandkids and just embedded in the AI. And he's like, oh, like, if they get, like, support and love and the emotional structure.

and fulfillment they need from that, who am I to say that that's not right? And I'm like, this is a very, like, seemingly, um, I, I would have said like a middle of the road opinion guy, except for the fact that he works on, you know, give him 10 years and he's going to want grandkids. That's just me. That's what I said. But I think this is actually like a really, um, I think this is like a more of a philosophical and political question. Like, do you, do you care about, um, uh, grandkids and continuation of the human race? and should you have any opinion on that for your kids? I want grandkids.

But this researcher made me feel like, I think, is a relatively uncommon feeling for me where I'm like, oh, man, I'm like really feeling very conservative on this topic. But how do you think about it? You know, there's a CEO of a well-known AI company. Actually, this is like his dinner question where he'll just ask people around the table, what proportion of you think that your kids will at least at one point in their life date in an AI bot or have them. like a romantic interest. You know, I think that will definitely happen. Yeah. But that's separate

from the degree to which people reproduce. You know, there's a broader question because sometimes people talk about artificial looms as well, right? There's advances in science where, you know, you can now grow, care if they use lamb or what, you know, up to some, you know, number of weeks of gestation, right, in a literal kind of membrane that's fed in nutrients and stuff. And if you'd asked me 10 years ago if I thought that was a good thing, I'd say yes. Now I actually think it's a bad thing. And I worry a lot. And it's good in terms of, you know, people can't reproduce and, you know, there's obvious application errors. I just mean everybody mainstream using it for everything in terms of how they have kids. And the main reason I have trepidation about it is I actually

feel like the movement over the last 20 years has been disassociation of people from society and dissociation of human-human interaction. And I personally think that's valuable. And there's a of people who don't think that's valuable, you know, particularly in subsets of the AI community. But I feel like that really breaks some aspects of how you think about kids and who they are related to and why and how. And, you know, it just really shifts things pretty dramatically. And Star Wars back to Star Wars again. That's actually the Clone Wars, right?

They basically just start cloning one person at large scale. And that's all the, all the Reagan file fighters on the side of the empire. But you do start wondering about human connection. where is it important and why? And I do worry that artificial wounds is kind of a step. And again, if it's mainstream, I think if it's like, hey, we are having trouble reproducing or somebody had a placental accretion or something else, I get it. But if it's like every single person should just do this, I'd just worry about what that means societally. And it could just be me being a little bit laudite-ish on this topic. And in five years, I'll change my mind and I'll

think the opposite. Yeah. But I think the implication you're describing where people just don't have as much attachment to other human beings because they don't need to, to get fed socially and emotionally. I think that will have an obvious impact of being less concerned about society. Yeah, COVID was a good example of that where I feel like a lot of people went a little bit crazy due to the lockdowns in isolation. Yeah. And in this case, it's a different form of isolation. It's no longer social isolation because you're getting social interactions with bots, but it is human isolation. And I think that's a interesting concept to explore. And again, in the extremes, when we shut down society, I think there's a lot of bad behavior and a lot of

anxiety and stress and unhappiness. And, you know, there's all sorts of reasons for that. But I think part of that was just shutting off human contact and making people scared to interact with other people. I do think even in the, like, very near term, no artificial wounds required, but some human AI dating or emotional dependence to come. I think the premium put on human-like communications is going to decrease, right? Just by volume of how possible that is. And so, you know, I definitely think like the quality and taste and all of that will still matter. But like as one example, I'm trying to cure myself of this. But when people email me, and it's a real human being, I feel some obligation to respond, even though I've never met this person.

And I think that, because somebody on the other end of trying to be kind, and I think that is going to decrease over time because you're just going to get an infinite volume of communications that feel. Yeah, this is going to be bots, emailing bots. I guess the other model product, et cetera, that came out very recently is 01 from OpenEI. And so, you know, I'd love to hear what you think about that. Let's just set context of the reaction from the world and why it might be important. The description from Open AI and the use of it is it can do longer term planning and thinking.

So it can scale compute usage at test time. And this makes it better at a certain category of tasks that require, let's say, like, iterative reasoning. and that tends to be like math, code, problems that you just need to think harder about. So like a favorite example of one of the researchers is crosswords, right? Because like all of these questions are interlinked versus it just being a like a recall-based answer or something that you can stream to the right answer with next token prediction. And so that's great because it opens up a category of use cases. that I think are, you know, less well served today, I think there has been both some enthusiasm

and some mixed reaction from the industry or end developers. It's still very early, right? And so I think, you know, every warning in the world from opening eye that, like, you can't use this exactly the way you use 40 is being ignored. But when people tested, it's not, you know, significantly better on every dimension. And so I think there's one school of thought that is like, well, like the base model is just not as good and that is the important thing. Another school of thought is, which I do subscribe to, by the way, is, you know, new scaling law, right? So will allow us to do an important range of new tasks and how good it is exactly at this moment is not the important thing. It's a new dimension of competition. Yeah, it's kind of like GPT2 for that approach and

the idea is that that could scale further over time. And it's kind of the initial early proof point that you feel like something really real is happening. And then it's just going to be subject to the same scaling laws of more compute, more data, et cetera. And it'll get dramatically better really quickly. So it's very exciting. I do think on how it is received and how people use it, as the labs, you know, attach tools and function calling to the ability to do this type of planning and iterative freezing.

I think people are going to get a lot more excited. Have you been using it at all? Yeah. Yeah, I've been using a prospect of stuff. Finding the bounds is really interesting. And, like, I don't know if you have this emotional response with AI models, but one of my colleagues, Prana of gave us, like, a brain teaser the other day as, like, one of those coin flipping games that, you know, you have to simulate and think through a bunch of scenarios.

And I got it mostly wrong, personally. And then I wanted to see, like, how much better is the model than I am? And the, I think the inability, I'm very bullish. on, you know, test time compute in general, but the inability to force the model to get the answer correct, even with clues is driving me insane. I spent like three and a half hours on it. And so I look forward to the labs innovating on the ability to better guide these models. The one other thing that's happened since we last spoke is the Nobel Prizes came out. You know, the Nobel Prize in physics was awarded to two people who've done some pioneering work in AI in AI models.

the chemistry prize went to Demis and other folks working on protein folding and the applications of AI to protein folding. What did you think of those prizes? It's the first time AI has been recognized with Noble's like, right? Turing Awards have gone to Jan Lecun and others, right? But I think this is the first time that a Nobel Prize has gone for AI. I think let's start with like the obvious but should be recognized premise of like these people are amazing and their work is world changing. That being said, I think the criteria of the Nobel is like moved that particular field. And so it's not clear to me that the award for physics was necessarily advancing the field of physics versus saying, you know, there are principles that apply from the field of physics, right?

Yeah, because there's a lot of physics that's been applied, right? So, for example, diffusion models are basically, which have been, were a lot of that really, really good image gen models were basically a statistical physics model applied to machine vision. image generation and understanding and things like that. So to your point, a lot of us almost gone the other way. Yeah. So, you know, what, I guess it begs the question of like, well, what's going on actually in physics that is interesting? And maybe it's just a loop, right? Like, we're going to go use these. That's what a lot of people believe about AI for science. We're going to use these models to make advancements in discovery, including in, you know, fundamental fields. So I look forward to that. I think the chemistry aside, I mean, you, you are actually a

biologist from a while back. So I want to hear your opinion on this. I think that is like much more obvious even, you know, I think those words are all deserved. Because it, in the ability to predict structure, predict function from structure and in, and in much more sophisticated ways beyond alpha fold, hopefully, changes like what the industry can do. But perhaps I'm being like too practical about it. Yeah, no, I think AI being applied to chemistry and biology, particularly around protein folding and protein drug interactions and things like that, really was a breakthrough. So I think that that probably merits it.

I was wondering who would get the Nobel Peace Prize from the AI community. Is it Adam DeAngelo for sort of bridging the different regimes at Open AI on the board? Is it Mark Zuckerberg for releasing Lama? Is it open? Like, who should get the, who should get the Peace Prize? We should just do an all AI nobles. Like, the Literature Prize, you know. Is that GPT4 for writing marketing copy? Hey, we're going to do live poll, guys. Just email into No Priors and nominate your, um, your Peace Prize winner. Yeah, I'm actually surprised that there weren't, that the, the AI community didn't sweep this year. I'm actually kind of disappointed. We can do better, guys. I think Gary Tan should get the Nobel Prize. He's done to work at why. I see. And in San Francisco. Yeah, exactly. Then anyhow, I really would have wished that we had a sweep. So maybe we can do that next year. A lot. One more macro business question for you. Given more recent advancements or just the investments, you're making, like, you know, sometimes we talk about what is most defensible. What do you think is most at risk? Like is that services vendors, IT outsourcing, last gen software, like who's most threatened?

Oh, sure, yeah. I've been involved with a number of companies now that are basically doing different forms of either horizontal or vertical applications of AI. And in some cases, they're really eating away at pre-existing SaaS companies or things like that. But I actually think the more interesting thing is we're augmenting or replacing people at scale. And so, you know, examples of that may be customer support and success. You know, we had Brett Taylor from Sierra. Decagon is a company I'm involved with that's doing a lot there in terms of, you know, really making customer service reps. dramatically more efficient. But I think in general, when I look at these sorts of areas, I don't really think of them as SaaS software replacement. Say there's a, there's an already existing software company like Zendesk and it's going to displace that, actually view it as augmenting the people that use the software. And so really the question is where there are large numbers of people effectively doing email jobs, right? They're cutting and pasting text. They're synthesizing information that we're reading it around. They're manipulating it. But I think more generically, I'm very interested in these markets where it's not even

even a you're displacing an existing SaaS provider. You're just changing how people are used relative to this stuff. And there's big bonus points if, for example, you need people who can speak multiple languages because AI can do that really well. And there's bonus points if you want people to be 24-7 available because AI is really good at that. And so you almost ask, what is AI really good at? And there's going to be a big shift as we see real-time voice models sort of kick in.

And it'll take some time. but people like Cartesian and others are working on that, you know, 11 labs and then the big model companies opening eye, Google, et cetera. And that's another area that I think will really get revolutionized by, hey, suddenly you have an agent that can speak 20 languages, 24-7, that doesn't get tired, that's always on track as customer support. So those are the kinds of things that I think are really interesting. Yeah, I think that makes sense to me.

I think it's a pretty nuanced question because you have companies where the value you is replacing some amount of labor or making it much more efficient. And I think the the dollar, like the value to businesses there is probably greatest. But I also, I also think you can take a lens that is like look at every existing category of, of software. And I think not even just the traditional like high growth, you know, venture-backed software categories that have like super high premium margins, but, um, uh, like thinking about it, like from a specific example perspective, just like what's good fit for capability. There are all these little vertical software companies that are essentially crud applications sold with some distribution to a segment

of the market that has no technical capability, right? Um, and think like, I don't know, like campground management software or whatever it is. Like some, some vertical like that. Yeah, it's basically Constellation Software did all this, right? They basically they buy something, I don't know, it's like 100 companies a year or something. Yeah. Some crazy number. And it's all these small niche vertical software companies and they just aggregate them up. And so they'll buy the golf course planning software company and the software company for window design. Yeah. If you build windows and this, you know, so there's all these, you know, thousands and thousands and thousands of niches for truly niche,

Social SaaS and they've just aggregated all of them and created a massive market cap. Yeah. And I think that there's like several interesting ways to think about what happens to those categories of software, right? One is just like if you think they're extractive and you can end users will be able to generate more of that software, like that could go away. If you think that a company can do it and like the distribution is the advantage, then you can replace it.

I think there's a like a sort of, you know, different, um, category that might face the same dynamics as like some, the web hosting companies that exist. I think they either need to dramatically change or you will have a new generation of them because, you know, web development is probably the first to be attacked by cogeneration to your point about like, you know, what is the work versus the software itself. Um, I think for some set of outputs, you won't want. the infinite control of something like Adobe, and you'll trade off those controls for cost and speed. And so I think it's mostly attacking the people aspect, but I think it's also taking away some of the tools aspect because you just don't need all of that power for some set of use cases, right? It's like the Canvafication of some of these industries. I think that's true. I think the other piece of it is there are going to be companies that I

call like AI durable, the ones where AI doesn't matter that much. And so the negative of it is that also means that maybe AI isn't that important for them as a tool, but the positive means that they're completely robust in the face of AI. The extreme example of that on the non-suffer world would be like a railroad, right? You have to lay down a track. You'll never be able to do it again in the U.S. due to a regulatory perspective. It's just there forever. You know, you probably have the same rough railroad for print for the next 20 years as you do today.

And then within the software world, there's a set of companies that just views very AI durable. Rippling may be a good example of that. They rolled out a recent AI product that we talked about with Matt McKinness, the CEO of a few. weeks ago. But the flip side of it is a new AI HR company isn't going to make headway against the beautiful bundle that they provide. And maybe the biggest risk to a company like Rippling is just if headcount starts dropping. And their customer base due to AI, then they lose per seat, you know, they lose some seats. But the flip side of it is somebody trying to attack them using AI seems like a tough lift. So it feels like there's companies that also are going to be very

durable in the face of this coming wave. I think Rippling is a great example of this. this, I do think that just categorizing, sort of categorically saying the system of record companies are protected is, like, I think at first blush, okay, but I actually think in some areas, you have a real chance at disrupting these anyway. Yeah, I think it's more vulnerable than that. I just think riffling is especially robust. But I do think there's some areas which are basically like a data store pretending to be something more. Yes. Yeah, I agree.

And they have an ecosystem wrapped around them, and the ecosystem is most of the value and most of the functionality. I think those people are much more at risk. I agree with you. I think a shape of company that I'm interested in is if you are coming at it from like a, I think this has to happen in the like SMB or midmarket first versus the enterprise. But if you're generating essentially the database that you described from the source material, because if what you had was a database and then people putting information into the database with. some like, you know, approval flows and ecosystem and transactions wrapped around it. If you can generate a higher quality data set from the source material, that might be, you know, communications, for example, or documents, then you end up with that software replaced.

But I think that has to happen in the low end first. And then I think in the high end, the, like some of the reasons the core systems of record have been so durable, and I think they're generally still pretty durable, is all of the, like, business process, maintenance, customization on them. And I think that's, like, generally still true, but I think you can attack the services spend with AI as well. Well, I mean, listeners, you heard it here first, like clone wars, artificial wounds. That's all we got. The episode in which you find out Sarah and a lot are Luddites. Good. The future this year is just not really distributed.

It's the takeaway, as always. Thanks. Find us on Twitter at No PryorsPod. Subscribe to our YouTube channel if you want to see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no dash priors.com.