No Priors: Artificial Intelligence | Technology | Startups - Public Markets, Image Gen, and Specialized Models, with Sarah and Elad

Episode Date: April 3, 2025

In this episode of No Priors, Sarah and Elad examine the current state of AI. They break down the recent dip in public markets, how tariffs could impact the tech industry, and where opportunities remain in large language models. They highlight the opportunities in more specialized models, new approaches to model development, and how the market is beginning to standardize with integrations like the Model Context Protocol (MCP). The episode ends with a look at early consumer AI applications and what types of expertise will matter most in the coming years. Show Notes: 0:00 Improvements in image gen 4:42 Public markets  8:08 Effects of tariffs on tech 9:42 Today’s large model market 11:34 Opportunities in specialized models 16:30 Research advances in model approaches 21:10 What expertise will matter? 24:30 Anthropic’s Model Context Protocol  26:30 Consumer applications

Hey, listeners, welcome back to No Pryors. Today, you've just got me in a lot again. It's a favorite type of episode. Sarah, Habibi, how you doing? I'm great. I'm so excited. Everything is adorable cartoons that are also, like, slightly nostalgic and sensitive. And tell me about how you react to Studio Ghibli and also just better image generation.

I mean, I'm a long-standing anime fan, so I think converting the world into everything anime or manga is a very positive step for humanity. So I view this is something I've been waiting for a while. I feel like every year or two, there's sort of this moment in the ImageGen world where people have a, wow, that's an amazing moment again. And the first version of that was like, oh my God, I think maybe even the GAN wave was the first wave. There was a GAN artwork in like 2019 or so or 2018 that went to Sotheby's for auction, which was one of the first sort of AI generated to arts back when people were doing these adversarial network-based approaches to generating artwork. And it was kind of these kludgy tool chains, but even then people were like, whoa,

look at what AI can do right now. It was super bad, you know, in comparison to what you can do today. And then there was kind of the mid-jurney, early staple diffusion wave where those models came out and people are like, oh my gosh, this thing is amazing, but everybody has seven fingers in the images, but oh, my God, it's amazing. And look at all the things we can do with it and it's going to transform society, et cetera, et cetera. I feel like we've periodically had these, and I feel like this is the latest version of that. And part of it is we're just on this amazing curve of quality and fidelity and this artwork and the ability to do. I mean, even back in the gang world, there was like style transfers and, you know, do this in the style of Van Gogh and et cetera. But the degree to

which it does it so well now and so cohesively and in so many styles and with so much aesthetic beauty and oversight is really striking. And I think we're just hitting another one of those moments where people are like, wow, this can really do it for forms of animation and other things. And all this is obviously in the context of chat GPT and OpenEI and sort of the 4-O models sort of incorporating a lot of this stuff directly. And so I think it's fantastic. We're going to see another thing like this in another year, I think. And then I think there'll be the very commercial versions of this, which already is sort

of happening. But look, we can use it for graphic design completely seamlessly versus it kind of works. And we can use it for all these different use cases. And so I feel like we're doing the horizontal version of it. and soon we'll have the vertical versions all come out. And obviously those companies like recraft and others working on the vertical versions directly. But I just view this as a super interesting evolution of the technology. So I think it's super exciting.

What do you think? I think it is funny how much, at least like our little niche of the technology ecosystem, but like, you know, anime and mango is pretty popular. The world reacts to like they want more cute. They want more beauty. I think it's really exciting. One of the interesting things this exposes is users, people overall are very good at projecting, like, where we are in terms of quality and controllability and how much more

room we have, right? I think, like, going from, you know, it's eight bits of gray scale to you have images that might be perceived as photos of real people was a huge jump, to your point of people being shocked at some point, you know, in, you know, two generations ago of image generation. And then, you know, I think one of the things that Mid Journey did was really have an aesthetic point of view and take a bunch of user feedback into account in terms of what was preferred. I actually feel like a lot of people thought of image generation, like end users, not researchers, as, you know, a little bit more of a solved problem. And I think just this is another data point of how much more like we're going to get and that people want, never mind

in video and everything. Yeah. Also text and logos and there's just a lot that's coming that people haven't done are sort of these truly integrative things where you can start truly clicking into the images and modifying pieces. And there's apps that are doing that or there's things like Korea that sort of do these real-time modifications as you're working on things. But I do think there's so much room still. We're very early, but it's still so striking. So it's a very exciting area. And I think ease of control ability is also going to give people a lot more creative power. Like one of the things that Hey Jen is demonstrated is going to come out with in product very recently is the ability to use natural language to describe emotion and voice, right? So you can like whisper ASMR and just, you know,

say, I want the whole video with this person in this way with three, you know, three words of text description. I think that kind of control ability is going to be really powerful. You can incorporate it into augmented devices. And then I would just be working through an MSR world. That's all I would, I would just live in that. Is that the ideal? No. Maybe the manga part, but the rest, not so much. Are you freaked out about the macro? You mean the NASDAQ or what? The markets. Yeah, the markets... Tariffs, inflation, which part of it?

You know, consumer confidence is at a multi-year low. The NASDAQ's down 8%. Tariffs on Chinese imports and on autos. I think there are investors and companies in market talking about how stressed they are about that. Yeah, I'm not very stressed about it. I feel like there's a degree of uncertainty in the world right now for sure, but from the perspective of people building technology companies

borrowing something truly existential happening, it's kind of business as usual. And I've been through a few of these cycles now where markets are way up and everybody's freaking out in a different direction and markets are way down and people. And, you know, the main place where it impacts the venture world or the startup world sometimes is if it soaks money out of the venture capital ecosystem and therefore valuations come down or there's less funding for the, you know, marginal startup or things like that. But other than that, these sorts of cycles tend to really wash away unless you're a super late stage company that's about to go public and there's some issue with your evaluation in terms of expectations

versus where you want to go out or something like that. But for day-to-day technology startups, particularly ones that are not doing hardware, which would be impacted by the tariffs, right? People who are just writing software, it should really be of minimal actual day-to-day impact, especially if your startup's working, like you'll be able to get customers to pay you or find funding or whatever it may be. I've been through a few of these, and every time it's been a bit of a shrug. I actually remember I went to the rest in peace, good time. Times presentation that Sequoia did in 2008. So back in 2008, there was the great financial crisis. And I was running a startup at the time. I was CEO of this small company. And Sequoia

did this big all hands where they pulled together all their founders and they had people come in and tell war stories from when the dot-com bubble collapsed and how it's time to batten down the hatches and do layoffs and the world will never be the same again and everything's over. And they were doing this as a service to the startup community, right? They were trying to help their founders kind of figure the stuff out. And I remember talking to one of Sequoia partners during it. I'm like, we're like a six-person startup. Like, who cares? And it's like, yeah, you're right. You shouldn't worry about this at all, you know?

And that's as all these financial institutions were collapsing around us. And so this strikes me as like very small in comparison to that. And I think back then that didn't have that much of a real impact to tech. You know, maybe Google did its first layoff ever. But other than that, tech just kept coming along. And if anything, the biggest tech companies in the world are now 10, you know, 20 times bigger than they were back then. So I think this is an even more minor blip that from a long-term tech perspective, like, who cares?

again, barring some unexpected path that's splitting off of this. I don't know. What do you think? It has like almost no impact on me, right? I think especially at the early end of the market, I'm like, well, the really high quality opportunities are just plenty of capital for them. I keep discovering that the capital markets are much deeper, and we should talk about this, are much deeper than I thought for very expensive, for example, foundation model plays. I still expect, like, capital availability and a lot of inflow there. I think it's probably, a little different for investors who have more public equities exposure. I bet pre-IPO crossover investors are getting more cautious, right? You have those sort of much more long-term

issues of liquidity having been starved for, you know, several years now. But I think, you know, return of M&A and like several companies ready to go public will help that somewhat. The place where the tariffs kind of matter that I think are interesting is for very specific industries where to some extent it's useful for America or the West to protect themselves. So I think automotive would be a good example where some of the Chinese car companies seem to be getting so good that if I was Europe, for example, and given the industrial basis so automotive dependent, I would probably be pushing for tariffs relative to Chinese imports of cars, right? Because the internal car industry may not be as competitive. And so I do think there's

some areas where the tariffs may be useful. There'll be some areas where they're probably being uses like a negotiation tool. And then some areas where, you know, they may be either net beneficial and net harmful in terms of actual costs passed on and things like that. But I think there may be a few areas where we should make sure that we actually have some in place. And then there may be some areas where it's going to be net negative or destructive. And then there may be some areas where it's just good for negotiating broader policy or relationships with certain external parties. People are kind of using a catch-off for all of them versus, you know, looking item by item. Yeah. I agree with that. And I think the

productive version of tariffs as, you know, a, I think there's a need for a broader industrial policy that is more supportive of the industries that we care about. And like, that's going to be a big investment, right? If we want to make key components for defense or automotive in the United States, like we are quite behind in many domains in terms of getting competitive from a skill and cost perspective. And some of those things are worth investing in on both the positive and the protection side. Yeah, I guess you mentioned depth of funding for models as part of all this. What do you think is happening in the foundation model world?

You and I were just talking about these artificial analysis charts showing convergence, like kind of monotonically more competitive market for capabilities and amazing improvement over the last 18, 24 months. But you just had the most recent Gemini release from Google. Like, they're clearly still in the game. I don't know who was doubting that. given they have infrastructure, not just researchers, but, you know, very smart people at the helm. They're competing here as well. I think one of the more interesting things is that you have

convergence, not just on capability, but also in the, like, product surface areas. Like, most people have search. They have a research product. They have reasoning in the models. I think, like, a lot of it is going to end up with, like, consumer surplus and distribution being the question. There's actually a really great website called Artificial Analysis.A.I. That shows different benchmarks that they've run against these various models for reasoning or for different aspects of, you know, how you test a model for knowledge base or for, you know, other forms of performance. Speed of tokens per time unit, et cetera, et cetera, et cetera, et cetera. So I think that's really worth taking a look at and you see that for certain areas, there is really strong convergence. And then there's almost like a cluster of models that seem reasonably within ballpark.

And again, certain things spiked dramatically in one form or another around coding or on reasoning or other things. And then you have sort of a longer tail of other models. And so at least for the core language model world, which those benchmarks are four, there definitely seems to be some forms of convergence happening. And then there's outliers, right? Like grok or X.D.A.C. coming out of nowhere with a roughly soda model in like nine months was super impressive. Or, you know, some of the things Deep Seek or others have been doing. And then, you know, they don't really have.

have benchmarks for Image General. Those obviously exist in a variety of sites in other places. But then there's a whole other suite of models that I think are discussed a lot less, right? And part of that is just the economic value. Part of it's what's in the market today. But that's things like physics, it's materials, it's robotics, certain types of science, right? It may be things that are more specialized in terms of post-training, like health-related data on top of some of these core models. And so I do think that there's a lot of other types of models that people spend a lot less time on, some of which are becoming quite interesting. Probably the place that gets the most attention outside of the foundation model world,

or the core all-l-lm world, I should say, the language models, is probably actually biology, right? I feel like there's a new biology model every week. But there's all these other fields and disciplines where I actually think there's some very big opportunities, and opportunities obviously are both societal in terms of impact, but also in some cases, actually I think there's very big markets behind them. And I think often the interest level of people working in the industry to build models is divorced from the economic value of these models, right? And sometimes that's rightfully so.

You know, there may be really interesting scientific applications that aren't very commercially applicable. And sometimes it's really misaligned where you're like, why are all these things getting funded when there's these wide open spaces for certain types of models that just nobody's working on? And so at least I've been looking a lot at what are these alternative models that are interesting from a market perspective that maybe are getting a little bit ignored right now. And then I guess there's the other question of like, is this, and I'd like to hear your thoughts on this, like, how many things do you get subsumed into these core LLMs versus their own

standalone thing? Like, do you think it's all one ring to rule them all? Or do you think it's going to be a fragmented landscape? And where do you think that fragmentation happens? Is somewhat of like two binary a distinction to say like it's a model company versus not a model company, actually. Even many of the companies that you and I in the industry would consider to be like model research companies, they are starting with some base of pre-training of like existing knowledge, which is more and more readily, existing knowledge and reasoning that is more and more readily available. In the case of robotics, you start with video pre-training. The case of other domains, if you were going to start separately focusing on code and we can talk about whether or not

that's a good idea. You want both language and code in terms of being able to interact with the model. I 100% believe that there are big opportunities in some of these domains. But one of the biggest distinctions to me is what does like the data collection engine for this look like? So if you are thinking about physics, chemistry, biology, robotics, and maybe even some more near-term commercial applications, the data you would want, the understanding for the model to learn from, it often doesn't exist yet. And so I think a theory of many of these companies that is interesting is our job is to go collect or generate it efficiently and use that to train the model.

And in that case, I think the question of like, does it need to be, you know, will it be in this single model to rule them all? There's a question of, well, is it reasonable to expect one of the existing large labs to go do that data generation, right? Like if you have to set up a physical lab with robotics to do experimentation on new chemicals, that feels more far afield than co-generation URL environments, for example. Anytime you go into the physical world, it's always harder to generate data. And that's one of the reasons that the language models, where you just effectively collect

the wisdom of the internet digitally, are the first places where we've really seen this scale of sort of breakthrough happen in recent. sometimes. And coding is a great example where you not only have a lot of the data resident, either online or digitally, but also you have very clear utility functions or things that you can test against in terms of code and its performance and et cetera. Is it doing what you think it's going to do? So those are always going to be the easiest areas. It's kind of funny. This is an odd pet peeve of mine, but it always annoys me when people who do really well as founders and traditional software and tech, start telling everybody else to go and do the hard stuff

in biology and materials and physics. And oh, you need to go be hardcore and you're like, well, you made all your money in fucking software. What are you talking about? And so I feel like there's been a long history of that, right? Like, I remember interviews with Bill Gates from 20 years ago. It was like, if I was to start today, I'd go into biology. So I feel like sometimes there's the model versions of this. You're so funny. I feel like you're the opposite. You're like, I actually have a Ph.D. in biology. That's why I know. That's why I know reality. I think the other distinction I would draw is, like, is it some, like, orthogonal, like, totally different technical thesis? Do I think there's, like, a research advance that is just very different, architecturally quite different?

I'll, like, describe categories of companies that could be relevant here. We had Karin and Albert from Cartesia on the podcast. I think states-based models are an interesting direction that are highly. efficient for certain types of data that are compressible, right? If you look, there are several plays on like formalism and like translating problems into lean and taking that as a path to increasing reasoning capability for math and code. I think there's a number of, there are a number of companies that are trying to train models that are better at taking actions in software and on the web. This is clearly also right in line of the large foundation model labs, but I think they're at least trying to work on a question that doesn't feel

fully answered in terms of consistent, generalizable RL environments for agents. And so there are spaces where I think there is a theory of why the company should exist, if true, versus just being straight in line of the OpenAI Anthropic X, like Steamroller, of course, and Google Steamroller. What did I miss? What else do you draw as a distinction or like where do you think there is opportunity? To your point in state space models, there may be advantages in terms of the speed and size of some of those models on a relative basis for very specialized tasks. And so usually I think of it as a two-by-two matrix where you have like one axis, which is sort of speed, performance cost, because those are roughly the same thing for many of these models, is inference time effectively. And then there's reasoning, fidelity, whatever you want to call it. And depending on where you are in those different quadrants, you have one quadrant, which

is like it's slow and it's expensive and it's not very smart. And obviously nobody wants to use those models. There's the, it's very slow and expensive, but it's very smart and very capable. And that's where you're like, I'm going to upload a hundred document Supreme Court brief, and it'll give me this amazing analysis I can use to argue a case or whatever, right? So high value. and it'll take a while to process and do it. And then there's the super fast, super performant

tends to just be these very specialized niche models for specific applications. And I think some of the space state models tend to work very well for that, some of the SSMs for very specific application areas. And then there's the last quadrant. And, you know, based on which of those quadrants you're in, I think it really determines the type of things

that you can build and some of the, you know, the really fast high performance tend to be more vertical focused or tend to be more focused on very specific types of tasks. And the really slow, you know, expensive ones that are actually very performant, you could imagine Birkly's versions, but it seems like the backbone for a lot of those are actually these very generalizable models where a big chunk of what you're getting is the reasoning and broader linguistic capabilities that you then apply to a domain. And then, of course, there's stuff that people build on top of it in terms of orchestration layers

and specialize bespoke things or route things at different models differentially relative to your use case. And it seems like everything that's quote-unquote agentic right now is basically doing that, across customer success and code and you go through every domain that has like a specialized approach and they always have this sort of orchestration layer built on top so you know I think I think it's super exciting to watch all this stuff and I do think some of the applications and some of the less just purely linguistic domains may be interesting in the short run I think going back to the question of like is the macro stressing you out um this there's like such a virtuous cycle in technology happening right now this is actually quite dominated by the fact that

M&A is alive again, and so we're going to have outcomes. But to your point, there's exploding surface area of stuff that these models can attack. You have, you know, research progress, people making different technical bets. You mentioned deep seek. I think model development and continued increase, like, of just more aggressive use of reasoning and test time compute is quite expensive and training continues to be more expensive. So I think the fact that there are now people trying to solve data and scale and latency problems. Like, that'll help everybody, too. Do you know if it's true that the deep seek researchers

are not allowed to leave China? I do not know if that is true. I think you in any country should want to hang out to your best talent, but perhaps not restrict people's movement. I think we should be trying to attract great talent here. We should keep all the AI researchers in the Mission District and just not let them leave.

Somewhere between Mission and Dog Patch. Yeah. Like, actually, we could just draw a line, In between our offices. They all have to go to Atlas Cafe every day. Let's talk through the, like, talent categories. Actually, for anybody who is not thinking about their kids and 10 years from now, but just thinking about the, like, next two, three years, like, what type of expertise is valued

where you should stay between, you know, my office and a lot's office in the mission and the dog patch. Okay. Like, you have researchers. You have infrastructure scaling and efficiency. We welcome all of you. Hardware software code design, right? Like, design, you know, the next generation, TPU or whatever.

There's a special visa for you to move into that region. Yes, we're here to sponsor you. Visa program. Yes. If you are ready to design chips to better handle sparsity or massive MOE models or something, like I've got a visa campaign for you. Kind of what you said, right? Like anybody who has got deep, like domain user understanding

combined with the basic product engineering, it's not basic, but the product engineering sense for this like orchestration applied ML area, Evales for agents, setting up RL environments, like still a very nascent area of like gather context plan, make like a bunch of model calls, paralyze, verify, retry, like this orchestration layer to describe. All of that, we've got a visa program for you. We're thinking about naming it. We'll hire somebody to run it.

It'll be great. We'll call it Gill and go. We're going to work on the marketing. I feel we're in the business's usual phase of AI. I think the stack is reasonably well defined. And obviously it'll change and there'll be new things in it. But I feel like, if anything, the last couple months have been very clarifying in terms of the consolidation of the things that are short-term crucial. And there's the model layer of that and all the various secretraments around agentic stuff and reasoning and et cetera.

And obviously that will only accelerate and get dramatically better in its own scaling curve. And then on the infrastructure layer, I think that's solidified a bit. I remember when Rag was a big deal about a new thing. You know, all these things, I feel like, are kind of falling into place e-vals. and how do you do them? And I think things are solidifying there with companies like brain trusts and others. And then I feel like on the application layer side,

I think bought into a notion we've been discussing for a year or two now around, you know, AI really starting to impact different services-related industries and vertebral applications in different use cases. And then I'm starting to finally see some inkling of consumer stuff again. And I think it's nascent and early, but at least people are trying.

You know, I feel like there was two or three years where nobody is really trying to do anything consumer, although one could argue that perplexity and chat GPT and Mid-Journey and all these sort of prosumery things were early consumer forays, right? And so maybe ChachyPT is the world's biggest new AI consumer product. I mean, Google was really the original one in some sense. It feels like a period of brief consolidation. And in a handful of verticals, I think we're starting to see some of the winners emerge.

And so I think it's an interesting, clarifying time. And of course, the thing I say about AI is that the more I learn, the less I know. It's the only industry where I feel like the more I learn about the market, the more confused I am, I feel like there's this brief moment of clarity. And then I'm guessing in a year, all bets are off. And all sorts of things will scramble again. But at least for now, it feels to me like a few things have kind of fallen into place for at least temporarily. This actually feels like a very comfortable time to invest for me, because to your point, it feels more like a, I don't know, maybe it's like inning three instead of inning one,

where there's a little bit of stability in the ecosystem. There's a real goodness around standardization, some standardization of integration with different, like MCP, I think, is going to accelerate a bunch of development for people. Like, I'm meeting companies where they set up a data source that is useful to the enterprise in some way that these models can interact with well. And they're like, oh, MCP server. Do you want to quickly explain to people, model context protocol and MCP and what that is and how it works? I'm going to fudge this, but I will try to describe it. So this is an attempt by Anthropic came from Ben Mann's Group in Labs. It's called Model Context Protocol, which is an attempt to spec out a standard interface for connecting, like, model capabilities to systems where you already have useful data.

That could be, like, documents, it could be logging. It could be business tools. It could be like the IDE, whatever. And Sam from Open AI said, like, they're going to support it as well. And I think this is not a complete solution. It has gotten a lot of popularity with developers over a very brief period of time. But it's just like how you expose your data to the model. And it's an open standard.

So it's not proprietary. Anybody can use it. And it's like a two-way connection between data sources and AI power tools. And big companies have done it. Yeah. I think there's still a bunch of work for developers to do in terms of, like describing, you know, their tools and how to use them very specifically and cleanly. But it does make it much easier. And I think it will accelerate agent development a lot.

But, you know, going back to this idea of like, what does it mean for the ecosystem? I think the fact that you have like, like you're accelerating the ways for models to interact with existing ecosystems. We expect agents to get better. You have a bunch of choices around model availability. As you said, there's this like clear pathway. way about how to automate certain types of work that is orchestration of these capabilities. And I think that's going to be super fertile. I do think it's very unclear what types of winning consumer experiences are possible here. There aren't consumer agents that don't look just like search or research, you know,

in the large model products that are really working yet that I've seen. But I expect to see them this year. I'm excited about it. Yeah, I think it's a cool stuff coming. When everything destabilizes, a lot of and I will. We'll be back on no priors. We'll talk to you all then. It's going to get destable again.

But I think it's a moment of calm. And calm is all relative, right? There's the enormous innovation, huge changes coming, big technology waves, new things every week. But at least there's a little bit more of a view of, okay, who are going to be some of the main players in some of these areas. And, you know, how do all these things fit together? So I think we should enjoy the calm while it lasts for, you know, the next week or whatever it is, the next few hours before the next thing drops. All right. Signing off, y'all.

Good to see you. Find us on Twitter at No Pryors Pod. Subscribe to our YouTube channel if you want to see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-dash priors.com.