Y Combinator Startup Podcast - Every AI Founder Should Be Asking These Questions

Episode Date: October 7, 2025

Jordan Fisher is the co-founder & CEO of Standard AI and now leads an AI alignment research team at Anthropic. In his talk at AI Startup School on June 17th, 2025, he frames the future of startups through questions rather than answers—asking how founders should navigate a world where AGI may be just a few years away.He surfaces the big questions startups should be asking in the age of AGI: Should you even start a company right now? What happens when software becomes commoditized? How do you build trust as teams shrink and AI takes on more responsibility?

Welcome to this talk. I'm extremely confused. I think maybe more confused than I've ever been in my entire life. Probably definitely more confused than I've ever been in my entire life. And I want to talk about that because I think, I don't know, when you're confused, that's the start of something interesting. If you're trained as a scientist, that's like the thing you want to pay most attention to is that moment where you're confused. Like, oh, this is interesting. I don't know what's about to happen. And I've been in technology for like my whole life. And I feel like I've always had this advantage where it's like, oh, I understand what's happening. I actually kind of think I know it's going to happen over the next five or 10 years. And I've used that to my advantage.

Like I've planned my career around it. I've founded companies around it. You know, I've kind of gotten ahead of the trend and gotten involved at the right time in a lot of different places. And that's like really been great for me. I no longer feel that way though. Like I can't see five years into the future. I can see like three weeks or less. And I really still know what's going to happen.

So I want to have a series of questions that hopefully help me figure out what's going on. Maybe it'll help you all figure out what's going on. Maybe not. Not all questions are useful. But I think asking good questions is extremely important to running a startup, running a research team, running your life. Whatever you're doing, I think we all need to stop sometimes and just ask questions. But I think the moment and time they're going through right now with AI is like, extremely challenging and fast.

And if there was ever a time to stop and ask questions, it's probably right now. All views are my own, obviously. I think we're required to say that. My day job is running an alignment research team at Anthropic. I've also been through YC. I've done multiple startups in my life. So maybe my perspective on like AI plus startups is useful for all if you're thinking about startup life or AI life.

Okay. That's enough of a preamble. Let's jump in. This is sort of like the main question that I want to ask. Everything's changing. How should that impact everything about my life? Honestly, like, should you even start a startup? Like, is a big question.

But let's assume that you're starting a startup or you're running a startup already. How should it impact your strategy? How should it impact your product? How should impact how you're building your team? Like these are big open questions. And I think AI is probably going to change how you answer them today and probably differently tomorrow. And there's like this kind of paradox that I've seen throughout my career running startups and talking to startup founders, everyone always tells you focus is everything. Like that's the

advantage that startups have. Focus, focus, focus. Big companies can't focus. That's why you can out-compete them and run circles around them. But despite the fact that focuses everything, the other truth of running a startup is you have to focus on everything. Right. Like it's your job to think about hiring and fundraising and product and strategy and go to market and just like everything, right? And then all the sudden in the middle of like a product launch, someone on your team quits and someone else threatens to quit and it's just like it's madness and it's happening all the time. This is just an extreme paradox. But I think it kind of makes founders suitable to this biggest question that we're all facing as a society, which is what's going to happen with AI and what

should we be doing? Because founders are the people that have to just answer every single question always. So I think it's a good positioning. There's been sort of like common, quote unquote common just like in the last six months best practices like what do you think about AI for your your product and people say like oh you should like think about what happens over the next six months think about what the next foundation models are going to be able to do and make sure you're planning your product anticipating what those capabilities are going to do don't like plan for the capabilities of today i think it's like extremely valid advice you should definitely take it seriously but i want to like up the ante a little bit and say actually you should be planning

two years in advance because it's extremely likely that we're we will have AGI in the next few years. Maybe it's not two, maybe it's three, right? But I think you should be planning your company and your strategy around this fact, right? And like, there's extreme uncertainty, right? So don't like take it too seriously. Don't have like literally a two year plan.

But if you're not thinking a little bit about how this is gonna change everything from hiring to marketing to go to market, et cetera, I don't think you're doing your job as a founder. Okay, so that's really the theme here today is. I wanna go through a series of questions and take this lens. of AGI arriving and also just take the lens of like what's changing near term in AI over the next six months.

I think there's been a lot of conversations around like, oh, actually the impact of AI is going to be slower than we think. And the reason is that big companies suck and they have to like, they take a lot of time to buy things. Right. They don't realize the trend and then, you know, the enterprise sales cycle is really slow. So all the Fortune 500 companies are just going to take years and years to like digest all the SaaS products that you all might want to be building. And I think that's actually extremely near-sighted. Because what's going to happen, I think, open question is the buy side, the enterprises, they're going to get armed with AGI or strong agents over the next couple of years as well, right? And that's not just via the SaaS products that they might be building.

It'll just be natively inside, right? Their teams are going to be using the next versions of LLMs to make buying decisions, right? And to figure out how to accelerate their adoption cycle. The force of AI is not just on this like product revolution that the startups are building. it's also on the buy side, right? And I think this is like an interesting, weird thing about AI is that it's going to rise, you know, the water rises and all ships rise with it.

It's not just the startups. The incumbents benefit from AI too. And I think we're seeing that in other places too where large enterprises sometimes aren't even going out and buying software from SaaS providers anymore. They're like, I can just like throw two people at Claude Code and they'll build it and it'll be dedicated to the capabilities that I need for my organization.

So buy side is going to be evolving really quickly. I think it's an open question, what does that mean? Right? I think another angle on this is, you know, when you have AI powered outbound sales, we think about what that is going to do to the marketplace, but there's going to be this in this by side too of like receiving those sale calls from AI's and trying to parse what's going on. So the dynamics aren't really clear how it's going to play out.

This is a kind of related question. Like is software going to just fully commoditize? Is it even going to make sense to run a SaaS provider in like two years or three years? Or is it like going to be the case that enterprises, really do just build all their software in-house because it's just one, you know, one prompt to cloud code, the next-gen version of cloud code, right? And actually, you just need in-house product managers and they're going to do everything for you. Like, that's one real outcome. That could happen to consumer side too. Like maybe consumers eventually are just like not downloading apps

anymore. They're just building apps on demand for themselves. They don't even think about it that way. Like, they don't even think about them as apps anymore. It's just, yeah, I want my phone to do something for me. I ask it. And yeah, I made an app for me. Sure. What's the point of downloading an app at this point. So that's one outcome, but I think another outcome actually is the opposite. It's like maybe all of this automation on generating code makes it easier to just raise the quality bar extremely high. And sure, you can make like the equivalent of today's apps very easily just by prompting, but can you make an exceptional app tomorrow, the equivalent of a great team that's working with AI to raise the bar? I don't actually know the answer to this question,

but I think it might be different depending on the vertical. And you should be thinking about this for your product. I think it's an interesting question too. Like if you can write software on demand, do you even make apps anymore? Or is it really on demand, right? Like why, like we're taking cloud code and things like it to ahead of time figure out what a user needs and then build as much software as we need to support all those use cases? But if you can do code on demand, why not do it on demand? You know, on the fly, here's this user, they're doing something in your app potentially, and the app realizes that the app can't support what they want. And on demand, you generate code for that user.

I think this is a really interesting pattern if you can make it work, but it raises all sorts of issues around like trust, right? It's one thing to do generative UI, where maybe you're changing the shape of the interface for the user, but like real new behaviors would require going down to the database level, right, or the back end. And if you want your AI to be able to do that on demand, you better trust that that AI can do its job, right?

And obviously right now, AIs are not trustable enough to make that happen. So I think trust is going to be a big theme over how these different questions play out. Similar question, like, what should UI look like? You know, I think people talk about generative UI hasn't really happened yet, but I'm like bullish to see when it does happen. Is it on-demand UI the right interface or is there something even more different that we haven't thought about? I think about this specifically in the context of multimodality, weaving together, auditory and images

and video and text, and what's the right input back from the user? You know, as a user, sometimes I want to speak, sometimes I want to use touch interfaces. It's really contextual depending on whether I'm in a crowded area, et cetera. right? I think you want to meet the user where they are. Like what's the easiest thing, what's the easiest way for them to interface with your product right now? That's the question you're thinking about. And I think all this ties into this big question they have, which is like, we have all these products today that have great distribution and people realize that we need to insert AI into it. All the major players are doing this, right? Slapping like a chatbot on,

slapping on some like agentic behavior or whatever, right? They're like retrofitting their products to try to enable something with AI, right? But that's like trying to like sprinkle pixie does. on something. It's natural to think. It's better to build it a new product from the ground up that's AI native. That's like my startup mentality, like new technology revolution. You need to start from scratch if you want to build this right. But that might not be true. It actually might be the case that retrofitting existing products with the benefit of distribution wins. And I think, again, it might not be universal. This might be vertical by vertical. But I think this is a really important question that you should be asking yourself. And like don't just have an opinion.

and like figure out the causal mechanisms that allow you to validate your hypothesis. Because I think these types of questions will make or break different products. Okay, so that's a lot about like products, product strategy. But I think every part of a startup is like extremely important. The most important thing is your culture, it's your team. Will team sizes get even smaller? I think like that's like the default that people are assuming. But I think similar to products where like you can retrofit or you can build from scratch,

I think there's going to be this question which is our like AI needs. native teams that were built from the scratch to be AI native, are they going to have some advantage over large companies that are like downsizing and finding ways to make themselves more efficient with AI, right? Are there like team patterns where you kind of like just operate differently if you started from an AI native place? And actually that might be different every six, 12 or 18 months, right? Because the capabilities of AI are changing. So like an AI native company today might be different than what an AI native company tomorrow looks like in 12 months. You might be outdated if you're not like thinking about how to retrofit yourself.

Like I said, I think trust is going to be extremely important, right? So like, how does the security model change? Like I mentioned this already in the case of on-demand code where you want your LLM to be able to go all the way back down to the database layer and do something on-demand for that customer, for that consumer. But you can't do that if you can't trust the controls that you have in place and if you can't trust the capabilities of the model to do the right thing, right? I think about this also in the context of like assistance. Like the whole point of an assistant is to be useful to do. to the consumer, to that user.

But we have all these walled gardens, and you're starting to see, like, maybe you need different agents in different settings, right? But that's not what you want. As a user, you want one agent for all of your things, right? So me, I want my personal agent and my professional agent at work to be able to work together, right? That raises all sorts of concerns.

Like there's stuff that I do on my personal time, or you do on your personal time, that maybe you don't want your employer to know about, right? So how do you make sure that the information is segregated while still allowing those agents to collaborate the right way. There's a hard questions. Can you do it, right? Can you get to the same level of of usefulness? Or do you have to compromise somehow on some of these security aspects? A deeper question about agents is, you know, we think about alignment as can we trust the AI,

but the truth is even if you have a perfectly aligned, you know, quote unquote, like intent-aligned model, it's going to be used by a corporation or a startup to build an agent for that user. And then And the question is, can you trust the startup that's building the agent, right? Is this agent that they've developed for you actually acting on your behalf? If this is like an ad-based company and you're using this agent to search for like a new brand of shoes or whatever, right? Like, is it going to be biased? Is it going to be like pumping you towards one direction? That's not what you want as the user, right?

But it might be what you want as the corporation. And then how do we start even asking these questions, right? I think going back to the previous slide around like personal versus professional. agents, I really care if I'm using that agent, I really care that it's operating on my behalf, right? If it's like secretly operating on behalf of my company and it's like optimizing things in my personal life that's better for the company than me, that's like extremely scary. And I think it gets more scary the more capable the models get. So I think it's important to start thinking about trust, right?

Like how do we how do we instill trust not just in the models but in the agents and in the companies that are building these agents, right? And I think ultimately what that really comes down to is trust. and you all are trust in the companies that you're building, right? How can a user trust this like company that's building this product, especially, you know, already today we have these extremely small teams. Tomorrow the teams might be even smaller.

You might have these like semi-automated teams almost. How can you trust that team? You might say, well, I don't know, like we trust companies today. One of the core reasons we trust companies today is because they're composed of a diversity of people. You can trust to some extent that if there's reasonable culture a company that if the company decides, if the CEO decides to do something bad, that someone in the company is going to raise their hand and say, no, I don't support this. I'm going to whistleblow.

I'm going to leak this. I'm going to quit. I'm going to take a bunch of people with me. I'm pissed. I hate this. Right. And without the people supporting the company, the company doesn't have a product, right? But in a semi-automated world, it's no longer true. And it could be the fact that a single person could make a decision that changes the entire impact of a product. And there's no single person that might be aware of that except themselves. It gets extremely easy for bad actors to do bad things. And if you followed along the history of Silicon Valley or the history of humanity, the truth is a huge majority of people are misaligned, right? Especially when money is on the line. So I think this is a really important question. And the truth is like we already

think about this. Like large enterprises, for example, already distrust startups, partially for this reason. There's a lot of reasons why large enterprises distrust startups. One of them is they're worried they're going to go out of business tomorrow, right? But it's just a a lot easier for a small startup to do the wrong thing compared to a big company, right? That's part of what makes small startups successful sometimes. So I think this is already like on the top of minds of enterprises, but it'll increasingly be on the top of minds of everyday people. So that kind of leads me to this next question, which is, well, how do we instill trust? Like, what does a new set of guardrails need to look like? If you don't have all these human

guardrails inside of your company where you've, you know, worked really hard to build great culture and build a collection of people who care about doing the right thing, if that's not the thing that makes a company ethical anymore, how else can we make sure that there are guard rails? Like what does that look like? A lot of people have been throwing around ideas on this. There's like this notion of AI-powered auditing. Like what should an audit look like in an AGR world?

There's this huge advantage that AI has over humans when it comes to bringing comfort to an audit, which is that they can be less biased. And they can also have no memory. For an example, right, if you agree as a company to let an AI audit you, And you can say, hey, if you don't find any malfeasance, if after your audit you've, you decided that we didn't do anything wrong as we claimed, maybe you have like a public mission statement that the AI is like confirming, then the AI deletes itself, right? All of its notes, et cetera, get deleted. And that's a big advantage over having human auditors where they could take information with them, right?

There's already auditing happening today, right? For legal reasons, for financial reasons, you know, or you want to be like certified organic or whatever it is, right? But you have to let people come into your company and audit you, and there's danger there, right? They might take with them like IP or they might find sensitive things and then like, you know, which were unrelated to the thing you wanted to get audited and all these bad things can happen, right? So there's like this potential, I think, for AI to give us a much stronger auditing system. That can be part of how we build trust, how we let people build trust in us and the companies that and products that we're building. So that kind of leads me to this next question, which is, should we be doing that?

Like, are there some other ways that you plan on having, on building trust with your users? Or should we be doing this? Should we be making, like, not just public commitments. Companies make public commitments all the time. Like, oh, we care about this. We care about open stores. We care about doing the right thing for the user. But are you actually willing to, like, make that a binding statement?

Are you really willing to, like, stick your neck out and say, yeah, not only do I say that this is what I believe to be valuable? I'm willing to commit to an ongoing audit from some neutral arbiter, some neutral AI powered system that will come in and inspect every single thing that happened in my company, every Slack message, everything, and ensure that actually we are making decisions that abide by the mission statement of the company. That would be something that might have teeth. Maybe this isn't the right way to do it, but I think things like this are going to be possible soon, they're not possible today, and they might be the flavor of things that we need to start building trust once we're in this world where we've lost a lot of trust.

Related question around alignment. So I think a lot about alignment. And I think there's this question of like, what parts of alignment do we have to solve? One reason we're working on alignment is because of the control issue. We want to make sure AIs stay under the control of humans. But I think there's this like extremely high pressure question for the next 12 months, which is what parts of alignment do we have to solve just to make these models more economically viable, right? Just to make sure that the agents that all the startups are building,

as their horizons get longer and longer, that we can actually trust those agents are not going off the rails, right? And if you use Claude Code today and it works like five minutes at a time for you, that's like one thing because you're going to review a lot of what Claude does. But if you're going to trust an LLM

to work for a day at a time or a week at a time before you intervene, you better have some degree of certainty that it's not going completely off the rails, right? So I think this is like a huge open question. I'm actually really positive and bullish that there is this economic pressure in a good way to make progress on alignment

because Long Horizon agents require it. But I think it's an open question how much and what aspects of alignment have to be solved for this. Changing topics a little bit. I think this is like a question that we all like maybe think has been answered, which is, is there a set of data that can give you an advantage? And if you go back in time just a handful of years before frontier models, before LLMs, the assumption, not just the assumption, the fact was that custom data mattered. Right.

Like if you wanted to build AI startup or if you're an enterprise and you're trying to deploy AI, You had this massive advantage if you had a massive data set that was custom to your need. And that was actually the only way to get useful AIs more than a handful of years ago was by training models on your custom data set. And then very quickly, what we saw was LLMs got extremely powerful and extremely general. And actually it was just better to use the general LLM than it was to train or even fine tune on your custom data, right? What I think is true is maybe it's true. This is my open question, is there might be industries where that's not the case where actually AI is maybe not great. An example might be like material science.

Can an LLM do really good material science? Does a company that specializes material science that has decades of data? Can they do better? And I think potentially yes, right? Like LLMs are great at everything that they've found on the internet, but are they great at all this like tacit knowledge that's locked up in companies that actually hasn't bled out? I think about like TSMC or ASML, right? Like they make these multi-billion dollar bets and they do an incredibly good job of keeping all that tacit knowledge they build in-house. it doesn't leak out. Frontier Ellums do not know how to build a cutting edge semiconductor fab.

That's an important fact, actually. And I think if you're a startup thinking about where there's a defensible position, and I'll touch on this again in a few slides, I think that's like an important question to keep in mind. Okay, different tactics here is you're probably all aware of like capacity issues. Everyone's trying to scale extremely quickly. I think there's demand from consumers, from your consumers, potentially from your startups, so the scale, extremely fast, right? Maybe we want to scale like 100x for the next couple years. That's faster than we can scale GPU production. So what are we going to do? And I think there's a lot of open questions around like, does fine tuning actually matter? I think a lot of people have abandoned

fine tuning instead, actually I'm just going to do better context management. Or is there like a role to play in like better routers between which model small versus large, etc. Right? So I think this is like a place where if you're interested in the technical details, you can have a competitive advantage, at least for the next year or two, because capacity, is really going to matter, right? You know, often from a product perspective, I like to say, make it great, then make it scale. If you're already starting to work on the make it scale part,

this is really important to you. And like this can be like some of the technical moat that you can build, that gets you ahead of the competitors. Getting ahead of the competitors, though, is like it's a rat race. So at some point, the models will get better, the capacity problems will improve, and then that advantage might go away. So you need to have a better answer to this question,

question to this question of like what's your moat how are you going to stay ahead what makes a durable advantage in a post aGI world right in two years or three years if i can just prompt you know clod seven or gpte seven to just replicate your startup what's your advantage going to be right and if i'm a megacorp and i have more money than you and i can throw more tokens at that like are you really going to have a durable advantage or are you just going to get clobbered by megacorp i think it's like a real question that is extremely serious. Even before AGI, I like to work on hard problems. Like, that's the moat for me. Everyone has a different type of mode. Some people are great at marketing, whatever, right? I like solving hard problems. But I think about what does it mean, like to be a

hard problem, like what's going to be hard in a post-AGI world? And I think a lot of things, actually, like TSMC, like ASML. Those are hard problems that eventually will get easy with robotics, etc. But robotics are lagging behind. So I think there's these hard problems that will exist even in two years. And if you're willing to go after hard problems, like if you have the guts to do that, then you can have a massive competitive advantage. So like what are what is that set of hard problems, you know, infrastructure and energy and manufacturing and chips? What else? Like those are the things that are top of mind for me, but like what are you, what are your answers to this question? Like what's still going to be hard and worth doing?

I think about this question a lot too. Like is there an intelligence ceiling to what you need for various different tasks, right? Like, you know, if you look at ImageGen, for example, you know, just recently with VEO3, with VideoGen, it's like, I can, I finally am getting fooled by videos. And like, my Instagram feed is starting to get full up of like Copybarra's singing songs and shit. And I'm like, actually, this is great. I love these videos.

That wasn't true three months ago, right? But can it get even better? Or is there like, for a specific task, for a specific use case, is there like some max where it's like, yeah, that's, it's. good enough, we've saturated, right? Like, that's the best you can get at writing a poem. That's the best you can get at writing a Git diff for a PR. That's, you know, an important question, because if there is a ceiling, then the commoditization for that task is going to hit much sooner, right? You actually can't stay on the edge longer by moving to the next model, figuring out how

to prompt the next model. The task saturates, right? So the commoditization pressure will be even more extreme. So I think it's important to think about this. And again, I think it's going to depend on the task in the vertical. Like some things maybe there won't be a ceiling. Some things maybe there will. This is changing tack a little bit here, but, you know, is there going to be a need for neutrality? You know, since the dawn of chatbots two or three years ago or whatever, like people have been complaining about refusals as like an example. Oh, like I asked the model to do this and it refused. If we end up relying on these models, that's a massive question for society, right? that there's going to be a handful of corporations that get to decide what is okay and not okay for an AI to do for you.

And if we start relying on AI to do everything, then those companies become arbiters of what gets built, right? That's extremely important. So is there going to need to be a notion of neutrality? Like we have today with some forms of infrastructure, right? Electrical infrastructure is neutral. Like if GE owned all the electrical grid and they were like, you can only use our electrical grid if you promise to plug in our toaster oven to it. like no other toast revenues allowed to work on our electrical grid, that would be a problem. And it's good that that didn't happen.

Obviously, we fought and lost this battle for the web. What's going to happen with AI? Do we need AI neutrality? Token neutrality. I don't know what we call it. I'll wrap up my set of questions here. I have an infinite number of questions. And, you know, I hope that these just, like, stimulated you to think a little bit as well about not just these questions, but your own questions, right?

I want to just, like, touch on one thing that's, like, always inspired me about Silicon Valley. And it's this, like, cringe thing that has always, like, you'll only see it in ironic sense, but actually I think it's really great, which is like, in Silicon Valley, we always used to say, like, what's your startup doing? And someone's like, oh, I'm trying to change the world. Everyone always wanted to, like, think big and do the big thing. And, like, at the end of the day, it was a cat sharing app or whatever, right? But, like, the desire to change the world was there.

I think startup founders really meant it when they said, I want to change the world. I think the thing that's concerning to me is, you know, I've been on the bleeding edge of AI for, like, ever. and I've, you know, been worried about AI risk forever. And, you know, five years ago when I talked to someone about AI, they'd be like, what the fuck are you talking about? You think these things are going to maybe kill us or whatever. Like, this is insane. Now when I talk to people, you know, I sit them down and I explain everything that's happening.

They already are aware, obviously, but I'm like, let me give you some more details. And suddenly they're like, oh, you can see the gears start turning. And just regular folks that I talk to in my life, they get it. And they're, like, viscerally aware that the things that were, to go through are extremely important. They're like humanity defying, society defining. Like people get this, right? It's not a nuanced thing, right?

We're for the first time ever bringing a second intelligence in the world that matches humanity and will eventually exceed human intelligence, right? Like that's just obviously extremely important, right? But I get a lot of people to the end of that chain of thought. You can see the gears turning and then they ask a question question and the question is okay then how do we make money off of this and I get extremely disappointed in that person and I totally understand why they're asking this question right like you know there's a lot

of fear in the world I think the fear of like all of us losing our jobs or not being able to build a startup that's going to be competitive anymore or maybe the door to build a startup closes forever in the next few years who knows right like there's a lot of fear and I think AI is scary right so like people start thinking, well, I need to make money right now. Like, if the economy is going to be fundamentally different in the next few years and I can't guarantee my place in it, like, I better like make my mark right now. I better make my money right now. And I think that's totally understandable. So like, actually I'm like, yeah, okay, like here, let's brainstorm some ideas for how your next company can make money. Or if you're, you know, an investor or you're an employee, whatever it is, let's brainstorm

your career. So I'm always happy to help people with that. But I also think that in that same moment, we, this is the last opportunity that we might have to make a difference, to change the world. If you're building a product, you need to use this moment. Right. Like, this might be the last product you build. This might be the last company you build. If you're inside of a company, the same applies. That might, this might be the last chance that you have over the next couple of years

to make that impact that could change the world, even in a small way. So if you have something you care about, now is the time to do it. I think about YC a lot and, you know, the slogan is first part. is YC slogan. Build something people want. That's what we all want to do. And I think the truth is what people want often is something that's good for society. But you do need to think a little bit deeper.

Like, I think people do want things they can trust. They want agents they can trust. They want bots they can trust. They want to know that when they use a product, it's not just going to like delight them for the next 20 seconds, that it's going to be good for their mental health for the next 20 years or the mental health of their children or their neighbors, right? Like we want good things. We as consumers and users, we want good things, right?

And when we say build something people want, don't just think about what people will consume. Like, what does society need? I think if you build the right thing, a lot of people will want it. So we'll wrap up and I'll say, like, I know there's a lot. Like, I get stressed out thinking about this sometimes.

But I really think as founders, there's people who are interested in being founders, like we have this unique perspective. Like, we think about things. in a way that most other people don't, right? Like, it's kind of the whole job of being a founder, is finding an edge. That's the whole thing.

And I think over the next couple of years, things are just going to move extremely fast. And the rules are going to change every six months. And you're going to have to think again, right? And if there's anyone that's positioned to stay at the bleeding edge of that, understand how those changes are impacting some of these questions and other questions that we're talking about, and then use the insights from the answers to those questions

to drive positive change. If there's anyone that can do that, it's, it's you all here. It's folks that care about thinking about these things and care about building things that people want. So I hope that that's what you all do. I hope you make money doing it too while you can. And yeah, thanks for listening. I think we got time for some questions.

Thank you so much for the talk. I think that was probably one of the more grounded and down-to-earth talks I've heard, especially the last point about build something people want. I've always thought it's build something that, the world needs is perhaps a bit more important for a time like this. Yeah. And so I guess my question for you coming up with all those questions, how to approach AGI and like the insights that come along with it,

what are a couple sources of information that like inspired you to do this, like whether that's people, podcast, books and things like that. What have been most helpful for building your mental model? Yeah, that's a great question. I hesitate to say it, but I think the honest answer is Twitter. I'm really religious about curating my Twitter though. Like, you know, if I see someone who I think has good takes, I follow them. If they have dumb takes, I unfollow them.

I think you really need to be like the master of your information diet. Not because like I'm not worried about like being influenced by bad opinions. It's more that I've just got a limited, limited energy budget to digest new ideas, right? So like don't just maximize for people you agree with. Maximize for diversity, right? I'm like in reinforcement learning context all the time. and there's this notion of like diversity in RL, like exploration versus exploitation,

you want to make sure you're doing a lot of exploration in your information diet before you're doing the exploitation, which is like starting a company, for example. Thanks for the amazing talk, guys, probably my favorite talk today. Something I think about quite a bit is, like, in a world where EGI is coming, say, in two or three years,

traditionally, like the questions I ask myself is if I have a couple of startup ideas, should I work on something that I'm really passionate about and have experience with expertise in, should I work on some place, an idea where the market's underserved or not as competitive? In a place where AGI is coming, should the most singular important question be what idea is the most defensible against AGI? I mean, it's a great set of questions. I think even before all this AGI stuff, a lot of those were good questions. Like, personally, my opinion is that once you're like six months into 100-hour

work weeks, I don't care how passionate you are about an idea, you're going to fucking hate it. And the only thing that's going to keep you going is like your desire to have the impact, your commitment to the company, your commitment to your founder, your co-founders and your team, right? So does it really matter to be extremely passionate about the domain that you're going after? In my opinion, no. But I think other people feel differently. So like that's a personal question. But I do think like being impact oriented is really important. And whether or not you're trying to have an impact or not, I think defensibility really is, in my opinion, one of the key questions, right? I mean, that's kind of half the talk, I guess, right? It's like, what's going to

change and, you know, is the thing you're building just going to be a rounding air over the next six months? And, you know, I honestly think there's a lot of money to be made on a six to 18 month horizon. If all you're optimizing for is like, you know, hockey stick curve, grow your ARR, flip the company and make a quick buck, like, you might not need something long-term defensible, right? But if you want to build something that's going to like stand the test of time and be part of this transition through the singularity and all the craziness of that, like I would say think harder about the defensibility. That's probably the most important thing.

Thank you so much. Yeah, good question. Some of those questions are really mind-bending. I've been thinking about them a lot as well. My question is sort of, I know you've tried to keep it open, but what's like your personal opinion on like the value of money? Do you think as the cost of goods and services go down, money will become like less valuable because, you know,

everything's free or it will become more valuable because we can do like way more stuff? Yeah, it's a great question. Yeah. I mean, I think there's going to be policy decisions that impact this a lot, right? like do we need some form of UBI or like, you know, maybe like slightly more weird like universal basic compute? Like if compute is the thing that powers everything, are we all entitled to some form of compute? These are like open policy questions that will start impacting that answer. And I think we should think seriously about them because they will also change the nature of our society and the sort of like checks and balances that we have over our democratic institutions, right?

Like if the government's giving out a UBI to everyone, that's an extreme amount of power. Suddenly the government has over the entirety of society, right? But on the flip side, if we don't do something like that, I do think that we might end up in a world where, you know, there's two primary forces at work today. It's capital and labor, right? You know, I won't say the names of people that I wouldn't want to work with, but there are people in this world that I wouldn't want to work with. And a lot of people choose not to work for those people, right? That's like a massive hit to those people, right? But once AGI arrives, you don't need labor buy-in anymore, right?

Like, you don't need folks like me to approve of the thing that you're building or of your morals or whatever, right? Like capital begets capital in that world. And that can easily spiral out of control, right? We already have a lot of concentration of wealth. That could really spiral out of control. So I think we're like in a between a rock and a heart space. I don't know what the answer is. But I think the policy decisions we make will have a huge impact on the question that you're asked.

Yeah. Thank you. Yeah, good question. Hey, Jordan. Thanks so much for this. time. I guess the question I want to ask is how do you think about alignment at individual at the level of individual users and I guess how important is that for trust especially given that you know preferences evolve over time and you don't

want to keep retraining model especially also when you have a lot of users. Yeah I mean I think like everything I have like a startup product lens on everything right and it's like startup mantra is that users don't know what they want right and I think that's true to a large extent right but I think users still have values and you want to discover those values and be an honest them to some extent, right? So I think like something that's top of mind for me is, you know, you probably saw like the sycophantic behavior from the recent, you know, other other AI provider. And like, I think if you put in front of a user two responses and one of them is more sycophantic,

one's like glazing, I guess, is what the kids say nowadays, right? Then a lot of users will pick the sycophantic response, right? Like in that moment, they're like, yeah, of course, like, of course the question I'm asking is a great fucking question. Thanks for recognizing it, right? But I think if you take a step back and you say, hey, hold on a second, here's two principles. And you can choose the AI that follows this principle or this principle. And the first principle is that we're never going to blow smoke up your ass. We're only going to tell you if we like something or if an idea is good, if it really is.

And the other principle is, actually, we're just going to, like, we're just going to glaze you all day. Like, that's what we're going to do. If you ask the user, which principle do they want? Almost everyone's going to say the first one, right? So I think doing what the user wants, you get to a different answer depending on what level of the engagement you're asking them. And I think that's really important because people can abuse that to their advantage by only asking the user the question in certain ways, right? And I think you need to really ask yourself, what's the right way to ask the user this question to get at the heart of what is going to be best for them? I don't know if I guess you. I kind of went off on a diet try a bit.

That makes sense. Thanks so much. Cool. Great question. Yeah. Thank you for the talk. I really enjoyed it. It seems to me like you value critical thinking a lot. So I was wondering what topics or what opinions does like the general crowd and like the tech field have that you disagree with. I guess my high level general statement that I'll tell you and I'm happy to like talk talk more offline. I think despite the fact that we say our industry is this like forward looking industry and we're like bold and we love taking risks and putting everything on the line and seeing what other people don't see. I think the truth is that there's an extreme amount of group think, right?

Like the products that we build, what gets funded by VCs, et cetera, right? And I think even today, like, you talk to a bunch of VCs and they're like, oh, yeah, like, I'm ahead of the curve. Like, I'm investing in AI. I'm like, no, like, you're two years behind already, right? Like, tell me about what you think needs to happen in two years, right? Like, are you asking these types of questions of like, what do I need to be investing in today so that it's resilient in two years? And I almost never see a VC asking that, that question, right? But if I was VC, which I'm not, that would be my investment thesis.

As a founder, that's my investment thesis. Like, I don't know if that answers your question, but happy to go to. Thank you. Yeah, cool. Great question. Yeah. You mentioned how trust is going to be a bigger issue in the future. So do you think that blockchain might be a part of the solution there?

Thank you. I will just preface by saying that I'm a huge blockchain doubter. I don't know. But nonetheless, the price keeps going up. and I don't get any of that because I refuse to buy it. But I do think, you know, in this world where we need trust, yeah, like, that's the right set of ideas. We need ideas like that and others, right? Like, you know, something that is I was touching on is like, you know, AI powered audits.

Can like two different AI companies audit each other? Like, how do we like get to places where we can build trust? If we do end up with some form of like universal basic income or basic compute, basic tokens, does that need to be mediated by a blockchain like so that it's not? not just at the behest of a central government. Like, yeah, I think those are like reasonable questions that maybe I could get behind a blockchain on. Good question, yeah.

Thank you. Yeah. So Google recently released an ATA protocol to standardize how agents would talk to agents. And we talked a little bit about trust on agents. So I'm curious how, like, you would envision a world of agents talking to agents and how that would affect applications

and will there be like a agentic premise scheme? Yeah. It's a great question. I could talk about this all day and I'm almost out of time. So maybe I'll just like, I'll give you one example where it's like not obvious why this was hard. But like think about like a personal assistant agent that's just scheduling meetings for you. Right. Seems trivial, right.

Like, oh, yeah, I'm just going to look at this person's calendar and then I'm going to like suggest times. Right. But actually like you've already lost the game of being a good personal assistant because there's a game theory component to how you schedule meetings. Right. And if you're like too liberal with like showing that there's free slots, you're like communicating this person's not busy. or you're communicating that like it's really important to have this meeting with the person that you're scheduling. Whereas like if you say, yeah, like I'm happy to schedule a meeting between you and Joe, but it's like two weeks out, three weeks out, right? Like there's a power dynamic that you're doing, right? And like the truth is like all these game theoretic things matter.

A good assistant knows all these things, right, and like abides by them. But it's all implicit, right? It's not like, oh, there's this like concrete piece of information that the agent has access to and that's the important part of the security. It's like way more subtle and semantic. So I think it's just hard, but I think it's a great question. Thank you. Cool. And I think out of time, unfortunately, but feel free to like shoot me a message online or I don't know how to get like my Twitter's Jordan Fisher, Jordan Ezra Fisher. Shoot me a message. Always happy to chat.

And thanks for the great questions and hope you all enjoy the talk.