No Priors: Artificial Intelligence | Technology | Startups - Building toward a bright post-AGI future with Eric Steinberger from Magic.dev

Episode Date: August 30, 2024

Today on No Priors, Sarah Guo and Elad Gil are joined by Eric Steinberger, the co-founder and CEO of Magic.dev. His team is developing a software engineer co-pilot that will act more like a colleague than a tool. They discussed what makes Magic stand out from the crowd of AI co-pilots, the evaluation bar for a truly great AI assistant, and their predictions on what a post-AGI world could look like if the transition is managed with care.  Sign up for new podcasts every week. Email feedback to [e-mail] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @EricSteinb Show Notes:  (0:00) Introduction (0:45) Eric’s journey to founding Magic.dev (4:01) Long context windows for more accurate outcomes (10:53) Building a path toward AGI (15:18) Defining what is enough compute for AGI (17:34) Achieving Magic’s final UX (20:03) What makes a good AI assistant (22:09) Hiring at Magic (27:10) Impact of AGI (32:44) Eric’s north star for Magic (36:09) How Magic will interact in other tools

So welcome to NoPriars. Today we're talking with Eric Steinberger, the co-founder and CEO of Magic. They're developing a software engineer co-pilot that will act more like a colleague than a tool. And Eric has a really fascinating background between work on meta, on different types of games, running climate science, which was a non-profit focused on the climate world, and now, of course, developing an incredibly interesting AI model and system. So welcome to NoPriars today. Eric. Thank you so much for having me. It's great. So you have a super eclectic background. Could you tell us a little bit more about your, you know, what you worked on in the early days, how that evolved into working on AI and sort of the path you take it? Yeah. Yeah, thank you. So I guess when I was 14, I just had my midlife prices and thought I had to do something important with my life and spent a year trying to look at everything. It was pretty stupid. And basically, I'd look at that things like string theory and like all the things that 14 year old would look at. I mean, like, okay, what can I spend my life on? And eventually, my mom got me a book on AI, and I didn't read it. I'm sorry, but it was like the idea was sufficient.

So it's like, okay, this could do anything. And so you should just do that, and then it does everything. Then it seemed plausible that you'd need to do reinforcement learning. So I didn't know how to code at that time. Then learn to code over a couple of years. This was sort of in high school times. And then it seemed plausible that you'd need to do reinforcement learning because otherwise you'd not be unbounded.

bit. So I sort of just started working on RL, played around with things for a bit, and eventually I reached out to someone at Deep Mind to basically do it. I was like pitching this like multi-page email. I was like, could you like do like a mini PhD thing where I'm like I'm a complete newbie, but if you can bash me, just please bash me like every two weeks and like tell me how to be a good researcher. And so eventually I got like reasonable and then did some actual research work and and work with, you know, a few other people, including Noam Brown, who had meta, on developing new Rural algorithms to be more sample efficient and just generally better and faster or whatever was the goal at the time to solve whatever environments we were interested in at the time.

So yeah, that's how I got into it. I have no background in language models when we started magic at all. It just seemed, I just was like totally not on my radar. I was like, oh, wait a second. Like, if you take this and this and put it together, like, maybe this works. and so then I sort of it felt like this huge relief of uncertainty relief of like where a GI would come from because you just put those two things together and then and then it will work was the sort of hope but yeah yeah I'm a very general background as an RL and trying to

come up with algorithms that sort of yeah just like have better structures to be more sample efficient than faster or better conversions and a lot of the emphasis on magic is for a twofold on the one hand, you're doing a large-scale custom model, specifically in part focus on code, and then you're also building sort of at the product suite that can really help address coding and working on the software development side. How did you decide to start magic? And why focus on that versus other aspects of AI? It sort of came from a place of working backwards from AGL. If your end goal is to have a system that can do everything, you can reduce that to building a system that can build that system. And so that minimal system is a system that writes code and comes

up with ideas and can validate those by writing code and running experiments, which is still like in the same order of complexity as the full thing. But at least we don't have to train Zora. And, you know, we don't have to think about 10 billion other use cases that everyone building general domain products has to think about. We only have to think about code. So it's a lot simpler in all aspects except compute and slightly simpler on the aspect and slightly cheaper on the aspect of compute, I think it's not a lot cheaper. I probably overestimated how much cheaper it would get on the compute site at the beginning. But the other things are simpler, I think. And you've taken a slightly different architectural approach, right, than what a lot of people are

doing in terms of just going with a traditional transformer model. Is there anything you can talk about there? And you know, you also had a very early emphasis on long context window. I think you were the first model that was publicly announced at like 5 million tokens. And I think that was like a year ago or something now, right? So you were way ahead of a curve on that. So I was just curious, like, how you chose a specific architecture that you decided to focus on and how you decided to focus on context windows before most people thought it was a thing. And, you know, I think you've been very sort of pioneering on a lot of these areas. Thank you.

Yeah, it seems important for models to have the ability to learn from long histories of their own and their collaborators' actions, as well as take into account a large amount of fast-changing data. And so if you imagine having 10,000 employees or everyone on Earth having their own model and wanting to feed in all their data, you can now fine tune everyone's model, maybe do some Laura tricks. But in practice, context just works better.

And that's, like in context learning is the magical part that came out of Transformers. Like this is what makes them great. I think of that as some sort of as an online optimizer in a sense that instead of compressing, a set of data, you're trying to learn an optimizer. So the perspective we take on models is instead

of, this is one of my colleagues put it this way. I find it very fitting, so I'm stealing his quote here, is instead of bringing the data to the compute, we're bringing the compute to the data. So you have a set of stuff and our model acts on that stuff rather than having like a giant

model that you have to sort of work around. The whole system is designed for this. So, yeah, like a year ago, we announced five million. And by the time this is out, we may have announced a larger number. The domain reason being that you would want to deploy these things for very long horizon trajectories, and you want them to spend a lot of time thinking, and you want the model to remember all of that. And you can't really do that by fine-tuning, because you'd have to fine-tune every, like, whatever many thousand tokens, your contact

spend always long. I don't know if people would think of this as a trick. Eric, but, like, you know, can you give us some intuition for why, you know, quality of output would be better than a retrieval-based system? Yeah, I mean, so you can comment this from two perspectives. I can explain it mechanistically, and I can, the other cheap way is to just point that Richard Souten's bitter lesson. Retrieval selects a subset of data for one completion. Our models use all the data all the time.

Clearly, a subset of data for the whole completion is a subset of all the data all the time. So if retrieval was optimal, our system could learn it. And it just turns out that it's not optimal. That's the sort of mechanistic, I guess, explanation or logical explanation as to why long context will be better. You could make arguments around the quality of long context if it wasn't sufficient. If it wasn't sufficiently high quality,

maybe having a short context window and pulling in some data. So you obviously have to evaluate this. But in principle, yeah, on the assumption of Richard Sutton's better lesson, you would want the thing that can learn your heuristic rather than a heuristic. The other area of expansive exploration right now amongst researchers for better AI code generation tends to be test time search. So, you know, more compute at inference time, you know, speaking of a mutual friend and know them, like, how do you think about this? Well, so you can think of model performance as some function of training compute times some function of inference time compute. Now, those are specific functions that are just scaling law things that you can like model, but the general

way to think about it is some function and some function. And then you would want to estimate how much inference you're going to do and how much what your total budget is. And then you would want to create the optimal trade-off in your allocation of money. You also want to consider the distribution of outputs. There'll be users who will want to spend less money and users will want to spend more money. And this is likely going to follow some sort of very sort of spiky distribution where there'll be like four users spending a million dollars and, you know, four billion users spending 10 cents and a curve in between. it seems strictly beneficial to be able to provide that choice.

So instead of training, putting all the computer to training and having that $1 million inference performance be purely from the training compute, which is just hilariously inefficient, you can allow the user to choose there, I think. Or rather, you can just deploy multiple things. The reason I can talk about this is not because everyone gets this. But basically, you clearly want to be able to regulate

the amount of computer users every time. Now, it turns out this is actually not trivial. Like, doing this is hard, like, finding, like, the right algorithms to do it. That being said, people have done it in RL for a decade. And so to those, you know, I don't want to name people. I don't know them obviously is one of them, but there are others at other labs as well. Like, there's a set of people to whom this is like the opposite of a surprise, but it's still not trivial because it's the general domain. There is no game.

Yeah, the analogy I've heard is sometimes when you're asked a question, you kind of pause and think about it. and that's almost like your inference time compute. So you're basically investing some sort of resources to actually consider the problem at hand versus just spot react to it. Correct. And there are things you have to learn during training. Like if you are asked to write a piece of code

and you've never learned coding, you can spend, the inference time compute you have to spend is ridiculous. Like you're going to have to, at inference time, learn programming, which maybe, I actually think this is possible. But it is also crazy. And the, this is like, clearly this is shared among, like, everyone who, like, posts the query.

So, so it's stupid not to make this into training. Even the best mathematicians in the world for the frontier of mathematics require a long time to solve the problem. And so, so, like, I would love to have a Terence Tao in my computer, but I would then still need to run Terence Tao for a year of human thinking time. And, like, Terence Tao will not just token by token spit out a proof for Riemann or whatever, you know. And so, so I think to achieve things like that, through pure true, training compute deployments, inference and compute work would be to drastically fail simply because it is a shortcut. You can also train this model for a quadrillion dollars. Maybe you can't actually because there's no data. But you say you could. Well, what if I just need a billion?

Like, you get the idea. So I think it's like this fundamental tradeoff that you want to be able to bake into. The humans can do this exactly as you say. And this applies to all parts of the workflow that you ask your model to do. And I guess if the goal of the company is eventually to build AGI, how does that impact your choices from a design perspective relative to some of these trade officers? Or is it more, we're going to iterate until we have a system that's very good at writing code and then it bootstraps its own next version? Or how do you think about your roadmap relative to AGI itself? Yeah, that's a great question. So I remember one of our

first conversations. We were talking about sort of the step function relevance of safety risks, let's say, right? Where like there's a lot of stuff people are panicking about that really doesn't matter in the short term for like the grand scheme of society. And like some people will get pissed from me saying this, but it's just what I actually believe. I just don't think this is like, the current complaints are like similar to what we have seen and like all other technologies and like totally resolvable. But then there comes like the evolutionary one. And I was like, okay, shit, like humans succeeded in the world because we're smarter than chimps

and we're smarter than bunnies. And, you know, like bunnies do not rule the world. And that would be cute probably, but also. It probably wouldn't be as nice for us. And like here we are turning ourselves into bunnies and apes and creating this thing that we're all thinking is going to be waste-minded and we are, this is insane.

And so the reason I think like the sort of recursive approach that you're mentioning here on Intingat, and obviously you know this is what, you know, this is what we're founded to do is exactly the, I think the only way to sort of reasonably approach this is to iteratively ask your model to solve alignment and safety at that stage. Surely you can also ask it to solve your product level problems,

but that's nice, but that's not the fundamental objective. The fundamental objective is to iterate towards AGI with a safety boundary, and there's just no knob like this. There is just no knob like this in the human world. You can say, like, oh, I'm going to like spend X percent of my resources on this, but that doesn't indicate an outcome. Like you can actually control it. But if it's like compute, you can't.

you can somewhat do it. So I think it's just the most promising path we have, both on the progress towards better models, because you can, if you just have like 100,000 brilliant researchers in your computer, that is better than having 100, especially if you can connect all their brains. And then if you can say, like, okay, we think this is safe to do, like, let's go do it, and then you're like, okay, maybe we should use that thing to do some more safety research. you can bootstrap. And so just very pragmatically, even if there was goodwill, I just don't know that we have the mechanisms to prioritize safety without automation.

So, yeah, that's the primary goal. Now, the good thing is, as we pursue this, if we're successful, if we're not successful, then I'm deeply sorry to all of our investors, including you. But if we do succeed, this will, especially if we are first or early to succeed, drop a beautifully profitable apple of a tree that happens to automate a good chunk of what we call work today which is another thing that I think is like a responsibility

of our company to do is to say look if this type of thing works it's not really an assistant it's it's just like you just put it there and like you talk to it and it's like the colleague and that's great like I think the economy does the same thing same outputs with less input or even more output with less input this is fantastic for the world if we just do it well like like capitalism and competition can be great and like progress the entire history of progress comes from this like we'd all be farmers if we weren't in favor of automation it's an it's like

it's scary i get it and and you know if you don't think about it all like for the whole way like it's very scary especially if you're affected that you're not in one of the lucky seats i'm in uh for example but but but ultimately it's good and uh so so yeah i just i just basically like when we started tried my best to think through this and and this seemed like the most productive path forward. And again, I think it's just like we're lucky into being a position where we both get to pursue an incredibly valuable

product that's a new generation of thing that just doesn't really exist yet. It's just there is no like thing that does your work for you. I think chat GPT was one of these monumental moments of a new type of thing. But you just did the first AI assistant. This is like mind blowing to all of humanity. And

this will happen once more. And then maybe once more when you like can just like query the solution. into remand. And then that's probably it, at least in this domain. And so I hope we can be a part of the second one, maybe the third one, and on the product side. But we chose the domain we chose because of what I said previously. Can you, let's talk about those two pieces separately. Like you said you can be slightly cheaper than a more generic AGI. Like the last conversation we really had one-on-one was about how much compute you might need. Right? Yeah, I was wrong. So like I guess Contrast this effort to like a more generic effort because clearly the large labs, they're using a great deal of human generated coding data.

They care about the use case. So like, you know, what makes the efforts different? I think a lot of it is direct competition. Now, I think that like there are totally things I could say here that sound like entirely believable as like major differentiators. But I think fundamentally are and I'm going to talk about them. but I just also want to highlight that I think at the core, the wider world has understood that code is very helpful, and there are ways to deploy compute to improve coding performance.

And so therefore, because a lot of compute is deployed in this domain, the need for compute is large. That said, also in parallel with the release of this, we're going to be announcing what is going to be one of the largest clusters to ever be built. So we are correcting for that. I was wrong when we initially talked. You just definitely need a lot of compute.

Now there's still, I think, a notion of enough. But it might be a lot. It's interesting, right? Because there is enough compute for search, for Google search. I don't think, if you took Gemini 1.5 Pro, and you put it into Google and you, like, did all the fine-tuning properly, and then you swap it with Gemini 4, I don't think anyone would notice

unless you go like prove remand, right? But like it's good. Like 99% of users use cases will never notice the difference so like this does the job. If like AI overviews works now. And I think this is going to like similarly be the case for some, for each thing.

Like there's going to be a model that can prove remand. You can make it a hundred times smarter. You have your proof, right? So for each thing, there's going to be improvements after a certain amount of compute. And I think I underestimated that number. and I underestimated how much others would focus on code. So anyway, just to clear our part, I think our 101, you were right.

So I guess there's the model side of it, and then there's sort of the productization of that model or the ways people access or interact with that. Is there anything you can share at this point regarding those types of things? As we have built the UX, each iteration of the, like, annexed UX internally, and thought about launching it,

we were at this interesting stage of it feels like an uncanny valley, where, like, well, you know, clearly you can see signs of life. First of all, I mean, completion is a trivial one, right? Like, like, we decided on auto-launch completions because it's just obviously going to get killed by the next thing. And then we're like, okay, it's going to take us a few months to get a prototype of the next thing. We got a prototype of the next thing.

And then, like, when you're looking at this, it was like, you can see signs of life, but, you know, like, you guys tried it when you decided to invest. So it's, you know, it's like, would you be using this to, like, write your idea? No, not yet. But maybe we can train the next model, and then that model can do it. But then that model can also do all these other things that would go into final shape.

So you just enter this stupid recursive loop. Until the point, we got to this, we get the point where we're like, okay, like, what's the final UX? Let's just make sure this never happens again. And so we're trying to meet the bar of that UX now, which I hope we will sooner rather than later. The closer you get to it, the sort of dumber it feels to launch the thing before it because you're going to replace it to a few months.

So if it's good enough for that, like, you know, how hard can it be to add these last level of few things? So I do think there is a difference between, like, you can launch an extremely capable assistant before you launch full automation. That's fine. But launching a sort of mediocrely capable assistant, like we might do it. We have a deadline internally by which we don't have the, like, you know, nobody has this right now. Like there is no amazing, can do everything. It just like feels like a true genius colleague on your team. But if we don't hit it by that time, we'll launch the other thing.

but I would prefer getting it. It's just the honest, you know, the reality is things are hard. Things are, some things, some projects are going great, and some are delayed, and some are just, you know, there are 100 fires all the time. This is just how every hard engineering project goes. Everyone who is listening to this, and has ever worked on an engineering project, it was like, this is just how it goes. I think we, you know, there are a lot of things we learned along the way.

and there's nothing we're stuck on. It's just things are, you know, oh, there's this thing we didn't think about, okay, let's fix it. And so I feel very optimistic. What makes a assistant, like, more mediocre versus amazing? Is it reliability? Okay.

Like, do you just trust it? My engineers, when I have one of them, write a piece of code and another one review it, like looking at it, why would I look at it? It went through these two guys, you know? So, like, what's the point? So is the Eval bar, like, I'm not going to do code review?

So that's like the fully automation thing, right? And then you can launch something where you're like, I'm going to do code review, but it's not frustrating. If you have to do code review, and it's, like, really taxing and you have to fix half of the problems, like half of the PRs or whatever. So I think the bar for this product is just high. It's not that, like, we are, like, so ambitious and, you know, I think that too, but I just genuinely think that there is a gigantic market that gets unlocked in a step function moment where users decide that they're no longer going to use a VS code to write code and send it to their colleagues.

They're going to use magic or whoever ends up doing this well first to write their code for them and then briefly look at it and correct every now and then what has been done. And eventually not, right? but that is a step function moment, I think. It's not, like, you're not going to use this for, like, 5% of your tasks. You're going to use this for 90% of your tasks or zero. You don't believe that this is something you can cut by use case, right? It will be trustworthy on some set of things. No, I think if you can, the leap to doing all use cases is small.

Like, you can build a UI builder, and then it's like a normal UI builder, or you can build a true, great UI builder driven by AI with some added features for that. vertical but then you can do the same thing for all the other verticals just add the features you know and so maybe your product team needs to do one by one that's feasible like that that's totally imaginable and maybe your go-to-market needs to be one by one but the model i don't think so i guess um to your point on having a very high trust team how did you think about the team that you assemble so now this is really easy um because we've raised an unbelievable amount of money from great people and we've got things to show uh that even risk-averse people who don't think from

first principles can understand that this makes sense, or who need that initial seat of trust. But I did find it very hard to recruit when we got started, to be honest, because, you know, when Dario Amode goes out and starts a company, this is trivial. Like, hi, I made GPD3. And when, like, you know, there are others like this. It's easy to establish that, that trust in the outcome. So the strategy we adopted at the start was to hire, actually kind of people that, like the ones you're referencing, people who might be, we have one guy who was just like the press that Amazon, working on like Alexa AI and they just hated it. And he was like so great. Like he just knew everything, every single paper. He was like, he is rag. Like I would go like, you know, let's talk about this and he would just know everything. And let's talk about this and he would know everything. And let's talk about this and he would know everything. And let's talk about this and he would know everything. And let's talk about this and he would know everything.

And you know, he studied engineering in college and then just got into him. He's one of the most brilliant people I get to work with. And so we hired him and like he did a bunch of stuff. Like he invented a new sharding dimension for a total training. And this is just like a random, you know, you have to like do, you have to really pay attention to identify these people. But then there is an amount of drive and loyalty that you just don't get if you're don't get, if you poach, like, the obvious guy, right?

And so that's the type of person we have. And we have a decent number of them now. We've gotten really good at identifying them. I would like to have, like, roughly four times as many if we could. But that's it. Like, again, I think with the series of announcements that's, you know, going out in the batch that this podcast is going life, that again will get, you know, easier and better.

But I love our culture. It's just everyone cares about the mission deeply, you know, what I said earlier about why we do what we do, that, you know, safety-bounded AGI recursion, like, it just takes a lot of brain power and understanding of the world to comprehend that this is the right thing to do and or a lot of trust to trust an organization with doing that in the first place. And everyone cares deeply about this, but, you know, we don't do it for like the, you know, years like your marketing. sign or whatever, you know, we don't have to, that's, it's just who we are. At the same time, everyone is deeply productive. They all, you know, when there's, one of our primary, like, one of our core engineers who writes, like, the inference engine, like, it's one of the two people writing the inference engine and stuff, and kernels. And when we need join, I was like, why do you want to join?

What do you want to do? And this was before we raised this giant stack of cash that's getting announced now. It was like the tiny amount still compared to other labs, or I guess compared to any lab by a large margin. And he was just like, well, you know, he saw this as an opportunity to. He wanted to be one of the best store. He said he wanted to try to be the best kernel engineer outside of Nvidia. And he didn't say this in an arrogant way. He just said, like, this is a ton of work. I've done this for the last few years. And I just need to be an environment where I am sufficiently challenged to do this. And he's been grinding every day. And just having like that level of

ambition, but not with like the typical San Francisco, you know, but but it's, it's like quiet with humility drive you just come into the office every day you're not like working until 3 a.m because you prove that you look like this is our culture we're to grow you do it sometimes because you're just so obsessed and you try to be healthy you do work all the time because you just care so much but we're not buying the IP by poaching someone from like a lab who tells us how they train GPT yet never done this we'll not do it we just do our thing we have our plan and you know we have brilliant people who I'm delighted have trusted us and spent their energy in their best years on our company.

What does AGI look like? I think you just talked to it and it does everything. And it asks two questions. That's important. And do you think the existence of that, I guess one could argue it can increase GDP, but it may also decrease a lot of sort of human-driven activities? How do you think about the eventual implications or impact of AGI? This is going to take longer.

I'm going to try to compress it, but it's actually quite complicated. One of the biggest problems with this question is that everyone tries to simplify it. By picking one side of the argument. For example, and this is just one example, centralizing power is terrible, so therefore you should open source everything. If I stop now, this is reasonable, right? Please don't make a quote out of this, because I don't believe this. And then you could say, well, I should not give these, I should learn how to do PR. I should not say these sentences.

Anyway, you can keep this in. So there's one way to say this. And then the other way to say this is, well, this is like nukes. We all fear this existential risk thing. Like maybe we care less about the, you know, or we care. It's not that we care less. It's just that we think these intermediate problems are completely solvable. But you can't open source how to build a nuke.

This is just terrible. So the problem is that both of these things are true. And the problem is that there are 10 questions like this. And both of the answers are true in all of them. And so what you get is people arguing on X, claiming one side and ignoring the other. And so I think there are like 10,000 possible futures and which one we end up with depends entirely on which answer we choose or whether we fight the sensible middle ground and we manage to have irrational debate.

The reality is I really truly believe that capitalism and competition are the only chance we have to provide an optimizer that is capable of. getting us to the right place. I think we need the right guardrails to do that. Not stupid guardrails. I'm not saying any guardrails. I'm saying the right guardrails. And then by the end of it, all work on a computer at least. They're probably like robot factory stuff. I know less about that. I haven't run the cost structure. But probably that too. I just don't know the cost structure will be automated. And humans will do other things. I don't think we'll be required to do work for financial gain, but we will probably be able to. Property will be a thing.

I think if you own apartments, like that won't go away. There's Etsy. Etsy is a great proof of concept for what happens after AGI. This is completely useless. Like, you could just buy the made-in-China product. It looks the same, but it's not made by the human, you know? So that's a thing, I think. Like, that will grow really big. Whoever owns Etsy, I don't know, but you will get rich. Josh, are you listening? So just do your thing and hold through if you don't get automated. Maybe you do actually, but your company doesn't. So that might be one way this could go.

I think games will be huge. People who are competitive, like all three of us, I would guess, will be deeply frustrated by the fact that they can no longer fulfill their desire for competitive interaction through work. Look, I care much more about the positive outcome of what I do than I care about winning personally. But this is a hell of a lot of fun. Like, I love what I do every day.

I get to build AGI. I mean, holy shit. And, like, I mean, isn't it, it's great to compete against other competent people. Like, this is, if it wasn't in such a serious environment, you know, I would just be enjoying it. Now, I have to be conditionally enjoying it, but enjoying it. And it's going to go away. So, so weirdly, I think, like, we're the ones who, like, aren't the most on, like, a meaning level. Because, because, like, we won't be able to contribute to society as much. Because the work I feel, like, at least I can speak from.

myself like I'm doing I feel tremendously for is fulfilling and meaningful and that's going to be deleted as much as it sucks to say and hear this I just it's going to be deleted and then and then my ability to be competitive is going to be deleted because you know deep blue beats everyone in chess and like hopefully magic or some AI system will beat everyone at coding and then like what am I you know and then like there'll be some CEO system and then like the the responsible decision will be to you know like have that thing be the CEO and then so that will happen at some point you can debate how long it takes it that doesn't really matter for the argument argument, to be honest. Definitely feels like 10 to 20 percent of society will be deeply frustrated

in a post-AGI world and there may be, you know, 70 percent that's indifferent or happy and then maybe another or 50 percent, whatever it is, and then the rest will be very excited and thrive. There's a book that is like worth reading, skimming from Ryan Event all the way back in 2016 called The Wealth of Humans and explores the question, like it's not like really focused on AGI, but it explores the question of in a time of abundance, where does your identity come from and how do we keep people like happy and productive in that society? I think it's really interesting because it goes beyond like a surface level question of just like, oh, like if we can make UBI work, like are we all okay? And the answer is no, right? Like I wasn't here for UBI to

begin with or any sort of income. Well, you can actually argue that abundance has created more issues in society than one would expect if you just look at fragilogy and other issues and you know, what people consider actual problems in the world versus real problems. And, you know, one could argue that's an outgrowth of abundance and relative peace that we've seen for 30 years. And so it's an interesting question to ask in the limit, what does that look like? Still on this abundance train. But yes, it's an interesting question. I have one more for you, Eric. So you mentioned the Riemann hypothesis a few times. Like, what is the thing that you really want to try in terms of new knowledge that you hope magic will be able to answer? Right. Because you could you can say

Riemann, you could say Navier Stokes, you could say P versus M. There's a bunch of like interesting problems in math or maybe climate or whatever else. Yeah. But if you're truly ambitious, like there's got to be a question. Look, my honest reply is that I think all of these questions are going to get answered. And my North Star, at least personally, and I think most is true for most of the company, at least, is that I just want the world to be in a good place in 30 years. And after all this is done, and thus said, and this is the past, and we talk about it, the way we talk about mobile phones, I just want the world to be in a good place. This is the largest transition we have ever faced, and work will get automated, and this is crazy.

We'll have to find new ways of finding meaning, and that's crazy. The economy will be, like, I don't even know. Governments are going to have to figure that out. But I'm curious about a number of things, but it's just not the North Star. It's a nice side effect that I think it's just, I mean, in a way, it is the North Star, right? Like, what are we building? We're building this, like, automation engine that can answer a lot of questions. But in a way, I think this is just going to happen, so you don't need to try. The thing you need to try is to make it go well. And if you make it happen and go well, all these questions are just going to get answered. Like, this is a side effect. Is it someone going to ask, like, go, what's

remote? I'm going to look it up. I'm going to try to try to understand the proof. I'm going to fail because I'm not smart enough, but I'm going to try. Like, it's been interesting. I'm going to spend like a few weeks on it and you know it's going to be fun but but that's just not that's just not my north star at least um they just want everything to be fine in 30 years if it is the world would be amazing and like because all the ways in which it could not be amazing are like terrible so so if we simply keep it not terrible I think it will be amazing because I can't come up with like a mediocre AI future it doesn't exist like I don't really like get a all the stuff we need, and we find a great way to, like,

live together and not use it as a weapon, and then, like, everyone has all the things. Nobody is starving, and, like, we all have, like, infinite computer stuff, and, like, it's a refined new meaning, and, like, all this is, like, and we're not dead, like, that's a good start. Like, that, the, that world is amazing. And so all the failure modes are terrible. So really, I'm sorry, it's just, the only thing I can think about is,

like the bimodal nature of this distribution that we're rushing into. And really what's happening is that this like smooth distribution of like this cloud of uncertainty is like slowly collapsing in like this bimodal thing. And everyone is sort of going to progressively understand this more. And we just as sitting in a chair that I'm sitting in, I just don't feel like I have the right to think about anything else. Like this is this is just the responsibility and like people look back in history and like all these questions are answered.

That's amazing, but, but, like, great that, like, this stuff did go wrong, you know? Distribution skews right. I'm sorry, this is not the answer, but. No, no, no, it's okay. The distribution skews right. It's going to be, it's going to be good. Like, I think an interesting product, like, user experience question is in, you know, in an era of co-workers rather than, let's say, completion and co-pilot type products.

Like, how do you think about magic interacting, magic is, you know, forced and foremost, a model company. But how do you think about it interacting with all of the other tools and interfaces that developers use today? Like the IDE and whatever. Like, does it matter? Great question. Very good question. I changed my mind of this like four times. Again, I was just like to be like, nobody has a clue. Everyone was like trying things. So it really just matters what the market wants in terms of product. We'll just do whatever the market wants. Our current state of belief is that you want the system to behave like an employee. So it uses the tool set that you give to your staff members in an interface that is either the same or specifically crafted for AI to be better

than a human could use a tool. For example, Grafana, log ingestion, I'm sure we can come up with better ways for AI to use this than humans are using it. So maybe I anticipate that companies will integrate into magic and maybe others. Hopefully, I'm guessing there'll be competition once this is a thing. But it'll be, I think it'll be such that systems, will be AI systems will be on a level with the human and tools will be below it. That we will not view this as a one-to-one integration. That we will view it as all these tools are being adapted for AI, the way websites had optimizations made for Google search, crawlers, crawling.

I think there will be tool optimizations made for AI. And for those that don't have it, the models will just use it natively. And the agent, the model, is the main thing that matters, and everything else will get sold for you by other companies. And it's the same way how, like, all, you know, this army of rapper companies is going to get swallowed by AGI companies doing their own agent stuff. The same thing is going to happen there.

Like, if you build your own tools, it's just going to get swallowed by simply the model learning to adopt everyone. This is just, I just like to think in the end point. So I think the end point is the model uses things the way a human does, and then maybe also more. Oh, well, Eric, thanks so much for the very wide-ranging and interesting conversation. Thanks for joining us on our Pryors. Thank you.

And, yeah, first and foremost, thank you again for supporting Magic. And thank you for giving me the opportunity to speak here. Great to see. Find us on Twitter at No PryorsPod. Subscribe to our YouTube channel. If you want to see our faces, follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-dashpriers.com.