a16z Podcast - Marc Andreessen and Amjad Masad: English As the New Programming Language

Episode Date: October 23, 2025

Amjad Masad, founder and CEO of Replit, joins a16z’s Marc Andreessen and Erik Torenberg to discuss the new world of AI agents, the future of programming, and how software itself is beginning to build software.They trace the history of computing to the rise of AI agents that can now plan, reason, and code for hours without breaking, and explore how Replit is making it possible for anyone to create complex applications in natural language. Amjad explains how RL unlocked reasoning for modern models, why verification loops changed everything, whether LLMs are hitting diminishing returns — and if “good enough” AI might actually block progress toward true general intelligence. Resources:Follow Amjad on X: https://x.com/amasadFollow Marc on X: https://x.com/pmarcaFollow Erik on X: https://x.com/eriktorenberg Stay Updated: If you enjoyed this episode, be sure to like, subscribe, and share with your friends!Find a16z on X: https://x.com/a16zFind a16z on LinkedIn: https://www.linkedin.com/company/a16zListen to the a16z Podcast on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYXListen to the a16z Podcast on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711Follow our host: https://x.com/eriktorenbergPlease note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures. Stay Updated:Find a16z on XFind a16z on LinkedInListen to the a16z Podcast on SpotifyListen to the a16z Podcast on Apple PodcastsFollow our host: https://twitter.com/eriktorenberg Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures. Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

We're dealing with magic here that we, I think, probably all would have thought was impossible five years ago, or certainly 10 years ago. This is the most amazing technology ever, and it's moving really fast, and yet we're still, like, really disappointed. Like, it's not moving fast enough, and, like, it's right on the verge of falling out. We should both be, like, hyper-excited,

but also on the verge of, like, slitting our wrists. It's like, you know, so the gravy train is coming to an ad. Right. It is faster, but it's not at computer speed, right? Right. It's sort of like watching a person work. It's like watching John Carmack on cocaine. The world, okay.

The world's best programmer on a stimulus. Yeah, that's right. Every few decades, programming takes a massive leap forward, and this might be the biggest one yet. In this episode, Mark Andrescent and I are joined by Amjad Massad, CEO and founder of Replit, to talk about how AI agents are changing what it means to code. We discussed the end of syntax,

the rise of agents that can think and build software for hours, and how reinforcement learning and verification loops are pushing AI towards something that looks a lot like reason. And finally, I'm Judd shares his story from hacking his university database in Jordan to building one of the most powerful developer tools in the world. Let's get into it. So let's start with, let's assume that I'm a sort of a novice programmer. So maybe I'm a student, or maybe I'm just somebody, I took a few coding classes and I've hacked around a little bit, or I don't know, I do Excel macros or something like that. But I'm like not, as it swell as I'm not like a master crossman of coding.

And somebody tells me about Replit and specifically AI and RELB. What's my experience when I launch in with what Replit is today with AI? Yeah, I think the experience of someone with no coding experience or some coding experience is largely the same when you go into Replit. Okay. The first thing we try to do is get all the nonsense away from setting up development environment and all of that stuff and just have you focus on your idea. So what do you want to build?

Do you want to build a product? Do you want to solve a problem? Do you want to do a data visualization? So the prompt box is really open for you. You can put in anything there. So let's say you want to build a startup. build a startup you have an idea for a startup i would start with a paragraph long kind of description of what i want to build the agents will read that it will you just you just type

in standard english you just type it in i want to sell craps online so you just like type in i want to sell craps online you can it literally could be that four words or five words okay or it could be if you have a programming language you prefer or stack you prefer you could do that but we actually prefer for you not to do that because we're going to pick the best thing for we're going to classify the best stack for that request. Right. If it's a data app, we'll pick Python stream that, whatever.

If it's like a web app, it'll like JavaScript and Postgres and things like that. So you just type that. Or you can decide. You can say, and I want to do it in. Yeah. I know Python or I'm learning Python at school when I want to do it in Python.

That's right. The cool thing about Replit is we've been around for almost 10 years now, and then we built all this infrastructure. Replit runs any program language. So if you're comfortable with Python, you can go in and do that for sure. Okay.

And then just again, I know this is obvious people have used it, but like, I'm dealing in English. Yes. Go ahead. Yes, you're fully in English. I mean, just a little bit of sort of background here. Like when I came here and pitched to you like 10 years ago or whatever, seven years ago, what we were saying is we were exactly describing this future is that everyone would want to build software. And the thing that's kind of getting in people's ways is all the what Fred Brooks called the accidental complexity of programming, right?

They're like essential complexity, which is like how do I bring my startup to market? and how do I build a business and all of that. Accidental complexity is what package manager do I use? All of that stuff, we've been abstracting away that for so many years. And the last thing we had to abstract away is code. I had this realization last year, which is, I think we built an amazing platform, but the business is not performing. And the reason the business is not performing is that code is the bottleneck.

Yes, all the other stuff is important to solve, but syntax is still an issue. Syntax is just an unnatural thing for people. So ultimately, English is the program language. But by the way, just to do it, good, does it work with other world languages other than English at this point? Yes. You can write in Japanese and we have a lot of users, especially Japanese. That tends to be very popular.

So does it support these days, like, does AI support every language or is it still, do you still have to do custom work to craft a new language? No, most mainstream languages that has 100 million plus people who speak at AI is pretty good at it. Okay. Yeah. Wow. So I did a bit of historical research recently for some reason. I just want to understand the moment we're in, because it's such a lot of a special moment. It's important to contextualize it. And I read this quote from Grace Hopper. So

Grace Hopper invented the compiler, as you know. At the time, people were programming machine code, and that's what programmers do. That's what the specialists do. And she said, specialists will always be a specialist. They have to learn the underlying machinery of computers. But I want to get to a world where people are programming English. That's what she said. That's before Kurt Patti, right? That's 75 years ago. And that's why I invented the compiler. And in her mind, like C programming is English. But that was just the start of it. You had C and then you go higher level, Python, JavaScript. And I think we're at a moment where it's the next step. Instead of typing syntax, you're actually typing thoughts, which is what we ultimately

want. And the machine writes the code. And the machine writes the code. Right, right. Yeah, I remember you're probably not old enough to remember, but I remember when I was a kid. There were higher level languages by the 70s, like basic and so forth than Fortran and C. But you still would run into people who were doing assembly programming, assembly language, which, by the way, you still do. Like, game companies or whatever, still do assembly to get up. And they were hating on the kids that are doing basic. So the assembly people were hating the kids doing basic,

but there were also older coders who hated on the assembly programmers for doing assembly and not doing direct machine code. Right. Not doing direct zero and one machine code. So people don't know. Assembly language is sort of this very low-level programming language that sort of compiles to actual machine code. It's incomprehensible gibberish to most programmers.

You're writing an octal or something like very, very close to the harbor. But even still, it's still a language it compiles to zeros and ones, whereas the actual real programmers actually wrote in zeros and once. Yeah, and so there's always this tendency for the pros to be looked down the nose. Yeah, and say the new people are being basically sloppy. They don't understand what's happening. They don't really understand the machine. And then, of course, what the higher level of distractions do is they democratize.

The absolutely irony is I was part of the JavaScript revolution. I was at Facebook before starting Replett, and we built the modern JavaScript stack. We built ReactJAS and all the tooling around it. And we got a lot of hate from the programmers that you should type Vanilla a JavaScript directly and I was like, okay, whatever, and now that's mainstream. And now those guys that built our careers on the last wave we invented are hating on this new wave. It's just, people never change. Okay, got it. Okay, so you're typing English. I want to sell grapes online. I want to do this. I want to have a t-shirt, whatever the business is. Okay, what happens then?

Yeah, and then a replet agent will show you what it understood. So it's trying to build a common understanding between you and it. And I think there's a lot of things we can do better there in terms of UI, but for now, I'll show you a list of tasks. I'll tell you, I'm going to go set up a database because you need to store your data somewhere. We need to set up Shopify or Stripe because we need to accept payments. And then it shows you this list and gives you two options initially. Do you want to start with a design so that we can iterate back and forth to get locked the design down? Or do you want to build a full thing?

Hey, if you want to build a full thing, we'll go for 20, 30, 40 minutes. And the agent will tell you, go here, install the app. I'm going to go set up the database, do the migration. write the SQL, build the site. I can also test it. So this is a recent innovation we did with Agent 3, is that after it writes the software, spins up a browser,

goes around and tests in the browser, and then any issue, it kind of iterates, kind of goes and fix the code. So I'll spend 20, 30 minutes building data. I'll send your notification. I'll tell you the app is ready. So you can test it on your phone, go back to your computer.

You'll see, maybe you'll find a bug or an issue. You'll describe it to the agent. And I'll say, hey, it's not exactly doing what I expected. or if it's perfect, you're ready to go, and that's it. By the way, there's a lot of examples where people just get their idea in 20, 30 minutes, which is amazing. You just hit publish.

You hit publish, a couple clicks. You'll be up in the cloud. We'll set up a virtual machine in the cloud. The database is deployed. Everything's done. And now you have a production database. So think about the steps needed just two or three years ago in order to get to that step.

You have to set up your local development environment. You have to sign up for an AWS account. You have to provision. the databases, the virtual machines, you have to create the entire deployment pipeline. All of that is done for you. And it's just a kid can do it, a layer person can do it. If you're a programmer and you're curious about what the agent did,

the cool thing about Replit, because we have this history of being an IDE, you can peel the layers. You can open the file tree and you can look at the files. You can open Git, you can push to GitHub. You can connect it to your editor if you want. You can open an Emacs. So the cool thing about Replit, yes, it is a vibe coding platform that abstracts away all the

complexities, but all the layers out there for you to look at. That was great. But let's go back to the side. You say, I've got my idea. You plug it in and it says it gives you this list of things. And then when you describe it, you said, I'm going to do this, I'm going to do that. The I there in that case was the agent as opposed to the user. Yes.

And so the agent lists the set of things that it's going to do. And then the agent actually does those things. Agent does those things. Yeah. That's a very important point. When we did this shift, we hadn't realized internally at Rapplet how much the actual user stopped being the human user and it's actually the agent programmer. agent programmer.

Right. So one really funny thing happened is we had servers in Asia, and the reason we had servers in Asia because we wanted our Indian or Japanese users to have a shorter time to the servers, when we launched the agent, their experience got significantly worse. And we're like, what happened? Like, it's supposed to be faster. Well, turns out as farce, it's because the AIs are sitting in the United States. And so the programmer is actually in the United States.

You're sending the request to the programmer, and the programmer is interfacing with the machine across the world. So, yes, suddenly the agent is the programmer. Okay, so the new terminology agent is a software program that is basically using the rest of the system as if it were a human user, but it's not. It's a bot. That's right. It has access to tools such as write a file, edit a file, delete a file, search the package index, install a package, provisioned a database, provisioned object storage. It is a programmer that has the tools and interface.

It has a sort of an interface that is very similar to human programs. And then, you know, we'll talk more about how this all works, but a debate inside the AI industry is with these, it was kind of this, you know, this idea now of having agents that do things on your behalf and then go out, you know, go out and kind of accomplish missions. There's this, you know, kind of debate which is, okay, how, like, obviously, you know, it's a big deal even to have an AI agent that can do relatively simple things. To do complex things, of course, is, you know, one of the great technical challenges of the last 80 years, you know, to do that. And then there's sort of this question of like, can the agent go out and run and operate on its own for five minutes, you know, for 15 minutes? You know, for 15, minutes for an hour, for eight hours. And meaning like, sort of like, how long does it maintain coherence? Like, how long does it actually, like, stay in full control of its faculties and not kind of spin out? Because at least the early agents or the early AIs, if you set them off to do this, they might be able to run for two or three minutes, then they would start to get confused and

go down rabbit holes and, you know, kind of spin out. More recently, more recently, you know, we've seen that agents can run a lot longer and do more complex tasks. Like, where are we on the curve of agents being able to run for, how how long and for what complexity of tasks before they break? That's absolutely the, I think the mean metric we're looking at, even back in 2023, you know, I've had the idea for software agents, you know, four or five years ago now.

The problem, every time we attempt them, the problem of coherence, you know, they'll go on for a minute or two, and then they'll just, you know, they did you compound and errors in a way that they just can't recover? And you can actually see it, right? Because they actually, if you watch them operate, they get increasingly confused. And, you know, maybe even deranged. Yeah, very deranged. And they go into very weird areas.

And sometimes they start speaking Chinese and doing really weird things. But I would say sometime around last year, we maybe crossed a three, four, five minute mark. And it felt to us that, okay, we're on a path where long, you know, long horizon reasoning is getting solved. And so we made a bet. And I tell my team. So it's a long horizon. reasoning meaning, reasoning meaning like dealing in like facts and logic in a sort of complex way and the long horizon being over a long period of time.

Yes. With many steps to a reasoning process. Yeah, that's right. So if you think about the way large language models work is that they have a context. This context is basically the memory, all the texts, all your prompt and also all the internal talk that the AI is doing as its reasoning. So when the AI is reasoning is actually talking to its health, it's like, oh, now I need to go set up a database. Well, what kind of tool do I have? Oh, there's a tool here that says

Postgres. Okay, let me try using that. Okay, I use that. I got feedback. Let me look at the feedback and read it and it'll read the feedback. And so that prompt box or context is where both the user input, the environment input, and the internal thoughts of the machine are all within. It's sort of like a program memory in memory space. And so reasoning over that, the challenge for a long time, that's when AI is just like went off track. And now they're able to kind of think through this entire thing and maintain coherence. And there's now techniques around compression of contacts. So there's still, the context length is still a problem, right? So I would say LLMs today, you know, they're marketed as a million token length, which is like

a million words almost. In reality, it's about 200,000. And then they start to struggle. So we do a lot of, you know, we stop, we compress the memory. So if a memory, if a portion of the memory is saying that I'm getting all the logs from the database, you can summarize, you know, paragraphs of logs with one statement or the database set up. That's it, right? And so every once in a while we will compress the context so that we make sure we maintain coherence. So there's a lot of innovation happened outside of the foundation models as well in order to enable that long context coherence. What was the key technical breakthrough in the foundation models that made this possible, do you think? I think it's RL. I think it's reinforcement learning. So the way pre-training works is, you know, the pre-training is the first step of training a large-in-Irish model. It reads a piece of text. It covers the last words and tries to guess it. That's how it's trained. That doesn't really imply long context reasoning. It, you know, it turns out to be.

be very, very effective. It can learn language that way. But the reason we weren't able to move past that limitation is that that modality of training just wasn't good enough. And what you want is you want a type of problem solving over a long context. So what reinforcement learning, especially from code execution gave us, is the ability to, for the machine to, for the LM to roll out, what we call trajectories in AI. So a trajectory is a step-by-step reasoning chain in order to reach a solution. So the way, as I understand, reinforcement learning works

is they put the LLM in a programming environment, like Replit, and say, hey, here's a code base, here's a bug in the code base, and we want you to solve it. Now, the human trainer already knows what the solution would look like. So we have a per request that we have on GitHub, so we know exactly, or we have a unit test that we can run and verify the solution. So what it does is it rolls out a lot of different trajectories. They sample the model, and maybe one of those trajectories will reach,

and a lot of them will just go off track, but one of them will reach the solution by solving the bug, and it reinforces on that. So that gets a reward, and the model gets trained that, okay, you know, this is how you solve these type of problems, so that's how we're able to extend these reasoning chains. Got it. And how, as a two-part question is, how good, how good are the models now at long, long, long reasoning? And I would say, and how do we know? Like, how is that established? There is a nonprofit called meter that is measuring useful, has a benchmark to measure how long a model runs while maintaining coherence and doing useful things, whether it's programming. or other benchmark tasks that they've done. And they put up a paper, I think, late last year that said,

every seven months, the minutes that a model can run is doubling. So you go from two minutes to, you know, four minutes and seven months. I think they've vastly underestimated that. Is that right? Vastly. It's doubling more often than seven months. We, so Agent three, we measure that, you know, very closely. And we measure that in real tasks from real users.

So we're not doing benchmarking. We're actually doing A-B tests and we're looking at the data that how users are successful or not. For us, the absolute sign of success is you made an app and you published it. Because when you publish it, you're paying extra money. You're saying this app is economically useful. I'm going to publish it. So that's as clear cut as possible. And so what we're seeing is in Agent 1, the agent can run for two minutes and then perhaps struggle.

Agent 2 came out of February, it ran for 20 minutes. Agent 3, 200 minutes. Some users are pushing it to like 12 hours and things like that. I'm less confident that it is as good when it goes to these stratospheres. But at like two, three hours timeline, it is insanely good. And the main innovation outside of the models is a verification loop. Actually, I remember reading a research. paper from NVIDIA. So what NVIDIA did is they're trying to write GPU kernels using

DeepSeek, and that was like perhaps seven months ago when DeepSeek came out. And what they found is that if we add a verify in the loop, if we can run the kernel and verify it's working, we're able to run Deepseek for like 20 minutes, and it was generating actually optimized kernels. And so it was like, okay, the next thing for us, obviously as a sort of a agent's lab or like applay our company we're not doing the foundation model stuff but we're doing a lot of research on top of that and so okay we know that agents can run for 10 20 minutes now or lLMs can stay coherent for longer but for you to push them to 200 300 minutes you need a verifier in the loop so that's why we spend all our time uh creating scaffolds to make it so that the agent can spin up a browser

and do computer use style testing so once you put that in the middle, what's happening is it works for 20 minutes. It spins up another agent, spins up a browser, tests the work of the previous agent, so it's a multi-agent system. And if it is, if it founds a bug, it starts a new trajectory and says,

okay, good work, let's summarize what you did the last 20 minutes. Now that plus what the bug that we found, that's a prompt for a new trajectory. So you stack those on each other and you can go endlessly. So it's like setting up a marathon or like a relay race. As long as each step was done properly, you could do in sort of an infinite number of steps. That's right.

You can always compress the previous step into a paragraph and that becomes a prompt. So it's an agent prompting the next agent. Right, right, right. That's amazing. So, and then when an agent, like when a modern agent, like running on modern, modern LMs that are trained this way, let's say it runs for 200 minutes. Like when you watch the agent run,

is it like running, is it like processing through like logic and tasks at the same pace that like a human being is or slower or faster? It's actually, I would say, it is faster, but not that much significantly faster. It's not at computer speed, right, what we expect computer speed to be. It's like watching a person. Like, if you watch that, if it's describing what it's doing, it's sort of like watching a person work. It's like watching John Carmack on cocaine work.

The world, okay. The world's best programmer. Yeah. The world's best programmer on a stimulus. Yeah, that's right. Okay. Working for you. Working for you.

Yeah, so it's very fast, and you can see the file lifts running through, but every once in a while, it'll stop and it'll start thinking, I'll show you the reasoning. It's like, I did this, and I did this. Am I on the right track? It kind of really tries to reflect. And then it might review its work and decide the next step, or it might kick into the testing agent.

So you're seeing it do all of that. And every once in a while it calls a tool, for example, it stops and says, well, we ran into an issue, you know, Postgres 15 is not. not compatible with this database or M package that I have. Okay, this is a problem I haven't seen before. I'm going to go search the web. So it has a web search tool, go do that.

And so it looks like a human programmer. And it's really fascinating to watch. So one of my favorite things to do is just to watch the tool chain and the reasoning chain and the testing chain. And it is, yeah, it is like watching a hyper-productive programmer. Right. So, you know, we're kind of getting into here kind of the Holy Grail of AI, which is sort of, you know, generalized reasoning, you know, by the machine. So you mentioned this a couple times with this idea of a verification. So just for folks on the listening to

podcasts, you maybe aren't in the details, let me try to describe this and see if I have it right. So like just a large language model the way you would experience, you would have experienced with like Chad GPT out of the gate two years ago or whatever would have been. It's like incredible how fluid it is at language. It's incredible how good it is at like writing Shakespearean sonnets or rap lyrics. It's really, it's really. amazing how good it is a human conversation, but if you start to ask it, like, problems that involve, like, rational thinking, uh, or problem solving all of a sudden, like, you'd, or math, or the math, the whole show. And, and, and in the very beginning, it was, you could ask, if you

ask, you know, it would, you know, it would, you would not be able to do them. That's right. Uh, but then even when it got better at those, if you started to ask it to, like, you know, it could maybe add two small numbers together, but it couldn't add two large numbers together, or if it could add two large numbers, it couldn't multiply them. Yeah. And it's just like, all right, this is, and then it had this, there was this, the, the, the strawberry, the The strawberry test, the famous strawberry test, which is how many R's are in the word strawberry? That's right.

And there was this long period where it kept, it would just guess wrong. It would say there were only two R's in the word strawberry, and it turns out there are three. So it was this thing, and so people were, and there was even this term that was being used, kind of the slur that was being used at the time was stochastic parrot.

Yeah, I always think you clanker. Well, the clanker is the new slur. Clanker is just the full on racial slur. I guess AI is a species. but the technical critique was so-called stochastic parrary, stochastic means random. So sort of random parrot, meaning basically that this thing was sort of a large language models

were like a mirage, where they were like repeating back to you things that they thought that you wanted to hear, but they didn't undertake it. And in the way, it's true in the pure pre-training LLM world. Right, for the very basic layer. But then what happened is, as you said, over the last year or something, there was this layering in of reinforcement learning. And then the key to...

And it's not new, crucially, it's like it's AlphaGo, right? So... Describe this. describe that for a second. Yeah, so we had this breakthrough before in 2015 was the AlphaGo breakthrough, I think, 2015, 2016, where it is emerging of sort of, you know, the, you would know a lot better than me, the old AI debate between the connectionists, the people who who thinks neural networks are the true sort of way of doing AI and the symbolic systems, I think, or like the people that think that, you know, discrete reasonings, the F-statements and

knowledge basis, whatever, this is the way to go. And so there was emerging of these two worlds where the way AlphaGo worked is it had a neural network, but it had a Monte Carlo tree search algorithm on top of that. So the neural network would generate, would like generate a list of potential moves. And then you had a more discrete algorithm, sort those moves, and find the best based on just tree search, based on just trying to verify. Again, this is sort of a verifier in the loop, trying to verify. which move might yield the best based on more classical way of doing algorithms.

And so that's a resurgence of that movement where we have this amazing generative neural network that is the LLM. And now let's layer on more discrete ways of trying to verify whether it's doing the right thing or not. And let's put that in a training loop. And once you do that, the LLM will start gaining new capabilities, such as reasoning over math and code and things like that. Exactly, right. Okay, and then that's great.

And then the key thing there, though, for RL to work, for LMs to reason, the key is that it be a problem statement that there is a defined and verifiable answer. Is that right? And so, and you might think about this as like, let's give a bunch of examples. Like in medicine, this might be like, you know, a diagnosis that like a panel of human doctors agrees with.

Or, by the way, or a diagnosis that actually, you know, solves the condition. In law, this would be a, you know, know, an argument that in front of a jury actually results in an acquittal, or something like that. In math, it's an equation that actually solves properly. In physics, it's a result that actually works in the real world. I don't know, in civil engineering, it's a bridge that doesn't collapse. Right.

So there's always some test of practice. The first two do not work very well just yet. Like the, like, I would say, law and healthcare, they're still a little too squishy, a little too soft. It's unlike math or code. Like the way they're changing on math, they're using this sort of like a program language provable language called lean for proofs, right? So you can run a lean statement.

You can run a computer code. Perhaps you can run a physics simulation or civil engineering sort of physics simulation. But you can't run a diagnosis. Okay. So I would say that... But you could verify it with human answers or not. Yeah, so that's a more...

Or I'll have in a way. Okay. So it is not the like... sort of autonomous RL, like fully scalable autonomous, which is why coding is moving faster than any other domain, is because we can generate these problems and verify them on the fly. But there's two, with coding, as anybody who's coded knows,

there's coding, there's two tests, which is, does it code compile? Right. And then the other is, does it produce the right output? Right. And just because it compiles doesn't mean it produces the right output. And you tell me, but verifying that it's the correct output is harder. Yeah, so Sweet Bench is a,

collection of verified pull requests and states so it is it is not just about compiling we so they they group of scientists so sweet bench is the main benchmark used to test whether AI is good at software engineering tasks and we're almost saturating that so last year at like maybe 5% early 24 or less and now we're like 82% or something like that with cloths on at 4.5 state of the art. And that's like a really nice hell climb that's happening right now. And basically they went and looked on GitHub. They found the most complex repositories. They found bug statements that are very clear. And they found ProQuast that actually solve those

bug statements with unit tests and everything. So there is an existing corpus on GitHub of tasks that the AIs can solve. And you can also generate them. Those are not too hard to generate what's called synthetic data. But you're right. It's not infinitely scalable because some human verifiers still need to kind of look at the task. But maybe the foundation models

have found a way to have the synthetic training go all the way. Right. And then what's happening, I think, because what's happening is the foundation model companies are, in some cases, they're actually hiring human experts to generate new training data. Yes. So they're actually hiring mathematicians

and physicists and coders to basically sit and, you know, they're hiring human programmers, putting them on the cocaine. Yes. And having them, probably coffee. Yeah. And having them actually write code, and then write code in a way where there's a known result

of the code running such that this RO loop can be trained properly. That's right. And then the other thing these companies are doing is, they're building systems where the software itself generates the training data, generates the tests, generates the validated results. And then that's so-called synthetic training data. That's right. And, but, yeah, but again, those works in the very hard domains.

It works to some extent in the softer domains. And I think there's some transfer learning. You can, you can see the reasoning work when it comes to, you know, tools like deep research and things like that. But we're not making as fast as progress in the, in the more soft domains. So it's quite, it's sort of a software domains, meaning like domains in which it's harder, harder or even impossible to actually verify correctness of a result. Yeah. In a sort of a deterministic, factual, grounded, non-controversial way.

Like, if you have a chronic disease, you could have, you know, you have pots or, you know, whatever, EDS syndrome, or, and they're all clusters. Because it is the domain of abstraction. It is not as concrete as code and math and things like that. So I think there's still long ways to go there. So sort of the more concrete, the problem, like, it's the concreteness of the problem that is the key variable, not the difficulty of the problem? Would that be a way to think about it? Yeah, I think that the concreteness, in a sense

of can you get a true or false verifiable app? Right. But like in any domain, in any domain of human effort in which there's a verifiable answer, we should expect extremely rapid progress. Yes. Right. Okay. Absolutely. And I think that's what we're saying. And that for sure includes math, that for sure includes physics, for sure includes chemistry, for sure includes large areas of code. That's right. Right. What else does that

include, do you think? Bio, like we're saying with a protein folding. Like genomic, yeah. Yeah, yeah, things like that. I think some areas of robotics, there's a clear outcome. Right. But it's not that money, I mean, surprisingly. Well, it depends. It depends on your point of view.

Some people might say that's a lot. So, and then you mentioned the pace of improvement. So what would you expect from the pace of improvement going forward for this? I think we're ripping on coding. Like, I think it's just, it's going. Like, I think it's going to be, like, what we're working on. with agent four right now is by next year we think you're going to be sitting instead of rep in front of replet and you're shooting off multiple agents at a time you're

like planning a new feature so i i want to you know social network on top of my storefront and another one it's like hey um refactor the database in in your running parallel agents so you have it five 10 agents kind of working in the background and they're merging the code and it'll take care of all of that. But you also have a really nice interface on top of that, that you're doing design and you're interacting with AI in a more creative way, maybe using visuals and charts and things like that.

So there's a multimodal angle of that interaction. So I think, you know, creating software is going to be such an exciting area. And I think the layperson will be as good as what a senior software engineer that works at Google is today. So I think, I think that's happening very soon. But, but, you know, I don't see them and be curious about your point of view. But like my experience between as a sort of a, you know, on the, let's say, healthcare side or more, you know, write me an essay side or more creative side, haven't seen as much of a rapid improvement as what we're seeing in code. So I think,

I think code is going to go to the moon. Math is probably as well. Some, some, you know, scientific domains, bio, things like that, those are going to move really fast. Yeah, so there's this, there's this weird dynamic, and see if you agree with this, and Eric also, I'm curious to your point of view on this, like, there's this weird dynamic that we have, and we have this in the office here a lot,

and I also have this with, like, leading entrepreneurs a lot, which is this thing of like, wow, this is the most amazing technology ever, and it's moving really fast, and yet we're still, like, really disappointed. And, like, it's not moving fast enough, and, like, it's right, like, maybe right on the verge verge of stalling out, and, like, you know,

we should both be, like, hyper-excited, but also on the verge of, like, slitting our wrists because, like, you know, the gravy train is coming to an end. Right. And I always wonder, it's like, you know, on the one hand, it's like, okay, like, you know, not all, I don't know, ladders go to the moon, like, just because something, you know, looks like it works or, you know, doesn't mean it's going to, you know,

be able to scale it up and have it work, you know, to the fullest extent, you know, so, like, it's important to, like, recognize practical limits and not just extrapolate everything to infinity. On the other hand, like, you know, we're dealing with magic here that we, I think, probably all would have thought was impossible five years ago, or certainly 10 years ago. Like, I didn't, you know, look, I got my CS degree in the late 80s, early 90s.

I didn't think I would live to see any yes, right? Like, this is just amazing that this is actually happening in my lifetime. But there's a huge bet on AGI, right? Like whether it's the foundation models, I think now the entire U.S. economy is sort of a bet on AGI. And there are crucial questions to ask whether are we on track to AGI or not. Because there are some ways that I can tell you it doesn't seem like we're on track to AGI because we, because there doesn't seem to be transfer learning across these domains that are, that are, you know, significance, right?

So if we get a lot better at code, we're not immediately getting better at, like, generalized reasoning. We need to go also, you know, get training data and create R.L. environment for bio or chemistry or physics or math or law or, so, and this has been the sort of point of discussion now in the AI community after the Dwarkish and Richard Sutton interview where, you know, Richard Sutton kind of poured this cold water on the bitter lesson. So everyone was using this essay that he wrote called The Bitter Lesson. The idea is that

there are infinitely scalable ways of doing AI research. And anytime you can pour more compute and more data and get more performance out, you're just, you know, that's the ultimate way of getting to AGI, and some people interpreted that interview that perhaps he's doubtful that we're even on a bitter lesson path here. And perhaps the current training regime is actually very much the opposite in which we are so dependent on human data and human annotation and all of that stuff. So I think I agree with you. I mean, as a company, we're excited about where things are headed.

But there's a question of, like, are we on track of AGI or not? Right. Right. Right. So, and, you know, I think, you know, Ily says cover makes a specific form of this argument, which is basically like we're just literally running out of training data. It's a fossil fuel argument. Right.

Like, we've slurped all the training data. Fundamentally, we've slurped all the data off the internet. That is where almost all the data is at this point. There's a little bit more data that's in, like, you know, private dark pool somewhere that we're going to go get. But right, we have it all. And then, right, we're in this business now, trying to generate new data. but generating new data is hard and expensive,

you know, compared to just slay-slurping things off the internet. So there are these arguments. You know, having said that, you know, you get a definitional questions here really quick, which are kind of a rabbit hole. But having said that, like you mentioned, transfer learning. So transfer learning is the ability of the machine to, right, to be an expert in one domain

and then generalize that into another domain. My answer to that is, like, have you met people? And how many people do you know? How are we do transfer learning? Yeah, not many, right? Well, because there's... Quite the opposite, actually. The nerdier they are in a certain domain,

and they kind of, you know, often to have blind spots. We joke about how everyone's just retarded in one area. They make something like massive mistake and don't trust them on this, but another's other topic, you know. Right. Yeah, well, and this is a well-known thing among, like, for example, public intellect. So this happens, there's actually been whole books written about this on so-called public intellectual.

So you get these people who show up on TV and they're experts. And what happens is they're like an expert in economics, right? And then they show up on TV and they talk about politics. And they don't know anything about politics, right? Or they don't know anything about, like, medicine. Or they don't know anything about the law. or they don't know anything about computers. You know, this is the Paul Greckman talking about how the Internet's going to be no more significant than the fax machine.

Facts, yeah. It's a brilliant economist. He has no idea how to computer. Is he a brilliant economist? Well, at one point. At one point. At one point. Let's get.

Even if, even if he's a brilliant, well, this is the thing. Like, what does that mean? Should a brilliant economist be able to extrapolate, you know, the internet is a good question. But the point being like, even if he is a brilliant, you know, or take anybody, take anybody. Oh, by the way, or like Einstein's like, like, Einstein's like actually my first. favorite example. I think you'd agree Einstein was a physicist. He was like, he was a Stalinist. He was a Stalinist. Yeah,

he was a socialist and he was a Stalinist. And he was like, well, he thought like Stalin was fantastic. He did what he's out. So, yeah, okay, all right. True socialism. All right, Einstein. You know, I'll take your word for it. But like, once he got into politics, he was just like totally loopy. Or, you know, or even right or wrong, he just sounded like all of a sudden like an undergraduate lunatic. Like, somebody

in a dorm room, like he, there was no transfer learning from physics into politics. Like, Right or wrong, he didn't, there was no, there was clearly, there was nothing new in his political analysis. It was the same rote, routine bullshit you get out of, you know. Yeah, so in a way, the argument you're making is like, we may be already a human level AI. I mean, perhaps the definition of AGI is, is something totally different. It's like above a human level that something that truly generalizes across domains. It's not something that we've seen.

Yeah, like we've ideal, yeah, as I said, we've, we've, and you know, look, we should, we should shoot big, but we've idealized a, we've idealized a goal that, um, that maybe idealized in a way that, like, number one, it's just, it's like so far beyond what people can do that it's no longer, it's no longer relevant comparison to people. And usually AGI is defined as, you know, able to do everything better than a person can. And it's like, well, okay, so if doing everything better than a person can, it's like if a person can't do any transfer learning at all, you know, doing even a little bit, a marginal bit might actually be better, or it might not matter just because no human can do it. And so therefore, you just, you just stack up the domains. There's also this well-known phenomenon

in AI, you know, typically this works the other way, which is a phenomenon in AI, AI engineers always complain about, scientists always complain about, which is the definition of AI is always the next thing that the machine can't do. And so, like, the definition of AI for a long time was, like, can it beat humans at chess? And then the minute it could beat humans of chess, that was no longer AI. That was just like, oh, that's just like boring. That's computer chess. It became an entire thing. It's just like boring and now it's an app on your iPhone and nobody, and nobody cares, right? And it's immediately the chewing test was the next one. And then we passed it and nobody. We blew, it's just a really big deal. There was no celebration. There was no

parties. It's exactly right. There was no party. For 80 years, the Turing test. I mean, they made a movie about it. Like, the whole thing. That was the thing. And, like, we blew right through it. Nobody even registered it. Nobody cares. It gets no credit for it. We're just like, ah, it's still, you know, a complete piece of shit. Like that's right? And so there's this thing where, so the AI scientists are used to complaining, basically, that they're, that they're being, they're always being judged against the next thing, as opposed to all the things they've already solved. But that's maybe the other side of it, which is they're also putting out for themselves

an unreasonable goal. An unreasonable goal. And then doing this sort of self-flagellation kind of along the way. And I kind of wonder, yeah, I wonder kind of which way that cuts. Yeah, yeah. It's an interesting question. Like, I started thinking about this idea of, like, it doesn't matter whether it's truly AGI. And the way I define AGI is that you put in a AI system in any environment and efficiently learns. You know, it doesn't have to have that much prior knowledge in order to kind of learn something, but also can transfer that knowledge across different domains.

But, you know, we can get to, like, functional AGI. And what functional AGI is just, yeah, collect data in every useful economic activity in the world today and train an LLM on top of that or train the same foundation model on top of that. And we'll go, we'll target every sector of economy and you can automate a big part of labor that way. So I think, yeah, I think we're on that track for sure.

Right. You tweeted after GPG5 came out that you were feeling the diminishing returns. Yeah. What were you expecting? And what needs to be done? Do we need another breakthrough to get back to the pace of growth? Or what do you have thought so? I mean, this whole discussion is sort of about that.

And my feeling is that, you know, GPD-5 got good at verifiable domains. It didn't feel that much better at anything else. The more human angle of it, it felt like it regressed. And like you had this sort of Reddit pitchfork sort of movement against Sam and opening eye because they felt like they lost a friend. Gypti4-0 felt a lot more human and closer, whereas GP-5 felt a lot more robotic, you know, very in its head kind of trying to think through everything. And so I would have just

expected, like, when we went from GP2 to 3, it was clear it was getting a lot more human. It was a lot closer to our experience. You can feel like it's actually, oh, it gets me. like there's something about it that understands the world better similarly three to four four to five didn't feel like it was a better overall being as it but is that is that is that is that a question there like is it emotionality is it partly emotionality but but again partly like i like to ask models like very controversial things um can it reason through I don't know how people want to go here, but like, what happened with the World Trade Seven? Right, right? Sure. It's an interesting question, right?

Like, I'm not putting out a theory, but like, it's interesting. Like, how did it, you know, and can I think through controversial questions in the same way that it can go think through a coding problem? And there hasn't been any movement there, like all the reasoning and all of that stuff. I haven't said, not just that, you know, that's a cute example, but like COVID, right? Like, you know, the origins of COVID, you know, go, you know, dig up GPD4 or other models and go to GPD5. You're not going to find that much difference of, okay, let's reason together, let's try to figure out what was the origins of COVID. Because it's still an unanswered question, you know, and I don't see them making progress on that.

I mean, you play a lot with them. Yeah, I use it differently. I don't know, maybe I have different expectations. I'm, I, the way I, my main use case actually is sort of, sort of PhD and everything at my beck and call. And so I'm trying to get it to explain things to me more than I'm trying to like, you know, have conversations with it maybe. Yeah. Maybe I'm just unusual with that. And that, that gets better.

Well, so what I found specifically is a combination of like GPT5 Pro plus deep reasoning or like rock four heavy, like the, you know, the highest on models like that. You know, they now basically generate 30 to 40 page, you know, essentially books on demand on any topic. And so anytime I get curious about something, you're just taking it, maybe this is my version of it, but it's something like, I don't know, like, here's a good example, when an advanced economy puts a tariff on a raw material or on a finished good, like, who pays? You know, is it the consumer, is it the importer,

is it the exporter, or is it the producer? And this is actually a very complicated, it turns out a very complicated question. It's a big, big thing that the economists study a lot. And it's just like, okay, who pays? And what I found, like, for that kind of thing, is it's outstanding. But it's outsideing at sort of going out of the web,

getting information, synthesizing it. Correct. It gives me a synthesized 20, 30, 40 pages. It basically tops out of 40 pages of PDF. Yeah. But I can get up to 40 pages of PDF, but it's a completely coherent. And as far as I can tell for everything I've cross-checked,

I completely like world-class, like if I hired, you know, for a question like that, if I hired like a great, you know, Econ PhD postdoc at Stanford who just like went out and did that work, like it would maybe be that good. Yeah. But then, of course, the significance is, it's like, you know, at least for, this is true for many domains, you know, kind of Ph.E. and everything. And so. But this is synthesizing knowledge, not trying to create new knowledge. Well, but this gets to the sort of, you know, of course, you get into the angels dancing on the head of a pin thing, which is like, what, you know, what's the difference? How much new knowledge ever actually is there anyway? What do you actually expect from people when you ask some questions? And so what I'm looking for is like, yes, explain this to me in like the clearest, most sophisticated, most complex, most, like, complete way that it's possible for someone. but it's, you know, for a real expert to be able to explain things to me. And that's what I use it for.

And again, as far as I can talk from the cross-checking, like, I'm getting, you know, like, almost, like, basically 100 out of 100, like, I don't even think I've had an issue in months where it's like, for sure. Had a problem in it. Yeah. And it's like, yeah, you can say, yeah, it's synthesizing is supposed to create an information, but, like, it's generating a 40-page, it's basically generating a 40-page book. That's amazing.

That's, like, incredibly, like, fluid. It's, you know, it's, it's, it's, you know, the logical coherence of the entire, like, It's a great, like, if you evaluated a human author on it, you would say, wow, that's a great author. You know, are people who write books, you know, creating new knowledge? Well, yeah, well, sort of not, because a lot of what they're doing

is building on everything that came before them and synthesizing combined, but also, like, a book as a creative accomplishment, right? And so, yeah, one of the thing, I don't know. I'm interested in, I'm hoping AI could help us solve is just, like, how confusing the information ecosystem right now. You know, everything feels like propaganda. Like, it doesn't feel like you're getting real information from anywhere.

So I really want an AI that could help me reason from first principles about what's happening in the world for me to actually get real information. And maybe that's an unreasonable sort of ask of the AI researchers. But I don't think we have made any progress there. So maybe I'm overfocus. Maybe I'm overfocus. Maybe I'm overfocus and arguing people as opposed to trying to get down the line truths. But well, here's the thing I do a lot. with this, is I'd just say, like, take a provocative point of view, and then steal man the position.

Take your COVID thing. So I often, I often pair of these. Steel man the position that it was a lab leak and the steel man the position that it was natural origins. And again, like, is this creativity or not? I don't know, but like what comes back is like 30 pages each of like, wow, like, that is like the most compelling case in the world I can imagine with like everything marshaled against it, like, the argument structured in like the most possible. Part of the reason that's not happening is because it stopped being taboo to talk about a human origin.

When it was taboo, the AIs would, like, talk, well, you know, talk down to it. It's like, oh, you're a conspiracy. And so, there's a, you know, a period of time. And so it takes something truly controversial. And they actually, they can't reason about it because of all our LHF and onset of the limitations. And as, you know, I will pick a specific ones here, but like, there are certain big models that will still lecture you that you're a bad person for asking a question. But, you know, like, I was just that there, some of them are just like really, really open now to, you know, being able to do. these things. Um, and then, um, uh, yeah. So, um, okay. Uh, yeah, so, okay. So, okay.

So, yeah, so, so basically, like, ultimately what you're looking for, like, the ultimate thing would be, if there's something that's like, I don't think anybody's really defined this really well, because it's not, because again, it's like, conventional, all the conventional definitions of AGI are like basically comparing to people. Yeah. And there, and there, it's always like, you know, it's, the conventional explanations of, of, um, of, of, uh, of, of, of, uh, of, of, of, of AGI, I always, for me, strike me a lot, like, the debate around, like, whether a self-driving car works or not, which is, is a self-driving car work because it's a perfect driver, or does it work because it is better than the human driver?

And better than the human driver, I think, is actually quite, you know, just like with the chess thing and the go thing. I actually think like that, that's like a real thing. And then, and then there's the, like, is it a perfect driver, which is, you know, what they're, obviously the self-driving car companies are working for. But then I think you're looking for something beyond the perfect driver. You're looking for the car who, like, knows where to go. So I'm off two minds, right?

So one mind is the sort of practical entrepreneur. And I just, I have so many toys to play with, to build. Like, stop AI progress today and REPLOR will continue to get better for the next five years. Like, wait, there's so much we can do just on the app, app layer and the infrastructure layer. So, you know, but I think that it will, you know, the foundation models will continue to get better as well. And so it's a very exciting time in our industry. The other mind is more academic because as a kid, I've always been interested in the nature of consciousness, nature of intelligence.

I was always interested in AI and reading the literature there. And I would point to the R.L. literature. So Richard Sutton, there's another guy, I think co-founder at Deep Mind, Shane Legg, wrote a paper trying to define what AGI is. And in there, I think that the definition of AGI, I think, is the original, perhaps correct one, which is efficient continual learning. Okay.

Like, if you truly want to build an artificial general intelligence that you can drop in any domain, you can drop in a car without that much prior knowledge about cars. And within, you know, how long does it take a human to learn how to drive? Right. Within months, be able to drive a car really well. Right. So generalized skill acquisition, generalized understanding acquisition. Yeah.

Generalized reasoning acquisition. And I think that's the thing that would, like, truly change the world. world, that's the thing that would give us a better understanding of the human mind, of human consciousness. And that's the thing that will, like, propel us to the next level of human civilization, I think. So on a civilization level, I think that's a really deep question. But separate from the economy and the industry, which is all exciting. But there's an academic aspect of it that I'm really... And so what odds, if we're on Kalshi today, what odds do we place on that? I'm kind of bearish on true AGI breakthrough

because what we built is so useful and economically valuable. So in a way... Good enough. Good enough as the enemy. Yeah, yeah. Do you remember that essay? Worse is better. Worse is better. Worse is better. There's like a trap.

There's like a local maximum trap. That's right. We're in a local maximum. Local maximum trap where it's... Because it's good enough for so much economically productive. work. Yes. It relieves the pressure in the system to create the generalized answer. Yes. And then you have the weirdos like Rich Sutton and others that are still trying to go down that path and maybe they'll succeed. Right. But there's enormous optimization energy behind the current thing that we're

hell climbing on this like local maximum. Right, right. Right. And the irony of it is everybody's worried about like the, you know, the gazillions of dollars going into building out all this stuff. And so the most ironic thing in the world would be if the gazillions of dollars are going into the local maximum. That's right. As opposed to a counterfactual world in which they're going into solving the general problem. But it's also potentially irrational. Like maybe the general problem is actually, you know, not within our lifetimes. Right.

Right. Right. How much further do you think, like, do you think we squeeze most of the juice out of LLMs in general then? Or are there any other research directions that you're particularly excited about? Well, that's the thing. I think that problem is there aren't that many. I think the breakthroughs in RL are incredibly exciting.

But we also knew about them now for like over 10 years where you've made. regenerative systems with with research and things like that. But there's a lot more to go there. And I think, again, the original minds behind reinforcement learning are trying to go down that path and try to kind of bootstrap intelligence from scratch. Carmack is going down that path. As far as I understand, Carmack, you guys may be invested, but they're not trying to go down the LLM path. So there are people that are trying to do that. But I'm not seeing a lot of progress. or I've come there, but I watch it kind of from far. Although, you know, for all we know,

it's already, there's already a bot on X somewhere. What's that maybe, you know, you never know. You never know, it might not be a big announcement. It might just be a, you know, one day there's just like a bot on X that starts winning all the arguments. Yeah, it could be. Or a code, a user Reddit and all of a sudden that is generating incredible software. Okay, let's, let's spend our remaining minutes.

Let's talk about you. So, so how, so, yeah, take us and start from the beginning with your life and how did you get, how did you get from being born and being in Silicon Valley. Okay. In two minutes. Yeah. I'm joking. But, yeah, I got introduced to computers very, very early on.

And so for whatever reason, so I was born in Amman, Jordan. And for whatever reason, my dad, who was just a government engineer at the time, decided that computers were important. And he didn't have a lot of money, took out of that, bought a computer. It was the first computer in our neighborhood. But first computer of anyone I know. And I just, one of my earliest memories, I was six years old, just watching my dad unpack this machine and sort of open up this huge manual

and kind of finger type CDLS, MKDIR. And, like, I would, you know, be behind his shoulder and just like watching him, you know, type these commands and seeing the sort of machine kind of respond and do exactly what he's asked it to do. Popping Tylenol, all is your. Bobbitts, I don't know. Exactly. Autism activated, of course.

You have to. You have to. What kind of, what kind of computer was it? It was an IBM, as far as I remember. IBM PC. So what year was the boss? 19903.

1993? Okay, so did I have Windows at that point? No, it didn't have Windows. It was right before Windows. Right before Windows. But I think Windows had been out, but you would. There was an add-on. It was an add-on.

You wouldn't boot it up. So I think we bought the disk for Windows, and you had to kind of bootloaded, you know, from the disk, and it will open Windows, and you can click around. It wasn't that interesting because there wasn't a lot on it. So a lot of time I just spend it in DOS and writing batch files and opening games and messing around with that.

But it wasn't until Visual Basic that I started. So, like, after Windows 95, that I started making real software. And the first idea I had was I used to be a huge gamer so I used to go to these Lang Gaming cafes and play CounterStrike and I would go there and you know the whole thing is full computers

but they don't use any software for on their business it was just like people running around just like writing down your machine number how much time you spend on it and how much did you pay and kind of tapping your shoulders like hey you need to pay a little more for that and I asked them like why don't you just build a piece of software that allows me to log in and have a time or whatever.

I was like, yeah, we don't know how to do that. And I was like, okay, I think I know how to do that. So I spent, I was like 12 or something like that. I spent like two years building that and then went out and tried a sell it and was able to sell it and was making so much money. I remember McDonald's opened in Jordan around the time when I was 13, 14.

I took my entire class to McDonald's. It was very expensive, but I was bawling in all this money and I was showing off. And so that was the first business that I created. And then when it came to, and at the time, I started learning about AI, you know, reading sci-fi and all of that stuff. And when it came time to go to college, I didn't want to go to computer science because I felt like coding is

on its way to get automated. I remember using these wizards. Do you remember wizards? Wizards, basically. It's like extremely crude early, bots, that generate code. Yeah, and remember you could like, you know, type in a few things, like,

here's my project, here's what it does, whatever, and then click, click, click, and which is like scaffolded a lot of code. I was like, oh, I think that's the future. Like, coding is such a... It's almost solved. Yeah, it's solved, you know? Why should I go into coding? I was okay, if AI can do the code, what should I do? Well, someone needs to build and maintain the computers.

And so I went to the computer engineering and did that for a while. But then rediscovered my love for programming, reading program essays on LISP and things like that and started messing around with scheme and programming languages like that. But then I found it incredibly difficult to just learn different programming languages. I didn't have a laptop at the time.

And so every time I go to wanting to learn Python or Java, I would go to the computer lab, download gigabytes of software, try to set it up, type a little bit of code, try to run it, run into missing DLL issue, are, and I was like, man, this is so primitive. Like, at the time, it was 2008, something like that.

You know, we had Google Docs, we had Gmail. You could, like, open the browser and partly thanks to you and be able to kind of use software on the internet. And I thought the web is the ultimate software platform. Like, everything should go on the web. Okay, who's building an online development environment? Right. And no one.

Right. And it felt like I found like $100. bill on the floor of Grand Central Station. Like, surely someone should be building this. But no, no one was building this. And so it's like, okay, I'll try to build it. And I got something done in like a couple hours, which was a text box.

You type in some JavaScript, and there's a button that says Eval. You click Eval and evaluates, it shows you in an alert box. So one plus one, two, I was like, oh, I have a programming environment. I showed it my friends. People started using it. added a few additional things, like saving the program. I was like, OK, all right, this is, there's a real idea here. People love it.

And then again, it took me two or three years to actually be able to build anything because, you know, the browser can only run JavaScript. And it took a breakthrough at the time. Mozilla had a resource project called MScripton that allowed you to compile different program language like C, C++, into JavaScript. And for the browser to be a browser to be.

be able to run something like Python, I needed to compile C Python or JavaScript. So I was the first to do it in the world. So built, contributed to that project and built a lot of the scaffolding around it. And my friends and I compiled Python into JavaScript. And it was like, okay, we did it for Python. Let's do it for Ruby. Let's do it for Lowe. And that's how the emergence of the idea for Replit came is that when you need a

Ripple, you should get it. You should replet. And so Ripple is the most primitive programming environment possible. So I added all these programming languages. And again, all this time, my friends were using it and excited about it. And I was on GitHub at the time, and just my standard thing is like when I make a piece of software is open source it. And so I was open sourcing all the things. I was, you know, years building just like this underlying infrastructure to be able to just run code in the browser.

And then it went viral. Right. Went viral on Hacker News. And it coincided with the MOOC era. Right. So massively online courses. is, Udacity was coming online, Coursera, and most famously, Code Academy.

So Code Academy was the first kind of website that allowed you to code in the browser interactively and learn how to code. And they built a lot of it on my software that I was open sourcing all the way from Jordan. And so I remember seeing them on Hacker News and they were going super viral. And I was like, hey, I recognize this.

What are you using? And so I left a Hacker News comments. I was like, oh, you're using my open source package. And so they reached out to me, they're like, hey, would like to hire you. I was like, I'm not interested. I want to start a startup. I want to start this thing called Replit.

And they're like, well, no, you know, you should come work with us. We can do the same stuff. And I kept saying no. I was like, okay, I'll contract with you. They were paying me $12 an hour. I was really excited about it back from Mon. But they came out to their credit. They came out of Jordan to recruit me and spend a few days there.

And then, you know, I kept saying no. And in the end, they gave me an offer. I can't refuse. And they got me an Owen visa, came to the United States. That's when you moved. So when was the first, because you were born, what year? 1987. What was the first year that you could remember where you had the idea that you might not live your life in Jordan

and you might actually move to the U.S.? When I watched Pirates of Silicon Valley. Is that right? Okay. Maybe 98 or 99. I don't know when it came out. That might be a good place to you. Yeah.

Is it worth telling the hacker story? Because there's a version of the world where you didn't actually, like if that changed differently, maybe you wouldn't have gone to America. Right, right. Yeah. So in school, I was programming the whole time. So I just want to start businesses. I'm exploding with ideas all the time. And the reason Replit exists is because I have ideas all the time.

I just want to go type it on a computer and build them. So I wasn't going to school. It was like incredibly boring for me. And part of the reason why Replit has a mobile app today is because I always wanted to program under the desk. It's like just two things. And so at school, they kept failing me for attendance. So I would get it.

A's, but I just didn't show up, and so they would fail me. And so I felt it was incredibly unfair, and all my friends were graduating now. This year, it was like 2011. I'd been, like, for six years in college, it should be, like, a three or four year. And I was just, like, incredibly depressed. I really wanted to be in Silicon Valley. And so I was like, oh, what if I changed my grades on the university database? And so I went into.

my parents' basement and implemented the polyphasic sleep. Are you familiar with that? I am. Leonardo da Vinci's polyphysic sleep.

I didn't hear it from Renaud da Vinci. I heard it from Seinfeld because there's an episode where John Cameron goes on polyphasic sleep. 20 minutes every four hours. Yes.

20 minutes every four hours. Yeah. And yes, and this somehow is going to work well in it. Yeah. And hacking, if you've ever done As the meme goes, this has never worked for anybody else,

but it might work for me. Yes. And a lot of what hacking is, is that you're coming up with ideas for like finding certain security holes and like writing a script and then running that script. And that script will take like a 20, 30 minutes to run. And so you'll take that, you know, 20, 30 minutes to sleep and go on it. So I spent two weeks just going mad, like trying to hack into the university database.

And finally, I found a way, I found a SQL injection somewhere on the site. and I found a way to be able to edit the records but I didn't want to risk it so I went to my neighbor who was going to the same school I think till this day no one caught him

but I went to him and I said hey, I have this way to change grades like would you want to be my guinea pig and I was honest about it I was like I'm not going to do it are you open to do it? He's like yeah yeah they call it human trials

this is how medicine works so So we went and we went and changed his grades and he went and pulled his transcript and the update wasn't there and went back to the basement. We'll turn out that I had access to the slave database. I don't have access to the mass of the database. So find a way through the network, privileged escalation. It was an Oracle database that had a vulnerability and then found the real database. And then I just did it for myself, changed the grades.

and went and pulled my scranscripts. And sure enough, it actually changed. Went and bought the gown, went to the graduation parties, did all that. We're graduating. And then one day, I'm at home. It's like maybe 6 or 7 p.m. I get a, you know, the telephone at home rings.

I'm going to us ring sound things. Well, hello. And he's like, hey, this is the university registration system. and I knew the guy that run it. It's like, look, you know, we're having this problem. The system's been down all day, and it keeps coming back to your record. There's an anomaly in your record

where you're both past, you have a passing grade, but you're also banned from that final exam of subject. I was like, oh, shit. Well, it turns out the database is not normalized. So typically, when they ban you from an exam, the grade resets to 35 out of 100. But apparently there's a Boolean flag. And by the way, all the columnings,

in the database are single letters. So that was the hardest thing. It's security by obscurity. Right. And it turns out there's a flag that I didn't check. So when you go over attendance, when you don't attend, and they want to fail you, they ban you from the final exam. So I changed the grades and that created an issue and brought down the system.

So they were calling me. And I thought at the time, I was like, you know, I could potentially lie, it'll be a huge issue, or I just like, I'll just, I'll just, that's up. So I said, hey, listen, look, yeah, I might know something about it. Hey, let me, let me come tomorrow and kind of talk to you about what happened. So I go in and I open the door, and it's the deans of all the, all those schools. It's like computer science, computer. They were all working on it for, like, days because it's like, it's a very computer-heavy, you know, university.

And it was like a problem. And they're all kind of really intrigued about what happened. And so I pull up a whiteboard and started explaining what I did. And everyone was engaged. I gave them a lecture, basically. It's an oral exam for your PhD. Yeah. This is great.

They were really excited. And I think it was endearing to them. I was like, oh, wow, this is a very interesting problem. And then I was like, okay, great, thank you. And I'm hitting. I was like, hey, wait, wait, we don't know what to do with you. Do we send you to jail? Do we?

And I was like, hey, we have to ask. to the university president and he was a great man and I think he gave me a second chance in life and I went to him and I you know I explained a situation I said like I'm really frustrated I need to graduate I need to get on with my life I've been here for six years and I just can't sit in in school the stuff I already know I'm a really good programmer and and and he gave me a Spider-Man line at the time it's like with a great power comes great responsibility and you have a great power and you know and it really affected me and I I think he was right at the moment.

And so he said, well, we're going to let you go, but you're going to have to help the system administrators secure the system for the summer. I was like, happy to do it. And I show up on all the programmers there hate me. I hate my guts. And they would lock me out. Like I would see them, they would be outside.

I would knock it in the door and no one would listen. It was like, they don't want to let me in. I tried to help them a little bit. They weren't collaborative. And so I was like, all right, whatever. And so it came to, it came time for me to actually graduate. It was the final project. And one of the computer science seen came to me and he said, look, I need to call a favor. I was a big part of the reason we

kind of let you go and we didn't kind of prosecute you. So I want you to work with me on the final project. And it's going to be around security and hacking. I was like, no, I'm done with that shit. I just want to build programming environments and things like that. And he's like, no, you have to do it. I was like, okay. So I thought I'd do something more productive. So I wrote a security scanner that was very proud of that kind of crawls the different side that tries to SQL injection and all sorts of things. And actually, my security scanner found another vulnerability

in this system. Amazing. And so I went to the defense, and he's like, you need to run this security scanner live and show that there's a vulnerability. And I didn't understand what was going on at the time, but I just, okay. So I gave the presentation about how the system works,

and I was like, oh, let's run it, and it showed that this security vulnerability. Okay, let's try to get a shell. So the system automatically runs all the security stuff and it gets you a shell. And then the other dean that turned out, he was giving the mandate to secure the system. And now I started to realize I'm a pawn

in some kind of bribery here. And his face turned red and it's like, no, it's impossible. You know, we secure the system, you're lying. I was like, you know, you're accusing me of lying. All right, what should we know? Should we know your salary or your password? What do you want me to look up? And I was like, yeah, I look up my password.

So I look up his password. And it was like, jib brush, it was encrypted. And I was like, oh, that's not my password. See, you're lying. I was like, well, there's a decrypt function that programmers put it there. So I do decrypt, and it shows his password. It was something embarrassing. I forgot what it was.

And so he gets up, really angry shakes my hand, and leaves to change his password. So I was able to hack into the university another time. Luckily, I was able to graduate, give them software, they secured the system. But, yeah, later on, I would realize that, yeah, he wanted to embarrass the other guy, which is why I was in the middle. Well, I think the moral of the story is if you can successfully hack into your school system and change your grade, you deserve the grade and you deserve to graduate. I think so. And just for any parents out there, you're children out there. You can cite me as the more.

You can say that I'm fat of me is the moral authority in this. One maybe lesson, I think that is very relevant for the AI age. I think that the traditional sort of more conformist path is paying less and less dividends. And I think kids coming up today should use all the tools available to be able to discover and chart their own paths. Because I feel like just, you know, listening to the traditional advice and doing the same things that people have always done is just not as, it's not working out as much as we'd like. Yeah, that's right. Thanks for coming to the podcast. Thank you, man. Fantastic. Thanks for listening to this episode of the A16Z podcast. If you like this episode, be sure to like,

comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcast, and Spotify. Follow us on X at A16Z and subscribe to our substack at A16Z.substack.com. Thanks again for listening, and I'll see you in the next episode. As a reminder, the content here is for informational purposes only. Should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16Z

Z.com forward slash disclosures.