Programming Throwdown - 189: Agentic Loops

Episode Date: August 24, 2026

Running AI in loop hands-free.

Programming Throwdown, Episode 189, Agentic Loops. Take it away, Jason. Hey, everyone, how's it going? Let me adjust my microphone here a little bit. So we should talk about programming. So, you know, I have been, for the first time since I was probably like nine years old, I have not coded for a number of days. And I was thinking a lot about this.

I mean, you know, it's like ever since I was, even before a teenager, I was constantly writing programs on the computer. You know, he started, I don't know if you started with Basic, but I used to do a whole bunch of stuff in Basic. I made like, Choose Your Own Adventure in Basic and a whole bunch of things on the Commodore. And then I went from Basic to, I used Basic for a very long time. I think I went straight from basic to C++, which was terrifying. I definitely like nothing worked for a really long time.

And I've just been coding ever since. And the weird thing is I've built so many more things. Like my, my, the things that I've built, my build rate or my efficiency or my progress is like super, super high. But all of it is in English. And it's almost at the point. where I don't even really look at the code except for very specific things that I'm trying to find and even that is starting to dwindle.

And it's just such a weird feeling. I mean, even the premise of the show is kind of learning every language. And now it's like everything you just code in English or in plain language. You could even code it in whatever language you're most familiar with by typing whatever that is into the DLM.

it's really starting to hit me that that coding and plain language is like really probably here to stay. It's kind of wild. Yeah, it is a bit of a mind bend. I mean, I think this is getting to the point where

I mean, especially with the framing of it's, people say that a lot. Like, it's the worst it's ever going to be. Like, this is the, there's no reason to believe it would be a rest. Yeah. I guess like if we, somehow ban all GPUs or AI?

Isn't that the Dune thing? They have a big war because, anyways, AI is banned. I don't think that's a real. I doubt it's going to happen. So, yeah, you're right, though. Like, it totally changes the way you approach things. And I think there's various people have lots of opinions on it. And, you know, I don't think we should get into all of them.

But one of the ones that you hear is like, oh, it can make bad code or it can make mistakes. And I'm just like, have you worked with programmers? Like, have you looked at your own code? Exactly. Have you looked at Mike? Like, how many times have I found a stupid, just like off by one error or just like, like, yeah, I did all the time. Like, there's this weird thing that like, I don't know,

somehow we were infallible or people were infallible. And I'm not saying like good engineers, bad. Like, I'm just saying random people. Like, everybody makes mistakes. And there's a lot of people I've read code from where it's clear like, I'd have preferred to read the LLM code than what they wrote. So yeah, it's a bit of an interesting transition time. And I think we'll be talking about for a little while,

but it definitely feels like there's been a knee in the curve or a change or some threshold has been reached where it's no longer like, oh, it's the, that was a chess and humans, right? The original idea is the chess would help the humans. Then it's like, now the humans are really holding the chess engines back. And so now like modern ones, there's like it is much better to use a chess engine directly than to try to do some. cooperation. And human computer interop is not advantageous.

And I feel that's not there yet, but it kind of feels like we're on that journey. There are definitely people who can work with the AI better and giving instructions because we're not going to get into a free will discussion. But like, you know, having the AI

puts something to do is still something that seems to be in the human domain. But, you know, certainly the tactical open the code editor, debug the code, like, yeah, I'm not sure how much longer that's going to be a thing. And I wonder what like gets lost a little. Like, you know, we can talk about it, but even one of the news articles I have this time, there's definitely things that I've framed in a certain way. We'll say at least in today's AI that helps it. It's like, I want you to take this approach or I want you to use this technique or I want you to do this thing. And I don't have to write the code for it, but I needed to know when and what I was asking for. Again, maybe that. It's just because of limitations of today. But certainly, you know, it has come up in modern, you know, usage. So we'll see where it goes, where the human computer interop boundary ends up being.

But yeah, it is crazy to do so much time, like typing text to get what you want. Yeah. I mean, now here's the flip side is I've been, the word I was looking for was productive. I've been orders, probably at least in order. of magnitude more productive, at least, if not more. So I'll give you like a number of examples. So I run this open source thing called MAME Hub, and it's basically a network layer built into this arcade emulator.

So if you and I wanted to play Pac-Man together, Pac-Man is a bad example, but was that 1944, that game where you're an airplane? Yeah, 1843. Yeah, 43, yeah. So if we wanted to play one of these, arcade games together. You live in Florida, I live in Texas. You can use MAME Hub. We'd fire it up. And it basically plays the emulator. It only works for emulators that are deterministic, right? But it plays the emulator. And when you press up, it actually cues that up command in the future for both of us.

And how long, how far ahead in the future it queues that command is based on, you know, an estimate of our latency. So I try to fit, I try to fit our latency to a distribution and then pick the upper confidence bound of that distribution. It's like 99.7

of the packets arrive on time. And, you know, some tiny percent of the packets don't. And when it doesn't arrive on time, we both have to wait. So if you, so if you say in the future, my joystick looks like this

and I don't get that future that I have to wait for it and then you're also having to wait, et cetera. So that's the premise behind MAMEHub. It uses some statistics to figure out how much to schedule those joystick updates so that we both get them and we can play the game

and it runs the same on both of our computers. So that's MAMEHub. And I used to spend a ton of time just rebasing it, you know, because it's a fork of MAME. It's literally a fork of the MAMEGit project. And, you know, they're making a Zembrising zillion changes. And every few years or so, I would bite the bullet. You know, people would ask for it on

our Discord, and I'd bite the bullet, and I would rebase it. And it'd always be a toll disaster, right? Because I didn't really have it factored out, maybe as good as I could have. So now I just have a Git job. I just have like an action in GitHub that uses copilot to rebase it. And when there's conflicts, whether there's conflicts or not, it like even plays a game and make sure that the game works. And so that's something that, you know, I told an LLM to go build some GitHub action to go do that. And so similarly, oh, here's an even better example. So this podcast, what you all are listening to right now runs on a ton of software, right? We have assembly AI for generating the transcripts. We use

we have a transistor for actually hosting the site and all of that. And with AI, I just basically brought all of that in-house. I said, hey, you know, and by the way, Assembly AI, amazing. They've given us phenomenal service. If you need to transcribe something and you don't want to ask AI to do it, ask

these people to do it, they're phenomenal. So this is the slide against them or any of these people who have helped us so much over the years. But I basically went to an AI and I said, hey, here's the system that can do transcripts. Go on the internet, find some open source models, generate some transcripts, and keep working until your transcripts look like their transcripts. And it took a while, but AI eventually did that. So my point is, you know, I feel like I'm building way more than ever, but I'm not coding in code. And so I think what we're doing is coding in plain language. I think that's really the terminology.

And so it seems like the feature is going to be how do we code in plain language more effectively and how do we build these harnesses that, that, you know, kind of make all of that as fluid as possible. So that's what we're going to talk about this show. That might even be, you know, the most important thing for a while. Talking about important things, maybe it's time to move. to our news section and I think you got the first I don't want to spoil it but but

you take it first. Yeah I mean this is a late breaking news I'm sure nobody's heard of this but RAM is out of control the RAM prices are totally out of control I think they're going to keep becoming out of control I actually think they're going to get a lot

worse and and so this is just let me keep an eye I mean this is a Tom's Hardware article I'm sure you can find plenty of others, but if you are, if you're saying to yourself, I'm not going to buy a computer now

because the RAM prices are really high. You might have to just do that, just bite the bullet and get the computer. You know, I just feel like the RAM shortage is going to take years, and it's not going to, it's going to get a lot worse before it gets better. So if you,

if you have a computer you need to buy or an upgrade, you make your hardware or something, you know, I really suggest kind of doing it now, locking in, even though it looks like a high price, locking it in now. Yeah,

it sounds like they're saying all of the RAM is spoken for for like the next year, over year at, you know, even higher prices. Yeah, yeah, I didn't buy a Steam deck as much as I wanted one

because I always, I actually, this is kind of sad, but I went to the Steam Deck page probably once a quarter and I got so close to buying it and I was like, you know, if I do that, I'll play even more games that I'm already playing

and so I didn't pull the trigger on it and now it's twice the price. So I decided that I'm fine with the Switch but if I wanted a, I don't expect the Steam Deck to come down in price. Yeah, I did get one but now I feel like I should sell it. Oh,

me and yeah I had one from early on but you're right like I don't know how people are justifying it at the price it's you know what I mean like I I definitely yeah even before it wasn't necessarily cheap but I've I've enjoyed it you know it's been good I definitely wouldn't have gotten the current price value out of it it would have been too expensive so you got a really you know what you're going to use the crap out of it I guess well and I'm not saying it's not here's a question I mean if the price of computers triples and stays that way. How is that going to affect you?

Like, are you still going to upgrade your computer? I mean, I feel like at some point you just kind of need to upgrade if it breaks or, you know. I mean, I think it depends, right? So what does it mean? Like, you need to upgrade your computer. So for me, I mean, I think I have a number of options. So for one, you could just like subscribe to the gaming streaming services. They have gotten a lot better if you have good internet.

And some people will call that it. But again, if it's whatever, I don't know, $20 a month, $30 a month to rent something, even one of the ones where you rent your own computer and install it on shared time. Like, that's a form of arbitrage I've considered, you know, for running some of the local models. Like I think we'll talk about a little later, like you could rent time and timeshare. And I think that's one way. You know, also we have also other like phones are more powerful.

So some stuff can. It's not ideal. It's just be done on gaming on a phone, you know, consoles. if you're having a console, you know, sticking with older consoles and playing games you just want to play before. I think the people are going to get hurt the most of people like want the max performance on the latest video games. Thankfully, I just not in that race. So, you know, for me, short of like something straight up breaking, you know, I'm probably like you, probably have like lots of electronics. We could share between members of the family laptops or something if we had to.

But, you know, at some point, yeah, you would just break down and buy it if you needed it. But I think then you have like a, there's a difference between it's broken and I have nothing and I need to acquire a cost. And I have something that is just less than I would want. Like, you know what I'm? Like the demand in both are slightly different. So for me, it becomes about that. But I've certainly wanted to upgrade the RAM and I just waited too long. And like that window is closed. I guess that's not going to happen. You know, at this point upgrading the RAM, the rest of the system is just basically free. like that's true by a whole no hard drives and specifically like SSDs and MVME have also gone nuts

that's not just RAM but if you try to buy it you're talking about like the steam deck I wanted to get a bigger SD card and I hadn't paid attention because everyone talks

about RAM but even SD cards have gone through the roof for people the shortage of nan chips so yeah it's pretty wild I think

um yeah I think it's probably a good time to replace your phone too because again phone has a lot of ram in it. I had an issue where my phone has started to degrade. It's about five years old. And I was like, yeah, I should, it's time. Plus, I was worried about the prices going up.

So I got a phone. I actually arrived yesterday. And about two or three days ago, the GPS chip, like completely failed on my older phone. So I really got super lucky. I timed it where a GPS ship fails. Next day the new phone is there. But there's about a day or two where my kids, I was hilarious that we were we were in the car and the the blue dot was just like teleporting all over the city and uh kids thought that was the funniest thing but but uh yeah i mean even for a phone i would say if you're on like a four or five year old phone you know and you could and you can you can stomach it you know get the phone now before the prices go out my next article is not per se about AI but it's kind of about AI all right so this is an article titled maria

meets Pareto. And this is about the Pareto Frontier, the efficient frontier. But it's in everyone's favorite competitive video game. Okay, that's a lie. It's the only one I play competitively with my family and not online because I'm not that good.

But my kids think I'm good, although not anymore because they've gotten a lot better. The Blue Turtle is Pareto dominant, right? Oh, in terms of weapons. But it's basically going through, which is a very actually, you know, an interesting thing, even if you don't care too much about Mario Kart, but the setup is you have some number of characters on. I know how many is like 16 characters you can choose from. Each of them have some number of, you know, vehicles they

can be in, you know, go cards or motorcycles. They have some kind of tires you can put on and then some sort of like a glider device in the more recent, most recent, no, nearly most recent Mario Kart. I think Mario Kart 8 deluxe. And each one of those changes various parameters. There's also some hidden parameters that even aren't shown. And so the question is like, how do you pick? And the answer is there is no one right answer. You know, growing up, it was always, you pick this, this person or this car, or, you know, this is the team you pick in Madden, you know.

The answer is like more complicated than that. Depends on like how you define. And this is something that comes up a lot in software engineering, which is why it's kind of an interesting article. And also in finance, but like, what are you optimizing for? And so if you think about to spend it in terms of like finance, is it better to buy stocks or better to put your money in a savings account? And it's like, well, how much do you care about risk and how much do you care about, you know, growth? Because, you know, there's no free lunch, right?

Okay, that's stealing another finance idiom, I guess, economics. But the idea is that there are strictly worse positions you can be in. So there are things you could do with your money that are more. risky for less expected payoff and you should not choose those things. Just like in Mario Kart, although there is no singular best answer, there are certain combinations which depending on what metrics or even all the metrics, choosing them is strictly a worse choice unless you're just trolling. I guess that's not a metric. But against the normal metrics, you know, there are certain combinations that are strictly worse. So if you care about, you know, the tradeoff between two variables,

like acceleration and top speed, which are very common ones, then there are combinations which are not, are strictly inferior to other ones. And so the idea is anything that lives for some combination, not in the interior, not inferior, we call those things perido efficient or on the Pareto Frontier, which means they're sort of like normally in the graph to the right and up, right? So there's this curve along all the choices.

and this applies through so many things we do. Like I mentioned, trading off what you put in your investment portfolio, how you choose characters in video games. You hear it a lot coming up about LLM models. So there are LLM models have various abilities and various price per token. And that forms an efficient frontier. So, you know, you may argue whether Open AI or Anthropic today has the best model. But they certainly don't have the cheapest.

So you can go cheaper if you need less, and there's like a different position. But there are also places you could pay just as much for crappier AI, right? So there is a Pareto Frontier. And when we talk about improvements, we're talking about moving that frontier forward, moving to a new spot that no one has been before and forcing others to play catch up. So a very interesting, well-written, lots of cool graphics. You know, everybody loves a good video game story. If you've played Mario Kart, which lots of people have, you can definitely read this.

but also an introduction to an important concept. Yeah, this is super cool. I always picked a Bowser, and now I'm learning that that was like Pareto-dominated by Donkey Kong, apparently. So. That's awesome. Yeah, check it out.

Yeah, this is a great visualization of a really important concept. I love this. All right. Mine is Quinn 3.8 versus Muse Glimmer. So it's interesting, like the sizes of the models have gone through different, you know, like hype cycles over time. So you basically have the people who are trying to build the biggest models. And so currently, you know, these are, you know, the closed labs and then and then people like GLM or like Kimmy K3, which has, I think, a two and a half trillion parameter model.

And so, you know, so at least for Kimmy, if you wanted to serve Kimmy K3, you would need, I think, like $200,000 worth of equipment minimum. So, you know, it's definitely out of reach for almost everybody on an individual level. And not practical for most companies, unless you think you can really serve that model, you know, keep it busy 24-7. Now, then on the flip side, you have models that are very small that are meant to really run on edge devices. So, for example, you have the Jemma 4 E2B model, and the E2B means effectively 2 billion. So there's basically a router, and without getting too much in the weeds here, there's kind of a router that decides which experts and which modules to activate. and and it's I don't know where they get the word effective from, but on average, you know, they're activating about $2 billion of the weights.

So that can run on your phone, especially after you quantize it, you know, even with the KV cash and all that, you know, it can run very comfortably on your phone. So what's emerged is kind of this sweet spot in the middle where, you know, it's too big to run on your phone or even on. an entry level machine. So imagine a machine with like 16 gigs of RAM. You know, these models are too big to run on those, but they're small enough to run on a single high-end machine, you know, consumer high-end machine.

So think like a MacBook Pro is 48 gigs of RAM, you know, or a Windows laptop and 64 gigs of RAM, etc. And so Muse Glimmer and Quinn 3.8 are both around 30 billion. I think the Quinn one is 27. And so, you know, as I said, if you have a high end, your laptop or desktop, you can run those pretty comfortably.

And they're very, very powerful. Now, I've tried it myself. I haven't quite got the level of performance for the tasks that I'm doing that people are claiming in the benchmarks. So it does feel like maybe the model's quite living up to the hype there, at least for Quinn.

Use glimmer, I think, is pretty good. The big thing that I've noticed from these smaller models is, and this is kind of interesting, is they don't necessarily call the right tool at the right time, and they also don't really know when to stop working. So they'll work for too little, or they'll think for too long. And I found this really fascinating, like, because if you, think about it, what is a bigger model? Well, one of the most obvious thing that it is is more parameters and so more facts, right? So if you say, what's the capital of France and it says Paris,

well, there's some parameters or some set of parameters in the model that are responsible for knowing that fact. And so it stands to reason that, you know, if you have a giant model, it knows a lot more facts and it can bring a lot more facts to bear on a particular problem. And that's true, but it just seems like the critical thinking is better. And I don't know if anyone's really, the closest I can imagine people have come to explaining this is this J-space theory paper from Anthropic where models like project problems into some high-level space and then try and solve it in that space. And so maybe the bigger models have a bigger space to work with. But even things like browsing the web and doing kind of pretty

wrote things on the web like pretty easy things that even a child can do on the web, you know, like change the dates on this set of assignments or something on this website. You know, the smaller models will tend to struggle with that. So it's just on the common sense level, there is a correlation there, which I found interesting. But the models are getting better and better.

These 30B models are definitely a huge, huge, huge improvement over the 9 and 12b models that people are running locally a year ago. So it's definitely a ton of improvement made. And I'll just end it by saying, I think 90% of the questions people are asking can be handled by this 30B model. You know, like, what are the kind of questions people are asking, chat TBT? You know, how do I fry an egg? Yesterday I took a picture of my kid's schedule that they gave us on a sheet of paper. And I said, you know, create a calendar ICS file for this.

You know, all of these things that 30B model could do, I'm confident of it. And so it's like on one hand, the gap is there. And it seems like it might not be easily closed. On the other hand, you don't need the more expensive model for 90% of tasks. Yeah, I agree. I mean, it's definitely an interesting space. In fact, one of like the most interesting spaces, because I saw, I won't, I don't know, I don't want to go too much into like personal beliefs,

but I still see this just like, oh, hi, our, you know, chat GPT, you know, Cloud, Gemini app is happy to ingest your data. Would you like to share your calendar, your health information, your whatever? And it's like, no. You know, whatever. Again, I'm not going to get into privacy as like a thing that's. personal and different for a lot of people, but local models to me are a way where you can say, look, I can do this.

There's not, like, I'm fine doing what I'm doing here, you know, but it's staying on my computer. And even just something like, hey, I have a, can you go through my tax returns and just make a plot of like my effective tax rate over the last few years? I would love to know that. But I'm not uploading. One, that's like a lot of data to upload.

And two, like, I don't really want to expose that level of detail for, you know, who knows what reasons, what happens. Now all of a sudden you ask, how much does Patrick Wheeler make or whatever? And then all of a sudden, it's like, oh, I know, that's what I'm like training. Yeah, so it's like, I think local models like important thing. And if you've not tried it, it's definitely worth trying. And like you said, I think, Jason, there's a lot of like questions that, yeah, it just,

I mean, I guess like to flip it around, Google search, like, things that. I just couldn't search for before or whatever, just get answered super fast by, but I assume it's a relatively cheap, you know, Gemini model that Google is using. And so a lot of those things can be done. And I do think the mixture of experts, the effective ones like you're saying,

are really interesting in that they give you opportunity for streaming from like an SSD into RAM if SSDs weren't so expensive as well, but like streaming the data off of disk and interram for like that stage of inference. and just in general what they mean as well. But I'm excited for that level of like what you can do and just speed as well. Like oftentimes you get a response pretty quick.

Whereas if you're if you've ever used one of those other services extensively, sometimes I think they just get throttled. Like there's just too many people using them. And so the responses are just so slow. And here it's a single user thing, right? Like I just get the response when I want it. Yeah. So I tried both of these.

I tried Muse Glimmer and I tried Quen 3.8 pretty extensively. I found that for, and this has been always a problem with these Quen models, is there tend to really struggle with text and formatting text and outputting text correctly. So for example, I had both of them work on the same task, which involved creating a lot of markdown. content. And what I found is that the Quinn 3.8 model, it wouldn't close the bold. So you know how in Markdown you double asterix to make something bold? So it'll start something bold like a word, but then it won't close the bold and that they'll get real confused. And you'll either end up with

like a giant amount of bold content or it'll just literally draw the two asterisies. So it just wasn't able to respect the formatting of Markdown. Muse Glimmer was, was able to respect it. The other thing I noticed, personal anecdote is, Olama, which is what I was using to serve these models, it has an MLX mode for almost any model. Like for every model you can put dash MLX, and it will, if you're on a Mac OS machine,

and it'll use like MLX, which I'll have no idea what it is, but it's some kind of thing on the Mac that lets it do these vector operations faster. But what I came to find out, and later on I confirm this, is the way Olamma does the MLX, they have some way of automatically converting a model to MLX,

and I don't think that that is, 100% reliable. There's something lost in that process because when I compared the MLX version to the regular version, the regular version was a lot better. So I guess all of this to say at a high level, it's kind of like Ender S1 3D printing days. You know, we're beyond like the, oh, you have to build your own 3D printer from scratch and it probably won't work.

but we're not quite at the like, oh, I buy a bamboo and everything just works. So we're somewhere in the middle where these 30 billion parameter models are powerful and you can see the future right around the corner, but it's not just like drop in yet. I'm here for that analogy. I figured you to appreciate that. All right. My next one sort of in Congress, I think with a lot of the stuff we've been talking about so far, but that's okay, is everyone should know SIMD.

So we talked about SIMD a number of episodes ago. Someone's going to be like, oh, it's 100 episodes ago. But I don't think it was. SimD is a single instruction multiple data. And in line with I saying before, this specific example is actually about Zig programming, which I've never done Zig programming. There's some nuance there.

But in general, just like I think we talked about in the other show, just walking through like what, what it is and why it's important. I think this is an example where, at least today, it can be difficult to balance, you know, implementation, I'll even say with the LLMs, of doing something one way and then like completely redoing it in another. So when you build a program sort of normally,

the most common thing is to have an array of structures, right? So you have an array and your array, you put your data that's intermed, leave does like, you know, field A, field B, field C, then field A, field B, filled C. And there's a very common. And that's how you would for loop through it. It's, you know, how an LLM would write stuff by default. But understanding when and where, sometimes you want to rotate to a structure of arrays where you say, I have all my A channels together, all my Bs together, and then all my Cs together is a huge unlock in some cases for doing things like in SimD and other

optimizations. And a lot of times, if you don't know that's possible, you don't know to dig into it. And there are lots of libraries. But again, compilers can do some SIMD, but there's only so much range they can act in, right? So they won't re-factor,

they won't recompile your whole program in order to get SIMD optimization. And similarly, I think by analogy, even in the LMs, I think this is where, at least for today, I'll say, I think understanding to ask for it, because it's not something that it's going to know to reach for immediately.

And it's also trying to which we all want, you know, we don't want them spending extraneous tokens just doing constant refactoring and trying stuff except when we do. But like, you know, they're trying to limit it. So doing massive refactors like this and not knowing is,

so sometimes you're going to need to push it to say, look, I want to implement some de-optimization. I want you to build it in this way. And when and how to ask for that and whether it even makes sense is something you really got to think about. What hardware are you running on? And these are things,

I'll say, normally doesn't figure it out. Like, you normally have to tell it, you know, like, hey, you know,

I want you, you know, write a simple, like I was doing something with the, just like a cheesy ML something. And it was like trying to run Kuda stuff. And I don't have GPU that's in video like on my door. It's like,

why are you like stop? No, like you didn't even think to check it. So anyway, SimD is a very powerful tool. See our previous episode. See this blog article. but just I think to talk about how useful SIMT is and a very efficient thing and the instruction

sets have gotten better over time from when I first use them to now they're really kind of awesome and libraries use them but also as a means to back into talking to I think this is the kind of understanding it still is important to have because I think it's something where we still need to help give direction to what we want out of our programs. Yeah this is a good point you know I think that the LLMs, they make a lot of category errors. And by that I mean, just quick recap, five second on what a category error is. The common example is you give a student, you give a child, a tour of a university. And you say, here's like the math building, here's the computer science building, here's the literature building. And then the student says, well, where's the university

building. And it's like, well, no, the university is like an abstraction. That's like, you know, a collection of these physical buildings. It's not a building itself, right? So that's a category error. And I've noticed the AI is, a lot of these AIs really struggle when it comes to categorization. So for example, you know, you'll tell an AI, make this code faster. And what you really want is for it to, you know, find the hot spot. and rewrite them in SIMD. But what it actually does is like delete like the slow parts of your code and now it's not functional or or you know rewrites the whole thing in Rust or something like that.

Like it rewrites the entire program, right? So it's like it doesn't know at what category at what level to operate. And so this is I think, you know, people, people use a term taste. I think taste is something different. but I think independent of taste, I think knowing at what category you need to operate to solve the problem you're trying to solve is still kind of firmly

in the realm of human beings. And so you have to know that, hey, here's an option. You know, if you have this Python code, you know, one option you can do is to take the slow parts of the code and use Scython, you know, to bring them into C,

and then use SimD after that, as opposed to like rewriting the entire thing. I think it's time for Book of the Show. I've got the first book. This is a book I have and am getting ready to start reading. So unfortunately, I can't give the full review yet. But this is Waves in an Impossible Sea by Matt Strassler. And this is a book that's, I don't know how you call it.

We've talked about some of these books number four. They have a name. It's slipping in my mind now. where it's like sort of a casual read, sort of like just a compelling story, but talking about facts. So I think Simon Singh does this for like, Fermat's Last Theorem and the code book cryptography book

where it's like stories, but through the stories, he's explaining like the history and the operation of something in sort of like layman's terms or whatever. Anyway, so this is that about sort of subatomic physics. And I do not know why through my entire life, I've just been sort of like fascinated by elementary particles, subatomic physics, even though like it's not my background. I never ended up studying it.

I know really nothing about it. I won't talk about it because I'll like horribly screw it up. But just like the way that people describe, there's a couple of YouTube channels where people talk about like the sort of math and physics behind sort of like discoveries. You know, wise spectrographic lines were, you know, befuddling people in the early times. 20th century. Just like all these things. And I don't know. It just always really excites me. So I'm, I'm happy to dig into this book and just sort of like go on an adventure because for whatever reason, this domain has always, is a nonfiction. I guess I should clarify. It has always been something that is awesome. There's so much more going around at like the very, very small level that

we kind of just never think about. And it's kind of crazy. But it's cool at the same time. So is this a fictional tale that explains a real phenomenon or is it a nonfiction book? It's a nonfiction book. It's not like an allegory or something. It's just a sort of talk through in an engaging narrative way through the history and state of these things. Oh, very cool. Yeah, I mean, I have just been diving into so many research papers that are not big. fairly interesting to me and my friend grew, my colleagues.

So I'm not going to make everyone suffer through the list of research papers I've been reading. But one of them in particular I wanted to talk about. So, okay, when we started with natural language processing with neural nets, right, people started with these recurrent neural nets. And so the idea is think about like a memory, a block of memory. but instead of it's storing very explicit things, like if you were to store the word dog in memory using Python or something, it would be a byte for the letter D, a bite for letter O, bite for large-g, and now you have dog, right?

But instead of that, it's this really abstract, really high-dimensional space where you're storing, you know, a whole library full of concepts, and then you can sort of pop things off of that. So, you know, the early Transformers did this where you had an, encoder that took what the user is asking for and the, you know, partial answer, which could be nothing at the beginning, right, encoded it into, you know, this memory bank, and then a decoder that popped off, you know, tokens off the memory bank and then also updated it. So, so the idea is,

you know, user input comes in, typically not with a partial answer. So the user input comes in. It gets encoded in this memory bank. And then there's just this really tight loop. It's like, based on the memory bank, pop off, you know, the first word of the answer, and then mutate the memory bank, and then ask it again, and just keep popping words off the memory bank until you pop the end of answer word, which is like this special token, right? And so the problem with that is that you have to compress your question into this memory bank, right? And then as you're popping words off, the memory bank, you're also kind of having to store what you've popped off so far. So if you say, what is the capital of France that has to get crushed into this vector, right, that represents that question.

And then if you pop off, you know, the capital is, now that same memory bank has to know your question and know that you've already completed part of the answer. And this is why early transformers would do things like, the capital is, is, is, is, is, is, because it couldn't figure out how to store the fact that it's already said that word, right? And so now transformers are decoder only, which means they

what goes in is the question and you know the part of the answer you have so far and what comes out is a single word so like what is the capital

of France what comes out is the and then this whole thing starts again so like all that work went in just to say the word the and then start the whole process over again

you know what is the capital of France the and then goes through this whole process and then outputs the word capital, right? And then so on and so forth. So now they've used KV cache and a whole bunch of tricks so that this doesn't waste that much computation, but it's still pretty weird, right? I mean, as a human being, I don't think we really operate this way.

I think, yeah, we internalize the question and then we sort of roll out the answer. So we've kind of deviated from the way a normal person or the way we expect a brain to think and reason. And it's pretty unnatural, right? So people are constantly trying to go back to this encoder-decoder idea. And world models have to be encoder decoder just because of their nature. We talked about world models on the last episode. And so the question is like why, how can you encode these things better so that you don't have to do that really expensive thing that we said earlier?

and I think part of it is, you know, the memory is flat, right? And so on your computer, you know, when you do malloc or something like that, and you get a flat block of memory, you can then go and put an image in it. Because, you know, you're storing some metadata. And as a programmer, you always understand. The code is an easy way to remember, oh, this huge chunk of, this huge line of memory. is actually a 2D image. But I don't think that the LLM internal is really good at that.

And so I think what we actually need are, you know, two-dimensional, three-dimensional, like n-dimensional hypercubes of information that they can read and write to instead of just a line of information. And so this paper kind of talks about that. So this is a paper where instead, of a single line of data that you can write to, you can now write to a grid or a hypercube,

and the space kind of matter. So if you write something to the left side of the cube, and then you write something else to the right side of the cube, those two locations are far apart, and it actually matters. It makes it harder for them to affect each other. So, yeah, so they basically are using, in this case, diffusion. Other people have used convolution. There's a bunch of different ways to do it. But I think that

storing information this way is going to unlock something really powerful in LLMs and a world model. So it feels like there's something cool here. We're just kind of on the cusp of it. That's exciting. Yeah. It feels like there's this plate, but it hasn't been true almost where like traditional, I think there's on the output of LMs, like this DSPY sort of like, instructions and things like that for attempting to kind of get to it, but almost where like you're talking about like memory storage or whatever, it's like tool, very limited tool invocation within the, you know, actual inference itself, right? It's like know that you can put stuff here in this way rather than just like learning it completely. But, you know, that complication becomes

incredibly difficult. So, you know, it is definitely exciting though for them to figure it out themselves and potentially even a better way of doing it, I guess, then maybe naively we would make tools to do it. Yeah, totally, totally. All right, time for tool of the show. Patrick, what's your tool? This is a very, no, I'm just kidding. I was going to try to make it funny, but it's the same every time.

It's a game. This game is available on mini platforms, a PC phone. I think it was popular for a little while. I'm probably past it, but I think I picked it up on a sale, and it is Nubby's number factory, which, like, the game is pretty good. Like, I got pretty into it. I like it.

It's a very casual. I think it's called like a Plinko-like, uh, rogue-like. Okay. Is that where you drop a thing and it pings off the, yeah, yeah, exactly. Yes, pings off numbers. Uh, you try to make number bigger.

And then there's various power-ups and that are randomly chosen. And, you know, your exponential growth in the, what you need to hit at each level makes it very hard, and very careful. There's not a lot of skill to it, I would say. I mean, like a little bit, but that's not the kind of main point. But the aesthetics are trip.

It's like a trip down 90s, you know, Web 1.0, I guess I was like late 90s. Like, it's so good. Just like, you know, just I just pull up a screenshot of it. And if you'll know instantly if it's like, this is my jam or not. So for that reason alone, just like, I don't know, decent gameplay, but incredible art aesthetics.

But I mean horrible, but like amazing. It sounds like Bellotro as far as the gameplay. Yeah, I mean, same kind of idea. Like a very basic game, like you're trying to play poker hands, but then like it's scaling. So you have like it's really all about the power ups. Yeah. In that way, I guess it's kind of similar.

I mean, definitely it's not at the Bellotro level. I was watching people do like speed runs of Nan Inf like Belotro. Okay, that stuff's crazy. Oh my God, yeah, the Bellotro stuff. Yeah, I still haven't beaten Bellatro. Like, even in one deck, I haven't gotten through all the challenges. And then I watch these people and they're like, oh, yeah, I have one card. I play this one card and I just win.

But I found out a lot of them are apparently using various mods, like, and then, you know, saying they don't or just to make it like easier. They really want to play a certain card. So even the like when I was watching, which, you know, I trust him, I was like trying to get Nan E-Inf like very quickly. And so there's a certain card that they want in order to be able to do it. And so they were like doing a seeded run where they had searched for seeds that would guarantee that card comes up within like the first or second shop or something. And so they've never played that run before,

but they know like the card they want is going to be there. Got it. Interesting. I'm not saying everybody does that. I just like, it's more common than I realized. So yeah, the game's actually not that easy. We're being diluted. Well, I think there are people who just play it a ton, which is probably like,

the same like Solitaire, like not all games of Solitaire are winnable. Yeah, it's probably like that. Like, I'm not saying not all games of Belatra are winnable, but some, you know, just you're going to have to play a certain amount of times in order to get 100% completion. And like, no matter how skilled you are. Yeah, that makes sense.

makes sense my tool to show is open code and actually patrick earlier you talked about doing your taxes with an lLM okay i don't haven't done that no no not doing my taxes reviewing my taxes reviewing your taxes reviewing your taxes within lm the IRS is on their way it's like do too do Patrick open up um so i um okay so our tax accountant which is this really nice lady that We met through church a long time ago, retired. And so we had this debate in the household. I think that an L.L.M. could do my taxes. Oh, yes.

Yeah. And I'm the only person of family who thinks this. So I was like, okay, we'll do this. We'll have somebody do our taxes. and I will also do our taxes with an LLM and then we'll see if they match up or mine's better even, right? And if they match up or if it's even close, then we know that we're good.

But similarly, I was thinking, you know, taxes, even like controlling the web browser where like all your cookies and your passwords are stored, like installing the Anthropic Chrome extension or having, you know, chatGBT do my taxes is where I draw the line. You know, like that's like, yeah. Like, and I feel like I'm pretty, you know,

open when it comes to privacy, but I feel like I draw the line at, you know, these kind of things. And so, and so I downloaded open code, which is one of many harnesses. I definitely don't claim that open code is better or worse than any of the other ones.

And maybe we should do, that could be a whole show, but I could do a deep dive on that. But at the moment, I'm trying OpenCode with these Muse and Quinn local models. And it's really, really good. As far as the harness goes, you can spin up subagents, you can do loops, which we'll talk about. All the things that you can do with Claude Code are pretty well supported with OpenCode. There is a little bit of jank. there was a situation where Muse, like,

when it returned the tool response, it wasn't quite formatted correctly. And instead of, you know, dealing with that or retrying it, open code just hung indefinitely. And so I had to kill it. So it definitely got some jank. You know, got some jank.

But, you know, the fact that it's free and open source and works reasonably well, I think it's pretty nice. I ended up writing a very simple watchdog timer to kill open code if it's processing and it hasn't finished

in a certain amount of time and restart it. But again, I think it's just like we talked about with the 30B models where we're maybe six months to a year away from having something

that's really polished here. And open code, it seems to be very popular. It's got a Python SDK if you want, want to non-interactively, you know, ask a bunch of agentic questions. So overall, really powerful piece of software. The other one, I mean, like you said, we would have an episode about it, but was the Pi coding agent, which tries to strip down the harness to

like kind of bare minimum and then have you add to it what you want. So you ask Pi to like improve itself, which I think is really interesting. I haven't gotten into it yet, but it's on my shortlists and so I wonder if something like that could be for you know open code by the name is probably built mostly for coding so if you wanted to use it as like a multi-purpose harness I wonder if something like that might you know also work yeah totally on the same page yeah I want to desperately try pie I've heard a lot of good things I went to lunch with somebody about a week ago who's really into it um so it's definitely on the list so I can try it and we can do a show on it Okay, all right, sounds good.

But you left us hanging. Who was right? Oh, well, so Janet, you know, our friend, she retired literally like a week or two ago. And so this would be for next year's taxes. I could back test it. It's a good idea, though. Maybe I should try to do my current year, you know, the ones that were already submitted. Try to do that without hindsight.

That way I wouldn't have to wait until January or February to try to. try this experiment out. Oh, one thing I did do, though, kind of related, I downloaded CSV files for all my credit cards and checking accounts and all that. So I got basically all my transaction data over the past 90 days. And I asked through OpenCode, I asked Mews, like, hey, give me a rundown of our expenses and what are we spending too much money on? Where can we cut down? Things like that. And it did a pretty good job. Actually, a really good job because you know that's when the LLMs are really good at if they see like a bill that just says Chili's it knows that Chili's is a restaurant right like that kind of stuff so so

that actually turned out great so I'd highly recommend I mean maybe there's even something there around you know like a open source project or something where someone can just install some desktop app and I don't know how it would connect to their bank they'd probably have to do that part manually, but some desktop app that just like goes through all their finances and, you know, just for people who have a hard time setting up open code and all that. You have more to say here, but well, let's go ahead and go to our topic. All right. We should do a show on, yeah, automatic finance management.

But we will. Brito efficient. Burrito. Oh, man. I, it's, you know, I've had this. same Twitter handle for like, I don't know, 20 years or something, but I think I need to change into Pareto efficient. Oh, man, so good. All right, agentic loops. So a bit of a history lesson here. So modern history. Okay, when LLMs first came out, when the technology first

came out, people, companies were afraid to roll it out. I don't know if you remember this era, but there was definitely an era where, you know, companies would dip their toe in the chat bot water and there would be so much backlash. People would find the worst possible thing that it could say, the dumbest thing it could say, and that would make the news headlines. The one I'm remembering of specifically was there was a Facebook, I think it was called, was it called Liberatus? We'd have to look this up.

But there is this Facebook project where it would create research papers. So you would say, you know, here's a bunch of related work. I want you to write a research paper on, you know, spatial neural nets or something, right? And it would go off and write a five-page research paper that looked like you could submit it straight to ICML. And so people wrote ridiculous research papers. Like, you know, they would like write a research paper on running a computer, with hot dogs and it would do it, right? It wouldn't push back.

And then people would submit it, not submit it to ICML, but they'd submit to the New York Times or something. It's like, look how dumb the Facebook bot is. And Facebook pulled it down. They were embarrassed, they took it down. And so nobody wanted to release the chat bot. The chat bots kept getting better and better and better.

Nobody wanted to press the green button until Open AI pressed the green button. And when the initial chat, GBT, came out, They got in so much, there's so much flack for it, right? It's like, oh, this is evil. It's going to teach people how to make bombs. It's also wrong. It's telling you to put glue on pizza and eat it, right?

All these things. And to this day, you know, whenever you go to any of these websites, the first thing it will tell you is, you know, Gemini makes mistakes, right? Chachu-T makes mistakes, right? I was driving behind a semi, and it's like sometimes, you know, the autonomous vehicle makes mistakes. mistakes. I'm just kidding about it. Oh, that's terrifying. But we just have come to terms with the fact that these models make mistakes, right? And even if they don't make mistakes, again, there's the category error where, you know, it did technically what you wanted, but it didn't follow kind of the spirit of what, you know, a common person would have expected, right, for that question. And so in comes agenetic engineering.

And the idea is we know we're not going to get it right every time. So what we're going to do is feed in our answer back in and see if there's more work to do and just keep doing this until we reach some kind of a conclusion. And that's what, you know, Claude Code kind of brought into the mainstream, right? So that all works out well. But the question there is when do you stop? And again, all of the same errors from before are still there. So if you might say something like, hey, get to 90% unit test coverage.

And this is about a year ago. It's not really true now because of loops. But you'd say get to 90% test coverage. year ago and Claude code would write some unit tests and then say, hey, I got up to 34%. Isn't that awesome? I'm going to stop here, right? And if you wanted 90%, you had to tell it to do it again. And I literally, about a year ago, had a bash loop that asked Claude to get to 90% test coverage in a loop and just ran that for 24 hours to get to 90% test coverage. So that's, that's is an extremely primitive agentic loop that worked.

It was successful. And nowadays, if you say something like gets a 90% test coverage, Quad code recognizes that you've set a goal and converts your question into a loop. And that's how it's able to run for much longer nowadays. So sometimes the LLM will do it for you. oftentimes it won't and and again even if it does it might choose a terminating condition that's not sort of in the spirit of what you are thinking of right if you say something like I want the I want the accuracy of this model to go up well like what does that mean do you want it to get to

90% do you want it to get to 99.9% you know if you let the LM decide that then you don't really know you're going to get out of that. So the rest of the show, we're going to talk about how you can build, how other folks have built these loops, how you can build these kind of loops and best practices there. So I think for like a half step back,

I think one of the earliest things people were figuring out, and this was popular with things like Lang chain and was other than laying grailgrath. Yeah, yeah. Where there were, you wanted to, and it's just kind of a loop, like, you know, pull, watch an email queue.

And anytime there's like a customer service question come in, you want to like label it, tag it, dispensate it, you know, whatever, and move it through. And I think for me, this just like turns into sort of like cron jobs. But it's actually a big unlock, which is, you know, hey, I, if you have a computer system, like, wake up and ask the LLM every so often if like the price on this, you know, website has changed.

You're like, oh, you can write scrape. for that. Yeah, try to write a scraper and then just watch like it not work because they move the stuff and the tags change, you know, versus giving it a screenshot, you know, and having it converted to text and then doing it. This is like a big deal. And so just replying to recurring events or doing the same task, which again, I think it was a half step before, you know, the introduction. But this was like one of the earliest places where, you know, we saw like the looping LLM sort of take a over. But then to transition to more, you know, what Jason said, for me, the big one, this loop there was, it was like the Ralph Wiggum loop that people would always talk about. Yeah, where they were like, write this special skill or whatever to like do the same thing over and over again. Like, you know, pick a, you know, bug and, you know, try to fix that bug, pick the next bug and try to fix that bug. But the first one to really like, I think go, I'll say viral with it was Andre Carpathy, which feels like he has an act for going viral.

Or maybe only a survivorship bias. I'm not sure. I actually don't know how he says otherwise. But he did this thing where he posted something that he titled Auto Research. I'm not sure if it was from the beginning or he eventually titled it that. But where he took a very small, I think it was nano-GPT, which is like a small GPT on a set training set and said, you know, basically can you make this run faster, train faster? and can you try various fixes to improve the metric, the output performance?

And to Jason's point, the terminating condition wasn't really present, but just this desire to go again and again and try something and then revert, you know, if it didn't work or it made it worse. Now, of course, like, it went viral because it was hugely successful. But there are lots of gotchas along the way as well. Like, there may be you need two, three, four things combine. So, you know, you take a step back to take two steps forward, you know, like there are all of these places is you have to be very aware, you know, of what the problem scoping and what the domain is and how much least you're doing it. But I do think there's this, and we talked about a little earlier, like the tension between, especially a cloud

LLM provider, not wanting to be accused of just evaporating your tokens. So therefore, they don't want loops that aren't going anywhere. And you who may be saying, like, I'm not paying the bill. I want all of it. And so, you know, you can end up with these tensions. And so specifying something almost external to even the harness or even a lot of these stuff is incredibly useful because then you're really crafting, sculpting, setting, trying it, encouraging, whatever you want to do and manipulating, you know, that prompt.

And normally you would need a human sitting there. But actually, like, you can ask an LLM between runs, should, like, should we revert that? Does that, you know, make sense? Is there something further along those lines? You don't need the pre-planned complete trajectory to go down. And so I think this has really caught, you know, people's attention. And now, and I think this has been popular before, but now it feels within and reach just to kind of like the final maximal extent of this

is when these allelims are, if they can, reach recursive self-improvement. So the idea is if one of these, you know, major AI providers or a new up-and-comer gets an AI that can improve itself by, you know, 10, 15% per training cycle, then they can basically just keep doing that without human intervention. And it's just bottlenecked by speed, compute, power, whatever, you know, and then they'll just be on a quote unquote escape velocity. I still think that makes a bunch of assumptions, but you'll hear it, you know, tossed around. You've heard. AGI, it kind of went by the wayside a little artificial general intelligence. But now you'll have this RSI, this recursive self-improvement where these loops are what is being done to try to say, can the LLM figure out what the LLM needs? And of course, the evil laughter one is, you know, hey, we need to make this thing more efficient. And then the robot decides the humans are the inefficiency. And so that annihilates all the humans because they're just consuming needless resources.

But yes, that's the Terminator scenario, I guess. Yeah. Yeah, it's, it definitely lends itself to going off the rails. I mean, I think, like, leading up to this was this idea of in-context learning. There's actually a paper that just came out not that long ago, or somebody showed that if you add periods to the end of a question, question or to the end of a prompt that it actually get better answers. So, and the more periods you add, the better an answer you'll get.

And the explanation was, well, you know, every single period requires the model to have to spend more compute. And so that's just more opportunity for the model to think. And so you literally had this graph where it's like, it's like, you know, statistically significantly smarter if you just add more periods of that. the end. And so, so clearly like, you know, if you were to add instead of periods, but add, you know, useful, you know, auxiliary content or a failed experiment or something like that, then clearly it's going to be, you know, more intelligent of an answer. If periods do it, right, then useful information

is going to do it even more. And so, and so that's sort of spawned this idea of, hey, like, you know, let's try something. And as long as you document your success. or failure, and that's going to result in a better answer next time. And, yeah, and so, you know, then you run into issues around the context limit and compaction and all of that. And so that's a whole other issue is how do you, how do you sort of give the right paper trail to these LLMs? That's maybe even another show.

but the first part of this is is kind of setting up, you know, appropriate terminating conditions and an appropriate way to step forward. So if you say, you know, hey, build Facebook. All right. If you just go into Claude Code and say, build Facebook,

well, you know, that's a very nebulous question. And probably what it's going to do is it's going to, you know, build a front end that looks a lot like Facebook and, and, and, and, you know, maybe a back end for, for doing some basic things, like posting messages and receiving them and all of that. But, you know, if you wanted, for example, live video, it's probably not going to build that feature, right?

At least not in the first prompt, right? Now, on the flip side, if you set up a loop and you said, you know, build Facebook and then also like, you know, search the internet for like the top 100 most useful features in Facebook and go through all hundred of them and, you know, tell me which one of those has have been built and haven't been built and get to a hundred out of 100, right? well that's very different because now the model has an easy way of knowing whether it's completed the task or not so it'll go to the web it'll grab 100 features based off i don't know reddit or whatever people people are talking about their favorite facebook features um and uh and so the loop will will know based on trying out your program that only 70 of the features are done and it will continue and at that point it's pretty determined. So, you know, there's like,

I've actually seen it where if it runs for too long, the harness will kill it and things like that. But generally speaking, if you give it a very easily verifiable target, you know, it will loop indefinitely to achieve that target. And there's things that you can do to make sure that it's committed, which we'll talk about. So there's kind of two categories of loops.

So maybe three categories. One is sort of firing on a trigger. So, you know, a GitHub issue has come in, an email has come in, et cetera. There's sort of a cron, you know, firing every day every five minutes or 15 minutes. And then there's this third one where it's almost like a while loop where it's, you know, loop until a certain condition has been met. And so it's kind of like we're reinventing basic all over again. But we're doing it with a much more capable of instruction set. Yeah, I wonder.

I feel chaining a lot of those things together is going to be really interesting. And I think, you know, obviously, there's lots of opportunity, but it's tough to build right now because in that space, you just don't know what will get scooped, what will get, you know, implemented by the major, major providers. But certainly I think that crafting very targeted directions and also doing some form of which I know you might have have some insight to but also like sort of space exploration, no, not like outer space, but sort of like exploring design space is something that I haven't

really seen a ton of progress towards. Like we talked about earlier sort of like, hey, you could please refactor this to use SMD. Right. It's like a very specific thing. But exploring that space in a like sensible way with you know sort of branching and picking up that's how humans or at least my brain ends up working you kind of like all of these potential things and you're sort of trying to balance the explore versus exploit in in the design space and that's not something that's something that's something that I still need to provide strongly but with loops and this is something that I think in the future as lower low low low smaller models get faster and faster and faster,

there becomes a point where you can just, you know, shotgun approach, try all the things in parallel, and sort of like, you know, obviously explore very quickly, but then choosing which ones to pick up and continue or not, you know,

yeah, I don't know. There are some interesting ways of approaching it. Yeah, I think actually that's a really good point. You know, we should talk about subagents. So,

so a sub-agent, is basically, think of it as, it's basically recursion, right? So think of it as clod code spins up another clod code and asks it to do something. And so the nice thing about that is the sub-agent has its own context. So here's an example. If you are, let's say you're coding up Facebook, so I want to build Facebook. and the ux the the the the the ui doesn't look quite right so you know it's the the columns are too wide and you know the heading is too too thick and doesn't quite look like facebook well so you're going to

spend a ton of tokens going back and forth about that right you're going to say well you know go to the facebook dot com look at their website get their columns right and it's going to go to facebook dot com and take a screenshot, take a screenshot of your version, and each of those screenshots are going to be hundreds and hundreds of tokens, right? And you're going to hit your context limit comparing all these screenshots, and then it's going to do something called compaction. And what compaction does is it basically summarizes all of the content up until now. And, you know, it's sort of a black box, right? You can't really count on it to do anything you know, as you would expect.

So to compaction might just delete all those images, which is probably what you want, or it might keep them and like delete all the interesting design work you did before you went into this rabbit hole, which is what you don't want, and you can't count on one of the other. So you could say, you know, spin up a sub-agent and have that sub-agent in a loop,

have that sub-agent compare visually, your site and the original Facebook site in a loop and iterate until it's complete. And so while that sub-agent is off doing that, you could even do other things. But concurrency aside, if nothing else, it manages the context. So when that sub-agent returns

and it says, okay, I've got it visually, you know, a match, when it returns, all of that context is deleted, which is, in this case, what you want so that you can move on to the next thing without polluting the main context. And so if you use Git work trees or other sort of technology where multiple programs can edit the same code at the same time and basically have a whole Git workflow locally,

then you could spin up. You come up with 10 ideas to make my website more flashy. and here's this benchmark, here's this black box you can run that gives you a flashiness score. So come up with 10 ideas, have 10 sub-agents, do 10 totally different things

to make the website flashy, and then, you know, tell me which of those 10 increase the flashiness score and then keep those and throw away the others. So you can start to get to this like simulated annealing kind of approach where you try a bunch of ideas, and keep the ones that are better. So I think we'll start to see a lot of this stuff

come to bear more formally. But right now you can build it yourself. Now I'm just thinking about Nubby's number factors flashingness score. I wonder if there's a way to quantify, you know, like how engaging something is. Because that's one thing, in LLM, I don't know if it can really,

could it look at that game, and the ET game on Atari and know that one is better than the other, you know, other than from just popular sense of it, right? Nobody could definitely hire people on Fiverr and do a poll. That's true. Oh, another thing that's worth of mentioning, we kind of wrap up here, but loops are early days.

I mean, this is now the third time that I've mentioned this, so it does seem to be kind of like a trope for this episode, but kind of like 30B models, loops are early days. And so one thing that I've found is if I tell it, let's say the 90% test coverage case, sometimes it will just end. Like sometimes it won't respect your terminating condition. Sometimes the loop just ends and you don't know why and it's not really clear.

the one loop that the one loop that seems to be reliable is the Cron job and maybe that makes sense right because it's the least ambiguous so if you say run something every five minutes it will almost certainly run every five minutes indefinitely so what I've learned to do is I have something

that says hey monitor this run like train a model monitor the metrics of the model and you know if the model is better then put it in this cat in this folder full of really great models if the models are regression then abandon it right and sometimes that loop will work and sometimes it'll just stop they'll say oh yeah i trained i trained my third model i'm done and it's actually not done right so i've set up another loop which runs in parallel which basically says you know wake up every 15 minutes and if if this other

loop has stopped than just, you know, started again. And, uh, and having both of those loops seems to be, uh, a way to like keep that first one, you know, uh, from dying. I was going to crack a joke about us being sub agents in the loop, but, uh, my loop got stuck. Apparently we don't have the other agent that kicks us every two weeks to make an episode. We need a, that's the agent. That's, we're missing. Um, But maybe just like a, you know, kind of a call to action here. I mean, I've done so much with loops in the past month. As I said, I've kind of gotten the podcast to transcribe and do all of that stuff locally with local LLMs for free. So many other projects, the MAME Hub thing, where it just goes off and does its thing now.

It's a super, super, super powerful technology. highly recommend folks learn it and leverage it. But it's also, it's just janky. It's just early days. Early days are actually kind of the best days. In hindsight, you know, when all of this stuff is solved and kind of frozen and we're all just using it, it becomes a little bit less interesting. I mean, at this point, you have a chance to actually shape the way that these things end up.

So definitely if you're not using loops, you should learn it. So people have gone further and done grass. and basically this whole like, you know, communication protocol between many different agents. And I mean, I think that this is becoming like maybe more efficient ways of doing this. But you're not really typing that much anyways. So if you have like three loops, it's probably fine at this point in time. But yeah, definitely, you know, something that you should keep your eye on. This space is moving so fast.

I don't know. I feel like there are topics for us to discuss instead of like the backlog, like, oh, someone suggested this a few years ago. I feel like we're in this stage of like this development happens. We should talk about it. Yeah. Yeah, it's wild. I mean, I definitely think we should cover pie.

I actually have it installed, but I haven't done a whole lot with it. I wrote it on my sticky. I'm going to go do it. Nice. And do something productive. Yeah, yeah. So folks out there, if you are coming across tech, you know, whether it's harnesses or really anything that you feel like we should bring to attention to the audience, just shoot us an email, hit us up on Discord. And as always, thank you so much for all of your support on Patreon and the other platforms.

And we will catch you all next time. Music by Eric Barn Dollar. distributed under a Creative Commons attribution share-a-like 2.0 license. You're free to share, copy, distribute, transmit the work to remix and adapt the work but you must provide attribution to Patrick and I and share like and kind.