Algorithms + Data Structures = Programs - Episode 293: APL or Assembly

Episode Date: July 3, 2026

In this episode, Conor and Bryce chat about AI vs abstractions, the language of LLMs and more!Link to Episode 293 on WebsiteDiscuss this episode, leave a comment, or ask a question (on GitHub)SocialsADSP: The Podcast: TwitterConor Hoekstra: LinkTree / BioBryce Adelstein Lelbach: TwitterShow NotesDate Recorded: 2026-07-02Date Released: 2026-07-03The Daily - Why Everyone Cares About This World CupADSP Episode 237: Thrust with Jared HoberockWhich programming languages are most token-efficient?cp.RawKernelCuPyJAXTritonNVIDIA CUDA TilecuTile PythonGPU ModeAI Pioneer Geoffrey Hinton: AI Is Conscious, Superintelligence is Coming, And We Should Be WorriedVALERIAN and the City of a Thousand Planets Trailer # 2 (2017)Intro Song InfoMiss You by Sarah Jansen https://soundcloud.com/sarahjansenmusicCreative Commons — Attribution 3.0 Unported — CC BY 3.0Free Download / Stream: http://bit.ly/l-miss-youMusic promoted by Audio Library https://youtu.be/iYYxnasvfx8

We were right. All of our critics were wrong. I'll put my premise out there. The premise is that if you want to get to the best performance or even the like just correctness or performance, whichever one, if you want to get there in the most token efficient way possible, higher level abstractions, higher level languages, and in particular functional languages, I suspect I'm starting to get early data that indicates that they are the superior option. Like, I think that the ideal programming language for some class of tasks, maybe a large class of tasks, is probably something that looks a lot more like APL than anything else. And like to APL also, you know, I always make funny, you Connor, that it's so, like,

compact and, like, symbol heavy, but, like, that's also very token efficient. So, so I think, I think that I think you may be justified. Welcome to ADSP the podcast episode 293 recorded on July 2nd, 20206. My name is Connor. Today with my co-hosts, we chat about AI versus abstractions, the ideal language for LLMs to target and more. It is, it is, I'll give it to an American. It's 99 degrees out, and it's supposed to feel like 111, with 99 degrees and, you Canadian. No, not in dollars. It gave me 99 degrees in dollars.

So it's 37 Celsius. The sea stands for Canada. But it feels like, it feels like 43 Celsius outside right now. So I'm not leaving, I'm not leaving the, uh, the apartment for a while. Yeah, it's, it is, it feels the same. in Toronto, but it's only 32, which converted to Fahrenheit is 89.6 feels like 109.4. Yesterday, I didn't go for a run because it was, it felt like 46, and I hadn't gone for a run the day before. Anyways, I think it was like two or three days in a row, but then today I was like, it's too much. I got to go for a run. And it was pretty bad. I'd like to say that it wasn't that hot, but it was my heart rate. I was just running like jogging, five and a half minute pace per

kilometer, which is definitely not that fast for me. And my heart rate was at 165, which is way too high for running that slow. So I walked a bit and stay safe, folks, stay hydrated. I think I lost about five pounds worth of sweat on a 13 kilometer run, which is ridiculous. Anyways, weather's hot. I was supposed to go for a run today with Eddie. You know Eddie, right? I do not know Eddie. Who's Eddie?

Eddie, Nolan? He's one of the C++ people down in New York. Anyways, we were supposed to go for a run today. And then he just texted me and he's like, it's too hot. Maybe we can take a rain check. And I was like, okay, all right. I guess we can do that. Oh, man, what a week.

What a week. I'm back in the U.S. for like two weeks, then we're going back to Europe for more stuff. Did you just get back from Europe? This is just like, I feel like the last three months have been like a year. Hang on my internet's still being really crappy. Yeah, this is much better. It's crazy that like my, I'm getting like one,

I'm barely getting one megavit per second from my home Wi-Fi right now. Holy F, that's so slow. Yeah, I know. There's something's got to be, it's the heat, it's the heat. The Verizon can't take the heat. Maybe one day I'll tell the story of how I spent a weekend trying to Jerry Rig with a Gigahub 2.0 Bell kit, a mesh network with like three different nodes and a bunch of stuff. Anyways, I didn't know what I was doing.

And it was only like, I think it was Claude. It was Opus 3.7 or 4.0. Oh, this is recent. And yeah, it was back in September. And it took me a while. But oh boy, oh boy, once I got that kitten purring, who it is. And I also, that was the weekend I discovered that Ethernet cables,

I mean, why I didn't think that, I guess I just never had fast enough internet for the cable to make a difference. But like at some point, I just had like, you know how you have that basket of like HDMI cables and Ethernet cables and a bunch of cables that like 90% you'll never use, but you don't throw stuff? You just throw it in like a box in case you need it one. day. Anyways, I needed a couple different

Ethernet cables and I was just like, how come, because I think we signed up for 3 gigabit internet. It wasn't the fastest, but I think it was like the second fastest. Anyways, and like I wasn't getting anywhere close to it. And then AI's like, well, show me your cable. And then I was like, nah, that's not going to cut it. You need to

go order a better cable. And I was like, the cable? I was like, I paid for the internet. Isn't that it? Well, I'm sure like 80% of our audience is like, wow, this guy doesn't know anything. Yeah, I was going to, I was going to say like before you started talking about internet, Ethernet Cables, I was going to say,

this is going to be a, Connor is solely a software person type of story. I used to put together racks for our lab when I was at LSU. So I've been there, had to deal with that. Yeah. Oh, God, what was I,

what was I said to say? What were you just? We were talking about heat internet. Oh, yeah, yeah. I was going to tell a great price story. Perfect. Which is when I moved to Berkeley, after I was in Louisiana, I moved to Berkeley, and it's really tough to find an apartment in Berkeley.

And I'm staying with my step-step-sister. That is my stepfather's ex-wife's new husband's daughter. So her and I share step-siblings. So she's my step-step-sister. Wait, say that again. It's your step-fathers' ex-wife's... ex-wife's... New husband's daughter.

So my stepfather's ex-wife's daughter-in-law. Well, Bryce is just frozen. So you got the connection, hopefully. I think I understand it now. You did freeze in the middle of that, but I'll stitch it together. Yeah. Anyway, so I'm staying with her. I'm trying to find a place to live.

And eventually, there's just like only really one place that's like in my... my price range and that like is still available. And so I just like, I go and I see it and then I sort of sign up without really like looking at it that deeply. And there were a couple of things that I didn't think to ask whether they had because I was moving from Louisiana. And so in Louisiana, everywhere has AC. It's just like you have to have air conditioning in Louisiana. So like I never even thought to ask, does this apartment have air conditioning? The apartment had no air conditioning.

It also had no wash dryer in the unit, and it had no dishwasher, all of which I suffered with for three years. And then I swore I would never be in another apartment without all three of those things. And the day I was moving, so I used to stay at Berkeley Lab in the summers until like 8 p.m. Because it would be too hot to go home. And there was like no cross ventilation. And the day I was moving out of that apartment, there was like this epic California heat wave. it was like 110 degrees in Berkeley or something.

And so I'm here in my apartment completely naked, packing up all my stuff because it's so hot. You can't even put clothes on. Anyways, that's the bright story. Well, I caught like 90% of that because you kept cutting out. We'll have to see whether we leave in the part where we talk about you being naked, because I'm not sure how many listeners we're going to lose over that, even if it's just an audio description. and I'm running speed tests on my phone to see what's going on with the internet. It's starting to come back.

All right. And I'll listen to the full story upon editing it. And I guess the listener won't hear the drops because they'll be listening to the locally recorded stuff. Anyways, you got a bright story. The internet is on. I may have to, I think we're going to, let's turn off our, let's turn off our video. All right, we're turning off the video, talking into the void. and all right we started off with a bright story a little bit of weather a little bit of internet woes

what else is new for the listener this is a uh if you're wondering why is this episode so chaotic or why is it so much more chaotic than usual it is july second happy Canada day and happy early America day what do you guys say happy 4th of july and we are supposed to release an episode tomorrow. And it's, sir, sir, it's America's 250th anniversary. Show some respect. No, that's okay. That's okay.

I mean, America's going through it. You guys can't even, like, clean your ponds or lakes or whatever mirror pools correctly without wasting a bunch of money. And although I will say, I've heard some very heartwarming stories. I listen to the daily this Sunday, or was it this past Sunday? one of the more recent dailies was talking about the World Cup and different, like, fans from different countries visiting. And apparently, like, there's a city outside of Kansas City, which a friend of the pod, Jared Hobrock, who was on a couple times talking about Thrust and Friends. He'll know this city.

I don't know of it. I only heard about it in the podcast. There's a city called Lawrence outside of Kansas City. I think I've got that right. And apparently it's hosting the country of Algeria. Algeria. And anyway, very heartwarming to see the people of America. Because apparently they got the local college marching band to like welcome them off the

plane and play them Algeria National Anthem. And like all of the locals, whenever they see the Algeria soccer players, they're always cheering for them. And apparently they're like rooting for, obviously they're rooting for USA as well. But apparently they're also rooting for Algeria. Anyways, so, you know, there's some positive stuff about America. So I have some irony. for you. What's that? I was

logging in. You're going to just love this. I'm logging into my Verizon account and it asked me a security question. And the security question is who's your best buddy? And can you guess what the security answer was? It was you, Connor. You were the security answer. I can't

even remember when I would have set this up. That's heartwarming as well. Maybe they'll call this the heartwarming episode. Oh, man. Oh, yes. Anyway, let me finish the preamble, though. So this is all to say, I don't even remember how I started talking about America Day. Oh, right, I was just saying the date.

It's July 2nd. You're listening to this sometime after July 3rd, but this episode is dropping July 3rd. And I only like this morning, because it's been such a chaotic last week and a half. I was in Ireland for a wedding and for a few days off. And then when I got back, my sister was visiting me from Calgary with her two kids, one's three and a half, one's one and a half. and they're a handful. Anyway, so it was like we landed.

We had a couple hours, then they were staying with us. And oh yeah, this was the weekend where you wanted to crash, but I wasn't in town. And then they only just left. And anyways,

and I realized, oh yeah, tomorrow I have to edit a podcast. And then like 10 seconds later was like, wait a second, I don't have any content to edit. Oh, well, I, boy, do I have content for you.

And then I DMed Bryce and was like, hey, And Bryce, both of us are very busy. So the odds that we have like a free slot for 30 minutes are low. So I thought it was going to end up being one of these. Like I talk into the recorder by myself for 10 minutes and recommend a couple podcasts. But Bryce magically was free. That's like 10 minutes out of the way or it says 16 on my audacity.

So anyways, now we're going to talk another 20 minutes about probably something AI related. You know, maybe it's going to be Codex. Although you're on Claude Code. I heard, I saw you got a hoodie on Twitter. I don't know where my hoodie is from OpenA. but I deserve a... Hang out, I gotta try connecting back to the Wi-Fi again because now my cell phone has no service.

Okay, now that's like fine. All right, can you hear me now? Yeah, I can hear you. Okay. Anyways, yeah, that's, you know, messed up. Anyways, you said you've got stories for me, so I'm ready for them. No, I have a topic, which is,

essentially, I think our entire life's work is vindicated. We were right, all of our... critics were wrong. And that is the premise of my story today, that I've, you know, been doing a lot of this agenic stuff for a couple months now, mostly focused on auto research. But the last week or so, I've come upon the question of programming language token economics, specifically like programming language token efficiency. And I'll put my premise out there. The premise is that, If you want to get to, you know, the best performance or even the, like, just correctness or performance, whichever one, if you want to get there in the most token efficient way possible, higher level abstractions, higher level languages, and in particular functional languages, I suspect I'm starting to get early data that indicates that they are the superior option. I think that the ideal programming language for some class of tasks, maybe a large class of tasks,

is probably something that looks a lot more like APL than anything else. And like to APL also, you know, I always make funny, you Connor, that it's so like compact and like symbol heavy, but like that's also very token efficient. So I think, I think that I think you may be justified. And so the actual, the logic behind this argument is twofold. One, a higher level of abstractions means that more of the control of how the program gets lowered is decided by the compiler, the deterministic compiler, which is the cheaper thing, right, in comparison to, you know, spending tokens.

So writing something in, you know, some high-level scripting language or something or, or, you know, something like modern C++, where it's going to be substantially shorter, substantially less code, substantially less time to develop than writing it all yourself in C. That's going to just be more, you know, efficient. And the other piece of this is around guardrails. So there's one, the question of abstraction layer. And there's the second question of guardrails. And these usually go hand in hands. And the premise here is that languages that have more guardrails where you catch errors early or where there's only really one correct way to write things, that they will be better for agents

because agents will spend less time making mistakes and having to learn from those mistakes. So languages like Rust is the premise are better for agentic work. And I think that if all of this turns out to be true, then that just vindigates all of us, all of us who've advocated for abstractions and, you know, generic programming, functional programming, all these things that we talk about that the people of the world have given us a hard time about. Anyways, I wonder what your thoughts are, about the potential that we'll live in a world in the future where everything will be written by agents and it'll all be an APL. Well, my first thought is like that's almost the not opposite of what I've been seeing lately,

but then again, the code that I've been generating has been code for authoring GPU kernels, of which no language like APL or J or K or Q or BQM has a good story for. And I'll find, I think I linked it before in the show notes, but someone once on LinkedIn DM'd me, or maybe it was Twitter, I don't know, some social platform, they messaged me a article that was talking about the most token efficient language is,

and this was like half a year or a year ago. And at the time it was closure, and someone, there was like an update to the article that said a few people have mentioned, did you measure APL? And APL wasn't great because it was Unicode symbols, so it was like multiple, code points in a single graph theme.

I don't understand unit code. Something like that. But then someone mentioned Jay, which is an ASCII-based APL, and Jay actually turned out to be much more token efficient than closure, which was the previous top one. Anyway, so those are my first thoughts. And then I can tell you more about the things I've been seeing. But like my newest thing is that I'm not actually sure.

It depends on what you're trying to do. do. But for the thing that I've been doing, like, kernel authoring, it doesn't matter if you're starting with like pie torch, TensorFlow, Jax, Kupai, you know, or you're doing stuff in Kuta C++ at the thrust level, the cub level. Every single time I've put it into my like, agentic loop research project, it always ends up somehow writing Kudac C++. If it's Kupai, it invokes like Kupai dot rock kernel. If it's jacks. But see, so sometimes, sometimes that's okay at the end. So I did some experiments last week of comparing kernel authoring with high-level abstractions like Kutai or Triton versus like where I run some experiments for this.

There's this GP modes, this community that's very into GPU programming and they run these programming contests. and they had one last week for a dense-batched QR factorization. And so I submitted a bunch of, you know, agentic solutions to it. But then I also did some, like, experiments with that problem where I would run, I ran 12 experiments and, you know, with, where each one ran for like eight hours. And in half of the experiments, I said use higher level abstractions. And in the other half, like right kernels with high-level abstractions like Triton and Kutile, etc. And the other half, I just gave the guidance of write kernels using Kuta C++.

And what I found is that it got a higher speed up per token by the time it was done with the higher level abstractions. And when doing the analysis afterwards, the reason was that with the higher level of abstractions, you were able to focus more, or the agent was able to focus more on different algorithmic approaches, different techniques, et cetera, different numerics, et cetera. Whereas if you write the lower level code from the start, then you're going to be more consumed with all the low-level details. Now, I suspect if you ran this through to completion, if you said, I'm going to let this run forever, that yes, eventually, you know, inevitably coup to C++ would come out ahead.

However, maybe what that tells you is that you should start iterating in the high level of abstraction, and then when you reach a point where you can't get any more performance out of the high level abstraction, that in those critical sections, you drop down to a lower level of abstraction. And that's exactly the message that everybody who's been focused on high-level abstractions, on productivity programming paradigms, has been saying for years, which is that this lets you iterate faster. And now we have, like, concrete, quantitative evidence of that. So, yes, maybe in the end of the day,

everything ends up being written in Kuta C++ or in C++, if you're just doing CPU programming, or that eventually, you know, maybe it's just it always ends up being handwritten assembly. But there's the question of how do you discover the right algorithm to write in handwritten assembly. And the best languages and paradigms for exploring different implementation strategies are the high-level abstractions, the productivity layers.

Yeah. I mean, it's a very, very interesting question. And I think the problems that I've, like the kernels that I've been authoring or having the AI authoring haven't been tricky enough. And so maybe that's why it's able to like drop. you know, down to Kuta C++. But yeah, I've, I have been, what is it the, what is the way to say this? My belief in higher level abstractions and the need for them has been like shaken significantly.

And I guess that's the thing is right now what you're looking at is token efficiency. What I'm looking at right now is not token efficiency. It's just like you've gone past the, I guess, trying to find speed of light to. how to find speed of light using the best token, you know, per... But I will say... I will say something in your defense, which is when I originally started doing this work, I provided the guidance that we give to humans,

which is I said, start by using the libraries. Use the accelerated libraries because the libraries are usually going to have better performance than, you know, then you're going to be able to write. And I still believe that is true. And I still think, like, if your goal is to land a minimal and maintainable change, right, into like an actual real-world

production code base, like a change that uses some optimized library is a lot easier to land and maintain over time than a change that adds 10,000 lines of bespoke written code. So there is a question of priority. If you're trying to land something in an actual production code base, then I think that the guidance of use libraries is, like, use libraries first, and then if they don't give good enough performance, then start to write low-level code, right hand-optimized code, or maybe hand-optimized is the wrong word now.

You know what I mean. But if you're competing in some coding contest where your goal is to write the fastest possible algorithm for something, then I don't know if you should start off with libraries. What you should use is high-level abstractions that are performance-oriented. So things like Kutail, things like modern C++, things where it's still focused on performance, but it gives you high productivity and high expressivity. And then the other thing is, I think the type of library that you might,

need to prioritize is different. That building blocks are far more useful than fully baked solutions. And this is particularly true for kernel authoring, but I think it's also probably true for a wide variety of domains, that having libraries that give you building blocks, composable building blocks that can be pieced together that are high performance or high correctness, whatever the thing you're focusing on prioritizing in your application, is probably more important in this day and age than having something that's just like a very easy drop-in monolithic replacement that does like a particular task. Because there's these agents that can piece together a custom solution for your particular needs. And then piecing together a custom solution is probably the

right thing to do if it can be done in a way that's easy to maintain and and easy to understand. Yeah, this is all predicated on the belief or the assumption that humans are going to need to be in the loop for these systems. No, no, no, no, no, I don't think, I don't think so. I don't think so. One thing I have certainly noticed in my GP mode competitions, the GP mode problems tend to start off with a reference problem that is one line of code. And then over time, you know, the model's making it more and more complex. It's, you know, coming up with special cases for different conditions, et cetera. And it's very quickly you end up with like 9,000 lines of code with split across Python and C++, et cetera.

And the larger the code gets, the more convoluted the code gets, the harder and harder it is for the agent to work on it. That, you know, the, yeah, I'm not going to look at the code. but I still care about how complex the code is, because even if it's not a human that's going to look at it, there is still the question of maintainability, right? Like, that still matters. And this is why people keep talking about this idea of direct to binary, that, oh, in the future, programming languages aren't going to matter

because the models are just going to directly write assembly. Guys, girls, come on, come on here. It lets, nobody's going to ship a software product. where it's the only artifact is purely assembly code in this day and age. I know that it used to happen in the past, and there's special cases, but nobody's going to go and ship a desktop app, you know, that's written entirely in assembly. And the reason is simple is, sure, maybe that's going to work great the first time, but then what happens when you need to add a feature to it, when you need to change something?

It's going to be a nightmare. air. It might pop you out a perfect binary the first time, but as soon as you need to grow and evolve that software, like software evolution, software life cycles will still matter. And all the things that were applicable to software maintenance for humans will be applicable to software maintenance for robots. I mean, maybe. I don't, the confidence with which you say that statement is like, just reminds me of the hubris of humanity. And I think that there's a ton of people out there every time you mention the word intelligent with respect to LLMs, you know, there's a number of comments and messages that I get either on the GitHub discussions that say, you know, these systems aren't intelligent. And I just, like, there's this great podcast episode on the big technology podcast where they had a conversation with Jeff Hinton, the interviewer there, Kantrowitz, I think his name is.

And basically there's this, like Hinton mentions at a certain point in the podcast that there's like, like these times in history when like the ego of man is challenged at one point. Like I'm not a student of history, so I'm going to get this stuff wrong. But like at one point we didn't think or we thought that Earth was the center of the planetary system. And then Galileo or some other guy came along and said, no, it's the sun. But it took like 100 years. It took like 10 decades for people to accept that.

And it was like considered heresy. And there was a couple other examples when. like this, this kind of like, either like theocracy was challenged or man was challenged. And it took like, it took years for society to accept that like, oh, maybe, maybe this is the case. And like, I completely agree with Hinton that we're at one of these like points in history where, where people think that like, oh, you know, it doesn't think like us. It's just token generation. And I just like, whether, whether this, you know, whether LLMs are the incarnation of like beginning of like a. new, sentient, like, I don't want to say species, because people are going to freak out,

but just like, whether it's LLMs or it's going to be like the sixth, you know, evolution of LLMs that will be called something else because it's not going to be like the transformer that is the basis of this technology. Anyways, I just like, when you say that we're not going to produce an artifact, I agree. I am, I have an ego and it makes me think that, like, of course we want something that is understandable by humans, but like, why does it need to be? What if we just have like a... I'm explicitly saying it doesn't have to be understanding.

You just said guys, girls, why would you ship an artifact? Like, come on. You said guys, girls, come on. No, no, no. But my point is that nobody's going to ship software that is very difficult and very expensive to maintain. And if you produce... Who says it's difficult to maintain if we're not maintaining it? What if we, what if like, you know, some company comes along builds basically like an amorphous, like entity that, like that is the product, you know?

It monitors crime or something. I think that directly, directly writing and modifying today's executable binary formats is not the, the, is never going to be the right way to maintain durable software artifacts. Like what precludes it for being, you know, I was just. talking with Michael about this. Like, why does it why? Because executable binary formats are just simply not designed for this. Like, what do you design? Design, design for what? Like, for one,

if you need to, if you need to change like one aspect of a binary, it might require you to move around a significant amount of data elsewhere in the binary to like, like if we're talking about actual like direct to like an elf binary, like if you're adding a new function somewhere,

that might require you to rebuild like a whole, you know, symbol table or something or like redo all of the relocations. So you can make a tiny change to a binary that actually requires like a, or a tiny functional change that requires a huge, massive, actual change to the binary code. And so I can't imagine that that will be the level that people will operate at. Will there be more high level, will there be more programs expressed in terms of high level, IRs, like MLIRs, probably. I could buy that. But I don't think that we're going to be free from the deterministic software piece parts of it. And in fact, in fact, the best uses of AI, I think are these combinations of deterministic software, traditional software, and

intelligent software. And you mentioned a lot of listeners question whether these things have intelligence. I do not think that intelligence is the important attribute of these LLM systems. I think there's three key elements that make them uniquely capable pieces of software. The first is the self-healing nature that, or maybe they're put it in another way. The first is that they're flexible, that they are by their nature flexible to the environment that they're in and the instructions that they've been given. And this flexibility comes in part from the non-determinism of these models. The second is that they're self-healing. So if something goes wrong, they can assemble a way out of it, not always successfully, but they can assemble a way out of it beyond the circumstances that was imagined

when the software was designed. And then the third principle is that they're autonomous, that they're not only capable of self-healing, of flexibility, but they do have some decision-making mechanism that can allow them to act without needing a human's input. And I think the question of whether they're truly intelligent, whether they're truly thinking or not, that I don't think is as important as the question of, like,

what can the technology do? and I think those are the three properties that make them very, very useful. And part of the nature of the current AI technology that we have and probably all future AI technology is that it is non-deterministic. It's not always reproducible. But that's okay because you compare it with reproducible, deterministic traditional software. You know, like I had a skill to, like, export and backup all the data on, VMs that I'm using. And I could just write the skill, but if I do that, then I'm not,

the backup's not going to always be in the same format necessarily. It's maybe going to have slightly different names every time, et cetera. I could just write a script to do the backup. But if I just write the script, then, you know, the script could miss something. Or what if the script, I'm running the script somewhere and what if there's a bug in the script? Or what if, you know, I'm out of disk space or it can't reach, you know, the place that I'm doing the backup. to, then the script just fails. So the thing that's most useful, the thing that I ended up building, was a combination of traditional software and instant software, the AI, just prompting, etc. And that's to have, you know, I have some core script that I use for the export, but then I have

a skill that drives that script and that does a scan of its own and looks for anything that was missing and has a way to add, you know, any missing files to that backup if needed. And so I think those are the things that are most powerful today. And maybe in the future, it will be that we truly won't need any deterministic software. But I'm just, I'm not, I find that hard to believe because, one, the economics of deterministic software are just so much better than of everything being AI, at least today. Token costs would have to go down 10,000 X for that not to be the case. And two, there's something to be said for reproducibility. I agree with a lot of what you just said.

I think future systems are going to boil down to deterministic scripts that are written by non-deterministic scripts and then like the non-deterministic LLMs. Or I shouldn't say non-deterministic scripts. Like you're going to have your deterministic computing, which is just, you know, Python scripts, choose your language, whatever. But then you're going to have your, you know, clod skills, non-deterministic things that are LLM functions. And the more you can push stuff into deterministic. scripts the better. That's what linting is. That's what evals are. You know, you know, you build loops around that stuff, fantastic. But I just, when I think about, like, what an LLM is, you know, it's ultimately like, you know, with a bunch of pre-training, post-training, all this fancy bells and whistles,

but it started out as just like a transformer, you know, and token generation. What is to say that, like, right now, we're building these LLMs that can generate Kudacet++ code and Python code, etc, etc. and there's linting and there's static analysis and we're building more and more guardrails which is a part of the reason that all of this stuff is getting so good if you play around with Claude Opus 4.8 you can look in the traces and they're all constantly if you're in Python

they're running rough and they're fixing things that it goes like there's so much so much of the stuff that like if you were using these models a year ago before the big jump back in December of 2024 you had to manually like you know add that stuff to your own loop in order to get good results.

That's all baked in now. And my point being is, what is, what is preventing these models other than the bias of humans to like be, like, you know,

the rule of six that, you know, our brains are capable of storing like, you know, six or seven plus or minus concepts in our head at the time, which is why functional programming

and array programming, you can get further with it. You know, what's to say that like, that's not a limit. Like, you know, sure, LLMs have context windows

and they have their own limitations but like what's to say that you couldn't take, you know, assembly code or P-Tex or SaaS and like train a bunch of really large language models build the same kind of linting tools which doesn't even make sense for us because like we program in programming languages, not in ones and zeros. But I'm just like I'm not convinced at all. Like I'm very skeptical. So I'm on actually your side.

Like I don't think that the future systems are going to be generating binary or executables. But that being said, I'm not like, I also thought the abstractions were going to be super important. And like I've seen now in some tests that I've run that like they, these models just, if you ask it for speed, it finds a way to basically invoke Kudu C++ at the end of the day, whether it's like FFI through jacks and a shared object file or like embedded like Kudac++ in a string in a Koo Pi like rock kernel or like anyways, it always finds a way to do like the fastest thing, which is in Kudu Kutu K++. and like it just discards all of the abstraction. Anyway, so if you take that to an extreme, you know, where do you end up? Do you end up in K2C++? Do you end up in P-Tex? Do you end up like in ones and zeros?

I don't know. But I would argue the, it depends on what you mean by fastest. A higher level representation that's tunable, something like a kutile kernel, is going to be faster across the world of every possible. input that you could test. It's going to be a faster way to generate the correct, to Jit compile the correct algorithm for the particular input in conditions that you're currently running in. A lot of the problems, you know, there's a lot of talk about reward hacking, and I think you and I have talked about that in the past and how a lot of our efforts should be focused on

reward hacking. But the other challenge with a lot of this like auto-optimization work is you can only benchmark for so many inputs. One thing I've been working on has been optimizing, you know, Cubs histogram. And I did a bunch of auto research with, you know, Cubs histogram benchmarks, which runs for maybe 10 minutes. If you want to test a representative set of inputs for histogram across a representative number of bin counts, number of element counts, data types, different input structures, you know, different like, like, you know, not just completely random, but different like structured input, you know, because that's what you'd see in the real world. It takes hours. It takes hours and hours and hours. You can't have an iterative loop that is

that is writing truly generic performance portable code with this like tight, keep, commit, keep and revert process. You, I think a lot of what we're seeing right now with people doing this, you know, loop for optimization is writing really good code for a very specific circumstance. The process of taking that and generalizing it to something that can be deployed in production is a different story. It's still an agentic process. It's just, it's not, you know, these type iterative things.

loops. You use those to explore and find techniques, and then you have to do some follow-on process to actually develop the algorithms that you're going to ship in production. And you have to deal with the trade-offs of accuracy, of, you know, code complexity, of the size of the binary that you're shipping, et cetera. Yeah. I mean, like I said, at the end of the day, I'm on your side. Like, I do think, I would be very, well, I would be surprised to a certain extent if, like, the best AI systems in the future are generating, you know, executables and binary. That being said, I don't know, I've, my wife has started to say that, like, I've, I've gone a little bit existential over the last, like, a couple weeks because the latest models,

they're just so good. And yes. And I've, I've started having, like, post-AGI thoughts. And I, at some point in this conversation, I was trying to think, in the trailer of a bad sci-fi movie in the last decade, it showed these like what looked like advanced servers and these like, they looked like they were self-healing. And I can't remember if they represent the like computation that runs society or something like that. At first I thought it was Jupiter ascending. But thanks to AI, I asked, it's like the instant free tier of Chad GPT.

And sure enough, it nailed it. Well, it actually gave me like seven different options, but I recognized it as the last one. It's Valerian and the city of a thousand planets, which I'm pretty sure is based on a book. I recall this movie being terrible, but I'll, I'll share my screen and I'll find a way to tweet this out or something. Let me know if you can see this. Yeah. And so if we play this, it's only like a two second clip, but this scene right here, you know, we go back, you know, one second. This is what I think. this is like this is what I have started to think like where we're headed this is like computing in the future these are all like algorithms that are running

and these little guys that are like soldering stuff that's like that's like the self-healing it so when you say like oh we need to be able to inspect I don't think so I think we're going to have you know algorithms no no I'm not saying that I'm not saying that humans need to be able to inspect I'm saying that other agents need to be able to inspect yeah but how do you know that the language that they are most like efficient in is the same language that humans like evolved to understand. I don't, but I, but, but the current, the current technology that we use for AI are large language models that are primarily trained on human language text.

There's a lot of interesting little fallouts of this. For example, I think almost everybody who's serious about auto research is running auto research in infinite loops. And one of the reasons you do this is that if you tell it to run for 24 hours, it's perception of how long it takes to write code is based on how long it takes humans to write code. And so it is just wildly inaccurate. It wildly overestimates how long it's going to do something. It'll be like, oh, here's a plan for how we can do something that's going to take two months. And you're like, no, we're going to do it tonight. And like, yes, maybe there are,

languages or intermediate representations that'll be better suited for these LOM technologies and the ones we currently use. I could certainly buy that. However, they are primarily trained on data that humans that we created, and they're primarily designed to interface with us humans. So I, just in the same way that like probably the most useful general purpose robot will be a humanoid form robot, simply not because that's the most efficient form, but simply because that's the world that we live in. We live in a world designed for humanoid-shaped things with machines and equipment that is humanoid-shaped thing, that's humanoid shape. Yeah, I do think that programming languages and programming models will evolve, you know, very drastically over the next few years.

But I don't see a fundamental paradigm shift away from the basic tenets of like software evolution and life cycles. Like I just, I find it hard to believe that we're going to be shipping elf binaries as the only software artifact. Maybe we'll come up with new binary representations or new types of computers or entirely new models. I could see that happening. But I think a lot of the things that make a computer system good for AI that make it agent-ready also tend to be the things that make it good for humans. Like, you know, having all your APIs, like, well-documented and, like, markdown,

like, that's good for agents. It's also pretty good for humans, too. You know, like, if you're a human, you can go read a skill file. You know, I don't see a huge divergence yet in, like, away from, away from, I don't know, software maintainability. Yeah. What are we going to call this episode? And it's so sad, too, because I actually think this is like, this is like the most interesting

conversation I've had in a while. And I feel like half of the people listening to us are just being like, these guys are out to lunch. Like, what drugs did they do before? That's okay. It's okay. I mean, we are very much at the epicenter. of this and we're to some degree like in an echo chamber, you know, maybe we'll be wrong.

And not everybody has to agree with us and not everybody has to like it. However, I do think if you're a software engineer, you do have to be aware of what's going on. Because this is a big change to the way that your industry works. And if you're not at least aware, it's going to be very tough for you to remain relevant in this field. And so I think you got to stay abreast of what's happening. And this is one of the best places to do that, this podcast. Yeah. And I mean, yeah, I mean, some people think we're a little kooky. I mean, you know, but it's also like you think about, I think about my mindset on AI a year ago versus two years ago versus three years ago. And I was pretty skeptical too. And I see a lot of people

who are skeptical and I think they just really haven't. In some cases, it's not that they haven't tried it. It's just that sometimes it takes a little while to pick up and to really drink the Kool-Aid. And not everybody wants to. Not everybody wants to. That's the other piece of it. Not everybody finds it as enjoyable as we do. Yeah, I feel like drinking the Kool-Aid is the wrong aphorism for that, though, because that alludes to, you know, people that ended up dying. So probably people would find that. that they would be like, yeah, it is like drinking the Kool-Aid, but I feel like this is like, it's like Excel coming out as an accountant

except that times like 100. And anyways, we got to stop. I would love to continue to chat, but I have to edit this. Be sure to check these show notes, either in your podcast app or at ADSP thepodcast.com for links to anything we mentioned in today's episode, as well as a link to a get-up discussion where you can leave thoughts, comments, and questions. Thanks for listening. We hope you enjoyed and have a great day.

Low quality, high quantity. That is the tagline of our podcast. It's not the tagline. Our tagline is chaos with sprinkles of information.