Programming Throwdown - 171: Compilers and Interpreters

Episode Date: February 12, 2024

Intro topic: Monitor setupsNews/Links:BlueScuti, Willis, beats Tetrishttps://www.youtube.com/watch?v=GuJ5UuknsHUPalWorld accused of being an AI Producthttps://www.forbes.com/sites/paultassi/2024/01/22/palworld-accused-of-using-genai-with-no-evidence-so-far/?sh=26a9651b42394 Billion if-statements to determine if a number is even or oddhttps://andreasjhkarlsson.github.io/jekyll/update/2023/12/27/4-billion-if-statements.htmlSeamless M4Thttps://ai.meta.com/blog/seamless-m4t/Book of the ShowPatrick:Foundation by Isaac Asimovhttps://amzn.to/3SrmgnPJason: Propaganda by Edward Bernayshttps://amzn.to/47JUCXJPatreon Plug https://www.patreon.com/programmingthrowdown?ty=hTool of the ShowPatrick: The Room Gamehttps://www.fireproofgames.com/games/the-roomJason:Incredibuildhttps://www.incredibuild.com/Topic: Compilers and Interpreters (Request by Jessica W.)Machine CodeArchitecture SpecificAssemblySingle vs Two Pass CompilerHigh level LanguagesIntermediate RepresentationJVM ByteCode vs Machine Code for portabilityScripting/InterpretersJITProfile Guided OptimizationResourceshttps://www.craftinginterpreters.com/https://nandgame.com/Turing Complete ★ Support this podcast on Patreon ★

programming throwdown episode 171 compilers and interpreters take Take it away, Jason. Hey, everybody. This is super exciting. We got this request from a listener. Maybe later on the show, I'll find out, remind myself who it was. But they said, how about you folks do compilers, interpreters? I can't believe we haven't done this episode. It seems so pivotal to everything we do in software engineering. But yeah, here we are to lead the way.

I have to confess, I don't know as much about Compiler's Interpreter as Patrick. So Patrick's really going to carry the water here. But I will add color commentary as appropriate um and before we get into that I wanted to talk about monitor setups for a little bit so I I was working next to somebody who had three monitors like side by side by side and they had three different things going on you know like they had one was for coding the other one was for email and i thought this was really compelling i thought oh this is a great idea and uh i just can't get myself to do it basically i just have to have just one thing

in front of me and uh and and just uh keep alt tabbing to go to different things. I haven't figured out how to drag from one desk screen to the other and do all of that. What is your setup? Do you have an array of monitors, or what do you do? Yeah, eight? Seriously? I do all my programming in VR, so I just have as many as i actually do look for it i think that'll be cool one day one day um that day is not today um i just have one big one uh but i

also have a ipad off to the side that i use for for my conferencing uh for video conferencing um that works well for me so i guess it's two, but one big one for sort of similar reasons to what you're saying, like that. Normally, if you have two monitors, you just get them extra wide or on top of each other, which is too much versus one big monitor, I feel you don't end up with that awkward split in the middle or have to shift one far to the side. Right? I will say for people who do like simulation race games or flight games, having three for a gaming setup would be really cool because you're one thing over three and the people who do of course the you know stock trading where they're actually just it's mostly focused on one task right like they're all of it is related there's just lots of you know dials and metrics it's akin to i think like a cockpit in an airplane

where there's many many indicators some are lower priority but they're kind of very related to a singular task to me it seems a little bit different like when you're doing your programming email is maybe that's not fair I don't know then I've never been those other things but it feels like a different task slack or email they feel different to the task of coding and so yeah I'm with you like i don't feel compelled to have them up on the side yeah you know i used to do um i used to have a my laptop so like my laptop right now is just mirroring the screen of of the main main display i used to have it kind of like what you're saying or sort of secondary screen but um i found myself not

paying attention in meetings even when i should be and uh um and i just i kind of couldn't do that i guess well i mean isn't this like a trope that people can't actually multitask we just sequentially singular task yeah yeah i mean i i just uh um but yeah i mean i do envy the people who can have like three different screens on and not be distracted i will say i i do find i it's not all bad i think if you occasionally need like a paper or document or you're implementing an algorithm and you want to consult something or the few times i've had the kind of i'm generating some kind of output or whatever that's visual um it's very cool to have a extra real estate where when i update something and click you know test i can sort of

still see the code but see the results and you know up inside you know screen real estate can be very very useful so i'll never sort of say something bad against someone who who wants to have more screens but the variety is yeah it's it is very interesting yeah i wonder um you know now that i'm working from home it's less of an issue but when i was going into the office i was running into problems where you know my home monitor and my office monitor aren't the same and so you know it wouldn't quite extend the desktop the right way and things would get all messed up that's the thing i've been picky about is i have my monitor pretty high i'm i'm fairly tall so um i want you know my monitor nice and high so i either put little plastic risers or put it on a monitor arm and lift it up so i have it higher than i think what i see a lot of other people do just because

it feels unnatural to me all right we have to have a we have to have a bake off here my monitor is on a i'm trying to eyeball it maybe like an eight inch riser what about you oh i'm trying from the bottom of my screen i'm like at a like a foot like oh wow okay that's even bigger i might be yeah a little bit more than eight inches but yeah that is pretty tall so like the when you look at your monitor at eye level are you looking at the center of it i wish it's not quite that high the monitor i'm going to go quite the high so it's uh yeah i'm looking at like just above something like 60 up from the bottom yeah okay it makes sense yeah i'm about maybe like 75 or so up from the bottom but yeah if i didn't have any riser the monitor like my eye level would be

above the top of the monitor it's just but i see people do that all the time i don't i i can't yeah it's like um i think your neck would get hurt i think if you do if you keep it like that what i want to try is one of the very wide curved monitors i think that would be really cool yeah you know i've i've tried it i i have one um um at work and it's it's pretty good it's pretty good i will will say that the one I have, the refresh rate isn't as good. It's not, it's not like,

you know, a deal breaker or anything, but you do like, if you have one monitor, it has twice the refresh rate of the other. It can kind of like, it's like when you're at the store and all the differences between the TVs are so noticeable

but you take it home you totally lose your frame of reference right so if you have like a four to three aspect ratio or whatever like not a 1080p widescreen 16 to 9 but like if it's curved left and right does it also curve up and down if you get a nice tall one like or i don't is like on a sphere i've never looked it up i guess i should the one i have i'm looking at it right now um it's it's curved but just you know it's just curved like like a cylinder like a cylinder okay yeah and but it's very wide and not very tall so the aspect ratio is not it's not hd aspect ratio it's much wider oh okay i think i would want one that yeah i think i just want like a very large like probably 8k but like you know big but then i wouldn't actually run it you know i would everything would be scaled up just so it's a comfortable eyesight thing i think 8k would be

too small at like minimum resolution or whatever yeah right um but yeah folks out there let us know your monitor setup i'm curious i do feel like i could do better um and uh yeah curious to know what what people's setups are out there just chat us email us post in discord um discord is is getting more popular so people posting some funny jokes there was this video of this professor pranking his class which i thought was hilarious so thanks wait wait what a professor ranking to their class okay i had to go on discord but pranking their class oh pranking i just said ranking i was like yes i was like oh yes we all know that they do that but they're not supposed to actually do it

rate my students.com new website all right well we'll jump over to the news of the show starting off since last time uh i mentioned this and then jason was excited about it as well so so i feel we'll talk about it here uh i the headlines everyone's probably seen them but but i feel like it's pretty cool for for something that is a memorable part of my childhood which is being absolutely crushed and being terrible at the game of tetris um but but now uh there has been someone who has beaten tetris actually as of recording this show i think two people have done it uh so the world record didn't last very long after lasting for years and years um but blue scutie i think the first name is is willis uh beat tetris first not at the earliest possible time which was accomplished now but uh did tetris first not at the earliest possible time

which was accomplished now but i did beat it and here actually the the reason why i think it it's worth mentioning on the show aside from just being a cool cool thing and i linked the the video there that is the first place i saw it um but it's interesting how they define beat here so when you know tetris was originally built they didn't actually have an end game in mind, which is kind of unusual. It would just play faster and faster. And I guess the assumption was just, it was so fast, you couldn't possibly keep going.

And so you eventually would die. A lot of games actually of that era are like that. Like Donkey Kong, actually, in Donkey Kong, the amount of time you have to finish the level goes down every time you have completed a full revolution of all the levels. And basically you get to a point where there just literally isn't enough time to finish the first level, even if you're frame perfect. And that's just how it ends. So Tetris, I guess, gets faster and faster. And then at some point it basically kind of caps out like it it just doesn't go any faster um and for a long time uh people would get to that area

that that speed and then just sort of you know die pretty quickly if their their row count was low they would be able to you know you know manage to maybe get a few more lines cleared and then die. But then recently, people discovered a new way of pounding, tapping the, you know, controller faster and faster with this crazy two handed technique. And they've been able to go faster and faster and began to discover, which I guess people had figured out via, you know, tool assisted speed running. So they had kind of known this would happen. But humans began to be able to get there, which is levels where, you know, incrementing the color palette left the predefined color palette and would start using actually machine instructions as the color selection, which would result in just really ill formed colors that made it even more difficult. But then eventually, a portion of memory would be accessed that would just cause basically uh you know a

corruption of of the data and so when that happens the game the game just crashes and stops and just hangs um and so this is what uh blue scooty managed to do was to to basically cause the game to try to execute code that wasn't code uh and then hung the game and so first person to do that it's just it's crazy it's very fascinating it's also and it's the same you know as jason was mentioning sort of frame perfect analysis for uh super mario brothers is some any sort of speed running game that is from around that era the amount of analysis and reverse engineering that goes into it and there are even patched versions of the game so that like it doesn't get weird and you can keep playing or you know just other things where people really really understand the mechanisms that uh you know

take place at this end game and the amount of analysis i'm sure the programmers at the time had no idea that you know for 30 years 40 years people would be uh sort of thinking about uh what that code that they wrote in the back of a cubicle you know somewhere and and turned out yeah it's totally wild right i think uh yeah yeah the uh yeah that that whole thing where they they do this crazy i mean you really have to see it almost in slow motion this way with two hands so the idea is every frame you know in tetris there's no inertia or momentum or anything like that with the with the pieces so you know every frame if you could get like a press left and a stop pressing left in those two frames then you could move it over um one whole column and uh and so yeah they do this weird thing where i guess they they hit hit the joystick from the bottom. Um, so in other words, like if you were imagine like just pressing with your thumb on a button, like you had another thumb kind of on the bottom of the

controller? And so one thumb is pressing and releasing, but then the other thumb is pressing the bottom into the first thumb. And like, I guess by doing that, you can get more presses than you could with just one, one thumb. It's, it's totally crazy. You have to see it i mean i guess tetris uh lends itself to that because you i i don't know i we we'd have to learn more but i guess you only have to move either left or right as long as you don't overshoot and you only have to rotate the piece three times in a in a direction and then you would you know just be back so you can only rotate at 90 degrees so there's like a finite number of presses if you knew exactly what you were doing your path planning uh you could be very minimal inputs

yep yep totally um all right on to pal world accused of being an ai product um so yeah so pal world is wild patrick and i were talking about it before the show. It really like is a great area, a great, you know, theme for a game. Basically it's adult theme, well, not adult theme, that means something totally different. Pokemon with guns. It's Pokemon with guns.

And they're just, I think they're also just really aggressive, you know, looking, you know, you could actually catch people in the Pokeballs and sell them. So it's basically crazy, much more like Mad Max. It's Mad Max meets Pokemon. It's a great idea of a theme. This article is really interesting patrick actually found this

um and it's it's it's claiming that a lot of the assets are ai generated um but they have no hard evidence and so i guess you know there's really like two things that i took away from this you know one is that you know the generative AI is really, you know, here to stay. It's not going anywhere. You know, I mean, the days of downloading like 30 gigabytes of MP3s when you buy a game, I feel like those days are very short lived. I think even if you did hire a voice actor, you know know you almost certainly want to do some type of capture of their voice and and if nothing else just to reduce the storage costs and you know generate their voice dynamically um so that's definitely here and then the other thing is

you know it's amazing that they they don't know because you know i've seen a lot of papers and other um you know reports where you can see how something is is generated by an ai um but but i guess you know they have a point that it's really hard to prove it um you know i i think um the examples I saw were interesting. Like, for example, if you see a, and this isn't a pal world thing, but if you see a picture of a face and the earrings don't match, you know, if it's a woman's face or man's face, anyone's face and the earrings, you know, they're wearing earrings and they don't match. That's a sign that it's generative AI because it's hard for the AI to, um, to, to get that symmetry. Right.

Um, but, um, but, but, you know, they haven't really found anything like this here. And it might be just that the person is,

you know, starting with, uh, an AI model and then just as inspiration, but then putting a human touch on it and it makes it very hard to to track so okay a couple things uh if you're going to commit a crime put on mismatched earrings and like do all the tips that are generative so if someone captures you on the camera you can be like that's clearly not me because it's like clearly an ai so i i sense a trend where people people do this uh second second a hat tip here to ace rola uh is role does a youtube video on shaders actually uh anyways and on twitter oh x is where

where i sort of first saw this uh controversy rising up um but if you've never checked out checked out uh him check it out ace rola um and then yeah i i think jason's right he's like wait first of all does it matter like if it's generative ai or not and the answer is maybe and it's sort of you know gets into one of those gray areas if it's fun it's fun but at the same time there are people trying to put work into the craft right it goes back to that debate which you know i don't think there's there is an singular answer to but you know the difference between ikea furniture and you know i heard anderson real wood you know piece of furniture they're both furniture but you know you could kind of talk at length about you know various pros or cons of each and so i think in the end you know as i don't know that even they

have to be forthright or honest about what they did they just you know probably shouldn't go around claiming that it's high art or you know that that you know they somehow that there's you know tons of people behind it right if you tried to say hey you should support this game because this took a lot of work and it didn't then you know there's like a sort of ethics problem but as far as you know as long as if i think right now they're not saying anything they're just just not talking uh it's you know been a kind of overnight sensation then i i personally yeah it's kind of interesting but i don't know that i have a specific problem with it. Yeah, I think as a coder, it's hard to really relate, right?

Like all of our code that we've done on open source, you know, like in my case like Eternal Terminal and MameHub and all these things, they're all public repos and so, you know, AI is scanning those and using them to build GitHub Copilot and all these things, right? So in a way, they're all public repos and so you know ai is scanning those and using them to build github

copilot and all these things right so in a way they're kind of like copying it from me but it's okay because i copied it from stack overflow anyway um so uh uh but but but you know artists don't don't feel the same way. But that's not true. But that's not true. But if I'm not, I'm not an artist, so I can't speak authoritatively. But it's very common when people are learning to take inspiration, to practice copying and to understand what it takes, you know, to make the paintings of paintings. And it's not good to copy too much, but to be derivative.

Or you can even see sort of the lineage of how artists build on other artists concepts and extend them and sort of push them forward and so there's this very fine line between ripping off you know uh pokemon characters although if you look other places like digimon there's a lot of you know digimon monsters that look very similar and the same kind of brouhaha didn't get that real maybe it did I was probably too young um to remember but uh you know it I don't know there's a lot of music it's the same way right everyone's sort of taking riffs and ideas and from each other and it's you know there's been court cases recently where it's I don't know the answer it sure sounds

really similar but then again you know it, it's a nice sounding thing. If you just sit there and play music until you find something nice, you may play something that someone else has already figured out before. So, yeah. Yeah. I mean, this is maybe, I'm trying to think if there's other examples before I say this, but I don't really think so. I mean, let's say this is all ai generated somebody basically threw all the pokemon characters

over of all time into some ai model and now they're able to spit out more pokemon characters um you know this is really the first time where we've seen like some huge commercial success come out of some generative AI thing. Like, you know, like people are so worried about, you know,

fake, fake things like, like you're, you're trying to impersonate a, you know, a politician or something like that. Um, but this is,

you know, not really that, I mean, you know, in terms of like the, the incentive is not to impersonate Pokemon. The incentive is actually the opposite it's to generate something that has plausible deniability that it came from pokemon i mean yeah how many go on the app store how many flappy bird but with a you know

oh so true right like and you but to jason's point those don't experience the sort of commercial success the numbers i've seen are absolutely crazy for how many people have bought this guy i haven't bought it yet uh i'm tempted because it actually does look hilarious uh but uh yeah my neighbors are actually going to going to play it so i'm feeling kind of compelled to buy it all right well hard pivoting off of this one i found a blog article here which is something that's meant as sort of i don't even know how to call it not a meme and like a picture with an image macro or whatever pasted over top of it but just a sort of meme like going around which is kind of absolutely silly ways to solve determining whether a number is even or odd and so i linked the link the blob here the title of the blog it says the

blog's name is blabbing but this is the only post i saw on on this blog so uh i think a new blog but anyways the link is uh will be in the show notes um but this person was saying they saw on tiktok someone get kind of ripped which again i'm not on tiktok so i i can't this third hand at this point but uh getting ripped for saying that they determined if a number is even or odd by basically a bunch of if else statements so if number is one odd if number is two that's even if number is three that's odd i'm gonna get messed it up already. Most people would say, Oh, do you just use a modulus operator, if you know what a modulus operator is, or, oh, you use, you know, bitwise, and with the, you know, first least significant bit just being one, and then that'll tell you, right, there's like a number of ways to kind of

do this check. And most people would be expected as like a test of your language knowledge to know what the modulus operator is at minimum or bit masking, depending on the context. But this person decided to in the blog decided to take it on themselves to for 32 bit number, which is 4 billion different numbers, right if statements and the interesting part and the reason I bring it up is first just the absurdity of it. They decided to write it in C with what I sort of sounds like some tongue in cheek justification initially. But what they did, which is an actual interesting thing to learn to do and, you know, on simple program far simpler than a four billion line program to generate all of the if statements automatically um and that way they you know could produce that output and what does that take which is a useful skill to be able to know how to do because there are times when uh doing not that but other code generation writing code to make code is very useful that didn't work the compiler c compiler didn't like

that many lines so then it decided to output raw assembly and actually did get it to work and i guess said it was very performant if you think about it i it's almost just a big lookup table and you can just there are jump tables so you can just basically jump to a set offset so if you know each if else statement is two bytes and you had a very clever assembler or whatever you can just basically jump to a set offset. So if you know each if else statement is two bytes and you had a very clever assembler or whatever, you could just jump, you know, based on the number that many,

you know, memory offsets times two or whatever. So I could imagine it being very, very efficient. And then just having a table where it either outputs zero or one. But yeah, so they actually did manage to make assembly that they say works. Of course it's a, I guess we could figure out four gigabyte file roughly um so it's like genetic

executable um 40 gigabyte ridiculous in that like there this is of course the i don't want to say this is a very silly way to solve whether a number is even or odd um but actually leads to some some kind of interesting uh computer science uh exercises and doing this and then of course people responded can you do it for 64-bit numbers and the answer is no yeah i love how this uh this this is the only post in this person's blog that's okay you gotta post something up when you have a cool idea and you want to show people like i i don't fault them no yeah no i mean i i didn't mean that sarcastic me i literally love that this person you know it's not like 10 posts about like like hire me or like self-aggrandizing or any of that just literally like here's this cool thing that i did boom blog i will say though that's like a tough act to follow. So I don't know what you make post number

two. That's true. Yeah. I mean, you really can't just start a new blog. It's been a month now and he hasn't made a post. Um, that is awesome. Um, all right. My next news is Seamless M4T. Have you seen this, Patrick? No, uh-uh. This thing is freaking wild. So this is what I use to translate the episode. So if you haven't done this yet, go to programmingthrowdown.com. Depending on a bunch of factors,

the very latest episode might not have this. Just go back an episode. And you can listen to us in mandarin chinese so it's it's totally freaking wild so basically the way this works is it um it does the uh speech to text now like you have to have a separate file for each person you have to have a way of separating or in our case you know we have um you know we have a separate mp3 for each person on the show um but you know it converts that to text and then it translates the text to other languages and then it goes from that language's text to, you know, one of several voices that they have for each language. And so the reason why they're calling it seamless is I don't think that it's end-to-end learned, but it might be.

But, you know, it is separate sort of modules. But the API and all of that is extremely simple. I mean, you basically just say, here's audio, and I want to get back text, or I want to robot us. Um, that was really entertaining. Um, and then, uh, I started picking other languages. So, um, this is amazing. I mean, it's amazing that it's free. Um, it's totally open source. Anyone can use it. Um, I actually bought for black Friday a few months ago, a 16, uh, GPU at 16 gigs of VRAM, that's the bare minimum you need to run this model. But if you don't have that, it's totally fine. I also got this to run on Google Cloud Platform,

and you pay about 50 cents an hour, I think, to rent one of those. Yeah, very reasonable. You can even now now they have something where if it's idle for so many hours, it shuts itself off. So you don't have to worry about forgetting about it and getting a $500 bill or something at the end of the month.

So yeah, all of that is super convenient. So you can run it on Google Cloud. There's also Google Colab. You can run it on google cloud there's also google colab um you can run it there there's some 16 uh gig vram gpus there i don't know if the free plan would work but um the paid plan is only 10 bucks a month um so yeah check this out i mean it's amazing i will say that the there's two modes there's a batch mode and a streaming mode the batch mode loads everything into memory so you know you can really only use it for like a 10 second file or something like that um you know the streaming mode is really where it's at um the documentation is still pretty nascent. What I found really useful was they gave a tutorial at NeurIPS,

which is an AI conference. They did a whole tutorial on Seamless M4T, and you can watch the video and follow along as if you were at the conference for free. I was able to find that. So check it out. Check out this tutorial. I feel like this is really on the vanguard

of something super exciting. I'm looking into, you know, can we clone our voices? Because if we could actually, if it could be Mandarin Chinese, but it sounds like us, that would be even better. It's like a, deep boy your earrings match it's gonna be like the deep fake jason but yeah speaking mandarin yeah so if uh in a few months you might hear patrick and i in mandarin maybe we'll swap out an episode just for fun and see if people notice i hope people

notice uh no no an old an old episode oh i see i see i see um you know i people focus like the machine learning stuff on the you know open ai and and i understand why um and the the sort of approach to agi i guess like you general intelligence. But I you're right, there's a lot of stuff happening, like out in the call it periphery that feels like we're really moving forward, especially once we start combining some of these things. So there's been a long work separating back speakers, people say they can't tell the difference between us. So I don't know, maybe that'll have to be like gen two of that, but you know, having a single file, separating it back and then, you know, running it through the system and combining it. But, but also, you know, we've

not, I don't know that we've talked about it, but the sort of, um, nerfs and Gaussian splatting around like making 3d models from, you know, pictures. And right now they need kind of a lot of pictures of a specific kind, but you know, that'll get progress as well and then you can imagine the the sort of stable diffusion feeding into those things like there's like a lot of interesting sort of like one two iteration away um and and you know probably requiring some leaps there because it's not sort of like gluing normal software together but uh yeah there's a lot of exciting stuff happening that I think gets missed for the focus on LLMs. Yeah, I actually am more excited about diffusion models and LLMs.

We should do it. We should do a whole show on diffusion models. All right. Well, we know a person who probably speak to that. It's not me. Yeah, I was wondering if we knew a third person, but, but yeah,

you know, have you seen the diffusion model with bomber man? No. Yeah. So basically, okay, let me just do a really quick and then we'll, we'll dedicate a show to this.

The way diffusion models work is you have a bunch of references. So in the case of Mario or bomberberman or any of these games, you have levels that were created by humans, right? That's your reference. And you want to create more of those, right? So what you do is you corrupt the references, in this case, the Bomberman levels. You corrupt them, but the way you corrupt them references in this case the bomberman levels you corrupt them but you're

the way you corrupt them is reversible so whatever you do you know the way the exact way to undo it and then you train a model to reverse the corruption all right and then once you have that model trained you give that model like random noise like levels that are complete garbage and you just tell it to reverse the corruption and you tell it to do it again and again and again and you tell it to reverse the corruption until it gets stuck in a cycle um when it gets stuck in a cycle you say it's done and uh and that's your bomber main level. It does. And it does.

It, it should have guarantees on like playability, like not having like something you can't reach or. So. It doesn't, you know, it's, it's all just AI. So there's no guarantees of anything.

You'd, you'd have to do some fine tuning afterwards, but, but it's like scarily like true to form you know like uh yeah i can't remember if they if they um i can't remember if they wrote something to throw out the levels you know like they wrote basically a validator and throw out all the bad levels or if they just like you know they they didn't actually make a real game with it. It was a research paper. So they might have not bothered to do that.

But you see the levels. It's like, yeah, that looks pretty cool. Like I could see how that could be kind of a fun Bomberman level. We just did it for the diffusion. But I guess like there's like some randomness that goes into how you select each operation. So I guess almost like backtracking you could just naively get towards the end and sort of try to go back and and you know assign some attempts or whatever and you know oh this you know do a validation at the end like did i

get a good level did i get a you know validatable level yep yep yeah exactly oh man it's gonna be cool the future is scary we're going to be out of jobs. The future is super scary. I mean, I definitely wouldn't want to be in the games industry. I mean, as an engineer, you would be fine. But if you are a concept artist or a voice actor or something like that, I mean, it's going to be different.

There are some heads of studios and stuff coming out and basically forecasting like reduced need for a lot of those things um people talking about you know for doing like cold calling and sales calls and informational sessions you know maybe reduced need for folks there um and then you know also some places are getting flooded there's always been a sort of like low bar kindle book ebook you know coloring book drawing just like you know low effort sort of written word kind of stuff and all of that too i think is uh this ai stuff is sort of competing with not the you know top tier stuff but the sort of like low tier low effort people turning those out hoping one or two people buy them or whatever which has proven successful i think i think there's a

threat to those not not that that's going to be a big problem but once they start targeting people who are you know higher up the the food chain there that's that's going to be a struggle and i will weirdly segue that into our book of the show from high quality books that are definitely not AI generated. That's right. So my first one, I tried, I might have recommended it before if I did. Well, it's worth a second recommendation. I tried looking, but after 171 shows, it's a bit of a process to try to search for it.

And that is Foundation by Isaac Asimov, which is actually a many book series. And the first one is a great place to start. And recently, this became a TV show, which I think renewed some interest. The TV show doesn't follow the book super closely. But if you've never read some of the classic science fiction, it's really like a fascinating trip to see how close some of the stuff got from people writing long before, you know, computers were a thing, much less mobile phones. Some stuff is just like laughably wrong. And some stuff is just like, wow, that's eerily the same issues we're dealing with today. And so reading classic science

fiction is something worth doing if you've never tried it before and the foundation series is a i feel like a pretty formative uh series of science fiction and uh just to give people flavor since i've not said anything actually about the book is the idea is there's a an empire ruling the galaxy and um there's concern that that this uh this these people raise that the empire is is going to collapse and it's going to cause you know humanity basically to be set back and you know and really be a struggle and everyone's going to be bad off and so they've created a a sort of way to help shortcut that to make it better but of course the empire itself is very unhappy with this and the whole prediction that this was going to happen is this of course fake but it sounds very

convincing psycho history which is this idea that you cannot predict individuals but that you know sort of like i think the analogy they use is like a gas you can predict a lot of statistics about gases even though an individual molecule in a gas is, is, you know, very, very difficult to predict. And so through that, they can kind of predict the course of human history and how things will go and make updates and help keep it on, you know, a plan that will, will, you know, sort of benefit the most humans. And it's just a very interesting read in the dynamics. And so if you've never read it before, and if that sounds at all interesting, although I probably did a poor pitch of it, or if you've watched a TV show and maybe didn't realize it was also a book,

I'll pitch Foundation by Isaac Asimov. Yeah. Did you read it recently or is that when you read it? I read it actually as a teenager. So I read it a long time ago. Yeah. Same here. I have to confess, I don't remember that much of it, but I do remember having a big impression on me when I read it a long time ago. Yeah, same here. I have to confess, I don't remember that much of it, but I do remember having a big impression on me when I read it. I think I also had kind of forgotten I read it. And then I like looked up the sort of summary and it was like, oh, I remember.

And then I actually have watched, I think there's a couple of seasons now of the TV show. And I will say, you know, it's a pretty dark, dark thing. So not watch with kids, but pursuing the same kinds of ideas. And I thought, I thought I like it, even though it doesn't follow the book that closely. It uses some of the concepts and the sort of setup to kind of examine some, some of different ideas, but in a similar sort of some similar vein. Cool. Yeah, definitely. If folks haven't read that, definitely give it a read. It's a great series. Um, my book of the show is propaganda by Edward Bernays. So this, uh, this is a fascinating

book. Um, it's, it's so wild that I almost, you know, I really, I wouldn't take it all as ground truth, really. I mean, a book about propaganda might be propaganda. You know, it's kind of like meta. get right into in the book is that is that um um and again this isn't my this is not my theory on life but that basically you know democracy can't work um and and so you really need to have just like a few few different ideas to follow that it can't just be a total free-for-all i mean a good example is the is the app store that patrick was talking about where there's like a million knockoffs like you could search pretty much any word in the dictionary and you're going to find a video game based on that word and they're all kind of cookie cutter and and there's no it's very hard to get

noticed in the app store or to find good content. And so you're really relying on the, you know, editor's picks. And so in a way, it's like you have this really democratic platform, but what it really degenerates to is just like a handful of editor's picks and folks who have a big audience already and all of that. And so the claim by the book, by the propaganda book is that, you know, there's just so much content and this was written a hundred years ago. 1928. Yeah.

There's just, there's so many clubs that you could be a part of. There's just so much, so many choices, so many brands, so many of this, I mean that, that really propaganda is the way

by which like 90 of your options get filtered out and you can just focus on the top 10 so it almost becomes like a um a way to ante in it's like okay you, you have to build this propaganda machine to be part of the system. And everyone who doesn't do that ends up just getting kind of lost, right? um some of them were pretty mind-blowing um yeah i won't spoil them but um maybe i'll spoil one there's there's this cigarette company that um they were under a lot of pressure there were a lot of moms who didn't want their uh sons and daughters smoking because they had heard about you know all these issues i don't know if lung cancer was a really known entity in the 1920s but but you know they they thought there were health issues and so they were able to like start up this like uh women's liberation march sponsored by marlborough and uh they basically tapped into the women's liberation movement of the 1920s as sort of like a back pressure, you know, against like this, like kind of like mothers against smoking group. And the whole thing is just blowing my mind.

I mean, it goes without saying that I think that's fair to say patrick and i are like what would you say is like task oriented in the sense like we don't we're not we're definitely not master manipulators uh we're not uh we're not like these kind of people um you know uh patrick doesn't even have a social media account i barely use my social media account um and so for me this is like a whole different dimension to you know humanity really um i mean it's something that i never would have really considered or thought of if i hadn't been recommended this book so i found it absolutely fascinating um highly recommend it uh very interesting yeah i think sometimes words get uh like artificially intense negative things so i think like some of these things

you're saying the other one i'll kind of by analogy or by similarity talk about is politics so like people have i don't want politics in the office. It's like, well, you need to be aware regardless of whether you want to quote unquote play politics. Politics is a like human way of, you know, that things happen and decisions get made and,

you know, backdoor backroom decisions, you know, these kinds of, you have to be aware of them. If you, you know, want to kind of like at least

have some modicum of control over over what happens and i think by extension propaganda similar it's you know it has a very very negative tone to it but it i mean it's it's a way of lots of people doing stuff they may not see it as propaganda right they just see it as hey i'm trying to get my ideas across or find a way to make people more open to this um but sort of calling it out and putting a label on it saying how common it is is a way of sort of engaging with it yeah and it turns out actually the negative connotation to prop to the word propaganda is also propaganda so the word they actually talk about is the root of the word propaganda comes from the Catholic Church in like the 1600s had a like a part of seminary where they were kind of teaching priests and bishops and cardinals um about how to kind of spread uh catholicism and i think that's where the word actually comes from and then uh you know and then i think uh and

so at that time it had a positive connotation and i think it the connotation flipped over time as a society we got kind of like uh you know not happy with like proselytizing and then and then and then now it's like really like commercial propaganda and and all that um so yeah like the propaganda i think collided headfirst into the enlightenment and lost when it comes to you know reputation as a word um but yeah it's it's yeah, the book is totally mind-blowing. I highly recommend people read it. And to Patrick's point, and we talked about this in the marketing

episode. We had an episode with a gentleman from marketing. I'll give you a really concrete example. I made that game AI Hero. I made it totally free. There know there's no ads there's no in-app purchases or anything like that um you know i told my friends and family about it all that and it has i didn't check recently but it has i think like 100 downloads or something like that

um and now you know i made it you know to frank, I kind of made it for myself. I mean, so I'm not out there trying to get a bunch of downloads, but I was even still surprised that it got such, such, uh, such little notice. Um, and it really just doubles down on that point that, you know, you have to get a propaganda cycle going, a marketing cycle going for anything that you want to have mass appeal. Otherwise, there's just too much content out there. So, yeah, it's not a propaganda marketing. These aren't words that you should be afraid of. I'll say that it's not something that, at least I don't know about Patrick, but it's not something that I personally am very good at.

It's definitely something where I try to partner with other people who are much better at it than I am. Um, but, uh, but it's something that, you know, I think we all should really appreciate. It is very important. If I did want AI hero to, to, to get, you know, a ton of people, you know, I would have to go about things very differently. I think I'd have to be posting regularly on Reddit. I'd have to figure out some way to maybe gin up some controversy. Maybe I'd make a fake post saying that it's generative AI, pretending to be somebody else attacking me for using generative AI. I don't know. But this is just from reading

the book. There's a million things you could do to amp up your thing, whatever it is. But if you don't do that, probably no one's going to pay attention. And that's true in politics. If you make a political party, nobody's really going to care. true in in in most things so so uh yeah it's just a fact of life yeah i think there's a lot to say here but but probably just uh move on for now as they get all right yeah moving forward so time for tool of the show all right what's your tool uh not a tool a game uh but uh classic patrick cop out mine is i've been getting into these recently but this is the one that i sort of encountered first which is the room this is pretty old at this point so most people probably played it before if not you know i would encourage you

to check one of them out often they're very cheap or free on sort of android ios i think there's a steam version probably a web version somewhere but the idea is basically uh kind of like an escape room in an app uh you know just like without without the time limit you're just like a series of puzzles um and you interact with them and for me i've actually never done an escape room uh why i i want to i just like it just hasn't been a thing that i've been able to do but i'm i'm sort of fascinated by the concept and these games are just like an easy way to spend a few minutes sort of playing with something and trying to figure it out i will say in the genre some games do it better or worse and i have my own particular sort of like i don't like

things that are you're just supposed to try random stuff until they happen there should be some clue that you can look for and it can be obscure or it could be that you have to pay attention or whatever but sometimes they fall into like how was i supposed to know that like you know there's no indication that you're supposed to you know touch the top of the windowsill and like it's going to be movable uh it's like okay but sometimes you fall into this but anyways um i guess a pitch for these kinds of puzzle escape room games in general and there's there's many in the in the genre but um for me the first one that i sort of bumped into was the room series i think

there's like now four or five in the series um but you know shout out to the the original one um and if you've never played one before uh you know check it out yeah totally i've played the room i think it's great a lot of fun um it taps into your common sense so yeah you do have some intuition into what the answer of the puzzle the answer to the puzzle is um my tool the show is also somewhat of a cop-out because we've had them on the show before we did in fact had a whole episode dedicated to incredibuild but you know i was building a new version of maimhub my house today, I guess, because it's like using all the desktops in the house. It's kind of the point of the tool. And I just thought

to myself, man, this is such an amazing tool. I've used it for so many years for free. If you're using it for your home use, it's not for professional use, it's totally free. I felt like they deserve another shout out after so many years um we've gotten tons of emails from folks who have uh you know got their company to buy incredibuild and all of that so we have been able to drive some business to them which i'm really proud of um but yeah it's incredibuild if it basically the way it works is it hooks into your build system. It even has plugins for visual studio.

So if you're building on visual studio, it works. Um, if not, it has, you know, it hooks into GCC and all that stuff. Um,

and you just, instead of, you know, choosing build solution from visual studio, you choose, you know, start Incredibuild. And,

uh, I have the Incredibuild agent running on all the desktops here in the house and they all just start the fans all start roaring and uh they all start working together to build stuff so maimhub takes i think like two and a half hours to build um but with incredibuild uh you know i can get it done in like 10 minutes or something. So it's super nice and highly recommend it. Very nice. Very nice. All right.

Well, I think it's time for our topic, which is compilers and interpreters. Yeah, this is a request by Jessica W. I won't say your last name. I know some people might sense that. But thank you so much, Jessica, for this idea. It's a great one. All right. I didn't know, you know, Jason sort of bowed out of like, well, that's not my background. So it kind of fell to me. And I struggled a bit with the tack to take here because there's a there's a lot to be said. This is a big, you know, request. So I decided to take a stab at what i'll say is a

sampler you know kind of going through a bunch of different topics in the area um with some sort of high level overview but you know choosing a interpreted versus compiled language i guess you could say choosing a compiler versus interpreter but most of the time if you choose a language and the pros and cons of language you're sort of of choosing one of these for that. But then in each of these, there's just an incredible depth you can go into or not. You know, most of the time, I guess, we're big believers and, you know, the right tool for the job. So despite the fact that I may throw stones at Python, you know, a lot of times, you're only going to run an analysis once or, you know, it doesn't really even really matter how long it

takes. And so you can get this just really big worry about speed or performance or, you know, selecting exactly the best tool. But I will say, most of the programs I write are probably run less than a dozen times ever, even to check them in and put them in at work, you know, and you, I may run it a few times, someone else may use it, we may copy and paste the code around, but any given instances is just not not that and then, and then a very few are, you know, run hundreds of times every day. And you know, or 1000s of times, inner loops and calls and those ones. And so there's a really a really wide range and so um find a few things that you're you're sort of proficient in and i'll say a way of disclaimer

before we dive in here is just sort of like you'll see a lot of uh internet flame wars about you know oh that that's slow because it's a you know scripting or you know don't if you're going to build something in godot don't use as a gd script because it's you know it know, scripting or, you know, don't, if you're going to build something in Godot, don't use as a GD script because it's, you know, it's going to be slow and it's, I don't, I mean, if you're finding yourself in a problem and maybe you want to listen to some of that or being aware is good, but I feel a lot of people hesitate to take the first step and get what, you know, I'll call analysis paralysis, you know, from making a decision and actually better to just build something and let it suck and be slow and and introspect why at the end of getting somewhere with it and then and you're

gonna make a follow-up choice than to never take the first step um yeah totally there's a balance there but okay off of my soapbox all right so i guess anytime we run a program we're talking about you know a processor and we were talking in the Christmas episode, holiday episode, Jason was mentioning, oh, now I already forgot if it was RISC-V or RISC-V. I think RISC-V. RISC-V. Yeah. Okay, good.

RISC-V. And RISC-V is an instruction set architecture, a sort of way of having computers run. And it has, I guess, what you would call machine code. So out of, you know, wherever it's loading the program, probably RAM, and then it sort of loads a set of bits, right? If it's a 64 bit processor, it loads, you know, 64 bits, and can kind of look through there. And part of that is the operation to perform in some instructions about to load it from a register or to load it from a place in memory or whatever. And then the processor sort of handles that.

And that is machine code. If you ever do like NAND to Tetris or there's another game where you build logic gates out of NAND on your web browser, or you take a class in university where they're sort of teaching you about sort of building up from like VHDL, you know, how to kind of like, you know, implement a processor, you'll end up realizing it really is just, you know,

hex numbers on a screen that you, you know, end up getting executed in there, that that's your program. And every kind of family of processors has a different set of instructions, a different way of organizing those operations. And to write a program in it, you know, you would have to sit there and really write, write, you know, a hex, you know, or you can write binary too. But you know, most people would write it in hex. And that's the way we sort of think about it, because it gives you the nice byte delineation. And you would not do that after about maybe one time of doing that you would not want to do that anymore and so as an example like when we did Motorola 68k processors which is just you know one of these families when I was in in university and we had to do this as a task like you know write some you know machine code

and I did that once before sort of realizing wait a minute I'm just going to write a program to be like a crappy assembler uh and you know do that and so then that moves us to assembly language and so assembly language is just like a very light skin over this machine code so you're pretty much have a one-to-one correspondence but you're writing english symbols so instead of hex code 47 being the move operator where you move you know one set of bytes to another location you would say mov move right that's a lot easier and then the registers each have names rather than just numbers like you know r8 or whatever for register 8 and so this is the first time we encounter what would you would call a compiler And the compiler takes those English letters or whatever language you're writing in letters and maps them directly and emits the bytecode. So does that mean

that it's reversible? Like, can you go from machine code to assembly code? Yes, great, great question. So you can, it's called disassembly. And for assembly, and I'll talk about a caveat here in a second, it's pretty straightforward so if you take a program that lives and you and you haven't gone we had oh that was way that was many years ago but um we had the folks from i think intel in here talking about obfuscating your program codes that people can't do that um and they can't sort of like patch it or do what would in video games would be hacking or you know and and more critical software trying to get your keys or something um but yeah so you can do disassembly so you can take the bytes flowing through your

processor and sort of convert them back to assembly because yeah it's it's a very direct mapping got it okay um so the caveat there that that i was already alluding to and you can can help here it's not too bad in assembly is what i've described far, you kind of do in a single pass. But that also turns out to be easier to do better than and you know, it's frustrating. So you wouldn't want to stay there very long, which is you may want, you may want to have an example where you have a loop. And so when you're on a loop, you want to go back in your program to somewhere previous, right? Just get forward if you have an if statement. And if you were doing what I described initially, and that's kind of like a single pass compiler,

you would have to calculate, go down, figure it out. It's eight instructions for my loop. So I need to go back to that branch statement and say, jump eight instructions down. And that'd be really annoying. So what people do is implement the ability to add labels. So you can put a label, end of loop. And you can know, jump JMP, and then you can put end of loop. The issue is

when you're doing a single pass compiler, you're scanning down, it says jump, end of loop, but it's never seen the symbol end of loop. So it doesn't know where to jump to, right. And so now you get what we call sort of a two pass compiler. So now you start to get what you would kind of think about variable names and labels and jump statements. It's not really that much extra, but it your productivity goes way higher. And for many years, people coded an assembly with two pass compilers. And the two pass just means first scan through to find all of the symbols that are names, and their you know, know locations and then on the second pass does the actual emit the the machine code and now it knows oh this offset is you know this many instructions

down and it's able to produce the the machine code but that's it says it does it literally do it in two passes because i could also imagine like anytime you see a jump forward you just keep track of it and then you go back and fill it in i i mean but that sure but that's still you can't do it in a single it's not one pass because you have to go back up right and right so i guess like the big o notation kind of thing like how do you want to call it i'm sure i'm sure they get pretty sophisticated there's probably you know it's just calling out that the progression i guess in here but still these assembly languages you're you're writing in um you know there's kind of some asterisk here but

for the most part you're writing it per you know kind of processor so if you want to write it on a you know if you're going to use an arduino which is i think based on a pick chip, then you need to use the pick flavor of assembly. If you want to do this in, you know, Motorola 68k x86 processor, arm processor, if you're, you know, going to run on a phone, every one of these has a different set of instructions, and therefore a different assembly language. And so again, that gets really crappy, if you want to ever write for more than one thing, or you want to go faster because you're pretty much still basically writing one line of code per instruction, um, which is going to make you very easily able to sort of intuit about sort of like runtime and complexity

of things, um, a little easier than my high level language. Um, but, but again, it gets, it's kind of frustrating and there are concepts for still everything that's in a high level language sort of has to be capable of being done in assembly language so you still see things like doing function calls so jumping to a function and pushing stuff onto the stack for some processors those things are strictly implemented by a higher language compiler but some processors actually have support for that even Even at the assembly language level does the new would sort of start to call those things complex instruction sets, CISC versus risk reduced instruction set computers.

And so there's a whole separate topic. We'll sort of leave this here because we'll go forever and move up, move up the, the family tree, I guess, to, to higher, higher level languages.

Yeah, that makes sense so so uh um so so every programming language like that you read about like c c++ java python they're all considered high level yeah right yeah yeah yeah i mean i don't i there's degrees to that i guess you know you but at least when i sort of started learning about this and you know they shift over time right there was early on it was you writing an assembly or you're writing in you know something that wasn't assembly and anything that wasn't assembly was you know high level and the idea there being in my mind at least the delineation um and it may be it's moved over time. But if you write a line of code, and it produces sort of an arbitrary number of machine instructions, then you're now writing in a high level language.

Got it. Okay. So in a line of C code, just as an example, or Java or whatever you can write you know a plus b divided by c times d you know to the power two whatever right that's that's one line but it's many many many you know assembly operations to do that depending on architecture um and the compiler now is is able to target different backends so if you write you know and we'll talk about sort of c c plus plus here for a second if you if you're writing in C or C++, you can compile against many different processors. So you could write one program that runs on ARM or on x86, because the compiler is picking up your code, and we're going to build a building, sort of finding the symbols,

building a syntax tree, and it knows how to emit the proper machine code for you know executing that program and it does it can do that with many passes depending on the compilers but by many compilers also emit a sort of intermediate representation so even different front ends so you could say like an llv llvm you'll see things that can target, you know, the LLVM IR intermediate representation, which is something which mean the sort of machine code assembly and the high level language that lots and lots of front ends can compile into. And then there's a specific set of operations and libraries and team working on taking that intermediate representation and compiling it down to your specific final machine. And so then that's called the backend. So you have the front end compiling to intermediate representation and in the backend,

taking it from intermediate representation to a specific class of processor. Got it. I see. So if you wanted to write a new language you could you could you could create a language that compiles to the llvm immediate representation and then you'd be guaranteed that it would run on all these different processors and not only that you're also going to benefit from a bunch of optimizations that the sort of like imagine loop unrolling so you write a loop in in this intermediate representation and now the sophisticated folks across all these languages economies of scale kind of thing can implement loop unrolling or some stuff we're going to talk about later but you know it can implement it at the ir level and then you can benefit from it got it cool it makes sense so now you mentioned Java as well. Java gets a special shout out here, which is that when Java is compiled and this starts to say, you know, these are words we use, but you know, requirement that it has to map cleanly. targets sort of an intermediate representation called the the jvm the java virtual machine byte

code and so it does get compiled and it has all this you know optimization and stuff that can happen and then it produces uh basically a machine code but the machine it targets is the the java virtual machine not an individual processor and when it was first developed this was a huge boon because now you talk about portability you can run on anything that there was a jvm for the jvm for and llvm wasn't really a big time so you know you know again times change but at the time this was like seen as like a really big thing and um sun microsystems at the time you know we're sort of sort of pushing that hey uh java really you know know, runs on I forget, they had an ad, you know, some obscene number of devices, and, you know, has this ability. And so there, you're compiling it down. And then the Java virtual machine has an implementation per, you know, kind of

architecture. So it has an ARM implementation, and it has an x86 implementation. And the implementation of that virtual machine is to take that bytecode and pretend it emulates right emulates this sort of conceptual computer that is like a almost like a lowest common denominator but on some processors it may be able to be very efficient with a certain operation and others not but you don't have to worry about that. Because, you know, it's sort of just handled for you. And all of the standard libraries can have sort of special treatment or efficiencies added. And so this was a very interesting approach. But it sort of threads the line here where you have, it's not a scripting language, we'll talk about that is in a second. But it's not a true compile in that there's still an emulator a virtual machine as it were that sits in the middle here uh and and sort

of that's the program that's actually running so that program is running your program um and so the compiler is not targeting a machine the compiler is targeting well it is targeting machine but it's not targeting a physical machine so if i understand correctly right so the difference is with llvm you're going to this immediate representation but then you're you're you're you're uh instantly going to machine code and so now you have machine code that can only run on uh that architecture but with java you're still going to this intermediate format but you're not doing that second step to go to machine code uh at compile time you're doing it at runtime yeah so the intermediate representation is not really executed right there's no program well again you could

could write one but the intention isn't that there's a program that runs the intermediate representation versus in java that there is a program the jvm that runs your you know jvm byte code got it okay so then now we get to to sort of uh scripting and scripting and interpreters the idea is is sort of similar it's very rhyming which uh you know basic was this way other things are this way and that is rather than a compiler you're running a program that needs to work per architecture but that program is doing all of the work at runtime of executing your script. So it's converting the symbols you wrote into the letters you wrote into symbols, it's building whatever it needs to to be able to execute it. So if you're, you know, setting a variable name, like Jason was mentioning, it's keeping track of that, but it's doing it sort of fresh every run. And so it's opening it up. And if you have,

as an example, you know, a typo halfway down, you're going to find out about it out about it a down your programs execution or whatever. And so it's not attempting to in this sort of level zero, it's not attempting to do anything other than just sort of execute your script, right? So if you've ever written Python, and you have a, you know, typo at the bottom of your program, and your whole program runs, and then doesn't output, you know, the answer, because it crashes at the end, this is happening, it's not trying to compile, it's not trying to look ahead, it's just there is the program that's running on per architecture is parsing the script input at runtime. Yeah, I mean, you can this this is starting to really boggle my mind now that i'm

thinking about it like in python you can open the interactive interpreter and you can actually have you can write if statements and for loops and anything you can write in python you can write in there and so you can actually like write a for loop in that for loop have a break if a certain condition is met and then keep going and so like python has to kind of remember that like you put that break there but it doesn't really know what's going to happen after the break because like you you literally haven't written it yet and so that's to kind of keep track of all of these things while you're while it's interpreting what you're writing it's also not that it's impossible other places but it becomes much more straightforward to things like python loves to do which is like modify definitions on the fly

right so like changing variable names to another type or to even modifying functions that are on a class because all those things are sort of held in memory right like the the sort of processing the compilation goes into memory and it's just held and is done sort of in real time like at execution and so modifying that stuff is more obviously you know straightforward to implement more it's obvious, but it's you can kind of envision how you would do that if it's like a thing sitting in RAM, versus if you've compiled it into something that you admitted to the disk, right, and you're sort of executing out of read only memory. And again, there's lots of things that blur these lines. But yeah, so scripting,

and if when people sort of think about it, it can be slower because at runtime, you have to do more, right? You have to convert the mapping. You don't need to know that, you know, your variable name is some complex, you know, set of underscores. When you're doing the compile, you just map it to this is the first variable I've seen. So it's variable lookup table entry zero, right? And it's literally just a pointer or just an index zero and then every time you see that you're keeping track of it but in the final program it all gets taken away right because it's just everything points to the right spot and all those things are resolved when you're doing scripting though when you're marching down the program you that table has to live in memory

and you have to go do that look up right right? Oh, Jason's most complicated, very long named, but not meaningful variable that has to get parsed, which is a lot of characters and has to get that mapping to take place every time it's seen sort of like at runtime. And so this gets to why people kind of say

scripting would be slower. But as Jason mentioned, the advantage is because execution is happening at like, you know, kind of while you're doing it and all these steps are you can also modify the steps and it becomes very easy to do and to interactive and to kind of see what's

happening and um you know very straightforward yeah that makes sense i believe i believe if you um your machine code is loaded into the process memory that like you can't touch like if you start writing in the chunk of memory where your machine code went it'll the the you know like they'll they'll assume that you're trying to do something nefarious or something or a mistake but but in python there's probably no way to catch that a lot of early viruses did this like self-modifying code so they you know people screen them they look, but then they get modified to do something that looks that is nefarious, you know, at execution. Yeah, a lot of people, a lot of systems will try to stop or prevent that and assume that

you were making a mistake and at best preventing you from doing something you really weren't supposed to do. Yeah, that makes sense. But yeah, that's it. Yeah, that's mind bendy stuff when you start talking about that. Okay, two couple other things I wanted to cover briefly, you'll hear about.

So one is a JIT, which is just in time. So for scripting languages, all is not lost about these runtimes. So in Python, as an example, or other things, you can have a JIT, which says, hey, I've noticed that this piece of script has been executed. This function has been called many, many, many times. So I'm going to rather than every time I get to it,

do all the parsing and all the work again, it can effectively do a compilation step, right? Can go ahead and resolve all of those things, things it can even go could do, you know, implementing them all the way out to machine code or the operations or whatever right um java has this as well well for bytecode but basically at runtime sort of observing that you're about to do something or that you've done it before and pre-computing and storing those results so that the next time you do it time you do it you get your execution that that's sort of the concept i'm probably doing a bad job but of just in time right it's just in the nick of time i've provided you a you know optimized thing to run um right and you could do the loop unrolling and all of that stuff like if you know that this four loops only get executed four times or you don't know it but you're confident then you could unroll it yep um

and you could even do code analysis to do it, right? Again, you know, extend these all have like research directions or implementations and various, you know, features. And then the last thing I want to talk about that also blurs the line, though, is something that's actually interesting you've ever seen, which is profile guided optimization. So some of these drag races, I guess you call them between sort of scripting languages and compiled languages will show, hey, actually, this Python code is a lot faster, because

in this case, the, you know, during runtime Python, the interpreter observed these behaviors and perform this optimization that you couldn't have known a priori, it had to be known, sort of like at runtime, and it gains this advantage. It's true, that can happen't have known a priori. It had to be known sort of like at runtime and it gains this advantage. It's true. That can happen. That is a thing. And it can be really cool. In compiled languages,

we're starting to see some work around profile guided optimization, which is run your program and you record a bunch of statistics about how many times various loops. By default, was this if statement skipped 99% of the time and even compilers do some of this sorry even processors do some of this to help with

optimization but sort of recording statistics about what was executed how much and and various sort of things and and that the sort of call it happy path you make a guess i'm just going to assume this is going to be true and i'll add a little check if it's true i'm going I'm going to gain this boost. And if it's not true, then I'll have to execute something a little slower, right? No free lunch. But it does this via a profiling. So you've profiled your code. And now this optimization uses that profile of the code as a sort of almost second compilation. So you run the compilation, you insert all these sort of like, you know, extra hooks to measure stuff. You look at the output, and then you run compiling again, that uses that as input, and it's able to sort of be more thoughtful about how to perform certain optimization steps of compiling. Cool. So if people are really excited about compilers and interpreters and they want to work on the Python VM or the C++ like GCC or something, where does most of that work take place? My guess is probably like government programs and giant companies and stuff. But like, what would be your career advice for them so there are folks working at you know big company big tech companies that you sort of hear right if we if we sort of name some um you know dart you know at google or swift at apple or

i think java is still at oracle um you know obviously those places have teams that work on these things and you can go work on them um but i think a lot of it also happens as hobbyists do stuff. You know, in university, this is compilers and interpreters is a course you can take. If you want to do it as a job. What I don't know is like how you as a job without working at a big company that has a vested interest in it, you sort of get paid to make a very cool llvm and intermediate representation optimization definitely something you can do and if you have a company that will sponsor their work which is you know maybe a hard thing to find i don't know that it's it's not like writing a game we just do it and then people pay you for for good output

be very difficult to write like a new language that someone would pay you just to like consult on um my opinion maybe i'm wrong yeah i think you're right i think even if if you set out to you know create a new language uh i think even then i would probably start by working for one of these big companies. I think like, I want to say one of the military labs is like the chief person on GCC, like Sandia or one of these labs, I don't remember. But basically, you know, for all of these things, LLVM, Swift, GCC, you know, look at who their biggest sponsors are, right?

Look at who the domain name of the email addresses that are most prevalent on the mailing list, right? It's not hard to figure out who is actually putting in most, you know, of the effort for that particular compiler or interpreter and then once you know that then then uh then it's a matter of you know creating changes uh you know basically doing that doing that job kind of pro bono for a little while and proving yourself to those folks um you know becoming ingratiated um by those folks and then and then reaching out to them and saying hey i want to do this full time um and so if that's something that interests you that's that's the path for it to do that. You don't need to be a master propagandist or anything like that.

It's a very simple formula. Just put some sweat equity into that. Talk to the folks who are also putting in that sweat equity and that's how you can do it. I'll give two shout outs quickly. I don't think this is the first thing you do, right? Write programs for a while before attempting to write your own programming language.

Probably a good move. But two resources that I think are really useful here, many more, but people would have recommended the Dragon compiler book before, but I never made it through it. Anyways, the first one is by and I might not say the last name correctly, but Robert Nystrom Nystrom, and he wrote a game design patterns, like design patterns for games that's very useful. Definitely check that out. I think you can read it free online, but also read free online crafting interpreters. So crafting interpreters.com. And I think there's like, you know, a web version, but I think there's

like actually now a physical book as well. And I haven't sort of gone through it and like implement myself. It's on my, my to-do list. Um, but definitely a way of talking about how would you add an interpreter, implement polymorphism, how would you, you know, handle variables and sort of building up your own programming language, uh, of you know not going to be i don't think state of the art you know pushing the envelope but but definitely giving you a flavor for this which i think is really useful to know how how some of this stuff works and then in the sort of opposite side of that i i mentioned it but nand game.com there's also nand to tetris but nandgame.com is really easy to jump in and start showing you like starts oh like one touch too low maybe for what we talked about today

where how do you build a nand gate how do you make an org gate from a nand gate but quickly jumps in basically how do you build um you know the actual processor components, the parts that do math, how do the bits get added? And why is it that a certain machine code byte evokes a certain behavior from the pieces of the computer, the actual silicon? Anyways, and so you can just like play it

in your web browser and, you know, just sort of go at it. You might have to Google some help. I did, sorry. But, you know, how do you implement registers? How do address buses work? Anyway, so that's nan game.com. We'll put them both in the show notes, but two useful resources. There's a modern take on this game too, called a Turing complete. Oh, I've seen this, but I didn't want to pitch it cause I haven't tried it. I haven't tried it either. So I haven't tried NAND to Tetris either, but it has amazing reviews.

I just looked it up. It's overwhelmingly positive. Steve reviews 2,200 reviews. So that is a really good endorsement by the community. Yeah, I've heard a lot of people say it's another good resource. So I'll put it in the show notes too. Cool. All right.

Thank you, Patrick, for going into a ton of detail here. It was an awesome episode. Thank you, Jessica, for recommending the topic. Phenomenal topic. And thank you to all of our patrons who are kind of supporting the show, helping us reach new people. I was looking through my email the other day about something kind of unrelated and i stumbled upon a email from a listener um a while back who uh who we

were able to help them uh you know find their first job i mean that stuff's really inspiring continue to send that to us we read all of them sometimes we read them twice sometimes we find them years later and read them again um so thank you everybody out there for listening for supporting this show and we will see you next time music by eric barn dollar programming throwdown is distributed under a creative commons attribution share alike 2.0 license you're free to share copy distribute transmit the work to remix adapt the work but you must provide an attribution uh to uh patrick and i and uh share alike in kind