Programming Throwdown - 153: ChatGPT

Episode Date: March 13, 2023

ChatGPT has made a mark on the world as we know it, but that’s only the tip of the AI iceberg. Join us as we discuss how the field of artificial intelligence is growing – including some developments that might not be on your radar!00:00:23 Introductions00:02:01 Jason’s attic adventure00:06:09 Comparing saws00:10:57 Patrick’s surprisingly useful thing00:12:21 SpaceX00:17:31 Human motion diffusion model00:20:47 37Signals00:29:30 Polars00:35:37 Books of the Show00:46:11 Neon00:50:33 Patrick’s player search00:53:47 ChatGPT01:17:12 The threat to Google01:28:06 Jason and Patrick’s future prognostications01:32:13 FarewellsResources mentioned in this episode:Join the Programming Throwdown Patreon community today:  https://www.patreon.com/programmingthrowdown?ty=hNews/Links:SpaceX Starship Static Test Fire Plannedhttps://www.cnet.com/science/space/spacex-prepping-for-first-full-test-fire-of-its-mega-starship-rocket/Human Motion Diffusion Modelhttps://guytevet.github.io/mdm-page/37Signals Leaving the Cloud and Details Cloud Costshttps://twitter.com/dhh/status/1613508201953038337Polars: DataFrames in Rusthttps://docs.rs/polars/latest/polars/index.htmlBook of the Show:Jason: Build by Tony Fadellhttps://amzn.to/3wpLnLWPatrick: Age of Myth by Michael Sullivan (Riyria)https://amzn.to/3HlEsJ5Tool of the Show:Jason: Neon: Serverless Postgres: https://neon.tech/Patrick: 7 Billion Humans (Steam): https://store.steampowered.com/app/792100/7_Billion_Humans/If you’ve enjoyed this episode, you can listen to more on Programming Throwdown’s website: https://www.programmingthrowdown.com/Reach out to us via email: [e-mail] can also follow Programming Throwdown on Facebook | Apple Podcasts | Spotify | Player.FM Join the discussion on our DiscordHelp support Programming Throwdown through our Patreon ★ Support this podcast on Patreon ★

Hey everybody, I have an exciting duo episode of about a topic that a lot of people have been talking about. You really can't go on. Well, depending, I guess I can't go on social media without seeing stuff about chat GPT everywhere on every app all over the place. Oh, OK. I shouldn't fess up that. No, I'm just kidding. I've heard of it. Yes. Yeah. ChatGBD is totally blowing up. Have you tried it? I have not. Okay. Yeah. It's pretty wild. I would highly recommend it. But yeah, we're going to dive really deep into that, but we're going to do a full duo episode, which is really exciting. I think that one thing I started to do a little bit of inside baseball here but you know we have an

intake form where um you know we talk to some folks if it looks like that it will be great content for you all um well we provide a link and people can fill out the intake form and we've had a ton of really exciting opportunities to interview people from all over the world about amazing stuff. But it means that, you know, if we do that, then there's no room for duo episodes. So I started filling out the intake form myself. So we'll notice that some of the times we go to record, it's actually an interview with with us so i think a great opportunity for us you know the two of us to just uh hang out chat about a cool topic that's uh that's pretty relevant and can't get much more relevant than that so you teed it up very well all right so we'll do as an intro topic you know i um i refloored my attic so what that means is is the attic has a bit of sort of ways of how

of how your house or these wood frame houses are constructed you have a bunch of these giant like thick wooden beams this is the attic over over the garage so so we have these wooden beams if you're up in the attic and then below the wooden beams is, I guess, whatever the ceiling of the garage is made out of, which is, I guess, like drywall or sheetrock or something, something like that. But it's something that's like really easily breakable. Like if you were to try to walk on it, you would just go right through and you would end up in your garage. And so it's not really practical to put things there. So I ended up getting just a bunch of plywood and putting plywood down so I could walk around and all that. And so, you know, because of that,

I needed to screw in a ton of screws and I wanted to do a good job because it's screwing right into the house. It's kind of hard to do a do over. And I found this thing on amazon it's basically it looks like a brick and patrick you probably have like a million of these you're gonna think i'm really done with this you're much more handy than i am but it looks kind of like a brick and it's got a bunch of holes of different sizes and the idea is you know you stick the drill like this brick goes up against whatever wherever you want to drill the hole and then you stick the drill through the brick. And then just because the brick acts like a guide. Oh, you're talking about a drill guide.

Yeah, yeah. Yeah, and then when you drill the hole, it goes at a 90-degree angle. Yeah, I mean, this is the simplest thing. It was like $5 on Amazon. And it really made me think, how many simple things are there like this that I don't know about? I thought you were gonna say you bought a screw gun, which is the like auto loading feeder where you just and you just like drive the screws like a nail gun, but for screws. Yeah, you know, I've seen that we had our deck redone. When we bought

the house, the deck was all rotted. And so we had to redo it. In this case, we paid somebody to do it. And they had one of those. And they actually had one where they could, it was long, like super, like four feet long or something. And so they could just walk and just as they're walking, put the screws in. I'm getting no end of joy, though, of thinking about you. I know probably wasn't. I'm just thinking of four foot by eight foot plywood sheets like balanced on your shoulders you know like going up through the attic injury and like over and drilling them yeah like well actually i have another cool purchase

now that you mention it so i had i bought the four foot by eight foot plywood which is the sort of standard size but i can't i couldn't fit that into the attic because when you take the ladder and everything into account so i had to cut it into the attic's really tall so the eight foot part was fine it just needed to be narrow so i had to cut it into two feet by eight foot sections and again like i do not have a steady hand at all like i'm not a good artist uh you know like with hand drawing and i. I like the least steady hand. So I found this thing where you screw your saw into it, and it acts like a guide.

And you choose how many inches thick you want the thing to be. And then it runs along the side of the plywood so that you can't mess it up. Don't go looking up the words track saw you'll end up with no end of tools for doing this sheet wood breakdown yes oh yeah but even that guide you know because i know they make table saws and things like that they're super expensive this guide was also like maybe 15 20 bucks just plastic thing and uh yeah the lines the the cuts came out perfect so yeah it just it made me realize like you know there's simple geometric geometry based things that probably the ancient romans used or something that are really cheap and really effective well okay i'll do so you you spoiled that you were going to do this as a topic.

And so I tried to prepare something. So on the topic of woodworking, I will say that I get no more pleasure than sawing using a Japanese pool saw, a Suzuki. What? If you've only ever used Western style, like in your... Okay, if you come from Europe or America, I guess,

and you're used to a push saw, which cuts on the push stroke, like a cheap construction saw from Lowe's. Those things are terrible. They're garbage. They're very hard to use if you're just like breaking down two by fours or whatever. But if you try to do what Jason said, it's very difficult. Instead, just buy yourself a very cheap Dozuki, which is like a pool saw. So it's actually really thin and flexible, so you can't push it. So it only tensions and then uses the tension of the steel to cut. It's a much more ergonomic.

It takes a little bit of getting used to, but the curve, the thickness of the blade, it's a lot thinner than a Western-style saw. You have to explain this to me. I thought that when you saw something that you saw in both directions, you push and you pull western saws cut on the push stroke and they have like thick blades like thick the the metal part and their teeth are sharpened at an angle to cut on the push push stroke and the cheap ones are often like electromagnetically hardened like they heat up the edges and make them very hard in a specific way so they can't be sharpened old ones can be and then the japanese have this like very thin metal plate saw that cuts on the pool the pool stroke so they're called you just blew my mind

because like you know when you watch tv you see people like saw saw saw you have to go yeah you have to go back and forth because your arm can't just keep going one direction but like you're saying when they pull they're not like actually doing any work in the pull part it's just getting them ready for the next push yes it does i mean we'll bruise so like when you're getting started you'll often pull to get started but then the actual cut the power stroke is the push wow my mind is blown right now okay oh uh so the pull yeah i mean yeah this explains why everything i cut looks like garbage so you only need to yeah put pressure and push on the certain stroke also watching like a youtube video about how to line up for the cut most people like me included when i was pursuing is like you line up the cut in front of your body like in front of your sternum but then you have to like

kink your arm to like make it Oh, right. It doesn't work. So you actually want to line up the cut so that you can drop your elbow straight down next to your body and just make a choo-choo train, like a little piston arm on the side of the train. You want to make that motion. And so if you line the cut up, set it up vertically, and then go back and forth that way that also is another kind of a related kind of dumb moment where my uh my axe was just kind of too messed up after use i have this like it's like a regular axe you buy from home depot so it's kind of flat on one end and it's got a blade on the other end and so when i would chop wood i would put the axe in the wood and then I'd take a sledgehammer and I would hit the top of the axe to like finish splitting the wood. And so I completely broke my axe doing that. And so then I went to get another axe and it actually, the axe I picked up actually has like a no hitting with sledgehammer sign.

Like it was pretty, it's kind of hard to put that into like having a stick figure swinging a sledgehammer at an axe with a no it's like kind of hard to do that so i have to commend them on that and i was like oh i was like oh i'll get the i'll get the the one you can hit with a sledge hammer so i picked up another and turns out like none of them yeah you buy a wedge yeah there's a special wedge you drop in yeah so i learned the hard way that uh you have to like replace the axe with the wedge once you have the little niche in there and then hit that with a sledgehammer. Oh, the, you know, lumberjack forefathers would be proud. See, I never did Boy Scouts. Did you ever do Boy Scouts?

I did not. I never did. Oh, see, yeah. So how did you learn all of this? Just through your parents and stuff like that? See, that's my problem like my family makes fun of me whatever take the most basic task if i'm not a hundred percent sure i know what i'm doing youtube yeah that is a really smart i mean pretty much so if

i'm not a hundred percent sure what i'm doing it's pretty much uh destroy things that's set off smoke alarms when cooking you know yeah I think some YouTube could probably do me some good alright so my surprisingly useful thing was gonna be these fancy $18 nail clippers but like I feel like we are totally out of woodworking theme now so wait a minute okay wait hang on now I'm curious about this like how is this uh better than regular nail clippers they're just like sharp so they just like it turns out you're not really supposed to just use nail clippers like your whole life like throw them in a drawer and you know just like never replace them turns out they can just be like really really sharp so you don't have to like

if you're putting pressure on the back of the nail clipper to make it cut you're doing it wrong oh like it should just be a very easy motion like it should be nice and sharp and very easy to cut so i yeah i learned this actually i'll i'll i'll riff off of that i recently replaced all of our cookware not all of it but a lot of it um so we be using the same cookware since we were dating and, uh, um, you just say pots and pans and all of that. And so, um, I decided to, you know, spend a little bit more money and I got this, uh, this pan that was really nice.

It basically, it kind of feels and looks like cast iron, but it's not, it's nonstick, you know, and, and the, the surface is just phenomenal. Like we cook an egg now and you barely need any oil or butter like the egg just slides around and uh yeah maybe the moral of these two stories is to you know don't have the same no tail no uh toe clippers and pots and pans for 20 years like you'll replace these things every now and then. All right. We're just dating ourselves now. Like we're, yeah. Okay.

We should move on to the news. All right. Time for news. Take it away. Yeah. Mine's up first. By the time you will have had listened to this, whatever the tent is messed up.

Hopefully this has gone very successful, but I'm excited. It's finally happening. It seems like this week, SpaceX is going to light up all 33 of the engines on their Starship. They have it all stacked up, the booster and the Starship itself sitting on the top. They're going to light off all 33 engines

in preparation for attempting an orbital test flight. This is super, super exciting. They went through, I won't get into it, just a lot of drama trying to get from their last test flights, which were a while ago, to getting ready to go to orbit. I'm super hopeful this will be successful. If so, they always say there's a large chance of failure, which is probably true. But I hope this will be successful. It would be great to see the program moving on again. It's also exciting to see that they have a lot of faith in it. They're ramping up production both in Texas and in Florida now. And they're going to sort of really try going for

it. And that's exciting. I live in Florida now. And you forgot from having lived here a while ago and now living here again, that it used to be like this, but even now it's more so. There are rocket launches basically like every week or multiple times a week. So you'll just be like this but even now it's more so there are rocket launches basically like every week or multiple times a week so you'll just be like walking your dog and like oh hey look there's a rocket going up uh just by happenstance which is awesome crazy it's so good to be back to where like that cadence is is really up there whenever we do the predictions i always make predictions about space but i really think it's underrepresented just how like big this could be so i'm pumped actually give us a background what is the starship oh i didn't i didn't get myself my notes together but uh make notes uh but the gist is it's you know spacex's next launch systems so they have new kinds of engines

the these like this full cycle float these like special kind of new engines different from the ones i have now a different what do you want to call like a chassis like the okay the rocket and instead of right now it's mostly the liquid fuel rocket and then a tiny little thing at the top right that is you know can't really hold all that much you can't put people on the top with the capsule that they have the dragon capsule or can put cargo but it's still even though it's it's much better than most of the other stuff going around and a lot cheaper still relatively limited but now they're working on the starship which is going to be on its own just like much more cargo capacity the engines

are much more efficient it's meant for this ramp up where they're attempting to have it all planned out so they can actually go to mars so there will will be, you put fuel up into space, then launch a rocket, then refuel the rocket from the previous rocket. There's a whole logistics chain that they're working out here and the right sort of stuff that they need to get that going. But fundamentally, it's just a much, much larger rocket. So I think it's right on par. I feel like it got debated last time I saw it,

but basically the most powerful or right at the most powerful sort of rocket to go up. But on top of that, it's not only that it's also going to be enormously cheap because it's far more reusable, far more efficient. So it's going to be much cheaper to launch that rocket, which drives down the price like per pound to get to space basically.

So it's only efficient to go to space right now. If you have like a super good use case or you're very well funded but if you can drive that down you need much less of a you know excuse to put something up which means the amount of stuff going up will will be a lot higher that makes sense i had this crazy idea over the weekend and i'd love for you to debunk this but so well okay the premise here is i assume that in space there are rocks with with precious metals in them like like yeah okay so so could you redirect one of those to like crash into the middle of the desert and then mine it when it hits the earth and like would you would you kind of destroy the atmosphere like is that how tenable is that is that ever something we

could see happen in the next like century or something uh people have talked about i think crashing certain makeup asteroids into mars for sort of the purpose of doing terraforming and like heat it up do various things i've never heard someone say that like that the for the purpose of mining i think too much it would either be so big, there'd be a huge problem or too small and it would incinerate from the crazy heat. But what they have talked about is moving asteroids into a better or more adjacent orbit,

mining the stuff and bringing it down, which would cause absolute chaos. Why would you bring this stuff down when you could just bring the whole asteroid down? Like, is it just so- Oh, because to get through yeah so you have you have this fine line between you either destroy what you're trying to bring down or you destroy all the people who live here it's kind of a kind of a balance to walk that's what happened to the dinosaurs they said they're dinosaur spaceship tried to redirect like an asteroid full of gold into the Earth.

That's what happened. Yeah. So I think being in a sort of Earth adjacent orbit is better. And then, you know, having a way to get up and down easily. And a more aerodynamic, you know, made for the task sort of vehicle. Or if you're trying to construct space stations and stuff, just leaving it up in space and mining it and using it on site as it were oh that makes sense yeah cool yeah that's super exciting my news first news story is this thing called the human motion diffusion model so the idea here is you know we've all kind of seen dolly and stable diffusion and these these other things where you type in

something like uh you know walking on the beach astronaut walking on the beach or something and it will draw a picture for you what this does is a little different you type um an action and it animates a character for you so they have this human body armature which is like a three-dimensional kind of think of it as like a skeleton right that you can animate so it's got a bunch of little joints uh kind of like one of these um you know paper puppets that we had when we were kids where you punch holes in it and they could articulate on those on those ambulate on those on those joints, but in 3D. And so the way games, for example, work is someone will wear a suit and that suit will have a bunch of sensors on it. And that person will do a somersault or jump into the air

or something like that with the suit. And then the sensors will allow you to take that motion and capture it. They call it mocap or motion capture, and then have your character in the game, as long as it's somewhat human-like, do the same thing. And so what someone has done is they've collected a ton of this motion capture data with labels. So for example, someone will say, you know, walking, and then they'll have a person in a suit walk, and then they'll capture that data and they'll make it so now this little 3D armature can walk and they have jumping,

they have waving, they have all these things. And so, you know, with that data set, someone put it into the same kind of technology that powers, you know, DALI and stable instability AI and all that. And they created this thing where you can type in, doing a somersault, then giving a high five. Like you could just type that word in, a sentence in, and it will generate an animation for that, which is really cool to watch. So they have some examples here.

A person skipping a rope, a person walks backwards slowly, and you can type pretty much anything and get this this little uh you know armature to do that for you yeah i went and viewed those clips that is that is pretty cool i think the yeah nuance of like do this thing but stumble around like you're drunk or do this thing. And then modifiers is pretty interesting that it seemed, it's always hard not to anthropomorphize it, but you can, it almost feels like it understands what you're asking for. Yeah, exactly.

And, you know, if nothing else, it's variational. So like if you had a bunch of people in a game or in the metaverse or something, a bunch of AI agents walking, they would all in the metaverse or something a bunch of ai agents walking they would all walk a little bit differently with something like this oh i didn't think about that so yeah the same prompt doesn't give you the same same result every time yeah exactly like there's element of randomness here pretty cool my next uh news article, I guess we're seesawing around here, but that's okay, is the 37 Signals, which is, I don't know, I would say is sort of partly most famous for its founder, DHH, and his interesting takes. And we were talking about that on the podcast the other day,

but he's famous for sort of Ruby on Rails and doing a lot of that stuff. I think, is that the person who made Basecamp, I think? Yes, that's right. Okay, got it. Basecamp, yeah, and a suite of tools. And so they've been very vocal about a lot of things, including not wanting to take venture capital funding, being sort of self-funded and sort of staying small, growing responsibly,

programming efficiently, you know, counter to, I guess, what they would say is a lot of the waste in typical Silicon Valley startups. And then obviously they've been pretty successful in exploring technologies, understanding, staying relevant, like all of these things. Okay, that's all set up.

So DHH on a Twitter thread has been talking about that they had gone to the cloud and been using AWS for a lot of stuff, but that it wasn't working for them. And that basically they were going to stop doing that. And they were going to run their own servers internally for just themselves. And they thought that they were going to be able to save a lot of money. As part of that, and I guess like what you might call radical transparency, he agreed to post how much they were spending and what they were using on AWS in a Twitter thread. I'm not

going to read the details of that. Please go read the Twitter thread if you're interested in those numbers. But he gives a lot of breakdown about what they were spending, why they feel like it was too much, comparing it to the costs of like heavy duty server infrastructure from just, you know, sort of Dell or whomever. And it sort of started a lot of conversation, well, a lot. It started a ripple of conversation in various communities that I saw about people either sort of agreeing with him and sort of espousing how AWS or Google Cloud Platform or Azure, they're very good at convincing you to spend ever slightly more money each time and just sort of grow your spend and have

negotiations. Other people saying that a lot of folks just don't anticipate how expensive it's going to be and really spend the time optimizing and being efficient. People pointing out he was storing a lot of data, several petabytes of storage. So if you look at the average customer versus him, the amount of storage they were using was a higher percentage of their budget than most use cases. A lot of people throwing random, maybe they don't know what's going on or that the person may know slightly more of what they're doing, just like random to, oh, go serverless, oh, do this, oh, do that. But it just started this interesting conversation because I think a lot of folks don't do this out in public and startups will say how

much they spend on cloud or whatever, but people kind of take that as, we don't know what you're doing, right? And here we have someone who it's pretty hard to just levy a, like, you don't know what you're doing. Although there are potentially some criticisms that, hey, you could have changed stuff around. You need to plan for it this way. You can't just bring in someone who's an AWS person because they're just going to do all the AWS things and not worry about the cost. But also just this broader conversation about how expensive it really is and that there isn't really a good way to do an apples to apples comparison. So as an example, if you say, oh, I have all this data stored in S3, several petabytes. Well, first of all, several petabytes is many, many, many, many hard drives that stick in one location.

Then you've got to add like backup redundancy strategies. You have enough that even that becomes like a logistical issue. Just like the rate at which you need to buy new hard drives to swap out failing ones. You need to have them in two sites. You know, all of this stuff, AWS just under the hood, right? So you say, oh, it costs X cents per megabyte, you know, and that's really expensive. I can go buy a hard drive at, you know, less than that cents per megabyte. And it's like, well, maybe for some use cases, that's the same, but it's not really. So there's just this big conversation opened up. But I think for so long, everybody's like, go to the cloud, go to the cloud, don't

self-host. Now people are like saying, well, maybe I'll self-host. And people are going, well, we see why. It's just a very interesting, I guess, pendulum swing back and forth and people just opening up that conversation. Yeah, makes a ton of sense. Yeah, just for a little bit of clarity, definitely, you know, folks should read the details. But yeah, he said for 37 signals, is that a company? That's his company. Yeah. Is that the company that makes Basecamp? Yep. Oh, okay. Okay. So for 37 signals, they spent 3.2 million last year in in cloud. And yeah, I think you hit the nail on the head. I think i think i think you know the

the thing that for me that makes this interesting is you know people will use the cloud as a way to not have to think about things it's like i don't want to have to think about um how many servers to buy i don't want to be a position where i have to rush and like emergency order a piece of hardware and set it up. And so, and I think the premise here is, well, there's so many other people using this cloud that they can just kind of package it in a way that saves everyone money and that savings will justify everything else. Everyone cringes at the idea of like, I have a hundred machines that are idle right when you know you could be time sharing that um but i think in practice you know the problem is that the unit of measure whether you're talking about storage or compute for a lot of these companies even startups is just so large that they can't practically share a lot um like

you know you might be using 200.5 machines and that other 0.5 can be shared, but who cares? Cause you're already using 200 whole machines. So yeah, it seems like there should be a way to, um, I actually, I know Amazon has, uh, things where you can reserve for a whole year, you know, reserve some amount of, of, of usage. usage yeah i really wonder if when these people try and do things locally if they're going to find out that it's actually not that not that much cheaper like you can buy the hardware but then there's all these other costs you're going to start paying for especially network and things like that yeah i think it's especially brutal also bringing back the like renting space and hiring people on your team to go manage those services and, you know, bring somebody to, like, put together the, you know, servers or, you know, go plug them in, screw them into the rack.

I mean, you can outsource. So it's like a spectrum, I guess. But I think what we'll probably see, and so a lot of this, and he even brings these things up about like reserve versus spot instances and long-term negotiated contracts and all of these things. So it seems like he's reasonably done his homework, but I think it'll be a very hybrid solution where I think when you're rapidly changing around, maybe you just go to the cloud. Once you sort of stabilize out, you bring internal and optimize some very steady portion, but then allow the flex to live sort of on the cloud. And it's higher per unit of computation, but the idea is you turn it off, right?

Or you don't use it all the time. And so there's this problem of sizing for the peak. Back in the yay old days of the internet, right? It was like even earlier, but like eventually this is where the slash dot effect, if your article got posted on slash dot, you know, millions of visitors would show up. And if your website doesn't respond, you're never getting that traffic back again. And so there's this like sizing for the peak. But if you size for the peak all the time, as Jason mentioned, then you're going to hover around like 1% utilization on average. Like that's not useful either. And so I think these sort of hybrid where you use a lot of it for overflow and you have it set up. But that's also a more complicated solution. So I don't know. Yeah, you need like a load balancer now that spans your internal like warehouse and Amazon and like redirects the traffic and all of that yeah that's

yeah there's got to be a product like that um but yeah it's it's you're right it adds complexity to everything but uh yeah that's fascinating um yeah i mean that that's that's uh i think that's going to be a big area i know actually they're doing a lot around this with storage. So there's a min IO and a bunch of S3 compliant storage where you could even use Amazon's SDK to read and write data from your own local data center. So it's like something like that for compute or for web services or something could get us there. My second news story is Polar's data frames in Rust. So Patrick, have you ever used Pandas? Yes, but not extensively. Yeah.

So Pandas, have you ever used R? No. Yes. Okay. I say, I guess like not extensive. I've scripted a few things and tried some stuff but i've never day-to-day programmed in it okay got it got you've never like checked in a pr at work that was our code for sure no okay so so r had this amazing thing or has this amazing thing

called data frames the idea is it's basically like an Excel spreadsheet, except instead of columns, which are just, you know, I'm on column three, I'm on column four. Maybe I'll put a letter on it. I'm on column A, B. Instead of that, each column has a name. So you can say, well, this column. Yeah, and this is what people do with Excel, right? The first thing they do is in the first row, they title all of the columns. So let's just like make that a first class citizen so so a

data frame is basically a set of series and each series is given a name and it's a nice way of working with data it's pretty much the way that r you work with most data in r and there's a bunch of support around that so pandas is a data frame library for Python. So the idea is you want to have that construct in Python, you use pandas. And so it supports like reading and writing in a bunch of different formats, you can serialize it to arrow to read into another program, has all sorts of nice utilities to it. The thing is it's pretty slow it's definitely not as fast as the r1 um and in general it's just it's just not very performant it's a lot almost all of is written in pure python so polars is basically a total rewrite of of pandas data frames in rust and um it's still a python library but under the hood it's it's it's

all done in rust and it's insanely fast so it's uh really really cool if any of you out there are using pandas my guess is just you know it's probably not you know totally api compliant so you're probably going to get bugs or let's say differences uh you know when you port over but but with a little bit of work you can uh you can uh you can use polars uh so polars by itself is just kind of pure rust so it's rust code but there is a like a python wrapper around it and there's a whole bunch of data showing how insanely fast it is this is interesting i guess here it says they're using the apache arrow stuff in parquet which is pretty cool so that i don't think we've ever really like gone over that but that's the

columnar sort of format so most structures in python or sort of c++ are what you would call i guess like an array of structures so if you had like your data as represented as you know has a time stamp and a label and uh whatever activity happened in enum whatever right and then you would have an array and the in-memory layout of that array is like bool int double, bool int double, bool int double, where each one of those is one of the instances of that structure in an array. And you get this very flat data model. The problem is if you actually want to query, like give me all of the doubles that are less than this value, that memory access kind of sucks because the computer pulls down like an entire contiguous block of memory, a row. Oh, row is ambiguous here.

An entire contiguous set of memory. And then it has to jump across and you're only using one third. It's not, but anyways, using one fraction of the data as you march across, right? So instead of this columnar format parquet arrow, and it sounds like this Rust library is trying to take advantage of that, is instead if it's a structure of arrays where you say, or an array of arrays, where I have all my bools together, all the doubles together, and if you're going to do these kinds of queries,

the advantage is their index match. So index 7 across each of the arrays represents one instance of your structure. But you can scan the array by grabbing those blocks of data down and going through them, you know, depending on how large your structure is significantly faster, which the pandas data frames is exactly trying to do. Like, you're often writing these queries, which is the cool part, saying, hey, give me the give me the items whose column this is equal to that and this column is less than that. And you're doing these almost sort of somewhere between SQL joins and Excel spreadsheet to come up with these very powerful things.

But like Jason mentioned, the underlying data access can be a gating factor there. Yeah, totally. Yeah, you hit the nail on the head, I think. Yeah, I think it's so natural to say like oh i have you know a list of users so let me just use an array of user objects but then as you said you get killed on performance as soon as you want to do some kind of analytics and so um this is a really nice way to complement that with another data structure that can handle all

of that and and uh, we've recently, that work switched over to Polar as we've been really happy with it. Do you use it from in Python or you're doing stuff in Rust now? No, it's all Python. Ah, ooh. Python on top of Rust on top of C++. Oh, it just goes all the way down.

Okay. Well, you know, I don't know if Polar uses C++. It might just be Rust all the way down. Oh, it's just Rust. Okay. Yeah, I think so. But it it's uh yeah it's actually totally transparent you don't have to know anything about rust when you pip install i guess maybe compile some code then i don't honestly even know how it works yet but uh yeah but it's like you just treat it like it's python code and i mean when's the last time you've ever needed to look at the pandas code to see what it's doing? Like never. It's like it just it just works or you go on Stack Overflow.

So so you could do that with Rust code just the same. Awesome. I think it's time for book of the show. My book of the show is Build by Tony Fadell. So Tony Fadell is maybe most famously known for two things. One is being one of the key designers and product managers of the iPod. And the second one is being CEO and founder of Nest, which is the company that makes a bunch of things now, but originally started with a thermostat. Do you have a Nest? I have a equivalent, but not Nest thermostat um do you have a nest i have a equivalent but not to nest thermostat i also have an equivalent but not nest i have an eco bee is that the same one you have yeah so you know what happened to me was my i talked to somebody oh so someone came to service our air conditioning and um i asked that that person you know, which one was better. And he said that Ecobee is more stable.

He gave me some reason at the time. So it's not an endorsement. You know, I just I literally just asked the HVAC person. That's what he told me. So so take that with a grain of salt. But so I also went with Ecobee. But the whole like smart thermostat is amazing. If you don't have a smart thermostat i highly recommend it

um i i installed it myself it was pretty easy and uh you could do things that are pretty pretty nice like for example um our heater is pretty loud and so it's it's pretty close to bedroom and so we'll do is we'll get the we'll run the heater a little bit earlier so that the house gets like pretty warm and then it doesn't need to run for a little while so people can sleep they don't have to hear it and just stuff like that that you can program different schedules put you're out of town all this stuff it's really nice if you have an ecobee before we go back to your book b stat b-e-e-s-t-a-t.io it'll show you like the full analytics dump of your ecobee and it'll like sort of tell you how long it has to be on in order to heat your house or cool your house by a degree outside temperature just gives you like the full

graph breakdown of like all the stats underlying like your houses, your house's HVAC system. Whoa, that's amazing. That's super cool. Yeah, I'll have to check that out. So okay, so so so smart thermostat amazing. At the time, it was very hard to break into that market. People didn't think that ordinary people would want to replace their thermostat by themselves. It was very rare for a homeowner to replace their own thermostat. And he talks about kind of how they went about, you know, they include all the tools. So there's this like, I guess I haven't bought a Nest, but if you do, it comes with this fancy screwdriver. And, you know, like they've thought about all the different types of home thermostat setups you can have. And they've made it so that you can buy the Nest. One big thing he talks about in the book is making it so you can buy the Nest and not have to go back to the hardware store or even go to a hardware store.

You know, just go to Best Buy, you get the Nest and you have everything you need to replace your thermostat without needing any expertise or tools or anything. I thought that was clever. And so, yeah, the book is full of these clever kind of product management insights. I will say on the downside, I felt like the book was very prescriptive, which is not good when there's not a lot of research behind it. So, for example, if you read a book by Harvard Business Review, they'll be prescriptive. They'll say, oh, you should try and manage in this way, and you should follow these rules. But it's based on this huge data set that we curated. We interviewed a zillion managers, and we decided to do this. This book's pretty prescriptive. It tells

you a lot that Tony thinks that you should and shouldn't do, but there's no data backing any of that up. It's just his own anecdotal experience. And so take the prescriptions with a pinch of salt. But I felt like the stories were really interesting and I learned a lot from that. I wonder if there's some thought there

where like if you do something, if there's some edge to be had by acting in a certain way or running your business in a certain way, leaving sort of ethics in your own personal desire to do it, if it becomes prevalent enough to be captured and shown to work in a Harvard Business Review kind of paper, has it lost its edge? I guess I kind of almost see it as if everyone does the same thing, you lose the advantage of being a unique culture. So it's not that you're... If you have a culture that specific people want to come to and flock, but then everybody offers it, you have to compete a lot harder for it. So if you offer something almost counterculture in a way or counter the average,

you'll attract people. Now, I don't know if those people will be good or bad, but if you find one that attracts people like-minded, there's a lot more agreement, you know, a lot less, you know, friction between team members and you can kind of curate that and go. But if it becomes, everybody believes that's the right way to do business and you don't question it. And then like you have a lot more competition in this space.

So I think famously, I mean, I think Tony came up in a regime at Apple that like was very opinionated in how businesses should be run. So he saw it be successful, at least anecdotally, and went and did the same. And I mean, you can't argue that what he did wasn't, I somewhat say he got acquired. He, you know, the Nest products are in many people's homes today. So, uh, agree or disagree with them. I guess it did work. Yeah. I mean, you know, a good example of what you described is, um, I don't want to swear that the example has the, so there's, there's something called the S sandwich and you can guess what S is. And so Jack Welch from GE, in his book, he popularized the S sandwich.

And the idea is you have two slices of bread. You say something. So imagine you have a one-on-one where you have to tell someone that they're getting demoted. So they would come in. You would say something really nice. That's the first layer of bread. You would tell them they're getting demoted. That's would come in, you would say something really nice. That's the first layer of bread. You would tell them they're getting demoted. That's the meat of the sandwich. And then you would end with something really nice, like, you know, hey, you know, bonuses are bigger

this year or something. And so that's the S sandwich. And in his book, he says, oh, you know, he does that and so successful and he kind of butters people up, tells them the bad news, ends on a good note and everyone's so happy so so then in um the hard thing about hard things um they talked about how the the s sandwich just doesn't work on senior people because it's it's now like 10 15 years later and everyone's everyone just knows like if you start saying something nice it's the s sandwich or whatever people just become cynical and they were saying how like if it's a junior person like straight out of college you might try it but don't try it anymore so it's it's your point you have this sort of arms race of communication you know tactics and and so yeah if you just follow what somebody said

worked for them you know maybe 5 years ago, it might not work today. I feel like there's a whole, a whole side route here about Elon Musk's philosophy on running companies as well, I think, but I don't, we better, we better keep moving. Oh my gosh. Yeah. That would, yeah. We're running low on time here. Yeah. My book is age of myth. This is a nice fictional book about a fictional place involving elves and humans.

A fictional manager in the S sandwich. No, no. So Age of Myth is a fantasy book by Michael Sullivan. It is in the same universe as the Rhaera Chronicles, but set, I believe, several thousand years before. So it actually isn't a prequel in that there maybe, I guess, are some Easter eggs or something to the later Rhaera Chronicles. But if you haven't read them, you can read this book. It sort of sits as independent. And I'm about most of the way through the book now. It's pretty enjoyable,

even though I have read the Rhaera Chronicles, coming back and reading this, just exploring sort of interestingly some of the world that you in the other book is just already there, already taken place. And here you're sort of almost reading like the firsthand account of what ends up being history later on. This is kind of an interesting dichotomy, but also just as a book, it's a sort of interesting tension that's built up between elves which are somewhat treated as like these mythically never die gods by humans and humans who are kept separate and seen as like basically animals by the elves only it turns out none of that is true like the elves do live you know for basically forever but they can't be killed they can you know hurt themselves and die it's just they don't die of natural causes so uh you know

it's just humans kind of i don't want to spoil but basically like you end up with the tension there and it turns out humans aren't these animals that know nothing and are ultra primitive like they can communicate and work together uh and so it's just uh not the normal trope but not not the normal trope either so it's a a good, I guess, like comfort fantasy. So not a super hard read either. So are the elves and the humans, are they the pro and antagonists? Or is there something else that they're fighting? Not exactly.

I think it's sort of like there's tension within both sides. And they're like forced to interact in ways that leads to the sort of like there's tension within both sides and they're like interact, forced to interact in ways that leads to the, the sort of buildup in the story arc. So there's, there's a problem in sort of the internal politics of how the elves like subcultures relate to each other. And then some humans get tied up into it and the elves get pissed off and

want to go do something about it. But not all the elves agree on like sort of what should be done and so you end up in sort of a well that makes it sound terrible like you don't want to read it though like a political i think it's actually i'm really intrigued political battle of sorts or at least that's how i the part i read from it or it sounds kind of like a game of thrones type situation a little bit a little bit so you know elves have like a very high culture and you know they are concerned about sort of different stuff than humans. But humans also have towns with their own culture and, you know, who's going to be chief and it's decided by, you know, a fight and, you know, that doesn't always go the way you think it's going to go. And so, you know, there's kind of a big difference between how the two, but, you know, the differences are more same than different. Very cool. Yeah, I'm going to check that out. Cool. Yes.

So we can jump to tool of the show. My tool of the show is Neon. So earlier I had a tool of the show called, actually we had them on the show, PlanetScale, which is a MySQL database that runs out in the cloud. And one thing that's very different about PlanetScale is you're basically kind of sharing compute resources with other people. And so it drives the costs way down.

It's a totally serverless MySQL. You don't have to worry about the machine dying or anything like that, or even how many machines to provision, it just scales up for you. This is the same kind of thing, but with Postgres instead of MySQL. So it's a totally serverless Postgres database. And so what you can do, imagine you build a simple app or a website on your computer. You have a Postgres database locally and you connect to it, write some data. It's looking good. You can just set up a Neon account for free and you get a database and it'll just always be there and they'll handle all the uptime and everything. I just love this model.

I think for databases in particular, especially for side projects and things that aren't going to use a lot of compute, these things work really, really well. I love the idea of having a side project that's totally free, that I don't have to continue to pay for. And every time I get the check every month, I say, oh, this is a side project that I never finished, right? This is a great way to just get something off the ground, to get your idea off the ground

and be able to show it to anybody without paying any money. So, you know, and I think then if it does take off, if you have that slash dot effect, then you can always like buy servers and all of that later or just, you know, take advantage of the flexibility. Awesome. I think there's a lot of it's being made a lot easier to kind of to get on to these these new technologies and approaches

and i i'm enthused i need to get back into making side projects start making side projects nice uh mine is a is a game it's available on um on many platforms it's actually been out for a while seven billion humans we've talked before about uh this is a zachtronics game we talked before It's available on many platforms. It's actually been out for a while, 7 Billion Humans. We've talked before about, this is a Zachtronics game, we talked before about several of those games, including Human Resource Machine. I think so. Or Human Resource Corporation.

I think Machine. But basically, both of these are visual programming games where they have a very fun animated style. And in Human Resource Machine, you are controlling a person and trying to accomplish various tasks in the office. This time, you're actually controlling a whole bunch of humans and attempting to accomplish various tasks in the office. And at first, I was, I don't know, this just seems like a little bit more complicated, but then I got pretty into it.

And it's actually very interesting because they begin to introduce things you can with many humans, like how do you accomplish something without like reduplicating the same thing, AKA concurrency and what kind of concurrency controls they give you? Oh no, there's not gonna be mutexes and semaphores. It's like, well, I mean, they give their equivalence of them if you want to think about that way. But if you just want to play it as a game, it works as well. And I will

say that in both games, there are very brutal side challenges where you attempt to limit the number of instructions you use or to try to accomplish it fast. And I'm not playing with those because I attempted to do it on a few of them. And it's something like I tried to write a really optimal solution, very compact. And I got like 10 instructions and it'll pop up and it's like, you could do it in four. And it's like, what? No, what? And it's like, I'm clearly missing something.

Or you completed it in 70 seconds on average and you were supposed to do it in 30. And it's like, okay, I don't know. Maybe the answer is like hard coding an entire thing. I don't know. But it's kind of a fun game that's almost a choose-your-own-difficulty level. Although I will say, if you're a programmer, probably like, I think accomplishable i haven't finished seven billion humans yet but you know could take a lot of time may lose your interest

that's fair if you're not a programmer i could see the difficulty being uh i would i would love to find someone who's not a programmer who made it most or all the way uh through the game and sort of understand about their background because i think you could definitely engage with it and be fun and i think you could accomplish it but you would learn a lot of programming along the way if you did. Yeah, totally. Yeah, I played both of these. And the first one, the human resource machine, I was able to pretty much blitz through it, do all the achievements and just knock it out.

And it was like, there was a couple of them, like some of the, you know, use the smallest number of instructions on the very hardest levels. It actually took me hours to really think about like, oh, how the heck am I going to do this? Like six instructions. And so I really had to think about it. But I eventually got my way through it. The seven billion humans, I mean, some of those are just incredibly difficult. I think the problem is, and I still haven't, you know,

gotten like the achievements, all the achievements yet. I think the problem is, once you start introducing many people, then some of the things become statistical. Like, for example, there was one in particular, I'm thinking of where you had a bunch of people and you had to, I think it was just like add to a block or something. Like if the block is less than 10, then double it or something like that. But there were a bunch of blocks that all needed to be doubled. And to do it in the fewest number of instructions, you basically had to have all of these people do a random walk.

So all of them just moved randomly. And if they found a block, they doubled it. And just some of the people would fall into pits or get stuck in corners or whatever. That's fine. As long as like enough people, you know,

we're there to cover it. And so as a, as a, as an engineer, it makes you cringe because you're just watching all these random walks, but like, it's the only way to do it in three instructions or whatever it is and so uh yeah it's a very very different way of thinking yeah i think i did get

one of those i haven't finished somebody but i did one we were saying where i like just implemented as like everybody walks around randomly no bias no anything uh and then i sat there watching trying to finish the level i'm like okay this is going to work but i have no idea how long this is going to take and rather than just like walk away i just like recoded it to like bias that oh i know the goal is like on the lower half so bias the random to to you know try to go down first before going up which made it twice as fast and i was just optimizing for my ability to sit there and watch this stupid game play itself out so yeah it's a phenomenal game actually when people uh if people want to learn about what would what would you call that skill i guess i guess like algorithms

maybe or just yeah ways of solving problems quickly i'm not sure not not sure exactly what the skill is i mean it's not a particular language or anything but it's a great skill to build up all right i think it's time did we ask chat gpt to uh give us an or anything, but it's a great skill to build up. All right. I think it's time. Did we ask ChatGPT to give us an intro? I tried, but it just keeps saying we're at capacity. Only even the we're at capacity message uses ChatGP to tell me various ways of telling me they're at capacity. Oh, my gosh. I haven't tried using ChatGBT today, but I feel like a good way to do this is to start by diving into how it works and RLHF, Reinforcement Learning with Human Factors and all these things. So I can give kind of an overview of how ChatGBT works and how it came about. And then we can kind of dive into cool things we've seen,

kind of implications for society and all of that. So we have to start with understanding language models because that's kind of the foundation for all of this. So a language model, or it's also called a forward language model, is where you take some text maybe from wikipedia maybe you have text just random text from the internet and you mask part of it and try and predict it so this is we've talked about this in past shows this is called unsupervised or self-supervised learning so the idea idea is, you know, you might say the dog went to the blank.

You know, you might know, you might have a sentence which says a dog went to the park and you replace that with the dog went to the blank and you ask your machine learning model to fill in that blank. If it fills it in with park, that's great. And it's not a sort of case statement like that. It's a fluid thing. So the model is download from Wikipedia a whole bunch of sentences that were written by humans, but you can just use all of that human writing to train a model to finish people's sentences. And so you don't need any humans to create any special labeled data. In a sense, they already have just by

expressing themselves on on the internet so gpt is effectively a really large model there's a lot of complexity to it but you think of it as a large model where you give it part of a sentence and it will finish the the sentence for you and then And then you can run it again to say, based on the finished sentence, what would the start of the next sentence be? They can cross sentence boundaries. It could write paragraphs. It can cross paragraph boundaries.

And so it gets to the point where you say, given a few words to kick off a story story it'll just continue where you left off that's a language model is there a way and if it's too complicated or if i missed it that's fine but years and years ago you could see something similar to this part you're describing where like you start a sentence and you would get something else or even just ask someone to generate plausible code or plausible text and it looked way worse than than what comes out here and those were using which i guess i and maybe i miss it but the look across a corpus of wikipedia or whatever and change and say what word on average starts a sentence or what distribution starts a sentence what word comes next and build the

in either word the sort of markov model stuff, like you move from each transition, each transition. And at some point you get a stop, right? And then you start again. Is there a fundamental difference between what that was doing and like what the setup is as sort of you described? Yeah. Yeah. And you kind of alluded to it. So I'll explain what Markovian is. So Markovian is a process where at any given place in the process, you don't need to think about the past. So for example, imagine chess. I could walk up to a chess board where the game is half over and that was played by somebody else and I can resume their game and it's not a problem. I don't need to ask that person what they were thinking. I might help, but I could just start playing that game and it's fine.

That's an example of a Markovian process. Chess is a Markovian game, so you don't need the history. Now, for example, let's say Monopoly where you're hiding your money. That is non-Markovian because if i jump into a middle of a monopoly game kind of need to know like who's been spending their money otherwise it's going to be really difficult right so the new models are transformer based and so the way that works is they build up a memory over time it's kind kind of like playing Monopoly where you know at the very beginning everyone started with $1,500. And so I know that, okay, this person's been spending a lot

of money, therefore they must have less money now. And so similarly, the transformers will, the encoders will keep track of everything that's happened. And typically this would not work because you would get just overwhelmed with the past. And it's very hard to know what things in the past are actually important. So you would just throw everything into some soup and then you would hope over time that the things that were important would stay in the soup, and the things that weren't important would be overwritten

by things coming later, right? This became much easier with this thing called an attention mechanism. So the idea with attention mechanisms are when something goes into the memory, you actually provide a bias. You actually say like, I want this to go into the memory and it's really important. Or I want this to go into the memory and it's not very important. And there's even like a positional bias on top of that. And so, again, and let me say this bias, you're not putting this yourself. Like you're not saying this word is important as a designer of the model, but you're sort of like making it very easy for the model to learn. This is important. This isn't important because you're

sort of giving that pathway. You're giving that lexicon and the model can can you know use that to to to learn attention versus having to to to do it otherwise is really hard uh so you don't influence it to say these like before you'd always hear about stop words or whatever words that just didn't mean mean much you don't teach it hey these words are low information english words you just say here's a mechanism for saying certain phrases or words are more important than others but that variable is left unset and then during training it attempts to figure out the values to set that that parameter to yeah exactly right exactly right and so right and so that was a big deal. There was a paper that came out from Facebook called Attention is All You Need, where effectively they showed that this attention part of the

network was by far the most important. You'll see this a lot in academia where people just add things, add things, add things. They'll just continue to build, remix each other's work. And then at some point, no one's really clear what is actually causing this to be successful. And so the attention is all you need paper is really interesting where they tore everything down and they said, oh yeah, attention is actually by far the most important thing. Everything else is either less important or completely unimportant. That allowed people to refocus on attention. You started getting these stacked attention layers and all these other things. And that caused this huge explosion in accuracy and all of that. There's some other tricks. So another really cool trick is, you know, the model will output a probability. So it'll say, oh, you know, the next word could be dog,

it could be cat, it could be fence, could be horse, like all of these are pretty likely. So let's say you pick dog. So now you try and generate the next word, and there's going to be a set of probabilities for that. And so you could almost think of this like a game tree, right? Like you're kind of searching. And so what you really want is not just the most probable word at every step, that's greedy, right? What you really want is a string of words where the sum or maybe the product of those probabilities is really high. So even though dog might be a really probable word, if you say dog, then like you can't really find another word afterwards that makes the sentence make sense. So you've kind of painted yourself in a corner. And so they actually, not only did they start doing beam search through all these probabilities, they actually created a differentiable beam search. So in other words, the neural network will actually get better at searching over time.

And so you don't have to exhaustively search the whole space. So there's all these tricks. And when you layer them all together, you end up with something like extraordinarily powerful. And so GPT-3 is basically that forward language model joined with a query language model. So they have a language model for things GPT-3 can say. They have a language model for queries that a person could type in. And they join them together so that, you know, you type in your query, like, you know, write an article about, you know, interest rates or something. And so GPT-3 has seen a million queries.

It embeds that query you made into a certain space. And then it uses that space to start generating content. That sounds a lot like the front half of how I've heard DALI described, though. It's similar, right? Yeah. So DALI, you have the query text, just like you have with GPT, except instead of a text generation, you have an image generation. You have a variational autoencoder. But you're right.

Yeah, the first part is exactly the same. Interesting. And then do they... Oh, okay. So they're taking building blocks, each of these building blocks, and then they're able to sort of like iterate by rolling forward in sort of a fashion by gluing these things together. Yep, yep. And so now the chat GPT, the thing they did there that is really interesting is this idea called reinforcement learning with human factors or RLHF. The intuition behind this is, you know, a GPT result is good if someone says it's good.

That's the premise. So what they'll do is they'll, then now this is where human raters come into play. They'll take like five results for the same query from GPT-3. So, you ask GPT-3, write an article about interest rates, you do that five times. All of these models are variational, which means that in addition to passing in the query, you also pass in a random set of numbers. And so every time you make a query, you're going to pass in a different random set of numbers, and you're going to get a different answer for that query. So they take five answers for the same query, they give it to a human, and they ask the human

to rank the five answers based on their own personal preference. And so then they do something, this is going to get a little bit technical, but they do something called log gumbel or Frechet regression. So the idea is you have a ranked set. And so you don't know how much better the first one is than the second could be a little bit better it could be a lot better all you know is the pairwise differences like you know one is better than two two is better than three you don't know by how much and so if you do something simple like say i'm going to give five points number one four points number two etc you're going to get burned because you're assuming that that number one is i guess 20 better than number two, et cetera, you're going to get burned because you're assuming that that number one is,

I guess, 20% better than number two. And that assumption is not right. So what you do instead is there's this trick where, yeah, it's gets super technical, but basically you want to look at like, can I recreate the whole ordering? Like like can i train a model which given these five paragraphs returns you know three five four one two you know like returns the ordering that the human returns and so that's a chain of decisions and so there's there's this trick where the log gumball distribution you know from from we talked about this in the stats episode. If you take two like normally distributed numbers and you add them, the result is also normally distributed. And so that's a really nice property because you're not you don't have to add any information.

It's like I have these two normally distributed numbers. You have a mean any information. It's like I have these two normally distributed numbers. You have a mean of three. I add them together. I get another normally distributed number with a mean of six. And so I don't have to keep track of what I did in the past. It's always like just another normal distribution, right? So what we figured out, we were at Facebook, we were trying to find something that could do that, but for multiplication. Is there a distribution where if I just multiply it by itself, by the same distribution with different parameters, that I get itself? Is there a distribution where I can do that? And there is one that looks close to the normal distribution.

Not perfect, but it's pretty close. And that's this log Gumball distribution. And so we took advantage of this trick where if I have a decision to make, like should I rank this item 1, 2, 3, 4, 5? And then based on that, should I rank the next item like 1, 2, 3, 4? And so you're chaining together all these decisions. You're multiplying all these decisions. And it's really convenient if at the end you just get one distribution.

So log gumball is the sort of trick that lets us do that. So yeah, all of that to say there's a nice way to learn from those ranks that came from those people. So you give a million of these, you know, five tuples of answers to people on like Amazon, Mechanical Turk and these other rating facilities. They go in and rank them for you. And then you train a model off that. And so now what you get is the answers are heavily heavily biased towards things that people would want to read and that ends up having like a huge impact on the quality so it sounds like from what you said i know after dolly which i guess like isn't available for just

everyone's use but the stable diffusion like the similar approach people kind of take, there was the like sort of guided version that would come out where like it would give you something. And I, from listening to what you said, it's like I gave this random set of numbers and then made these choices, probabilities, whatever along the way. And if you say I want something more like this one, like take this, or I guess you can even do that in Dahlia, like this picture, like choose this one and I want something close to this but a little bit different and it sounds like the same mechanisms would work here so we could see chat gbt where like if you were trying to curate a story

and it wrote you one you could ask for one that's really similar and it could produce you something really similar as opposed to just something else random uh that had nothing to do with your first one yeah that's right and uh yeah i'm not sure at what point the chat gpt part comes in like like i'm not sure if they are seeding the random number or if they're actually retraining the entire model based on uh you know the people's preference They could go either way there. But yeah, that's basically how it works. And so the cool thing, actually, we'll finish the technical part with this. The novelty here is reinforcement learning is very, very slow.

It requires a lot of data to do anything. This is why it's really good for things like Go, where the machines can play each other without any human in the loop or anything. But it really struggles in things where there's... You wouldn't use it for protein folding unless you could... Well, actually, you could do that in simulation. What's a good example? If you had to do some chemical test, where a human actually in a lab coat had to do a test you wouldn't use

reinforcement learning for that because it would just take uh too too much time and it's we call this sample inefficient the trick that they used is you know they only used reinforcement learning at the very end when they already had something that was pretty good and so because of that um they were able to get a lot of bang out of every example um and that was a really interesting novelty oh cool so yeah i think uh that's that's basically how it works kind of in a nutshell and uh people have been using it for all sorts of of wild things have you what's the craziest use you've seen, Patrick? Or the coolest, maybe. Yeah, I mean, I've seen a variety.

I think the one that everyone keeps saying is sort of spam generation. That's the one that I keep seeing, which is like... Have you gotten any chat GPT spam? I don't know. I mean, I guess that's the debate, right? It's like, would we know or not? But that's the one that I see where it's like, someone needs to generate a lot of variations, right?

Which is what spam traditionally wasn't. Which is, hey, I'm a whatever international prince who's had a lot of money and gonna send it to you. You just send the same thing to a million people. Now you send the same idea to a million people, but each one is different. You can't write filter words to try to keep it in your spam inbox spam folder yeah i'm surprised that hasn't

happened yet or if it is gmail is still figuring out how to block it i think maybe people aren't able to generate like them in bulk in the way they would need to to do it quite yet oh right that's a good point yeah one of the most interesting i saw were people using it to cheat on on tests on like uh oh not tests but like on english homework like write an essay about picasso and they would just tell chat gpt to do it i you know i saw a lot of that and i you guess it has ethical implications, but I wonder, though, if that's the only thing you ever had to write, fine. But most students, I feel like you would know pretty well that the student used to write in this voice. It would be very difficult for ChatGPU in its current state to generate a book report that looks like it's in your voice yeah that's a

good point you know like write me a letter c a graded c paper because i'm a c student like i maybe i can't get it it's blocked right now but like you know i i'm not sure that that seems a bit hard but i do see this as like a big concern that people will just generate at least an input for them to start with, right? Rather than doing the research. Yeah, I mean, I would imagine like what could make this really powerful is if you were writing, let's say, a marketing email or a sales email and you wanted ChatGPT to write variants of what you wrote. But then, you know, if when I think about it, it's like, what's the real value in that? You know, because you're sending it

to two different people anyways, or maybe actually, oh, here's what it would be. It would be amazing if you wrote like, hey, I wanted to introduce our new product. It's called, you know, FUBAR security. FUBAR security is so much better

than like FOO security has these five things. So you write that. Then you give chat GPT like information about your clients. And then it like rewrote it for each client, which is like, yeah, like, hey, you know, your daughter would really love this this because like, you know, it's like you knew that this person had a daughter. So like that could be wild. I think the cool use cases I've seen, and I think they're kind of similar to what I would say for the like, if you're going to turn in a term paper, you probably need to edit and curate down what it says, which I would argue if you can curate and edit down to like what's supposed to be, like, you probably could have done it in the first place had you really wanted. And so what was the point of the exercise other than to just like, consume an amount of time? Like, it's the that's the anyways, I'm gonna get outside. But yeah, so if you're able to like curate, edit,

combine and produce a paper faster, like that really a problem like you're demonstrating like understanding of the subject and the similar one i would say is we we in a holiday episode we got brought up but like a lot of people stack overflow right you'll just go ask chat gpt instead what's wrong with my code or how do i make this and the few that i've seen go through that it it's eerily how close it gets but it normally also doesn't work quite yet so like if you ask it how do I do you know like I want to run an edge detection filter on a png in python and it's like before that would have been like a sequence of stack overflow queries now you're going to get something that's 80 of the way there for someone who would

know how to do it given enough time. And so they can take the chat GPT output and they now have probably like, oh, there's this open CV. What is open CV? Oh, it's, oh, okay. And then, you know, oh, it's this call and this is it. And you may need to swap some lines around,

but you have the like, to the, you know, initial approximation, you have the structure, the steps, the libraries, even function calls. But it probably doesn't do exactly what you asked it for either underspecified or it just didn't get it quite right. But you're a lot way further along for someone who could sit down and write it, but just may not be familiar in that language. Yeah, that makes sense. A lot of people are saying that it's a big threat to google search i don't quite understand that do you do you have a better handle on that like what's the connection between

this generative ai and google search i think it's the a lot of people go to so a lot of people go to google and type in facebook right uh so they use it as like a navigation i don't think that's a danger i don't think that's a danger. I don't think if you type Facebook to chat GPT, you get the response. Right. Yeah. Or maybe you do.

But that's not interesting. I think the threat there is what Google tried to already do by its sort of knowledge bar on the side where they give you answers to the questions without you going to the website, right? Oh, I see. If you type in what movies has has i don't know what's an actor's name leonardo dicaprio yeah sure there we go thank you leonardo dicaprio what movies has leonardo dicaprio been in or how much is he worth or whatever right and all he's doing is going and scraping the first few pages and like showing you the movies but it doesn't actually go to those pages right so first of all the question is like what was in it for those websites that google got that information

from them but now google doesn't send them to those websites problem number one problem two is like now chat gpt it is more easy for a competitor to stand up against google for that kind of question and answer oh that makes sense makes sense. Can you ask, what's like chat GPT's fact corpus like? Can you ask it people's net worth? Will it be accurate? The second part is harder. So it probably understands the United Cup as an actor, and they probably, plausible actor ranges of salary or this, that it gives you exactly the right number when it's already not a great number is interesting. I guess that'll lead to something that I did see,

which is several articles, including from the Wolfram Alpha team themselves, but saying that, hey, ChatGPT, you should look at like hooking up what you're doing to Wolfram Alpha because Wolfram Alpha says, let's take a bunch of like hard facts and try to make them easily

queryable by roughly human language. So things like math, things like time zones, population statistics. Net worths may or may not be in there. I don't know. But things like that, which are just facts discernible from authoritative sources, compile them and make them accessible. If chat GPT sort of like, understood it needed this fact, and it could go and get it authoritatively from Wolfram Alpha and then feed it back into its output, right? It's like this sidecar thing. I don't

know how that works in the training and the like how all that would go. But it's this sort of almost database storage of facts that they could use. And there was on Wolfram Alpha site, they were sort of saying how like it would merit. Maybe that's in response because they were worried from the inevitable. Why do we need Wolfram Alpha anymore? Why do we need Google anymore? And they were heading it off. But I did think that was an interesting pairing that could happen. Yeah, you know, this is really fascinating.

Like if you, this is tricky, you could for example like you could have let's say you had humans go in and every time there was a fact actually went in the corpus like they'd have to go through all of wikipedia if wikipedia is a corpus and for every fact replace it with a sequel query chat gpt would learn to like occasionally drop into SQL and then drop back out. That would be pretty freaking awesome. And it's totally doable. I think, uh,

yeah, that might be what we see next. Oh, now we're giving them ideas. I just don't know. Uh, you forgot to scream trademark at the end. I think the hard part is,

uh, is, is the curation part. So you would have to, you know to either write algorithms or by hand, you'd have to replace all of those pieces of data with queries to retrieve it. Yeah, I wonder if there's some way to do it more directly in isolation. Like, say, teach a portion of the system, hey, these kinds of things

are answerable here. And then like either, like you were saying, it's like a follow-up step. The earlier stage says, hey, insert token that represents, I know this is a fact, like number of dollars, right?

Number of dollars. And then the second part of the system can understand, okay, it's looking for number of dollars. I see here salary then the second part of the system can understand okay it's looking for number of dollars i see here salary and i see a actor's name therefore like i need to hit this system to fill in the blank of that token based on the context right right right it seems totally doable you know you could even automate a lot of this like you could you could for example net worth like you could go in by hand of this. Like you could, for example, net worth. Like you could go in by hand and say,

every time you see someone's net worth, replace the dollar amount with like some special token that means a SQL query to this database. Oh, and then start training the model to do the replacing itself and then get that accuracy up and go from there. Oh, another thing I should mention is one thing goes under rated, but I know from from I do know some folks at OpenAI and they said a ton of time

is spent curating the data. So, you know, there's a lot of bad data. There's things which confuse the algorithm. I mean, you know, people talk a lot about misinformation that's a little different that's where you know someone's intentionally like saying the wrong thing but they could be saying it really eloquently or whatever but this is just like you know there's parts of wikipedia which just like mess up your training and stuff like that and so that you know people think like like uh like oh you know eventually there'll be just like a million open source chat gpts but beyond the electricity costs and all of that of training the model the data curation just in terms of human labor hours is extraordinary and people don't really know that or they take it

for granted yeah i think that's the you know to transition i guess from uses into like looking forward i think that's one of those things where oh we're gonna you know it's here the matrix like we're all gonna get sucked into powering chat gpt by like whatever like i think the there is still a lot of human bias that gets introduced in like how you clean the data and where you do and a lot of human effort will that be forever no probably not but like i think there's a lot of selection even wikipedia style itself and being a big corpus like that that is more standardized than normal and then i think there's a lot that goes in into that um and then keeping it fresh. But I also think that they'll make progress on it.

It'll get better. What will be interesting to me is right now, you can go onto the internet, go onto Wikipedia, whatever, and pull this stuff in. You almost need a snapshot of Wikipedia before it comes out because now you can start to get into the problems we could see when you're talking about misinformation, which is if you're just generally doing a web crawl and looking for things, it's very

straightforward for people to say like, hey, someone posted an article saying Jason Gauci is actually an alien. And you're very frustrated about that. So you want other articles about you to appear. So now rather than you having to handwrite many articles, you take on many ghost pen names, whatever, and you write articles that flood it with similar or dismal information, so searches can't do that anymore. And Google would normally have a lot of tactics for filtering that, in part including the quality of the words. But now that part, that signal goes away, and they'll have to fall back to other signals but at the same time if then at some

future you know company or chat gpt themselves goes to the web and tries to consume data that was generated by their own tooling then there's this whole like hey you're what are you really getting you're getting information that you made up and now you're feeding it back in as true facts and so right right i think there's this interesting going forward. If someone trains a bot to spam Wikipedia with chat GPT trained Wikipedia articles, does that ruin the source? Wow, that is wild. You know, I've always like laughed at the, what's it called the singularity, you know, AI singularity and all of that but like but like you could approach a point where where the ai you know creates a bunch of content on the web consumes it learns to create

slightly different content and actually kind of like moves forward in some weird way oh another thing related to what you said um i noticed with chat gpt they've they've put in some hand-coded answers. Like, for example, if you touch on some really sensitive political topics or things that are just dangerous to their business that they don't really want to give answers for, it'll basically say things that were written by a human. And the way they've done that is they've just put hand-coded answers into the ranking system and then told to the human rankers or just forced the human rankers to always give that the number one score. And you do that enough times and now you have something that can stay away from really delicate topics. So I thought that was really interesting the way they did that.

Yeah, I think Open AI has come under some interest for, I think, Dolly the same way, like trying to avoid certain things, trying to maintain a modicum of ethics, I would guess, about this and sort of, you know, try to say, hey, look, we don't want this making fake news or you know denying the holocaust or saying these things that are are known but ultimately it's also sort of research that once people figured out how it's done it's expensive but people can redo it right people will make their own versions or you know we saw that with dolly like they'll do things and they won't they won't put those restrictions in so it's a interesting game for like whether or not this this is more useful as a tool than research or whether at some level it's being completely closed and no learning taking place, but you also don't have the ethical concerns. You end up in a hard decision space.

Right. Yep. It's a really good call out. I think it's really interesting the fact that this is all private, like in the private sector where people have to worry about investors and all of that. And so, uh, and so the, the restrictions are going to be a really interesting space um there's some like you know unicorns already in this in this space the unicorns a company worth a billion dollars um there's some companies that are worth a billion dollars and they're literally just chat gpt uh like in a chrome extension or target targeted to a specific market or something like that so you, you know, I think it's like anything where, you know, there'll be a lot going on here, a lot of energy,

and then we'll see sort of what rises to the top. But there's definitely like huge, huge value here. I mean, we have to, as a society, we have to figure out exactly where it belongs, but it's obvious that this is something really, really big. And the other thing that I wanted to highlight, it's not just for text and audio. I mean, look at that stable diffusion thing

I talked about at the beginning of the show. You know, you could do this for just about anything. It can cross different media. I mean, video is a natural extension, but even to other sort of latent spaces like robotics. You know, imagine like a chat GPT where you tell a robot to pick up an arm, sorry, pick up an arm, pick up a block, tell a robot arm to pick up a block, and it turns that text into robotic motions.

I mean, totally, totally plausible. So you tell robots like wave, like really vigorously or something. And it just knows what that means. That would be cool. I guess we should have made predictions for where we think this future is going. Yeah. What's your prediction? And then we'll call it a day. There will be a lot of articles by very angry teachers saying that like all their students do is ask chat GPT.

I know that's already happening. Oh, wait, hang on. My prediction is I actually think robotics man i think robotics is a place where this kind of technology could be really disruptive in a good way i i would say i think memes i think we're gonna see a flood of memes out of this and people are gonna say are these funny one thing we yeah that's true one thing we don't have yet is something that can replace text in an image like imagine if you had something where there's like a speed limit sign and or like a traffic sign and you could change the text to say like like speed limit infinity as a joke and it would look like it really

belonged there you know like it would replace it appropriately yeah. And it would look like it really belonged there, you know, like it would replace it appropriately. Yeah, I think that feels like a little bit of a natural extension, but saying like, let me take a video of something and using not like we're seeing a lot of text, I think, but I wonder, and you were saying audio, that seems naturally just, but like,

I wonder like either like signaling to chat GPT or one of these for outputs, like hand motions, like, but basically like video, right? Like either make a scene like this or some description of what you want to happen. Like, Hey,

I want you to make something this big, right? Like I'm holding my hands up. This is audio. It's a podcast. Size of a postcard, you know, I size of a postcard,

you know, I, you know, make an image where something's like this and you're just talking to it and using hand gestures. And then it's able to sort of like accomplish the same sorts of things yeah actually with that in mind you another thing that i would like to see i expect to see in the future is partition decisions like here's a good example like a scrapbook so so imagine if you could say like here's a bunch

of photos i don't need you generate a photo or anything but i want you to lay out a scrapbook so that's a set of like very discreet decisions that like ladder up to a scrapbook but like it's not fluid like text or image you know um like things that like layouts floor plans another example you know things where yeah there was oh plans. That would be good. There's also that, oh, that's, that'd be good. I bet it'll happen. If you recall when the iPhones, there was that iMovie and there was that trend for people making trailers

where like the beginning and then like input picture of yourself and then like, and they were having an adventure and like a little clip of, I feel like those kinds of things, ChachiB2 would be really good at. Some of the photo services try to do them, but they're always kind of hokey.

They're like, hey, I had my, can you take all the pictures from my vacation and like make a highlight reel set to music in the style of Evel Knievel? And then it's like, you know, here's whatever, I don't know. Yeah, that'd be freaking cool. Also like if it could do vector images. So like, again, it's like partition based decisions. Like, can you compose, you know, circles and crescents and lines to make like a vector logo? That's really cool. That would be awesome.

All right. Well, thank you, everyone, for listening in to another episode? Yeah, thanks, everybody. A lot of folks have emailed us and chatted to us in the Discord about ChatGPT. So this is for you all. And yeah, I thought this was a great time. Thanks again for sponsoring the show and being a patron if you're one of our patrons on Patreon.

We sent out the t-shirts from the holiday episode. Folks were really excited to get those shirts. So I'm glad you all like that. Congratulations to our t-shirts from the holiday episode folks were really excited to uh to get those shirts so i'm glad y'all like that um congratulations to our t-shirt winners this year and uh we'll catch y'all in a couple weeks music by eric barn dollar Music by Eric Barndollar Programming Throwdown is distributed under a Creative Commons Attribution Sharealike 2.0 license.

You're free to share, copy, distribute, transmit the work, to remix, adapt the work, but you must provide attribution to Patrick and I and sharealike in kind.