Python Bytes - #385 RESTing on Postgres

Episode Date: May 27, 2024

Topics covered in this episode: PostgresREST How Python Asyncio Works: Recreating it from Scratch Bend The Smartest Way to Learn Python Regular Expressions Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/385

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 385, recorded May 27th, 2024. Did I get that right? Yeah. I am Brian Ocken. And I am Michael Kennedy. This week's episode is sponsored by Mailtrap. More about them later. If you want to connect with us, we're all on Fostanon, the show, and Michael and I. We're Fostadonians.

Fostadonians, yeah. So those links are on the show notes. Also in the show notes or at pythonbytes.fm, you can go to live if you want to join us, join the audience. Usually it's Tuesdays at 10 a.m. Pacific time. You can also use that link to get older videos too if you want to watch what happens live.

And finally, if you'd like to receive an email after a day or two after the show to find out all of the links and to have a link to the show. But make sure if you listen to this and you want to make sure that you don't miss any links and be able to go check those out later. You can become a friend of our show and we'll send you those emails. So you had an announcement before we get started. I do. Well, I just wanted to say, Brian, the reason that we're not just doing this at like a normal time, we're just kind of late. And I got some messages from folks in places like New Zealand, like, awesome. This is the perfect time. So there's always a perfect time somewhere in the world. But you've got this thing going on tomorrow. And then for the rest of the week, I'm going to this thing called the giant loop rides, this off-road rally that's at a hot Springs and maybe 300 other off-road riders out

in the Southeast desert of Oregon and riding the mountains and the desert and open bar, live music, all sorts of shenanigans going on. Anyway, if somebody- Yeah, that swimming pool is a hot springs, by the way. How crazy is that? If anyone is listening out there and is also going, just come say hi. That's all I'm saying. But also why we're doing this in the middle of the night for some places. So I would say if you have a podcast that kind of runs late,

sometimes it'll make you tired, wouldn't you think? Would you have to rest a little? Yeah, yeah. Let's talk about PostGrest, PostGrest, as in Postgres Rest API. So this one comes to us from Mark Little. So thanks, Mark, for sending this over. It's a good one.

This is basically a web server, web front end for your database. So you've got Postgres, and you might think, all right, well, I would just like to expose the tables as endpoints with ability to do queries over HTTP endpoints. You somehow pass the query over. You want to have security, things like that. So this is an app that you can run

and it automatically with zero API work for you turns your Postgres instance into a somewhat restricted API basically. Cool, right? Yeah. Yeah. Now, obviously these things, I don't know, I always have mixed feelings when I talk about these things.

On one hand, if you just say, I would like, say my view front end or something, or even my Python code to just have access to the database as an API just read you know the crud things of all the tables you're good to go this is what you want right on the other hand sometimes I want to have more specific things instead of just here's the user table I want you to go through the user the create new user API endpoint that uses I don't know arggon2 for memory hard password hashing and other checks. And maybe, you know, like every time I come up against these things, I'm like, oh,

they're really cool. But at the same time, I kind of want a little more specific control over how it works. So if you're just like, I just have a database. I just want to be able to expose it real quick. Let's go Postgres rest sort of thing. Cool, right? So the way it works is it's written in Haskell,

and it's 4% Python, so I guess there's a little Python in there. But it doesn't really matter what it's written in. It's just a thing you run, and then you talk to it from your Python or from your JavaScript or whatever it is, okay? So there's a couple of things you care about. One, you probably care about performance. I'm not super familiar with the performance of Haskell,

but I believe it runs on top of the JVM. Sorry if I get that wrong, but I think so, which makes it pretty quick. It says it'll do 2,000 requests per second on the Heroku free tier, which is pretty fast, actually. That's a pretty wimpy server. And 2,000 requests a second is pretty high-end. Three reasons for that. Haskell, and it's written used in the warp http server a compile link

compiled language with lightweight threads i also learned about warp so i'm learning about all sorts of things it delegates as much as it can to the database so it's like a really thin layer over the database as i was describing so serializing j spawnspawn, JSON responds directly in SQL. It does the data validation, the authorization, combining row count and retrieval, data post in a single command. All those different things are happening in the database, not in this thing, right? And finally, it uses the Haskell instead of SQL or SQL. It's the Haskell library, which manages database connections, a binary protocol for talking to it,

and is stateless for horizontal scaling. Also, very important for security. You don't want to just go, and the whole thing is read-write on the internet. How could little Bobby Tables doesn't find it? So it uses JSON web tokens. And basically what you do is you create a user in the database and then you set authentication

like this user can only read and they can only read these tables, but they can write to that table. I don't know. And then that's what you can do when you authenticate over this, right? So it basically leverages database users and their implicit permissions

to control what you can do with the API. There's ways to version the API, which is kind of interesting. As the database changes, you could keep it stable, it seems like. And it documents itself through OpenAPI. Well, this thing's pretty neat. So if you have a Postgres database and you're like, look, we really just need rewrite, update. It's maybe even an internal app. You just want to have it real simple. Check this out.

It's kind of cool. Cool. I didn't know that Heroku had a free tier anymore. This may... Let's check out. Let's go back and do a little... It may have been from previous times. Yeah, that statement, it must be from previous times because this was updated

a month ago, but it didn't. Yeah, I think that's just whatever the Wimpy free tier was, that's giving you a sense of what you can do. Okay, cool. Neat. I want to talk about asyncIO a little bit. So there's an article called from Jacob Padilla, how Python asyncIO works, recreating it from scratch.

And I kind of like this idea because, I mean, I love async IO, but sometimes it just feels magical. And to sort of walk through some of the, how it might be working in the background is kind of cool. So it goes through generators review, talks about the event loop, talks about sleeping, then yielding to await and await with asyncIO. And I just kind of like the format that this is written in to just sort of get a good mental model of how asyncIO works. So this is, I guess I highly recommend this article. Looked really fun.

The colors are nice too. I, you know, I got a sucker for a good webpage that's easy to read, but this is good. I skimmed it briefly already and like, yeah, this kind of goes through

how a way async works. Async just sort of feels too magical sometimes, but. It is pretty magical. Andync just sort of feels too magical sometimes. It is pretty magical. And at the heart of it is generators, right? So generators themselves feel super magical. And once you kind of understand how those work, you understand, well, okay, well,

that's how you kind of partition the work and then you just run it in steps, each fragment of the generator and off you go. But yeah, it seems pretty wild. Yeah, so it kind of walks through all the little steps and then it shows what the so it has like the the final bit and then it rewrites the whole thing in actual async io so that we can see what it looks like um so that's great uh along with that when i noticed the link that some there was um camera where i got it from there was a comment also about if you want to like learn async io from trying to recreate it from scratch there was also a uh david beasley talk from the from i think it's from pycon yeah

pycon india from 2019 i hadn't watched this i still haven't watched this but it's on my list now um build your own async uh um from david beasley instructor. So I'm sure this is a good video as well. So we'll link, we'll link to that. So some, some learning async IO. Awesome. Good stuff.

Indeed. What, what else is good is our sponsor MailTrap. All right, let's talk about them. This episode is sponsored by MailTrap, an email delivery platform that developers love and email sending solution with industry best analytics,

SNTP, and email API, as well as SDKs for major programming languages and 24-7 human support. Try for free at MailTrap.io. Thank you. Thank you, MailTrap. Let's move on to something a little higher order, shall we, Brian? Sure. Perfect follow-on from what you were talking about. AsyncIO parallel programming, it basically scales the weighting. Now, this is going to be a dated statement, I believe. But at the moment, Python is not super good at doing computational parallelism, right? But you know what is? GPUs. GPUs are awesome at doing parallel programming. So, you know, your typical Python program has one thread, or if you don't do anything in most languages, they have one thread. If you scale out your CPUs, maybe you got 16. My Alienware gaming PC has that. My M2 MacBook Pro, I think, has 10 threads, you know, but there's way more. It's get a lot more work done if you

scale those out, right? But if you scale the GPUs, they have 16,384 threads, things like that, right? Incredible. So I ran across this thing called Bend, B-E-N-D. And it comes from Higher Order Company, which has a couple of, it's like a runtime for Bend, and then has Bend the language. And it's all about taking things and running them in parallel and making them basically be Python.

So it says with bend, you can write parallel code for multi-core CPUs and GPUs without being a C or CUDA, respectively, expert with 10 years experience. And it feels just like Python. So they give us this example. Here's a function called sum. Not the best example because it shadows a built-in,

but whatever. Let's get rolling with it, right? And so it says def sum, take a def, and there's some value that it starts with and then adds them all up, right? So sum all the numbers from zero till two to the n and to the depth, right, as a switch statement that once it gets to the end, stops, and then sort of recurses down as it would. And there's nothing that looks parallel about this whatsoever. But what happens is this code actually runs in parallel potentially on those thousands of GPU cores.

Oh, wow. Wild, right? Yeah. So the way it does that is, there's a picture here, is it breaks these things up into what are called computational graphs instead of work. And then it just goes and processes them to see if that part of the graph can be parallelized, which is kind of nuts. So you don't even write the parallel code. It just looks at your code and goes, oh, that loop that we just saw

or that recursion type of thing we just saw, that actually could be done recursively by just changing the parameters or something, right? So pretty wild. And so, yeah, they have this HVM, highly massively parallel runtime that achieves near ideal speedups as a function across the available cores.

And then Ben is the programming language that runs in parallel, powered by this thing. Looks like Python. I don't know how compatible, say, its standard library is or things like that. Yeah. And then you have this thing called Kind,

which is a parallel proof checker. I have no idea about that. I just said it's coming soon. So this is available on GitHub. And you come over and you can find their language. And guess what? It has 15,000 stars. I'd never even heard of it

until recently. It's pretty popular. Written in Rust, not that that's super relevant, but it's like powers a language that looks like Python. That's pretty cool. I'm cautiously optimistic about that.

It could be good. So it looks like the real, you know, the real implementation for it is here, although it looks pretty short. I don't know. Updated a couple days ago. But here it says, you know.

I don't know. We'll have to see. How large of a thing is it? I guess that's just some pretty big scroll bars. So maybe the whole runtime is here. We'll see. Maybe the number of files might be lower, but could have tons of stuff in there.

Yeah, that's true. Anyway, is it going to be a thing? I'm not sure. Maybe, maybe not. But it's cool to see people being creative with this kind of stuff, you know? Yeah, actually sounds pretty cool. Yeah, it does.

And to, I guess, simplify the way you get your code ready for massive parallel running, if it can just look at your code and go, oh yeah, I got it. Yeah, if it can look and say, this is what's often referred to as embarrassingly parallel, right? There's like no state shared.

You could just take the pieces and break them up and run them on cores. Yeah. If it just finds those automatically and does that, which is what it claims to do. Yeah, that's great. That's really great, yeah.

Really neat. All right. Next up, i've got uh regular expressions so we're getting all sorts of uh deep topics today we are we sure are so um i ran across i was looking at reddit um and so this isn't is this book is an older book about a year i think it's a year old um. It's called the smartest way to learn Python regular expressions. And I don't normally pitch other people's books or anything on the show, but I thought it was kind of nice that it's a lean pub thing.

So it's there, they're doing their own thing. And I guess I'll get into why I think this kind of neat, but the minimum price is free. So if, and since they're allowing that, I encourage everybody to throw a few bucks their way if they if they can because why not it's good to have people teaching things um anyway ran across it on reddit um uh and uh the topics sound neat so they're uh they go through an introduction of regular expressions they do an application i talk about puzzle learning and stuff, and then I skim the rest. But it looks like great stuff.

And some Python-related thing to teach regular expressions would be good. And then I went and watched their video, and now I am even more intrigued. So the idea around this book is that modern ways to learn are different than possibly people my age might be used to, just reading a book and trying things out. But this is set up. There's a little video introduction to the book. They're using the book itself as a long form teaching people. But it's also there's a video course attached to it. So if you get the book and you're reading through it, there's links to the different course chapters within the book, which is kind of a neat idea. And then there's puzzle solvers for each thing you're learning.

So you can go apply it in a puzzle environment. And they're using a thing called Finxter. It's an awful name, actually, I think, but F-I-N-X-T-E-R. But it looks like there's a whole bunch of other people using it, too. And I've never tried it. But it's this online trying out some code platform, I think. So I'm going to try this out. I'm curious about the whole idea of reading a book and clicking on it and doing that as well. Yeah. That's, you know, mileage may vary for me because I'm probably going to download this onto a Kindle and I can't click on anything with the Kindle,

but at least the one I'm using. But anyway, it might be fun to go through. Yeah. Looks excellent. Give us a report when you get back. Sure. See how it went. Yeah.

And, you know, not enough regular expression material out there for just deep, diving deep and really getting your head around it. Yeah, they're very powerful if you're good at them. Yeah. Big if. Those are our items. Do we have any extras?

I have one that's quite interesting. And I don't, somebody gave me a bit of a tip towards this i don't remember who it was so i think it was a macedon but thank you and uh sorry about forgetting who to credit here on wikipedia we have the python conference aka pycon and what does it say about it says the python conference also called pycon is the largest annual convention for the discussion and promotion of the Python language. Scroll down a little bit, Brian, you see things like, where was it in 2003? It was

in DC, had 200 attendees. 2012 was in Santa Clara, 2,300. Oh, 2016. Yeah, baby, Portland, 3,391. 391 almost almost the record almost 3 393 two more people uh beat that number uh came over that number in the from cleveland ohio in the before days and then covid hit knocked it down and it's slowly building back like in pittsburgh it was up to 2500 again so what is that it's uh still got about 50 more to add or a third looking backwards uh it's down that's not what i want to talk about i want to show you what is in this wikipedia table right below it where we're going next the long beach baby going Going to California. Let's go. Going to the beach, going to some warm weather, going to a vacation spot. I will bet you that this significantly beats those numbers

because it's just a destination, right? People want to just go there to be there, right? Well, it's kind of nice to see it on the West Coast. I mean, nothing against Pittsburgh, but it's only two hours away from, where was it? Cleveland. Yeah. Yeah. Yeah.

So we did have Salt Lake City in the intermission times, but Salt Lake City was still under a lot of funk. It was the first one that came back in person after COVID. So it was kind of funky. I went to the 2023 one. I know that, and I guess I'm going to go on the record of saying, I think that like, there's a lot more cool destination things, destination places on the coasts and Midwest,

obviously, but, but I think doing it kind of in the middle of the country makes sense so that people can get easy flights from anywhere in the country. Just saying. Yeah. What are you optimizing for right are you are you trying to move it around are you trying to make it as centrally located are you

trying to make it as cheap as possible you're trying to make it a destination um all these different things play into this and this is the biggest funding for the psf so it's it's not just like a curiosity right it this is the funding for the psf to a significant degree it's interesting though that portland was way back in 2017 and this in 2026 is going to be the next time that it's going to be on the west coast i don't think that i know that everybody on the east coast thinks that salt lake is in the west but it's not really not even the same time zone brian not even the same time zone, Brian. Not even the same time zone. Mountain time, Pacific time. Let's go. Okay.

Anyway, this is my extra. Okay. Planned tentatively the PyCon after Pittsburgh. So next year is still Pittsburgh. It's round two. But then Long Beach, California. And I'm here for it, Brian. Also, I've always been curious about,

sometimes I'm curious about the numbers of like, how many people showed up in Montreal? And it's cool to see. You can just look it up on Wikipedia. Yeah, yeah. It's all right here. I've got something completely random. Prince of Persia was a video game from 89.

I remember it. Oh, not only was it a video game, it was one of the best video games. It was so good. Yeah. it was oh not only was it a video game it was one of the best video games it was so good yeah well it was like this one of these cool like uh you know racer through all sorts of stuff we had like the little runner thing and had to jump over all sorts of stuff it was a fun game uh and then there was a bunch of a bunch of different um uh permutations of it and it's still going strong i think um still stuff going on there's's a... Anyway, a cool game. But I ran across this...

There's a book about it. So Jordan Mechner is the person that wrote it and originally wrote the Prince of Persia game. And he wrote a book called Replay. It's a memoir of an uprooted family. And apparently it goes back for 1938 um when somebody's family um his parents uh fled austria to escape um the nazi persecution and uh then you know he traveled uh they traveled to the u.s eventually they went to france then they went to eventually think,

actually, I'm just making this up. I don't know if we ever made it to the U S. Uh, I think so. But, um, uh,

then there's, there's, then he goes back eventually. Yeah. Because he goes back eventually and, um, and, uh,

works in France for a little while, but then did this, this, uh, this book about, about his life, his family and his life story. And then about the Prince of Persia. So I'm kind of excited to read this.

I really like these looks inside the storytelling of the history of some of these famous games and famous platforms. Yeah, they're very interesting. Yeah. And apparently his dad wrote the music for the original video game. Okay.

It's kind of cool on a on a i think anyway one of the one of the older computers in 80s so that's it that's my extra awesome let's close it out with a work hack joke i don't know what you call this brian so we have this woman developer sitting here pretty happy and this boss comes in manager says i don't like this bash script you created write it in python instead commands the boss import sub process sub process call work.sh well that's a day's work done nice how about that just if you want me to write in python i'll just call the script from python let's go yeah well what are the what are the times where i'm like where early on uh i guess not early on anyway 20 years ago or so um i'm i find myself in a new job and they've got all these

builds make scripts or for for c++ code and i can't remember they were they were in uh batch files and and bash and all sorts of stuff. And I said, can you rewrite this? Because it's kind of a mess. So I rewrote it in a combination of, I had a combination of bash, Perl, Python, sed, a whole bunch of things, whatever I wanted to,

to what I thought was an elegant solution. And the team was horrified. They're like, we're not learning that many languages i'm like okay so i rewrote it all in python um and that was the first time i i really grabbed python for do whatever i want as a shell script language so yeah that's very cool yeah i love it it's just subprocess call yeah subprocess.call that's right yeah nice yeah don't wait for it to complete just let it go well yeah no error handling needed let's let's just let it run now it's a good joke and that comes from your credit where credit's due that comes from linux handbook linux handbook nice

cool well thanks michael for another wonderful episode as always thanks for everyone to listen