Python Bytes - #394 Python is easy now?

Episode Date: July 30, 2024

Topics covered in this episode: Python is easy now Trying out free-threaded Python on macOS Module itertools overview uptime-kuma Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/394

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is 394, recorded July 30th, 2024. I am Michael Kennedy. And I'm Brian Ocken. And this episode is brought to you by Scout APM. Thank you, Scout APM, for sponsoring the show. We will tell you more about them later. If you want to connect with us on the socials, the best, but not the only place is over on

Mastodon where all the links, me, Brian, and the show all have our accounts listed at the top of the page of the show notes. You'll see in your podcast player or the website. Also, if you're not joining us live and you would like to, we record Tuesdays at 10 a.m. Pacific time. Check that out. Python by side FM slash live. You'll see all the older versions under the live section

on our YouTube channel there. And finally, you probably do want to also get this, a summary of this episode in your email. I'm sure you do. A handcrafted, artisanal, Brian Ocken special email. So just visit pythonbytes.fm, click on newsletter, enter your information. We won't give it away.

Share with other people. Resell it. Retarget you. Do evil things. We just will communicate with you about stuff we're up to, mostly the show. Brian, I learned Python. It was a good weekend. Is that the same?

Is that what you're telling me? For your first item here? Python is easy? Where are you going with this, Michael? I think you've known it for a while. No, it bundled the joke a little bit but yes that's all right I thought Python is easy I hear yeah Python is easy now apparently no I it's I love Python anyway what I wanted to talk about was beyond

hyper modern or postmodern I'm not sure what the tagline was really so this is an article by Chris our dear Ardern, Chris, sorry about I'm probably mangled your last name. But I really liked the article, Chris. So what is this? So it's the title is beyond hyper, hyper modern Python is easy now. And this is a reference to hyper modern Python. That was he says it came out like uh as a series that came out four years ago or something um interesting um but it's uh uh what was i who is the hyper modern person again can't remember oh um oh i should have looked this up anyway uh claude claudia claudio claudio oh i'm not gonna try claudio's book hyper modern python tooling is now as a book which

is cool uh good job but that's not what we're talking about we're talking about uh beyond hyper modern or uh from the url post modern so what is this basically it's talking about all the tooling and it's all in rye now so this is what chris's idea, is you could do so much in Rye. And actually, I knew Rye did a lot. And but Rye actually does more than I thought you could. So Rye is a tool that's like kind of does a lot of stuff around Python. So there's setup. It goes through setup, linting, typing, testing, documentation, CICD. And then there's a bonus section on monorepo.

But I really liked just walking through, like if you wanted to try out Rai, all the different, like the workflow, going through the entire workflow of building a Python project, like the setup part is setting up an initial directory, including setting up your pyproject.toml and setting up some different directories, which is great. Just sets up the structure for you. There's a lot of files here

that I don't often, to tell you the truth, I don't use lock files a lot. Partly because I'm often developing Python modules or packages, which I don't really lock stuff. Locking is more regularly around applications. So do you have lock files for some of your application stuff, Michael? All of them. All of them? I use the pip compile workflow.

Okay. So I've got like one thing that specifies the top level requirements and then another one that is the lock file using uv-pip-compile. Okay. One of the things that I didn't think about was like requirements for the requirements-dev-lock. So instead of requirements.txt, these are the dependencies are in your PyProject.toml,

but there are a lot requirements lock files. But a dev-lock file, that I might actually have even for a package like a yeah to keep make sure all the developers are using the same thing so that's kind of interesting it makes a lot of sense i was just had um when i publish it this is that this week's episode i can't remember very very very recently uh david lord from flask on talk python yeah and one of the things they're working on is creating a more repeatable environment for contributors and developers to Flask.

So like you, Flask, you don't want to just pin the versions of everything because that wrecks everybody else's options to work with different versions. But you do want to make sure that people who do PRs have the same setup as people who are reviewing the PRs. So he talked a lot about having pinned requirements for developers of the project

but not the project itself. Yeah, that's a cool idea. So let's set up. You can do linting with it. The testing section talks about PyTest. Yay! But one of the things that I thought was kind of neat was this, and I think I knew this, but anyway, this idea of scripts. So within and I think I knew this, but anyway, this idea of scripts. So within your PyProject Tomo, you can set up RY scripts. So they're basically extra commands that do things like, so you can say like RY run test. And so the run command runs your scripts. So you can say run in a test is a thing that you've said, that just like defaults to write test.

But things like right, right, run check might be running pyrite, or for linting, or for type checking, or linting is a right lint fit with a fix or something. So all these different if you if you've, especially if you're, you've got other people on the team having some way to do all of this is good and then a kind of a neat thing that i that's probably as nice as being able to chain uh scripts so if you've got like uh running your form auto formatting and linting and uh they're type checking and also running your tests but then if you want to do all of them uh there's a way to just chain them up within the Piper.tomls, which is kind of cool.

So it kind of takes the place of where I would use either where I used to use makefiles. Now I use talks commands for. So a lot of this around, if you don't need it, I'm guessing this doesn't create a new virtual environment. But I haven't played with it yet. But that's where talks would do that. So there's a discussion around documentation.

It isn't really a Rai thing, but just talking about documenting things well. But I really kind of like this article about like if you want to try out mostly because I do want to try out Rai. And this kind of walks through all the different things it does within your workflow and where you might want to pull it in. And I do appreciate that Rye has some configuration. So apparently the default packaging tool is Hatch. But this person, Chris, wanted to use PDM. So you can set that. But I'd probably leave it as Hatch because I like Hatch.

But it might do something different. But you can even Set a dependency operator. So whether you want it to be like approximately equal to or greater than equal or whatever Kind of neat the the bonus is really kind of fun I don't I actually just ran in into a project where I have to help support, which is a large repository with multiple Python projects in it. And so I'm looking forward to reading through how possibly this documentation

around dealing with monorepos in Python. This is on my to-do list, to read next list. That's cool. That's next level stuff right there. Yeah. Well, very,'s cool. That's next level stuff right there. Yeah. Well, very, very cool. I, Brian, am happy to report, I think, that the gill has been vanquished. We are now in the epoch of free-threaded Python.

So how might you test this out and see if it's actually worth it and so on? Okay. So Simon Wilson, one of his Today I Learned, is trying out free-threaded Python on macOS. There could be others as well. But you can just go to, this is interesting. I didn't realize that you would be able to get free-threaded Python this way from the PEP.

It's like you've got to get a special build of it and it's just like its own thing, right? Yeah. But if you get the macOS installer, right? I don't know about the others, but presumably the Windows one has something similar or will, I don't know. Go to install this.

You can set up the options, custom install, and you can just check off a checkbox to install a separate binary for three-threaded Python 3.13 instead of Guild Python. Wow. And that's just from the python.org download. Yeah, exactly. Wow. Exactly. Yeah, just to get that downloads beta 3 of it and there it is so you just hit uh customize your install

and boom so that alone is kind of interesting yeah and then once you do that you get not just a python 3.13 but a python 313 t which interesting interesting, but not FT. I mean, the other one should have a T on it. Come on, if it's gonna be threaded, I don't know. Anyway, T whatever you get one that is like, that's the one you can run in the command line for free thread Python. And Simon points out, like,

didn't really put this in the path because it's beta. I don't wanna mess stuff up, but nonetheless, there it is. So let's test it out. Comes up with a simple little program that just does some cpu bound thing this is the important thing we've for a long time since python 3 4 had really good options to parallelize io bound or waiting types of things i'm talking to a network i'm talking to a database i'm talking to whatever with async io right yeah the cpu bound stuff the gill would

stop it yeah no i don't care how many cores you got you got one right yeah and so this is just summing up a bunch of numbers the squares of the numbers just like doing some computational busy work right so it says okay well let's set up a little app that'll take some parameters so we can feed it different information and see what we'll get and then run it on a thread pool executor with as many threads that is are specified so start out just to skip ahead so look i ran it once i get one core it says 99 cpu usage on mac os that it uh reports this as a hundred percent basis as being one core fully busy so you you've got eight cores, this could go up to 800%. It's kind of weird. Anyway, 99% CPU if you run the regular one, but if you run the free threaded one, he was

getting some number that was a little low, there's a bug, it's not super relevant about him fixing the bug or not. I don't really think that matters. But what the takeaway is, if you run it with four threads, that whole workload ran in 1.3 seconds. You run it with 12 threads, it went down to one second. I don't know how many cores he has. I don't believe it said. But with the GIL, that's four seconds versus one second or 1.3 seconds for the basically four times as fast.

Presumably this is a four core system. I'm not sure. Anyway, pretty cool, right? That is pretty, pretty cool right that is pretty pretty cool so yes that's faster four times faster is uh you know 400 that's no joke that'll make a difference for some people but i think more relevant it's just like hey look apparently the installers have it right that's kind of cool yeah okay i was chuckling because uh so testing it out it says uh i asked claude three and a half sonnet to write me a quick test script, but

the script had come up with is something I know that he can write on his own. Yeah, exactly. Exactly. That's funny. Yeah. The, the, it didn't come up with something all that interesting and it introduced a bug. He should have just written it himself. Oh, well, but this is neat.

Actually I, I saw the announcement that there were some builds of the FreeThreader one, but I didn't know you could just do it, like click it from the installer. That's cool. Yeah, that's the big news I took away as well. Very cool. Neat. Very neat. And you know what you don't have to build yourself is an APM platform.

Right. We could let Scout do that for you. Let me tell you real quick about Scout APM. They're big supporters of Python Bytes, so we appreciate that very much. So if you are tired of spending hours trying to find the root cause of issues impacting your performance,

then you owe it to yourself to check out Scout APM. They're a leading Python application performance monitoring tool, APM, that helps you identify and solve performance abnormalities faster and easier. Scout APM ties bottlenecks such as memory leaks, slow database queries, background jobs, and the dreaded N plus one queries that you can end up if you do lazy loading in your thorium and then you say, oh no, why is it so slow? Why are you doing 200 database queries for what should be one?

So you can find out things like that. And it links it back directly to source code so you can spend less time in the debugger and healing logs and just finding the problems and moving on. And you'll love it because it's built for developers by developers. It makes it easy to get set up.

Seriously, you can do it in less than four minutes. So that's awesome. And the best part is the pricing is straightforward. You only pay for the data that you use than four minutes. So that's awesome. And the best part is the pricing is straight forward. You only pay for the data that you use with no hidden overage fees or per seat pricing. And I just learned this, Brian, they also have, they provide the pro version for free to all open source projects. So if you're an open source maintainer and you want to have Scout APM for that project, just shoot them a message or something on their pricing page about that. So you can start your free trial

and get instant insights today. Visit pythonbytes.fm slash Scout. The link is in your podcast player show notes as well. And please use that link. Don't just search for them because otherwise they don't think you came from us. And then they'd stop supporting the show. So please use our link pythonbytes.fm slash Scout.

Check them out. It really supports the show. So please use our link pythonbytes.fm slash scout. Check them out. It really supports the show. What's next, Brian? I want to iterate into the next topic. I want to talk about iter tools. So this comes from Rodrigo from mathspp.com. And he writes a lot of great articles. And I know we've covered iter tools before. And people have heard of iter tools. And it writes a lot of great articles. But I and I know we've covered inner tools before

and people have heard of inner tools and it's built into Python. But there's so many goodies in there that I like to have people tell me what their favorites are and reshaping. So this is there's this is an article that groups all of the it's they I don't think I think there's more than 20 in there. But anyway, 20 tools that every Python developer should be aware of from inner tools. And it's broken into five categories. There's reshaping iterators, filtering, filtering iterators, combinatorial, infinite, and iterators that complement other tools. And there's, I just did a quick list of this. And I really think there's a few, few of these in here that I need to use more. So I'm glad, glad I ran across this article.

One of them, so right off the bat, it says the three most useful iterators are product. And what product does is it simplifies, like, let's say you've got a nested loop of like width and height for instance for x in range of width and for y in range of height you do something with x y well you can use iter tools product and and have it just be one does a product of two ranges or product of two iterators and and gives you the two like the all the all of the combinations in order. So that's a really cool thing that I always forget about, that I probably should write that more, especially if you've got more than two, when you get like three, it gets ridiculous. And so using intertools to simplify that, it's a really great idea. And because, you know, with Python, every time uh indent into a for loop you're going to indent your code um it helps you keep the uh the the horizontal width of your code a little bit more reasonable

okay nested all that kind of stuff yeah um chain i always remember about so chain is a is a is a way to combine two lists and iterate through all of them instead of adding the lists uh adding is kind of it creates a new one you might not need that, whereas chain will iterate through both of them. And I have to remind people if you're not aware, all these really cool things within the iter tools, they're a way to manipulate iterators such that you're not copying things, you're not making a new list of something, you're just iterating through existing iterators without doing the copy. I think that, I'm pretty sure that's true. Yeah.

The the one that I the the third one on is you everybody should know about that I thought was great was pairwise. And I actually apparently never knew about this. And this is a way to like, let's say you've got the example is a bunch of names and you want to be able to make sure that everybody says hi to the person on their right you can go through and it well this may be a silly example but if you want to go through an entire sequence and make sure each pair in a like you're just walking it down like the first like number one and number two and then two and three and then three and four this is an

easy way to do that and actually there's an old way of using zip with the you can you can combine two lists and do slices but that's a little confusing whereas pairwise just gives it to you right away it returns to two pointers to objects so that's really cool a couple more things i wanted to bring up that I thought were great in here was Zip Longest, which I do remember because I use it frequently. But if you're zipping two things together, which means you're going to go through two sequences using Zip, by default, Zip picks the shortest and it just chops it off, I think. Actually, I don't remember what it does at the end.

But ziplongus will have a fill value. So you give it a fill value, and it goes through everything, and you'll have something empty. So you make sure you go through everything. And then the last thing is T, which is kind of neat. It allows you to do – normally iterators are exhausted once you go through it, so you have to like regenerate an iterator but t allows you to do to iterate through something a couple times um it allows

you to to iterate without exhausting the iterator which interesting so uh so nice and then uh at the end a discussion of um which is which i think is cool that he did this so a discussion that there is a is he wrote a book on iter tools uh so there's a little book of iter tools um he's not paying for this ad i just think it's neat that somebody wrote a book about iter tools that's nice neat yeah yeah it's pretty neat so there's a lot of a lot of powerful stuff there yeah it's all a lot of simple hard problems in simple ways. I really appreciate that he picked out those first three, though. Even if you're not really wanting to go through all of this,

take a look at these three. You probably need them and you just don't realize it. Indeed. What do you got for us next, Michael? You would love to know if your website was down and not up or your Docker container or your other service, and you would like to be notified in interesting ways. And there are services you can pay for this

or free services that have weird limitations or I don't know, just funky stuff, right? Some of those, in order to answer really good questions, you'd have to give them pretty serious internal access to your code or to your infrastructure potentially. So I present to you the Uptime Kuma, which is trying hard to resist uptime karma, but no uptime Kuma, a fancy self-hosted monitoring tool. Okay.

So this thing is pretty cool. It comes with a Docker variant, so you can just Docker compose it into life. And I'm thinking about setting this up for Python bytes and TalkPython and other things because I've already got that Docker cluster and all the infrastructure to run it. So adopting it should be really easy. It just takes one container to run, and I believe it stores its data into a SQLite database in there. So file copy is kind of a backup.

It's kind of the backup option, right? So anyway, check this out. Doesn't this look neat, Brian? You tell it different things you want to do. Like I want to check that the port is open. I want to check that this example is, you know, some example.com website is up. You can see that MySQL is running.

You can ping the server to see that the server is up. All sorts of things you can put as tests in here. And you can even have an anti-test. Like for example example if you're ever able to directly connect to the database that should be flagged as an error that like a down moment not an up moment because something's gone wrong with the infrastructure and you need to go fix it but in a reverse sort of way so what does it say first of all 54,000 53,000 g GitHub stars. So really, really quite popular. It has a little test, a little test server. You can set up, point at your website for 10 minutes and play with it.

That's a really nice feature. Well, it says uptime monitoring for HTTPS, TCP, HTTP keywords, JSON query, ping, a DNS record. You can make sure your DNS records don't go insane and like your mx dcam or i don't know whatever um game servers and even check that your certain docker containers are still running so it's got a nice ui and it has notifications for telegram discord slack pushover email and 90 other notifications can have as fine grained as every 20 seconds checking. It comes with status pages. So if we had a Python bytes monitor set up with this, I could set up status.pythonbytes.fm

that people could go and see if the RSS feed is live. Is the Docker container running? Is X, Y, and Z, right? So it has that for you. And it'll also, one thing I really like, because I use Let's Encrypt, but the auto renewal is a little bit janky. Long story, I don't want to go into it,

but it's a little bit janky. And so it works, but it requires a little fiddling. So having it will actually check SSL certificates to see if they'll expire. So it'll say like, your certificate is good for 29 more days. Maybe you want to get on that before it goes like, you know, something terrible. So all of that before it goes like, you know,

something terrible. So all of that in that suite is cool. Yeah. There's a bunch of sponsors and it's got this nice UI. You can bump over to the status page. It'll prepare you a little demo app and it runs for a certain amount of time. So how, how long are certificates good for?

If you do a let's encrypt, it's a year, I believe. Okay. So anyway, you can go over here and create this and actually go and add a little monitor and pick all the different kinds. And look at all these things. You can monitor MongoDB, SQL Server, Postgres directly.

Yeah, it's really neat. So I'm thinking of setting this up. And if you want to play with it, just click on that little thing. You can see here, Brian, I clicked on it. Now it says I have nine minutes and 21 seconds before my app gets blown away, but I can test it out, which is pretty cool. Anyway, it, uh, will give us, uh, potentially a really nice way to sort

of monitor and get notifications about it. So services, but don't like it. I could set it up once a day on the free one and make sure that my website at least is live for 10 minutes a day. Yeah. Yeah. If you don't mind reconfiguring it every time. Yeah.

Okay. Yeah, sure. All right. Pretty cool. How about some extras? The only extra I got was last week I talked about making a new PyTest course. I've been working on that. And I'm on version 3 because I'm a perfectionist and want to make sure that it's smooth and quick.

So I do hope to get that out sometime soon. I was going to say maybe next week, but I don't want to put too much pressure on myself. Yeah, cool. That's very exciting. I know people are looking forward to it. I also had to basically put too much pressure on myself. Yeah. Very cool. That's very exciting. I know people are looking forward to it. I also had to basically put DaVinci Resolve down to come record the podcast.

So I was in the middle of editing some videos that I got coming on for a new course as well. So very exciting stuff. All right. There's a never-ending stream of things to talk about, right? Yeah. All right. I've got a couple.

Kenneth writes, talks, pointed out that, you know, of requests and more fame. Talks about this move that Switzerland pulled. Switzerland now requires all government software to be open source. That's cool. Pretty cool, yeah.

So what does that mean? You gotta read it carefully. It doesn't mean the government has to only use open source as in Linux and Firefox, but the thing, the software that the government makes has to be open source. So what this is. So we bump over to omnivore. You can see my notes. A Switzerland has taken a major step forward with its federal law on the use of

electronic means for the fulfillment of government tasks. M bag is the acronym groundbreaking legislation mandates, releasing open source software of the federal government. The law requires all public bodies to disclose the source code of software developed by or for them unless third party rights or security concerns prevent it. So that's pretty neat. There's actually, there was some pushback. For example, one of the main areas this was around is the justice system this uh was trying to get all the the software that runs basically runs the justice system the court system for switchland open source but guess what the proprietary legal software company weblaw wasn't happy. No, they weren't. But after a decade of fighting, they came out.

And there's one other point to make. In addition to mandating open source software, it also requires the release of non-personal, non-security sensitive data as open data. So it's like open software and open data as well. So pretty cool. Interesting. Yeah, it is. We do have some stuff like that like over at talk python what was it elections i think if i look for that yeah python at the u.s federal elections uh commission and talk to laura uh beaufort and i believe they

are open sourcing their stuff as well a lot of things going on there, like yeah, open source at the FEC, for example. So we have some stuff in like spotty examples of that, but not as a overall statement. So it's pretty interesting, right? Yeah. I'm curious to look at some of this stuff. Like, yeah. How clean's your code? How many AWS keys will be in there? We'll find out. Thanks indirectly, Kenneth, for that. One more. Remember last week I talked about the PySimple GUI and it's switching to a commercial license, which again, I'm not super against, but sort of the rug pull aspect of it, like, hey, it's open source for a long time and we're going to take everybody's contributions and then change the license. That sort of thing got some people's's nerves up well um f falcon 31415 on massad i'm pointing out hey check out free simple

gui which i linked to in the show notes which is an actively maintained fork of the last release of pi simple gui with an open source license so if you were like oh i was going to use that you still can yeah actually it's got a really good good, the readme on that has some really good to like, has some good tutorial examples of how to write a simple GUI on it. So not only is it the open source free version, but it's also some pretty good documentation on there. Cool. All right. How about some jokes? Yeah, joke time.

A singular joke, a singular joke. So this one about some jokes? Yeah, joke time. A singular joke. A singular joke. So this one is kind of a meme with Will Smith and it doesn't really have that much to do with Will Smith. I think it's just the attitude. I like Will Smith. Anyway, it says just because I know Java doesn't mean I know JavaScript. I mean I do but not because I know Java.

That's good, right? Yeah, yeah. I was just talking. Whenever you talk to kids trying to learn software, it's still confusing to people. I mean, it's not confusing to us because we've been around it. We knew the history, but it seems weird now to tell people, yeah, the JavaScript that everybody uses, it was named that because Java was popular at the time.

So anyway, are you muted? I was, sorry. Fun episode as always, Brian. Yeah, fun. And yeah, thanks, man. Thanks for being here. Thanks.