Python Bytes - #354 Python 3.12 is Coming!

Episode Date: September 26, 2023

Topics covered in this episode: logmerger The third and final Python 3.12 RC is out now The Python dictionary dispatch pattern Visualizing the CPython Release Process Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/354

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 354, recorded September 26th, 2023. I'm Michael Kennedy. And I'm Brian Ocken. I am good. It's been a busy week. And it's only Tuesday. I know.

And it's starting strong. If you want to follow the show, stay in touch with us. Be sure to check out Bastodon, overall Mastodon. We've got all the show, stay in touch with us. Be sure to check out Bastodon over on Mastodon. We've got all the links for that in the show notes. I was just thinking today, I just booked a couple of episodes through Mastodon for TalkPython. I just was working on another video presentation thing on Mastodon, like how much work gets actually done on Mastodon. So people should be there with us, Brian. Yeah, I think so.

Yeah, for sure. For sure. It's the place where all the cool kids are. Indeed. Indeed. So hopefully you'll connect with us over there as well. Very exciting stuff. See some people in the audience here.

I'm gonna have to give them a shout out later because some things that we're sharing. And so that's gonna be very exciting. So speaking of exciting, what's the first thing you got for us? I'm actually really excited. I may be jumping the gun on this, but Log Merger is from Paul McGuire. And it's a project that he's working on

and it's pretty fun. So I don't know, maybe I'm not, I think more people, a lot of people deal with log files that they have to deal with more than one so you have maybe two processes or oh yes three that that are all time stamping and saving well um so this is a project that uh is a textual based project but it combines them all and sorts the multiple logs based on timestamp. So you've got, maybe you've got a timestamp and then content, and it splits it out and shows you the timestamp on the left

and then the different log files in columns. And the example just shows two, but I think you can pass it more than two. I think you can do multiples. So it's called log merger, but it's not, it doesn't, I don't know if it actually merges them, but it makes this combined output thing. And it's really kind of fun. It's more look at them in a merge, what a merge view with timestamps would look like, rather than actually about trying to unify them or something.

And I had some concerns right away, and they were already like, I downloaded it, and I'm trying this out. You can pipe the output to standard output, or you can use the textual thing. So you can do either one. So the output is nice. But sometimes, you know, log files can be huge, and you only want to look at a part of it.

So there's a cool, there's some nice flags that are included. There's a start and end timestamp. So where you can say, yeah, start here and then go to here and don't do the rest of it. Um, uh, there's an interactive mode, which is I think the default. Yeah. That, um, that shows you the little two and you can do error boxes and stuff. Uh, you can save them to a CSV. I don't, I think that'd be great if you want to throw it in Excel or something like that. And then multiple timestamp formats are supported. So to got like, there's a handful that I think this is pretty good. This is the standard ones. I didn't even pass in a timestamp when I tried it out. And it just worked fine. So I'm going to be using this a lot. I'm excited about some of the

upcoming work. So there's possibly, oh, maybe it's not here, but somewhere else. There's a possibility of using PCAP files. I think that's in working on those are, I don't even remember what that stands for, but I deal with them for protocol analysis uh but the um the i maybe jumped the gun because you can't pip install it yet you have to right now you uh download the code um but it's totally usable and runnable uh so i i think it's worth checking out but it's oh gosh it's new it's like you probably could pip install with the git oh yeah with the git plus exactly yeah with the git plus exactly um i forgot about that so you can do yeah so um i guess uh paul um people are using this at least i am already so maybe like a little uh maybe i'll do a merge request to add the readme to say how to install

it for now so cool fun yeah nice and prior guest on the show, Michael Larson says, whoa, packet capture files. Is that the extension you're looking for? That's awesome. I need to check that out. Yeah. And Jeff out in the audience says, does it support time zones? It does have a time zone display feature, but I don't know if the sort considers time zones or not.

Well, my thing is going to be that they're going to be a little off anyway because sometimes the different... I've got log files getting generated by multiple computers that are the timestamps off by a little bit. But it's usually close enough to line them up. Yeah, close enough. Yeah, I do stuff like this as well.

Like for all the talk Python web properties. You get lots of requests and the Nginx files have like a two week rolling average. The most recent one is a straight log file, but the rest are gzipped by day. God, okay. So I have a Python script that downloads all of those, unzips the ones that are zipped, puts them all together, then loads all of those files, sorts them by timestamp, and then like creates one super log file, maybe appending onto a previous one. So I have like single log text log files that are the full month, month by month for analytics. Because, you know, we turned off Google Analytics and all sorts of other things because those are evil. But you still want to know how many requests were there to this page or to this episode page or stuff like that. Those kinds of things are relevant.

So yeah, this looks interesting. I had to write a bunch of code to do similar stuff. Although for 14 log files, you wouldn't try to visualize them. I just want to turn them into one mega log file so that I can run analytics across them. It's not on this readme, but I think when I was looking at it, I think there's, there's recent stuff being done and I think that you can pass it GZIP files. I think it pulls them out.

It seems reasonable. Those are frequent formats, I suppose. Anyway. All right. Cool. Good find. Very interesting one. People in the audience like it as well. This next one comes to us from Hugo VK, and not sent in, I just discovered it, I believe. He says, hello, Python library maintainers. So if you have a PyPI thing,

if you twine or what, upload those types of things, you probably want to pay attention because I said, because everyone was so good at finding bugs last time in the Python 3.11 release candidate, the third and final release candidate is now out. Please give it a spin. It's your last chance to find significant problems, right? Critical problems. And goes on to point out, here is a nice article on how to test

it, how you can get the release candidate. And basically, how do you test your libraries with this? And it goes through like, how do you set up a GitHub action so that it will use the 3.12 pre-release, right? Yeah. How do you do it from dead snakes? How do you do it for Travis CI or other CIs and different things in here? So that's cool. That'll help some people set it up into their CI. There's also a big long discussion here. So Thomas orders points out that release Canada 3 is here as a reminder until it's Until the final release of 3 12 0 the release 3 12 branches set up So the release manager has to merge to the changes right so reach out at this point unless something critical comes up it should really only be documentation other changes will be

pushed to 312 one and somewhere in here there is yeah the intent this is exciting the intent is for the final release of 312 0 scheduled0 scheduled for Monday, October 2nd. That is like the coming Monday. Yeah. Unless you, if you're on top of your Python Bytes playlist, don't let that thing fall behind. Then the coming Monday, October 2nd, is going to be when this thing releases.

So that's pretty excellent. I guess bringing back to the first part, this is basically you have a couple days to make sure that your library will have no hiccups when Python 3.11 ships. Or see, 3.12. Kind of gets used to saying 3.12. When that ships in a week, you want to make sure that your stuff just works. So here's your chance and a few steps on how to do that.

Yeah, because you know how nice people are on GitHub. You'll get issues right away. Hey, your stuff doesn't work on 312. you might even get like some hot october amazingness like hey you were using the wrong preposition here i changed it like well that was not wrong but this is not a real commit seth is back to say highly recommend the setup python allow pre-releases feature that lets you set up a forever pre-release testing job if you use a python version 3.x that's pretty pretty good advice there yeah cool neat i don't think i use that i'll also check it out well now now you know you can all right all right what else you got as exciting for us okay other good advice do you have uh what do i got i got um oh good advice i i think i have questionable

advice um but i like this pattern so i'm linking to uh james coffee blog with um the python dictionary dispatch pattern and i just like this because i don't know if it's not discussed much in like beginning teaching python stuff as to but um you know if you're a c programmer use jump tables or maybe you shouldn't use jump tables but jump tables are fun in c and you can kind of use them in python too but they're they're a little bit um easier to use you just have um dictionaries uh that dictionaries can store anything and they can include functions so uh he has an example of just add, subtract, and multiply, which you wouldn't really use for that, but it's just an example.

And I kind of forget that you can throw lambdas in because you can assign the lambdas have a value, so you can put a lambda in as a function. But you can use a function also. Then you can call them later. So you could just uh dereference the um or dereference we don't call it like that you look up the key the element uh in the dictionary and then actually just call it with parameters and it works so um kind of a fun

thing to do in python uh here's a little bit of a uh he said uh as part of a vision script programming language does that um interesting but i mean there's there's a lot there are times where you you just have lots of things that you're switching on and an easy way to do the switch is just to have a dictionary lookup so yeah exactly that's it is pretty interesting um it's a traditionally it has been a way to do switch statements, which at the time it was pretty much all you had to work with. Right. Since then we've had the match statement added to the language, right. Which is a little bit better.

So not switch, but match because it's this whole parser mechanism on top of just the switch statement. But people used to traditionally say, okay, well, we'll just make a dictionary. And then like your case is the key. And then what you do is the function that gets returned from that. Um, one thing that's really missing from that pattern, that dictionary dispatch pattern is that the default case gets clunky looking, right? Like if, if none of the cases match, what happens like you just get none an exception like none type is not callable or some weird thing like that right oh yeah you do yes you could

you could upgrade it you could upgrade it with a default dict where the default dict returns the default case right um i for a while i don't remember when i added this but i added switch laying which was fun and switch laying um uses context managers to simulate a switch with a default case sort of thing. Yeah, or as Seth is pointing out, if you use get instead of a dictionary lookup, you can give it a default to return. You can, but you got to give it the default every time you use it, is kind of you can but it is a little bit weird right like that's why the default dick makes it a little clear because if you're going to use it more than one i don't know you use it more than once it gets janky right so i but so i've used this and i i don't use i don't put a fall through but what i the reason is because it's all my code it's all i, I don't, it isn't something that's coming in from the outside world.

And I have test cases around all paths. Right, so you know there's not gonna be a missing case. And if you do, an exception's probably a good idea. It's not something you like, ask a user, which function would you like to call? And then, you know, pass it through. Indeed. Yeah, don't do that.

Anyway, oh, reason why i wanted to bring it up not is because i not i didn't want to bring this up because i wanted more people to use a dictionary dispatch uh you don't have to do that what i brought it up for which is it's still kind of cool to reference just to let people know you can do it but um this guy's got a blog roll how cool is that um when's the last time you've seen a blog roll? Dude, a blog roll. Yes. Should we bring blog rolls back?

I don't know. Maybe. Maybe. I don't know. Anyway. That was when your blog was your little home on the internet. Yeah. Yeah.

Yeah. How interesting. There's no hit counter, though. So, you know. Oh. Does it have a little dig, like a little digging? This part is under construction digger. Oh, cool. No, sorry, James. If you're listening, um, we're not dissing you.

It's just fun. No, no, no. It's awesome. It's it's nostalgia is what it is. It is. So anyway, um, All right. Back, back to the, uh, Seth Michael Larson show. Yeah. So his, uh, his article is very timely that he just put out a couple weeks ago, but I finally got around to featuring it now, given everything that's coming together, called Visualizing the CPython Release Process.

So it was awesome. He's the Python security manager, security dev, as opposed to Mike Fiedler, who is the PyPI. Both of these roles are somewhat new, which are awesome, right? This one is made possible by funding from OpenSSF Alpha Omega Project. So very, very cool. And Michael, or Seth, is pointing out security developer in residence is the official title. Yes.

Right there. And then we also have the developer in residence, which is Lucas Schlenke. So anyway, they planned out or he went out and wrote out what it actually looks like to release Python. And boy, oh boy, this is more than just some build CI at GitHub to make sure that the wheels or the binaries get built. So here's the process. See how close I can get this right. And also, Seth is going to be on TalkPython on Thursday in terms of live stream. And then a little bit later to talk about, we'll talk about this and a bunch of other security things.

Anyway, so start out, freeze the release branch. And then the CPython upstream repo goes along, and the release manager forks the repo, takes that, feeds it into the release tool manually. All these things have, there's a bunch of colors in this diagram. So each color, there's a legend, like the hand ones are human tasks, the blue ones are release artifacts interesting from a security perspective um there's a source of risk is red and so on all right so the release tool goes along and git commits get tags things to uh set the tag to version you know 3.12.0 also it creates the source ball the tarballs from that and And where do we go now?

We go over here to Azure Pipelines for the Windows people. The Azure Pipeline, there's a Windows Release Manager. The Azure Pipeline over there builds out the Windows embeddable packages, the Windows installers, updates NuGet, Windows Store, maybe even Winget. I don't know about that. There's also an equivalent for the macOS Release Manager, who does the Mac build and Mac installer. And all of these things come back into testing artifacts and creates the, they all get a GPG signature.

And then we get everything signed and ready to go. That gets dropped on python.org. Pretty cool, huh? That is, wow. Yeah. It's a lot. Last year they live streamed this and all the people involved were on Zoom. I think it was.

I'm not sure. It was live streamed to YouTube, though. And you could actually see step by step. So but I don't know how well people really knew what was happening. You can be like, all right, we're on step five. Everyone, let's go. Let's go. So I think, you know, if you're going to watch that live stream, if it exists, I i'm not sure it does um people in the audience if they know it's going to be live streamed

shoot out a message um yeah okay so too early too i don't want to slow down the release this year but maybe next year we could have this graph but like animated with like little little things what's going on yeah i have like little little little video cutouts of people in the different sections and they're talking. Yeah, that'd be fun. Why not? Why not? No, this is cool. And then there's more about, there's like more of a description below that Seth put together, as well as the sources

of supply chain risk and examples of that happening outside of Python primarily. So, pretty cool. Thanks for the visualization, Seth. This is great. Yeah, this is cool. Nice. Yep. Over to you. No, wait, that's it. Over to you for extras. We are in extras. Do you want to do your extras first or?

Yeah, sure. They're easy. I'm going to be at Pi Bay over in San Francisco, October 8th. So all I want to say is if you're around, come say hi. Right. That's like primarily the reason that I go to conferences. I don't go to that many talks or that many other things, but these kind of like get to know people in open spaces and so on. So this is all one giant open space, this Pi Bay thing. Yeah, I went last year. It was fun. Maybe we should just bounce back and forth and do every other. Yeah, absolutely. The HTMX plus Django course is going really well. People are loving it. So if you want to do HTMX with your Django code,

it will make it so much better. You don't have to do front-end frameworks unless you really, really want to. So HTMX is awesome. This is how you use it with Django. Each framework has its kind of own thing. The reason I bring this up is we are closing in at the end of the early bird discount.

So if you think you're going to get it anyway, you've got until September 29th until the price goes up. A few more days. So check that out. Yeah, that's it for my extras. Nice. I'll try to go through mine a little quickly. Anyway, of course, the complete PyTest course,

I just updated part three is started it's um not part three part two i've started with chapter seven uh testing strategy so that's going along strong um i wanted to uh mention some podcast names uh so of course you're here with python bytes that's us and my other other one, a couple others, Python People, I've got a lot of great interviews coming up. And then Test and Code is now Python Test. So you get the, just here we have Python Bytes, Python People, and Python Test. And then now Talk Python to me is the odd man out,

so we have to get Michael. You got to change your name to Python Talk. Python Talk? Yeah. Python Talk. So we have to get Michael. You got to change your name to Python Talk. Python Talk? Yeah. Python Talk. Anyway. I could do that. I could change it.

I can change whatever I want. It's working great. Don't change it. If it's not broken, don't fix it. No, it's going pretty good. Speaking of broken, let's talk about the final thing. A joke here. Okay.

Maybe this has happened to you. Here's a woman developer trying to sleep, but her mind is running. And she's like, okay, mind, stop. I need to sleep. Leave me alone. There's like a little conversation. Brain, woman in bed, right? Says, hey, are you sleeping?

Yes. Now shut up, she says to her brain. The brain says, I know how to fix the bug on line 255. Boom. Eyes wide open open no more sleep i can't sleep uh we've probably all been there right you're just like somewhere else you're like i know what it is i know what it is yeah so this one got the database index no it's one of the downsides but also one of the upsides of working being having a setup for working from home so that

you can just uh run in and actually fix it and then go back to sleep. Right. Or you can sleep at your desk. And then when this happens, you just wake back up. It's keep typing. Okay. There we go.

Yeah. Awesome. Well, I mean, I have a standing desk. Why not? A like a sleeping desk. Oh,

white. Yeah. Why? You know, it goes, they go from sitting to standing. And it could just go one more level. And you're like lazy boy just drops you into like sleeping position yeah with a lazy boy white noise kicks in as it gets below a certain height yeah monitor goes up to the ceiling

exactly yeah be good anyway all right um well awesome podcast again michael yeah thank you thank you as always and thanks everyone for listening and some of you for attending