Python Bytes - #355 Python 3.12 is Out!

Episode Date: October 3, 2023

Topics covered in this episode: 3.12 is out! Trouble with virtualenv caching, a tale of 3.12 update Python Developers Survey 2022 Results Scientific Python Library Development Guide Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/355

Hello and welcome to Python Bytes where we deliver Python news and headlines directly to your earbuds. This is episode 355 recorded October 3rd, 2023 and I am Brian Ocken. And I am Michael Kennedy. I almost said the second because I thought we were going to record yesterday and we didn't but had it off. Anyway, welcome everybody. It's good to have the show. Thank you everyone for supporting us through purchasing courses at TalkPythonTraining or purchasing the PyTest course, of course, or supporting us on Patreon. We love Patreon supporters. And if you want to reach us, reach us through Fostedon on Mastodon at MichaelKennedy, at Brian Ocken, and at Python Bytes.

And if you're listening to this, you can also join us live sometimes. We usually are Tuesdays at 11, Tuesdays at 11-ish on Pacific time. So join us. And you can just go to PythonBytes.fm to see the link. But we have some exciting news. Yesterday was an exciting day. Tell us why. We've been waiting a year for this one.

Python 3.12 is out. Absolutely. Absolutely big news. If you look at how much stuff, it's easy to just go, oh, what are there? It's like these four new features I care about or something along those lines. It's like, okay, well, there's a new Python. If you look at the changelog, Brian, if you look at the what's new and you scroll this, I threw this into omnivore.app.

And it said it's 48 minutes of reading to read the what's in this release. Wow. That's a lot. That's a lot of stuff. That's a lot of good things. I think it's, I don't even know how much it's even called out in the what's new because it's a little tricky to put in the not new, it's just more, but the faster C Python initiative, I think, you know, is, is going strong for three 12 as it was for 11 and it will be for 13.

So not even covered in the what's new, I think is Python three 12 should just be faster. That's really cool. Yeah. But if you look at the, you know, they kind of, it's really nice, even though it's 48 minutes of reading, they do put the headline items at the front here. So there's new type parameter syntax and pep 695. So, uh, Brian, how do you like your generics? Uh, I haven't thought about it actually. Or your, your templates, if you're a C++ guy, they don't call them generics. Like C Sharp and Java, they call them generics.

But C++ calls them templates. But Python calls them generics. We have that. We had it before, actually. So this is not new, new. Like there were ways, if I scroll down somewhere in this big long list, there was like a here's how you do it a, here's how you do it before.

And here's how you do it now. Honestly, meh. I mean, no disrespect to people working on it, but it's just not like a thing that, that I'm going to be focusing. It's just not a huge deal to me. Right. It's not something I feel like I'm going to do a lot of. Maybe. I feel like most of the places that i would have wanted

to use it it's kind of like the self parameter sort of takes care of that for me but anyway very exciting for people who do it do need to use this and if you're building a library like a package possibly this will give you a lot of flexibility for extensions and sensibility and whatnot but i don't know. What do you think? Exciting? I think I'm not excited about it yet, but I think I'll be using it probably pretty soon.

So yeah. Yeah. F strings are all grown up now. Oh boy. They can do all the Python. So F strings, I don't know if people know, but F strings have their own separate implementation, separate to like regular strings and stuff.

And a lot of it was done in C. They're really wild. Because of that, they're somewhat limited in what you can put inside the curly braces. There's like different kinds of expressions. You can do ternary like if tests, and you can just say like thing.u you know as what goes in there but you can't write full python in f strings well most you can and the most obvious was you couldn't put

quotes in the little curly between the coiled braces or you couldn't put the same quotes that you had on the outside so yeah i i'd always be like okay this this one's getting single quotes because the string is a double quote or vice versa versa, right? You have to kind of think it out. So now it's like, a little more freeform, what can go in there. I don't know, that's not really a recommendation of writing your app inside the upstream. But you know, it gives you more flexibility. So there it is. I am super psyched about 648, which is a unique per interpreter Gil, Eric snow, man, you've been on this for a while. Congratulations. And that this is out. So this has mega possibilities to unlocking multi-threaded performance and Python.

Don't know that it's actually has anything to do with threads yet i think it's way more manual and maybe even just at the c level uh yeah it said that the pep says it's at the c c api right now 313 might have a python api access to it yeah i mean what i would love is like i'm going to create a thread and I'll pass a flag that says, use your own Gil. Or I'm going to create a thread pool executor and say for every thread that you create, get a new guilt. That means full on parallelism in Python. I mean, I'm sitting here on my, well, what you're talking about then is, is basically forking off a new interpreter from Python. Yes, in a sense, but without a new process. Okay.

Right? Yeah. So. Why not? I don't believe you can share. Maybe you can share data directly. I don't think you can share it directly because it would have to share the gill. I think you've got to like serialize the data over but you can get better performance if you don't have to start a bunch of processes potentially and do some message passing. But you

know here on my M2 Pro Mini I have 10 cores and all the Python I want to write I'm only going to ever be able to do one cores worth of work for computational things unless I go crazy with like the no gills, scython, or write some C code or you know, things like that, right. So this, this has the possibility to take advantage of, of, you know, modern hardware, right? Very exciting. Low impact monitoring. So if you want to like hook into events for like profiling, debugging type of stuff, you can do that with less Heisen bug effects. And also one of the big things that came in 3.11 was, did you mean type of recommendations for, we couldn't import request. Did you mean requests, plural? Things like that, right? And so that's further improved.

There's a buffer protocol, which allows direct access to memory of things like byte arrays and other low-level items that you don't have to go through the Python level. You just go, give me the direct access to the memory. That's kind of cool. Nice. Good for embedded.

You can have, yeah, I bet, pathlib.path can be subclassed. That's kind of cool. Hey, OS module gets Windows things. That's nice. Let's see. There's a command line interface that adds a SQLite 3 module.

So, you know, you can do like Python-M HTTP server or something along those lines. You can do that for SQLite now as well, which is kind of neat. Yeah, this is exciting. AsyncIO has some improvements with benchmarks showing up to a 75% speed improvement in certain areas, I'm sure not generally, but yeah, a bunch of other things. There's also some security updates, some the API, there's performance improvements with comprehension inlining. And there's a Linux Pro performance profiler. Yeah, bunch, bunch of different things.

And that's pretty much the highlights. I mean, there's some binning, I'm calling up the standard library, like there was an async chat example implementation type of thing in in the standard library. So it's not anymore. Good. Yeah, so that's not there. But generally, you look at some of the things that were deprecated or removed, but yeah, that really much covers a few more type things and that's that's your new your new python awesome are you excited

how long do you switch to it i'm i'm working on it right now man um are you yeah so yeah wait a sec uh and i'm gonna tell you a story. Okay. So yesterday, I've been slowly becoming one of the maintainers for a plugin called PyTest Repeat. So it had been tested up through 3 three nine, I believe. So so I wanted to, but it worked fine on the newer pythons, but I just wanted to make sure it was updated. So I, I did some things like move the move the the continuous integration to GitHub actions, it was on Travis before, so I moved to GitHub Actions. And so 3.12, I wanted to update the test

to 3.12, right? So this is using talks. And so I just went through and updated the talks any file to add 3.12, I installed it on my machine. What's the problem um ran it and created a new virtual environment installed talks whatever and ran it and it blew up i don't know yeah it what what happened is i got this uh python attribute error module package util has no attribute imp importer did you mean zip importer i'm like i didn't mean any of that stuff so no idea um so i'm like trying to figure this out like clearly other people are releasing at 312 they surely have come up with this i dug through a long i was researching a ton on this um and uh basically i thought there's something weird going on between pip and talks and python 312 because i knew that the imp importer thing was deprecated i don't know

what it is but i knew it was deprecated and it was taken out in 312 i think so it's not there um and no mostly you don't have to care about it, uh, except for I ran into this. So what was happening? I didn't know. So I did come up with a fix though. And I found out this morning with the, what really was going on. So it's a long story, but I'll try to make it quick. Uh, the, the fix I came up with was there was a in, in, uh, talks in a, you can say

download equals true. So when it creates a virtual environment, it creates the, it updates to the latest pip. So what was happening was, but without that, I was using an older pip, but I don't know. I didn't know why. Why was it using an older pip? That's the part where I didn't know why. So this fixed it, but it's kind of a bandaid. It's a little bit of an ugly bandaid. So what's the real fix? The real fix is vert talks uses a package called virtual env. So that's not the built in vnv. It's, it's the third party

package virtual and, and virtual env has some cool features. One of the things it does is it keeps a little cache of third party packages to store to load up. And it doesn't update them all the time. It doesn't like, I don't know, every 14 days or something like that. It's a kind of a slow update thing. And it makes things run faster because you don't have to keep downloading all the time but i had an old pip cached so the real fix was um i went through and i probably didn't have to do the reset but i i called virtual env with reset app data um and then did an upgrade and i probably could have just done the upgrade upgrade embed wheels and it updated my pip cat cashed pip to the latest. Anyway, so I'm going to, I wrote this article about how I patched it. I'm going to update it today to say, ah, the real fix is to

clean out your, uh, your virtual lamp pip cache or virtual lamp cache. So yes, I'm working on updating to 3.12. So, anyway. Nice. Yeah, that's cool. I also have to shout out to Juergen. I don't know how to pronounce his last name. G-M-A-C-H. Juergen helped me debug the problem today.

So, that was awesome. Excellent. One more shout out. Henry Schreiner says the buffer protocol that I described is not new, but the pure Python version of it. 312 is. Okay, excellent. All right. Over to you. Oh, I just did this. This is my thought that was just a follow-up to my topic.

This is your topic. Okay. Excellent. Yeah. Yeah. Cool. All right. Let's talk about the present by talking a bit about the past.

So very exciting news. The Python developers survey results are out. It's slightly confusing in that it says the Python developer survey 2022 results. Like, wait a minute, it's September 2023. Is this last year's one or the no, this is the one that just came out on the PSF blog, they just on Python, they just announced like, these are finally put together, analyze a report from the end of 2022 is now out. So very exciting and also new. I did a video on YouTube actually, where I had a first reaction to it. So this is an 18 and a half minute video. I'll link to it. You can check it out if you want to go deep. And because this video exists, we'll just kind of skim it, Brian, we won't go over too much of it, but, uh, I basically didn't open up or read the survey results until I

hit record for this video. And then I just dove into it. So that's fun. People can, can check that out if they want, but let's see, we can hit some of the highlights, uh, general Python usage, 85% of the people who Python Python primarily that way so it's not like a second language you know but it's mainly what they do which is pretty cool there's a lot of these that add to over a hundred so um take it for what it is there's also like a little gray um like a light gray and a dark gray um i don't know word or legend and that will they often overlay these things to show them side by side. So for example, says Python usage with what other language like if you're not, you're going to use another language, what do you also use in addition to Python,

primarily being Python be the main language and it has like this one is the prior year and this year again, with the offset by one, a little bit. So guess what the top three are JavaScript, HTML, and CSS. Are those languages? No, no. Can you make it here's a rule for what is a language in my mind? Probably can I make a thing that runs on its own from that set of syntax JavaScript? Yes. C++? Yes. Java? Yes. TypeScript? that runs on its own from that set of syntax javascript yes c++ yes java yes typescript yes

css no but nonetheless um yeah the reason i make that distinction is like it's a it's um i feel like it's a trade-off of like i do java or i do python or i could choose one or the other but they're not if it's not interchangeable i'm not sure i don't know it's somewhat interesting but i'm i'm on the feds it's they've got they're both complicated enough that um i think it takes training to learn it so why not i mean i wouldn't think i wouldn't think of like any files as a language but so they're a lot they're a lot more complex than any file or something. True. All right.

So I think I go on that every year. So the top, the top three languages used along with Python, three of the four, cause there's a combo. I don't know. I would really call it a separate language, but anyway, JavaScript, HTML, CSS combined and SQL. And those basically that block tells you, guess what? People build web apps with Python, right? That's what it says here. And web apps are often five, six, seven different languages, which is partly why the web is hard.

Right. But there's that. And then a bash and shell. I mean, that kind of speaks to the DevOps automation side of things. And then C, C++, Brian, you're kind of down in that realm, right? Yeah, that's most of, half my time. And that's interesting that that's that large still.

So that's cool. Yeah. I think another thing you would want to consider here, if you like, look at this is TypeScript and JavaScript, those should be the same. If you say HTML slash JS, HTML slash CSS, you should say JavaScript slash TypeScript, which bumps that up. Even higher, right? Because that script is just like a better JavaScript. All right, let's go. If you're going to do data science, most people use SQL as their first language that they're going to do. And yeah, it's interesting. Let's see.

Some of these things are about like, how do you do work? I think it's interesting, this one here, what do you use Python for the most? Web development, number one. Data analysis and machine learning, which is kind of like the data science block. And then other, whole bunch of other. And I think, I kind of think of Python as having a one-third, one-third, one-third kind of partitioning where the web development,

API development stuff, service functions, all those things live in one third and then data science lives in another third. And then there's like the catch all block of everything else. So this kind of says that maybe the web development side is a little smaller, but the, and the random section is a little bit bigger,

but roughly I think that's a good rule of thumb. Yeah. Yeah. Yeah. And I think finally legacy Python has been vanquished. It's been vanquished for three years now. Python three versus Python two. Basically. Um, it's it's Python three.

That's like 94, 95%. Ironically, it went down a little bit to like Python two made a bit of a comeback this year. That's interesting. It's probably within the variability of the number of people and the type of people that responded to the survey more than it is a real trend. It's probably like the plus or minus one or two, like plus or minus one would account for that just to be flat. Yeah, that's true. Anyway, it's like 95%.

And that last 5%, they're never changing. You know what I mean? That's like the 2 million lines of Python 2. It runs the trading system. You don't touch it. Like that kind of thing, right? Well, yeah. And I know some people that have frameworks built up on top

with an embedded Python interpreter that it has Python 2 inside of it. And you'd have to completely change the whole architecture to rip it out and put 3 in there, which I say just throw the whole thing away and start over. But anyway, I know that. I'm going to do the big rewrite. I'm going to throw the whole thing away and start over. But I know that. We're going to do the big rewrite. We're going to do the big rewrite.

That's actually an incredible joke. It's not a joke. It's a parody video, like music video song. We're going to do the big rewrite. Maybe I'll put it in the show notes. It's not the joke for this week, but boy, it should be. It's good. All right. Let's see. I think that's, I'll probably leave it at that. a joke for this week, but boy, it should be as good.

All right. Let's see. I think that's, um, uh, I'll probably leave it at that. Like I let people look through here, but like you mentioned virtual EMV, that's number one there. What do you want to cover? Uh, unit testing frameworks. Oh, is that a question here? Let's see.

Just kidding. So yeah, 51% by test. Yay. Coming in second, 35%. None. This is a problem, people. Yeah. The second most popular framework.

Is none. Is to not ask the question. And I think unit test is largely there because you don't want dependencies, right? If you've got a whole bit of code that's only standard library and you want to have a couple tests for it, that makes a lot of sense to just not add a dependency

potentially. Well, so, and I also got to talk with the JetBrains and PSF and have them change this question because unit testing framework, I think you mean automated testing framework because it might not be a unit test and people get confused by that maybe. I don't know.

Yeah, potentially. All right. Okay. So what's next? There's some interesting comments out in the live chat here about like, Oh yeah, let's cover some of them. build system, talks is not a unit testing framework.

Yeah. Does Turing complete make CSS a language? Is CSS Turing complete? If it is, then yeah, totally. I think it does. But I don't remember what Turing means anymore. Uh, specifically, I mean, I kind of, yeah. Yeah. My, my rule of thumb is, can I make a thing that runs with that? I don't know, but, uh, that's, that's the Michael role, Michael role, but anyway.

Yeah. Uh, on, onto the next one for you. But that's the Michael role. The Michael role. But anyway, yeah. On to the next one for you. The next is brought to us by Henry Schreiner, who is in the audience. Hey, Henry, thanks for putting this together and also letting us know about it. So we've got the Scientific Python Development Guide. This is a big guy. So this is really cool, actually. So this is I'm, we're going to post both

the the announcement post and a link to the guide, but it's both the same place, blog scientific python.org. And then there's a learn scientific Python.org. So this is pretty awesome. It's very comprehensive, too. So this may have awesome. It's very comprehensive, too. So this must have taken a while to put together. So you've got basically how to develop in Python for scientific people. And it's pretty comprehensive. I didn't go through too much, but I was like, let's look at some of the tutorials to see what we got.

And intro to development, that's nice. I was curious what they were doing for packaging because it's kind of one of those things I follow. And the packaging tutorial is pretty great. It touches on a whole bunch of stuff. Some at first, it's just talking about how to use Python packages, but then it's got a packaging guide where there's two forms of packaging. You've got simple packaging, and I'm glad they split this up because if you just have a bunch of Python stuff you're sharing,

you just need simple packaging. It goes through Hatchling, Flitcore, PDM backend, which I haven't played with, but setup tools. I've used three of the four so far. And I like that it highlighted that really, if you click through, all of the project table stuff just stays the same. And mostly what changes is you just change

the build system settings. Pretty awesome. But for scientific, you kind of have some complicated stuff too. So this is great. We've got descriptions on how to do Python packages with C++ and Fortran and C. And there's CMake. There's Maison and Rust with Cargo, with Maturin, and then in Sconce. I haven't played with that.

But really great stuff. And then it even, it talks about classic packaging, but don't do that. Stick with the compiled packaging or simple packaging. That's what you want. This is a great write-up. So thanks, Henry. And probably lots of other people working on this, too. it, you read it all yourself. I'm pretty impressed. But anyway, pretty cool. Yeah, that's a good one. Then this is how to package up these, your packages is really cool. Also, doing things like entry points. So you basically, once you pipx install something,

it becomes a command line. Oh, yeah. Capability is really awesome as well. There's a lot of neat things about packaging that I think not everyone is like, well, I don't want to put it into PyPI and have it open source. Well, maybe there's other interesting aspects still

I'm sharing internally. Nice. I hadn't checked out, this is kind of daring to do it on live but i checked out the test uh section and yay they talk about pytest so well i don't check that they're participating but good job guys just suggesting none the none for none we suggest not testing the world's simplest way to write tests. No. Yeah. Uh,

what about extras? You feel an extra? Um, I am feeling extra. Uh, do you want me to cover mine while I'm already up? Yeah, go for it.

Um, just to just, I was going to like do this extra attribute thing as an extra, but decided to just run with it. Um, the, uh, quick extra is, uh, I'm continuing to work on the course.

Chapter eight is up. Configuration files. It's a short one, but please don't skip it. It will mess you up if you don't understand this. So just a few minutes to cover some configuration. And one of the questions I always get is, what do the Dundra net files mean within a um in a test directory and i'm gonna keep it a secret you gotta watch the course um but it's nice it's a they're good

things and you're making good progress on this i am and it's a blast i'm getting great feedback from people some some people from uh pie bites uh are reviewing it and giving me feedback some people from jet brains um it's been it's been really good good community there so cool how about extras from you well i have some mastodon extras and i came across mona m-o-N-A app, which is a really cool client for Mastodon. I believe this is Mac only. Optimized for voice over. Optimized for Mac.

Native design. iPhone, iPad, and Mac. So sorry for people not on that one. But if you're on these, this is really, really nice. I tried Ivory. I know you and I spoke about it like a couple of weeks ago, maybe off air, but Ivory drives me crazy. It's like the big recommendation everyone has.

And this is super nice. It has a free version or you can pay like $15 once forever and then you have it, which is kind of nice. So people can check that out. I think I'm going to move to that. Another thing that we talked about the other day, Brian, is why are these like weird domains so expensive, right? Like we have pythonbytes.fm and to renew pythonbytes.fm is $170 a year. I'm like, why didn't we just go with.com?

Like what were we thinking? Come on people should have just done.com. But you know, when we came up with pythonbytes and we started it, the.fm was all the rage for all the podcasts. So there it is. And it's not a huge deal, but we were talking about where does that money go? And the biggest winner of these is the.ai, right? There's so many.ai domains these days, right? With all the AI startups.

And it turns out that the island of Anguilla is a tiny British territory with around 16,000 inhabitants, but its domain name is.ai. And so the software developer who manages the domain told Bloomberg it could generate $30 million for this 10% of the GDP of this tiny island because of the.ai domain. Anyway, I just thought that was interesting

and I'll just throw this in as an extra. What's the.fm, Dino? I don't know, actually. That's a really good question. Okay, on to the next one. Yeah, Vivaldi. We're both Vivaldi fans. And I really like a lot of things about Vivaldi. It got a lot faster on the desktop recently, which is super cool.

So now it's super, super fast. Not the kind of thing you can demonstrate well over screen sharing because the latency of the screen sharing outweighs the, yeah, that, but it has not been on iOS. And now finally, finally, finally it is out on iOS. How exciting. Yeah. I've just downloaded it recently.

I'm loving it. Yeah. So five days ago it came out. I've had it on my Android phone and I'm just like, ah, why don't I have this on my iPhone? This is in my, in my iPad.

It makes me sad. And I know people say, oh, why don't I have this on my iPhone? This is in my, in my iPad. It makes me sad. And I know people say, Michael, Apple's evil. This is really just WebKit wrapped in a Vivaldi shell. And yes, I know. I know. Thank you for letting me know again. I know that. Is it?

However, it's still really nice that if I open up Vivaldi, I could say, carry on where I was on my other machine. Show me the history from my Mac or from my Windows machine or from my other, right? The synchronizing is awesome. And the ad blocking is like next level. So if you go over here, you go to the ad blocking section, you can say block trackers and ads. Yes, that's cool but you can actually go through and check off a whole bunch of additional sources for tracking and blocking so you can say i want the easy list and i want ads from um allow action partners you can uncheck that if you want you want

the abp anti-circum uh circumvention list and a whole bunch of others. And you can even add, if you run like a pie hole, you can even add your own blocking list or ads and stuff. So like super, super cool. Sync plus ad blocking. I know it's web kit. I wish it weren't, but you know, it's better than not. So there it is. Yeah.

Uh, but I mean, it isn't really, I'm, I'm not, I don't really care what the, maybe I should, but I don't really care what the rendering engine is and all that stuff. It's the who's, who's taking the data and sharing it with who is the part that I really care about. Yeah, I, of course. And this, this checks that box, right? Yeah. I do hate when you go to a place and goes, you need to have Chrome to have the best experience. You're like, gosh, no way I can have Chrome on this device. You know what I mean? I, I have a, um, I, uh, brought in a guy to interview once because I was intrigued that

his website said, uh, uh, his personal website said best of feud with Netscape navigator. And that's awesome. I love it. Uh, so I love that little, and it even had like the animated little globe thing that was going on. Oh yeah, the little, like the little, like a ship's Janewell.

I remember that. Oh my God, those were good times. Yeah, it was great. Yeah, David out in the audience says, federated states of Micronesia, apparently. That's what.fm is. Yes, exactly. Niceronesia, apparently. That's what.fm is. Yes, exactly. Nice.

Thanks, David. I love having guests here helping us out. I know. It's awesome. Our audience is excellent. All right. You ready for a joke? Those are all my extras.

I am ready for a joke. All right. Let me set the stage. So there was the JFK speech, right, about going to the moon in the early 60s. And it was like, you know, Sir Edmund Hillary was asked, why did he climb Everest? He said, because it was there. And the moon is there and the stars are there. We're going to climb it, not because it is easy, but because it is hard.

Right? Like that? Something like that something like that yeah yeah i'll do that again that was fun no we do these we choose to do these things and the other is not because they are easy but because they are hard no the joke is this is the software one we do this not because it is easy but because we thought it would be easy exactly dang it why is it still three months and we're rewriting this section? It was supposed to take a week. What a bad idea.

Do this not because it's easy, but because we thought it'd be easy. Yeah. Okay. So David shares with us, there's the old school. Dot AM domain for Armenia. Dot AM instead of.fm.

Should we retire the.fm and go like talk radio.am? We could put like a phonography type of filter on our voices and make it sound like staticky and tinny. What would be great is to, yeah, do a filter and have like both. Have the.am and.fm versions and have it sound like you can simulate simulate going through a tunnel and have it get like a little staticky and then come back just periodically right why not yeah uh filter the high end and low end so there's just like the mid-range and that's it so i love it i love it with AM, you have to have the bad car speakers also. Oh yeah.

Anyway. Nice. Cool. Well, thanks again for a wonderful episode.