Python Bytes - #346 Have you lost your GIL?

Episode Date: August 2, 2023

Topics covered in this episode: A Steering Council notice about PEP 703 (Making the Global Interpreter Lock Optional in CPython) Google's post-cookie world could turn into DRM for the internet How ruff changed my Python programming habits pathlib api extended to use fsspec backends Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/346

Hello and welcome to Python Bites, where we deliver Python news and headlines directly to your earbuds. This is episode 346. I'm Michael Kennedy. And I'm Brian Ocken. And this episode is brought to you by us. Check out all of our things and support the show by taking part in them. We have our courses, the PyTest one Brian wrote, a bunch of other ones. I just did the Async Python and MongoDB combo with a little bit of fast API and load testing in there. That's a good one to check out.

And if you want to be part of the live stream, just Python by setup.fm slash live, and you'll see the next scheduled one that we got set up for you, queued up for you. Usually 11 a.m. Pacific on a Tuesday, as it is today, but that's not always the case. So be sure to check. It is August, Brian.

Yeah, the summer is moving along, and so is the Python. You have some big news to share, I know. I have a bunch of fun small things, but you got some big ones. You want to kick us off? So I think we've covered this before. There was a PEP 703 to make the global interpreter lock optional in CPython. And, well, we have some news from the steering council. So the steering council has come back.

There's a Thomas Wooders posted a steering council notice about PEP 703. And there's a lot of news in here that I think I'll let you help me dig through this. So I'm gonna jump down to the, so there's some assumptions and some discussion around this gill removal thing, but I'm going to jump to the short and midterm

and longterm actions. So it's going to be in stages. So the idea is in the short term, there'll be a no gill build as an experimental build um and that will be a part of possibly python their version 313 or 314 um looks like they're hoping for 313 but that's um so that what that's like next year or something like that so um and then okay so then we'll have two builds what do we do with that well Well, midterm is to have, and the default version is going to be the GIL version, of course,

just as usual as things go through it, as they figure out, and everybody deals with, like the rest of the Python environment and the community and all the third-party libraries figure out how to deal with the no-GIL version. So in the midterm is there's going to be um uh community hopefully community support to help uh help get through this uh no-gill part and and then long term the no-gill version becomes the

main gill or the main not the main gill the main version of python and we deprecate the gil version so the time frame looks like uh about the long term isn't super long it's like in the five plus year time frame well the plus means like who knows how long it will be but i think that there's a lot of energy around getting this done and getting a lot done in the next few years so yeah the the time frame of it being an option in 313 i would put it in october of next year of 2024. yeah and some some alphas and betas earlier then so yeah for sure it's pretty quick and then so there's just some caveats in here saying they're this everybody's excited about it, but they reserve the right to just cancel the whole thing

if it's really not sound, if it's not working out. So, but I don't think that'll happen. Yeah, yeah, we'll see how it goes. But we talked about this a little bit on our AMA episode about the value and basically what the other languages do. Most of them have some constructs,

but assume that developer is kind of responsible. You pointed out that it's not as burdensome as people might think, because most code is not parallel code and you don't have to worry about it. It's just kind of a no, no big deal. Don't have to think about it unless you're writing libraries where you know that will be used in a thread safe way. You know, things like that, then, then you do have to take that into account.

I think the trade-off is worth it. We're not trending towards a world where there are fewer CPU cores, right? It's only getting to the point where, for example, on my Mac, if I run some Python program, it has access to 10% of the capacity. If I run it on my sim racing computer, it has 1 16th access to 1 16th of the CPU capacity, right? It's just, if you want to work in modern hardware, you really need to have access to parallelism when you need it. So for me, this is a real positive. They would come along and dedicate three experienced engineering engineer years to this project to help make not just it happen in CPython, but to make it happen in the important libraries outside there like NumPy and those types of things. So I think for me, this is a super positive.

It's always been a little bit weird that Python has been so restricted. It's also one of those reasons that might, it's like one of those springboards that might eject someone from the community that they would otherwise love, right? You might be told, well, we got to get better parallelism. So we're not doing this in Python. We're going to do it in Go or some other place where you like, kind of like Python, but I guess it's not a fit anymore, right?

This would mean that it's a better fit in other places. So this is awesome. I know it's going to be a lot of work it's going to be another legacy python versus non-legacy python sort of deal hopefully not too bad i you know maybe having gone through it once the community go i will do it again it's not as bad as we thought yeah but i mean like the goal is so that you can have like if you're just writing a little uh a little single threaded thing or a little script or whatever the small sort of things you do with python also that those things are just as easy

as they are they've always been and just as fast uh that's what we want so yeah that's certainly the goal i'm just thinking about you know the code that uses the python api for important libraries that you know run calls the API, release the gill, take the gill. You know, when there's no more gill, does that library become unusable? Does that become a no op and it's just undefined behavior? You know what I mean? Like there will be some kind of consequence in the ecosystem.

I just, I'm not sure what the knock on effect will be. Yeah. Yeah. And it'll be interesting to see how like applications like Django and things like that deal with it. Yeah, for sure. And Liz out there points out, get through this long winded breakup with a gill. We have a semi toxic and ambivalent relationship with a gill. We love it and hate it at the same time. For sure. Yep, definitely. All right. Well, Thomas, thanks for the update on this.

Yeah. And it's going to be an interesting journey. Brian, it'll give us plenty to talk about. So that's cool. Yeah. Stick with this. Make sure you subscribe to Python Bytes to keep up on everything that's going on with the Gil breakup. Yeah, exactly.

It'll be fun. All right. I got to adjust my browser before we switch or I'm going to spoil the joke. All right. So I wanted to just a quick one. This is not Python specific, but it's super important for web developers. So we had Google with flock. Remember that federated, uh, cohorts of, uh, learning of cohorts.

Oh yeah. And then we had the sandbox groups, which was kind of like that. I don't remember exactly what that was called, but that was another technique because the flocks met with such resistance. Like you could really do some bad stuff by putting people into these groups. And so the, I'm not sure where the sandbox stands, but it turns out there's now a new proposal to try to get rid of third-party cookies. My feeling, I don't know for sure, but Google is trying to get rid of third-party cookies. My feeling, I don't know for sure, but Google's trying to get rid of third-party cookies because they're easy to block and it hurts their advertising and their retargeting

if you can just easily block it by checking a box, like, do you want to be tracked? No, thanks. You know, in your browser, right? Like Firefox does, like Vivaldi does. You know what? Oddly, I don't remember that setting being a default in Chrome

that it just offers to block the Google network. That's interesting. Must have been an oversight. So anyway, they're now out with a new thing. And over on Bleeping Computer, they say browser developers push back against Google's Web DRM. the different websites can choose or be made to choose to force you to verify certain things that basically prohibit things like ad blocking, like prove that you don't have ad blocking on so you can visit my website or I'm going to tell you you can't, right? New authentication system could let websites block extensions, jailbroken devices, and other important things. So the headline is,

Google has been trying to implement plans to move beyond cookies for years. That sounds great, comma, without denying its partners and itself the means to sell targeted ads, which form the backbone of the company's revenue. This new one comes, proposal to guarantee user privacy and security comes at the cost of freedom of functionality, aka the open web. So there's a bunch of, it talks a little bit about how this works, but basically the website has to come along and decide what browsers it trusts and the web browsers themselves have to implement a mechanism to guarantee who they are. Not surprisingly, Vivaldi comes along as one of the first people to push back against this and calling it dangerous.

If an entity has the power to decide which browsers are trusted and which are not, they say, there's no guarantee that they will trust any given browser. And a new one that comes along would not, by default, not be trusted until it's somehow proven itself to the people who run that website and what about the next website what about the next and so on you know Vivaldi's kind of in an interesting position like this they they're just they use a fork of chrome that's you know stripped down of a lot of things but when you go to a website or you look at the user agent that's how browsers nowadays tell people who they are, right? You can just send whatever you want, but it usually is a consistent thing. Vivaldi lies and doesn't say we're Vivaldi version, I don't know, whatever version we're on. It says I'm Chrome and whatever the current version of Chrome is, it just says that. That

way websites don't go, oh, you're going to need to use Chrome because your web browser isn't one of those. It's like it's like it actually is internally identical. Just it's got a different name, right? So they're in this situation where they kind of lie to the world about who they are, the web server world. And this would kind of break that as well. The Brave team says they don't really care. They're not shipping it.

We'll see if that's a problem or not. And Mozilla doesn't have an official opinion, but one of the folks said the mechanisms to attempt to restrict these choices are harmful to the openness of the web ecosystem. And this one might be interesting too, Brian. Additionally, the use cases listed depend upon the ability to detect non-human traffic, which as described would likely obstruct things like web assistive technologies or automated testing. So anyway, this is one of those things that probably deserves enough interest from people in the Python web world to just voice a little bit of no.

No, the answer is no. Yeah, and yeah. So anyway, it's not super actionable. It's not a thing that's out there yet, but like the previous two, it's, it's the folks behind the website of Google, the web browser side of Google, the web standard side of Google saying, how can we basically change the web so that it's so that we will be able to completely still control and track and sell you right right now The third party cookie thing is not looking good even you know Parliaments and Congress have gotten into it and said oh you can't have these cookies We have these stupid pop-ups everywhere this would make those all go away, and you would just be like part of the machine You know what I mean? This is just the third the third version of how do we make them part of the machine? And this one, I think, is probably the worst, because not only is it something that would go into Chrome, which was the previous two,

but this is something that would basically make the websites reject browsers. You might say, Michael, why would a website do this? I mean, obviously, terrible places like CNN that have some like 45 trackers they put on you every visit. You might see why they would do that. But why would, you know, random little website not want you to come? Well, there could be things like, oh, do you want to have Google ads on your site or any double click or any ads from any ad network? Well, we're not going to allow you to have them at all unless your website has this, you know, please reject the ones that we don't like sort of setting turned on. And here's the script will help you do that.

So it's not ideal. Yeah. It's like nobody can make money off of traffic except for me sort of thing. Yeah. It's, it's, I mean, it's pretty bad. It's kind of a monopoly type of situation. One in the browser space and then two in the ad space. So kind of both sides of that markets. It's, it's pretty

not good i wonder how many people are turning off ads uh because i mean i've gone i've got vivaldi running and it's turning off ads and i i run into websites that say i did you can't read this unless you turn on if turn off your ad blocker and i'm like well i'm not gonna read it then i'll go somewhere else um yeah but how many people really do i mean it can't be that many many of us i would say for developers the developer audience looking at when i used to run google analytics before i decided they were evil and turn them off it was about 40 to 50 percent of the traffic would not show up yeah okay so that it's not quite all blocking there could be robots like you know requests doing a request like python requests

do a request that also wouldn't register but that's that's pretty good and just you know yeah we'll go back to like magazine ads never had tracking in them and they did fine yeah exactly so i mean it's not just we're trying to be jerks to websites and we don't want to pay for stuff or we don't look at ads like this headline how long is this this is just you know six months ago or nine months ago hackers abuse google ads to spread malware and legit software right there's just stuff over and over plus what you know there's all like the reselling stuff the nsa cia stuff uh yeah buying your i mean there's just lots of it's not just i don't want to see ads there's really negative effects to these things and it's

honestly it's a little bit disappointing that google is doing this right and we have advertisers and we put ads on our uh website for the advertiser and it's but it's there's no tracking so and they're not even blocked because we're not trying to send it through some creepy network with retargeting. We're just, we tell you things that we've, we've evaluated and thought you might like, and people have paid us to help spread the word. And, you know, there's certainly ways you could do that, right? Like this last thing, this page we're on here, if it had ads, I don't know if it does or

not, cause I'm blocking them like you. But if it did, it could easily show you ads about privacy. It could show you ads about browsers. There's like certainly relevant ads that can be put on here just based on the context of the page, not who is viewing it. And those work almost as well without all the negative stuff. So anyway, one more thing to keep an eye on folks. And Christopher on the audience says, interesting how Google has lately continuously, continuously been trying to do the exact opposite of don't be evil. Like shareholder value, Christopher, shareholder value. Let's have it. Well, all right, right over to you's it's pretty rough news man so it's it's okay the last two

sounded bad too and they both got you know yeah pushed out of existence i suspect this one will as well but i was just trying to do a rough transition um oh that that was pretty rough so anyway um so there's a there's an article oh Oh, I lost the author. It's from Mattia, Mattias from Mattias, how rough changed my Python programming habits. And we've covered rough, I think several times on the show, but I I and I've used it on a lot of projects. But, but I was excited to see this because it's some stuff about rough that I, I haven't been paying attention to. It just works and it's really fast, but there's a whole bunch of cool stuff. So this, in this article talks about he talks about using pre-commit or has

long been using pre-commit and flakade and black and eyesore and all that sort of stuff. And I knew that rough was going to start picking up some of these extra things, but I didn't know how far it went. So in this article, he talks about using different rules within, uh, within rough,

like, uh, like pie flakes and pie code styles. So that's, uh, and the McCabe for, uh, cyclomatic complexity checker,

uh, just to make sure that you don't have too many nested huge loops and different things like that. I love that magic. So all these things you can turn on within, just within rough. Isort, pep8 naming, that's cool.

PyUpgrade, love that. Flake8 2020. And I'm going to shortly show where you can look these up. So there's a whole bunch of really cool things. really like bugbear uh flake 8 bugbear also so apparently that's been supported for almost a year within rough um like a django pi simplify but these these are just the the ones that this person likes or is he trying out and using um so i went ahead and kept looking so that's that's this

article which is a pretty decent article about using how it changes one of the things i want to highlight and i've been trying to use i when i tried to use pre-commit a while ago i haven't been using it lately because a lot of these things take a bit uh they take a bit of time and it's um he writes uh there's always a trade-off between development speed, i.e. waiting on git commit is very boring, and strictness. And I have to agree. So what I've been doing is mostly putting these tests in CI and not doing it as part of pre-commit. But having it within rough is super fast. And you can go back to using pre-commit with rough and just turn on a bunch of these things so uh link also linking to the rough configuration uh documentation and

for example these aren't all these things aren't turned on by default we get um uh i think the air the uh pi code style warnings oh it does not turn on it doesn't turn on pi code style warnings. Oh, it does not turn on, it doesn't turn on PyCode style warnings. By default, it turns on PyCode style errors and PyFlakes errors. So those are turned on by default. And you turn it on with this, like within the Toml file with like a select equal.

And then you have like just a string of characters, like E for PyCode style, F for PyFlakes. And then there's a whole list of these. So there's an example of you do B for like a bugbear. And then there's another page for rules, where in the rules list all of the different different ones. So like the PyFlakes start with F. So if you would add F for that, there's a whole ton of these things that I'm excited about trying to turn, turn more of them on. And I have on one of my projects and it's just super fast. So you just like within pre-commit. So there, when I try to commit something, it'll run these on the files that I'm changed and try to run, or I can't, I actually don't know it's happening so fast I don't care maybe it does the whole thing but it's just super fast now and I don't have to wait to commit but I know the

checks are being done super fun the other bit that I wanted to the one last link I want to share is astral which is the now company around rough has a rough pre-commit hook that you can set up for get for GitHub actions. So if you just add a this little snippet that we have a link to there's just a astral astral dash sh rough pre commit repository with a river with a version and with the hook, and that will make it so that your project runs within GitHub Actions also. So super fun. Okay. Yeah, very cool. I also like having it set up so not as a pre-commit hook,

but also as a PyCharm plugin. Yeah. Oh, and I just misspoke. This isn't a GitHub Action. It's a pre-commit hook. So it hooks rough in with pre-commit. Sorry about that. Yeah, yeah, nice.

Well, so that way, as you're typing and you see the stuff, it just puts a little squigglies in there and suggests sometimes it can even auto-fix it, you know, alt-enter, fix this thing by, for example, applying strict equals true in the zip. Okay. Stuff like that. So you just have rough running in PyCharm.

It just runs constantly as you type. But again, it's so fast, you don't notice. Well, what are the fun- 10 cores, I can do it over there. One of the fun things with having it be a commit hook also is you can set it up to auto-fix. So ref isn't just you have errors, but it can find the errors and fix them

and not even tell you about them. It just fixes. That's nice. Yeah. Um, so like, well, I turned on bug bear or I can't remember what I turned on one of the extra rules and it showed some, some, like some spaces at the end of the lines. And then I added the command to just fix them for me. And then it just didn't do it. It just would fix them without even telling me, which is great. It's way better than complaining. Yeah, like I sort stuff.

Well, if I'm just sort them, if I got them in the wrong order, just reorder them, that's fine. Yeah, and Henry says Rupp also has a VS Code language server plugin as well. So either side of the fence that you sit on for those, that's all good. All right, on to the next, that's all good. All right.

Onto the next, onto the last. All right. Onto the last. We've discussed FS spec, file system specification, I'm guessing. And the idea of FS spec was that what you can do is you can have all these different file systems that when you would do like open, you know, open some file name, encoding is this append, so on that kind of work, you could point it at different places besides the hard drive, you could point it at S3, you could point it at some kind of web dev or some other, you know, blob storage, network drive, all these different things that you could possibly do. And you have exactly the same API as just opening a standard file, right? So that was like with open as file stream,

you go do your thing, right? Well, a lot of us are fans of using Pathlib, right? So create the path, you do like a cool thing, you do the slash, the division is overridden to look like you can separate the file names and all that. Right. So there's this thing called universal path lib now that's looks like this project was created two months ago and this was sent into us, I want to say thank you to Justin Flannery. Remember he had that, that camping CLI thing that we could find campsites

that are sold out and stuff. Yeah. And so we talked about that a few months ago. So he sent this in and said, look, FS spec is awesome, but now there's a universal path lib and it extends the built-in path lib. So the path lib module dot path API to use basically FS spec, which is awesome.

And so if you want to talk to things that are not file systems, but you really like the path class, then you can use this one. Either saying, you know, import it as upath. You could also just import it as path if you want, as long as you're not mixing it up with the other and just treat it the same, right? So for example, you could say upath instead of giving it a directory name or a file name, you would say like S3 colon, you know, slash slash some path to your S3 information slash some file, right? And you could ask, what is the name? What is the suffix? Does it exist? And read it and so on, just like

you would with a normal file. But now this thing is in S3. Here we have file. We azure storage we have http and https we have hadoop google cloud s3 web dev and one i think that would be really cool for testing brian is memory yeah right you want to create a file work with it make sure you know pass this path like object to other apis let it do things and and then it just goes away when it goes away right that's That's pretty cool. Yeah, I love using memory file systems for testing. Yeah, absolutely. Like SQLite has one memory colon, I think it is, or maybe two colons on the front. The same thing for the database. But yeah, this is really handy. You might be wondering, like, why would I ever use memory? Because you don't want to touch the file system. You just need to pass a file, and there you go. So super simple. There's a little example notebook that I blasted past somewhere. People can check that out if they want to just see how to use simple. There's a little example notebook that I blasted past somewhere.

People can check that out if they want to just see how to use it. Okay. A bunch of examples play with it. So if you are working with these alternate file systems, this is definitely worth giving it a look. Yeah. Cool. Nice. All right.

Well, I have no extras. How extra are you feeling today? I guess I don't really have any extras either. Just thinking something funny might be nice. I think so. And I know that you've done, you do a lot of C these days still, and I've traditionally done some, although not too much lately. So I want to have both a joke and something to help people think about pointers

and pointers to pointers. Because Python is all about pointers, even though we don't realize it, right? When you create a variable, it points out to a thing. You've got a list. It doesn't hold the value of the thing that's in the list. It holds the pointers that point to the things that are the values. So here's the joke. There's an int just written out in memory, just out in space floating there. And there's an int star, so a pointer to the int. It's like an anime character pointing out to it. And then what is a pointer to a pointer a pointer well it's just the anime character pointing at the it's all right that points over to the other one what do you think of this i i think it's just a great i agree with the comment it's just a great way to describe pointers um actually somebody pointing yeah yeah it seems completely non non-scary whereas thinking about

int star star avoid star star something like that is pretty crazy. Wow. It's got 18,000 likes on, I don't know what it's on. What website is this, Brian? The company formerly known as Twitter. Yes. And yet what is the title of the pages? So and so on Twitter and the domain is twitter.com, but the logo is X and I don't know what a mess this is. But anyway, the joke is twitter.com but the logo is x and i don't know what a what a mess this is but anyway the joke is funny well the next so the the and the icons there's an x for closing which is

still confusing and yeah there's two x's in my my tab up there yeah the left x and the right x yeah we just hopefully nobody renames their company greater than greater than and weird things like that. Yeah, absolutely. So, hey, I want to add one more funny thing. This is not really programming related, but I've been watching a lot of like streaming, catching up on some TV shows. I'm watching. I can't remember the name of it now. Catching up and I want to skip through the beginning.

Have you ever done that on streaming shows? You skip the intro? Right. So I saw this the other day. It just cracked me up. Apparently, it's rude to poke someone in the head, the forehead and say skip intro when they start talking to you. I know people like this.

I've heard the first out. Yeah, like, okay, can you skip the preamble and just go to the whatever you wanted to talk to me about? Pretty funny. I imagine it probably is a little rude, but you know, sometimes you're in a hurry. You've heard it three times already, so get to the end.

Awesome talking with you as usual. Indeed. Thanks for being here. Thank you to everyone who is listening and everyone who participated in the live stream. See you all next time.