Python Bytes - #286 Unreasonable f-strings

Episode Date: June 3, 2022

Topics covered in this episode: The Python GIL: Past, Present, and Future Announcing the PyOxy Python Runner The unreasonable effectiveness of f-strings and re.VERBOSE PyCharm PR Management Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/286

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 286, recorded May 31st, 2022. I'm Michael Kennedy. And I'm Brian Ocken. And this episode is brought to you by us. If you're looking to learn Python, check out all the courses over at TalkPython Training. If you want to get better with testing, check out Brian's PyTest book, second edition. Yes.

Yeah. Yes, indeed. Indeed. All right. Well,Test book. Second edition? Yes. Yeah. Yes, indeed. Indeed. All right. Well, let's jump right into it, Brian. Okay. Tell me about the gill. I do want to talk about the gill, but I was just realizing that we're at 286.

I just started watching Big Bang Theory with my youngest kid, and there's 279 episodes of them. So we have more than – it went on forever and we have more episodes than them. So that's, it's quite the milestone. I mean, you divide that by 52. That's a lot of years. We've been doing this for a while. Yeah. So Python and the GIL.

So the GIL is a thing, right? It's like something everybody knows about. I think people know about, but not too much. So, I mean, you kind of, it means we can't do multiprocessing, but we can, but we can't. So what is it? And so I really like this article. So this is, this is an article called the Python GIL past, present, and future, because we have been talking about some of the no GIL options that are coming. And so this is really nice. It's, um, who was it? Gary, uh, Barry Warsaw and, uh, Pavel Polowicz, I think anyway, uh, really great article on, uh, the Backblaze blog. And it starts out and

goes through, um, sort of what is the gill, uh, why we have it. It's a lot around, um, the way in, in some of the history is around around reference counting. And maybe you knew that off the top of your head, but I forgot that reference counting is something that's how we do memory management and all sorts of stuff within Python. But to make it an easy implementation, there's this global interpreter lock. And that just basically instead of lock like other code you lock individual items shared memory that you're using we just say the interpreter let's just lock that and all of the variables that it uses it's not just reference counts but other things too and and this is just it's a fairly quick read, but enough detail that you really kind of understand what's going on.

It does talk through reference counting, talks through some of the advantages, why it's really kind of handy to have it around, but then attempts to remove it, including, like, I didn't know about this one. There was a free threading one in 99. From 1999. Yeah, that's, I didn't know about that one. There was a free threading one in 99. From 1999. Yeah, I didn't know about that one. It actually was successful. It just made your Python code really slow. So not good. And then Larry Hastings' Galectomy, he's had a lot of PyCon talks or a few PyCon talks around this.

So that's an interesting thing to read about there's a little more oh there's quite a few interesting bits in there but it never really got back into main python but we're getting a little closer where with this no gill stuff so there's no GIL stuff going on with Sam Gross. Sam Gross, yeah. And then also we've got Guido and others in Microsoft now working on this problem of speeding it up and everything. So really great discussion if you want to catch up on all the history of the Gill and where it's going. And then a little prediction at the end of we're going to see speed ups because of these efforts. We're going to see speed ups in the next few years. It's just going to get faster. And then if we get no gill, it might be maybe five years from now-ish.

Yeah, it's pretty exciting. There's some very neat work. The stuff that Sam gross is doing is is quite interesting yeah what's really cool is it's not just theory or people playing right the python 311 is a lot faster than the previous pythons already and it doesn't even really touch the gill side of things so there's actual progress being made yeah um and it's it's actually it's pretty exciting pretty exciting time to be part of uh part of python so like it it's never not been exciting but you know yeah it only gets

more exciting how's that yeah uh so hey let me ask you a meta question so this is on the backblaze blog so backblaze.com is a one of the i think better sort of backup my computer somewhere off-site sort of things yeah and the fact that they're blogging about the python gill is kind of interesting i wonder if they're a big python shop and we just didn't know it well uh there's like i didn't know it's on their blog but i mean we got uh barry warsaw is written by Barry Warsaw. I'm Powell. Yeah. So. Are they, is he, he's, he says he's currently a senior engineer at LinkedIn. So. Yeah.

Yeah. I'm not sure. Sometimes life's confusing. Yeah. Not sure how this ended up there, but yeah. No, but it's cool to see Backblaze support in the Python world with some, some dives into that. Yeah. Maybe somebody from Backblaspace can reach out to us

and tell us what's going on. Yeah, that'd be great. Also great, wouldn't it be cool, Brian, if we could take our Python program, whether this is a CLI or some other app that we need to have for people to run, and instead of saying, well, make sure you have this version of Python at least

and make sure you've set up a virtual environment, you've installed these things, and then you've got this in the path path and then you've got this environment variable. Now you can try to run our thing I gave you. If you could just give them a single binary that contained all the Python, contained the standard library, and you said you run this and that's your app. That's not bad, right? So let me introduce you to PyOy the pyoxy python runner now a while ago

we spoke about this thing called um pyox oxygen no pyoxidizer that's what it was pyoxidizer and this is kind of like an advanced version of py2exe or py2app or or pi installer but what's really nice is that it bundles the stuff up just sort of all into one thing and it runs out of memory instead of creating like a file structure of the files that you need and then you know sort of altering the path so like a copy of python will run that and so on so this pyoxy attempts to make the features of PyOxidizer more accessible and simpler for people. So here's the deal. It is an executable program for running Python interpreters. So you can just give somebody this one PyOxy file, and they can, instead of saying Python

this, they can say PyOxy some Python file, right? You can say like run Python or run Python dash dash and give it a file or something along those lines right and it's just a thing you copy around you don't have to worry about installing it or configuring your system or anything like that so it's kind of a really easy way to pass around and executable that is python and the standard library without all the overhead or setup of actually installing a python runtime okay well okay yeah yeah so let me go back up here that's pretty cool that's one it's a a single file it's in a single file that is a highly

portable distribution of c python so for example on mac os and linux it statically links in a lot of the files that are dependencies that you might find you know so if you try to run something in python it might go like oh you have to install this other like C library in order for it to work correctly. So all those things are also like its dependencies are statically compiled into it, which is pretty cool. Another thing that's interesting is it gives you a lot more control over the interpreter and how it behaves. Like you can pass it configuration files and do all sorts of crazy stuff to really tweak the behavior of the runtime, the Python runtime that it comes with. Yeah. And it's just to sort of make PyOxidizer apps more accessible.

Okay. So, yeah. So it gives you, it's built with PyOxidizer. And basically it gives you its own standalone one. Like I said, it has the whole interpreter. It has the stuff statically linked. It has the standard library. And I think you can link your own libraries into it as well, which is pretty awesome. And you could also give it, like I said, more control. So you can create

a configuration YAML file. And when you run it, you can say, I want you to debug allocations, or I want you to run this command to initialize the interpreter before it starts to run any code and things like that. So you can give it this YAML config file and so on. So anyway, it's a pretty new project. This is from Gregory Sork. And he says, it's very young. I hacked it together to try to get better exposure for PyOxidizer for people who want to build things that they can distribute more easily and so on. And for me, he's talking about, oh, there's some certain circumstances where maybe you want to have people run your script so you can send the script and then this thing and so on. It's supposed to be sort of on par

with zip app, which lets you ship the zipped up thing of your app and then run it. But the benefit is it already comes with Python. Like a lot of these zip, shiv, pex, et cetera, they all require that Python is installed on the system. And then you can run that right with its dependencies. But this is like, no, it even comes with Python. So all that is pretty good. What I would love to see out of this is could I take this and build some sort of UI framework, pair that, I don't know, maybe with PyScript, and get a UI framework

I can ship to people and just go download this, double-click it, there's your app. Who knows? But anyway, if you're shipping Python to people, this might be worth checking out. Kim out in the audience asked uh it looks very handy does it make cross-platform uh cross-platform executable i think you might

have to run a build command or something or ship the right binary for each platform but it does support mac os windows and linux so that's pretty cool now um maybe you covered this and i just missed it um what does it does it deal with my dependencies? Like if I have third-party dependencies? That's what I'm not 100% sure how to do. And I looked through it. I was looking for that. There's this oxidized importer.

Okay. Which it talks about like this embeds a copy of the Python standard library and it imports from memory using the oxidized importer. I think that you could use that as well for your own dependencies and i'm pretty sure that's the goal of this this pi oxidizer project in the first place so i would think so but nothing from this article was like really clear like and here's how you bring in a request or httpx or whatever you're going to bring along you know yeah yeah okay so or pi test um i was actually just thinking if i you know yeah yeah okay so or pi test um i was actually just thinking it'd be if i you know run these tests well what what version are you using okay just fine here just install this and

run run run this test with this and that would be yeah exactly yeah alvaro's you know when i was talking about ui stuff he says what about PyOxy plus textual? That'd be pretty neat as well. Yeah. Just around the corner. Yeah. Cool. All right. Well, that's PyOxy. People can check it out. All right. Well, how about regular expressions? Do you like those? When they work, I love them. When they don't. I'm like, what is, what is this madness? I've gotten myself into. Okay. So I was almost afraid to like, talk about this, but it's a really awesome article because, because sometimes you need regular expressions. And, um, and one of the things there's lots of that I like about this article. What I'm talking about is an article called, um, uh, the unreasonable effectiveness of F strings and verbose. So regular expressions and all languages, including,

I think, including Python have, have a notion of verbose mode. And I've always been enticed by this, the verbose mode idea is that you can like instead of having this weird you still have the hard to read uh regular expression stuff but you can put comment you can put spaces and comments in it and uh and then it could be multi-line right what it could be multi-line yeah it could be multi-line and you can have a bunch of uh comments in there to say oh this part of the regular expression what it's doing is this, you know, and then so it makes it little little snippets of regular expression are easier to read, you're like, Oh, yeah, I can see that it's doing that. Great. When it's when you clog it all together, which makes it really hard to read. So and then, and then, of course, you can build up

strings in Python with F strings strings and do string replacement how awesome so that's what this article is about is really talking about uh using um uh taking some of the complicated bits of your regular expression or duplicated bits where it shows up several times in your regular expression and pulling that out that string out and even maybe commenting that and then uh dropping it in with f strings to build up a complex regular expression it's it's brilliant and it also is one of the best explanations i've ever read on on uh verbose regular expressions so it's pretty cool yeah i never really thought to refactor the pattern so that you have multiple strings.

Yeah. We like, here's the part that's repeated. So we'll have a pattern that that's that thing and then F string it into the other pieces. Yeah. But it makes perfect sense. Of course you should do it. Yeah. And now, I mean, I don't use it.

I mean, actually I kind of do. I use regular expressions on a regular basis. But not like every day. But it's always something that whenever I put it in there, I'm like, is anybody else going to be able to read this? And so I put comments around it. But I've never really embraced verbose mode. And I'm definitely going to use this now. Yeah, for sure. The other thing real quick is you can have f strings and you can have raw

strings and you can have raw f strings apparently in python so the prefix is fr triple quote multi line like that's that's some serious string action happening right there man yeah um raw f strings nice yeah yeah and um neil out in the audience asks you, Brian, do you like your regular expressions, greedy or non-greedy? Okay. It depends on what you're trying to do. Yeah. I don't want mine to be greedy. Let's share. Share kindly. All right. Are we ready for the next one? Yes. Cool. All right. So this one I want to talk about, and I'm going to start off from a weird place, from a tweet that I gave because I can't find anywhere else to talk about it.

Like this is why this needs talking about. So I just want to sort of highlight some really cool stuff for people who are using PyCharm and doing open source stuff that involves pull requests. And these can be pull requests to your private repo. Like one team member gave a PR back to the repo. It doesn't have to be open, open source. Okay. So if you're using PyCharm, check this out. Let me pull up this screen. Come on. Oh, I'm not logged in. Oh, there we go. I thought it might not. Let me get the image up. Anyway, so check this out. So people

are probably familiar with the Git support in PyCharm. you know vs code pycharm they all have really good git support you can see your changes a lot of times you have this um sort of oh this gray little over overlay that says who recently edited a file or a line it's really cool to go back with like git lens turned on for c python because you'll see stuff from like 30 years ago, Guido made this commit, and here was the commit message. You're like, what? This is crazy, right? But working with PRs is one of the things that doesn't normally seem to fit. And so I don't know when this got in there, but not too recently. There's now a pull request tab. So if you open a GitHub repository, if you're logged into GitHub, and that GitHub repository has a pull request, and you open a GitHub repository, if you're logged into GitHub and that GitHub repository

has a pull request and you open up the section, it gives you a list of all the open, and I guess maybe you can toggle it closed as well, but all the lists of the open pull requests. If you click on one of them, it'll show you all the details of the pull request, the whole conversation if you click on that, who's the reviewer, you can change that, who's it assigned to, whether there's labels,? You can change that. Who's it assigned to? Whether there's labels, whether it's been merged. Wow. And then like on the right, you have this whole conversation.

You can see sort of the flow. This person commented this and they made these few more changes. Then they commented again here. This is all really pretty neat. But the thing that is noteworthy, the reason I brought this up is there's a button that says check out. So instead of trying to create a branch or fork their repo and checking, like you just, I don't know where this repo or this PR came from or what branch it was

against. I just want to check it out and run it locally, run the test against it, see what it looks like. Yeah. Go. Isn't that awesome? Yeah,

that is really, really awesome. I like it. Yeah. Cause normally you're juggling a lot. Maybe you have CI containers integration that tells tells you the linting pass. But other times you want to actually just get it and run it and play with it and decide,

did this make the code slower? Did it make it faster? Do I like how the result of this feature looks, especially if it's like a UI thing or something, right? Yeah. So I think this is super cool. And the reason I bring it up is there's nowhere in the documentations.

Like if you go over to PyCharm here and you look for pull requests, they don't even mention this, right? So I don't think people necessarily know about this. It's a stealth feature. It's a stealth feature. It seems totally worth highlighting because dealing with PRs in that way is actually kind of janky, right? It's like it takes a little bit of understanding Git maybe more than, and GitHub more than a lot of people would off the bat. So if you just go click, check out, play with it, nope, I don't want it,

or yes, I love it, I think that's great. I would still actually manage the PR in GitHub, but the ability to explore it super quick and easy within your project, I think is super cool. Yeah, and quickly grab the code because I mean, I know in GitHub, you can get it, you can clone their branch. Yeah, exactly. But it's hard.

It's like not super easy, right? It's several clicks at least to find it. Yeah. So, yeah. Right, and do you actually want it cloned to your account? Maybe not. I just want to check out the code, you know?

So grab the right commit, hash, shot, whatever. whatever um also really quick while we're on this topic i just want to give a quick shout out to this so if you go to just the jetbrains.com slash pie charm right now across the top there's this oh here there's this cool uh upcoming webinar 10 things and get things you wish you knew in pie charm very oddly that that is me. So anyway, if you want to, I'm going to do a webcast next week on June 7th about a whole bunch of fun things in PyCharm and UI tools and whatnot. So if people want to

check that out, they definitely can. Yeah, alright. So that's it for my thing. Check out, if you're using PyCharm, check out this pull request thing. It's even in the free version of PyCharm, so it's worth noting. Let's see, a really, really check out this pull request thing. It's even in the free version of PyCharm. So it's worth noting. Nice. Let's see, a really, really quick real-time follow-up.

Roman Wright, hey, Roman, says, yes, it helps me sort out PRs so much. I totally agree. And then Wolf says, is it available in IntelliJ and does it work with GitLab? I don't know if it works with GitLab. I'm quite sure it's available intelligent in intelligent because like pycharm is just basically specialized intelligent so i'd i'd be very surprised if it

weren't available in intelligent but i i can't say for sure in gitlab i would check the documentation but i also can't find that so um anyway so on the pycharm side there's isn't there supposed to like a new some ui changes that happened recently or you know anyway possibly oh one of the things that changed recently that was uh i actually kept stumbling into is like on the get stuff is they have little overlays of who made changes when kind of like that get lens from vs code i described yeah they started adding that and i kept clicking it by accident trying to click on the line so that's like why does it keep popping up a diff oh i see okay so i kind of crashed into it but yeah okay anyway if you're doing prs check this out uh if you're doing prs and using five charm check

this out it's definitely useful all right well so uh we have no guests so that's our items for today right it is it is well we would be remiss to not have extras though of course so yeah you want to do the extras first your extras first sure um i just got one um so the it was a a couple weeks ago we i think we talked about um um about panda's tutor uh switching to uh piodide i think um yep um so there's a there's an article that came out on the piodide blog. Um, that is a panda about pandas tutors switching to, uh, uh, Piedide Piedide. Uh, and so, uh, one of the, the reason why I bring it up is because there's some cool stuff in here. It's a, it's not just, Hey, we did this, but it's also why did we do it and kind of walking through some of the stuff, how they did a self-contained Pandas tutor wheel, you know, dealing with JavaScript and imports and all that sort of stuff and the fine tuning and what work has to go further. So other people that are like, hey, this Piedide actually proof of concept, it's starting to look cool. I want to do this too. This might be a nice thing for people to check out.

Yeah, this is super nice. And I definitely think the, what are the rough edges and what worked really well? Those types of lessons for now are going to be really valuable for people trying to adopt this. Yeah, that's all my extras. Awesome.

All right. Well, let's just make it a straight run for Paiodide and and PyScript right out. So last week I did two videos on PyScript, which is based on PyEyeDyne as well. So first one is a 30 minutes video. I sort of hinted that I was working on this, but remember I said it wasn't quite released yet, Brian? Yeah. Last episode.

So now it's out. So now you can check out, I'll link to it. You can check out my 30-minute Python, PyScript, WebAssembly, Python web apps running locally, which is super fun. It's all about not just, you know, let's like take some Python and put it in the browser, but how can you use some like PyDash config settings to override where PyScript actually comes from. So the Pyodide, the runtime comes from, so you can like put that locally in your app and then download it offline and all sorts of fun stuff there. Cool.

So basically building an offline progressive web app that is like, can be almost 100% offline. This one calls an API, so it still needs its API. Anyway, that was really fun. And then the next day after releasing that, it occurred to me, do you know what supports progressive web apps? iOS and Android. So I created a second video called Python iOS Web Apps with PyScript and Offline PWAs.

And so here is, if you're looking at the screen, there's a, on my iPad, there is a web app that runs PyScript talking to a Flask-based API over the internet. And it is basically, you can go up and basically say, install this as an app on my iPad. And when you launch it, that install process already downloads the WebAssembly and the standard library and all the WebAssembly stuff that makes it large. So you just, boom, and it opens up within like less than two seconds running Python client side on iOS. That's so cool. I'm seeing lots of uses for this. I mean, people could do like dashboards of pipelines and all sorts of stuff and have them be on tablets now. Yeah, absolutely. Yeah. It's I mean, once you've already made it a progressive web app, it's like a few button clicks. This whole video about this iOS thing with setting the stage and everything, it's three minutes. It's not a hard process to make it happen, but it's just not obvious and it's really cool that it works that way.

Cool. For my items, I do have a joke. Nice. And I know you might have a joke as well. Yeah. Awesome. Okay. I'll go with mine first because mine's easy. And then I have a bonus joke for you based on your topic that I didn't see coming. Okay. Okay.

So this one comes from Brian Skin. Thanks, Brian. I just sort of pointed out a joke from Seth. And it says, people are out here just dangerously throwing around SQL abstraction layers. And if we aren't careful, someone's going to lose an arm and orm it's pretty good yeah it's really good uh yeah actually if you are careful you can lose your arm for good yes and still be still be all right yeah all right so here here's the here's the real-time joke uh follow-up that uh based on your your um okay reg x1 ask me why i'm looking like this why well that's just my regular expression Here's the real-time joke follow-up based on your Reg X one.

Ask me why I'm looking like this. Why? Well, that's just my regular expression. That's terrible. It's really bad. All right. I wish I had somebody in my life I could share that with other than you. Okay.

If other people listen to the show, we could share it. Okay. So there's all sorts of AI and stuff going into, uh, code editors now that make them like do all sorts of stuff to help you out. Right. So this is, uh, um, Kermit the frog looking at a, uh, out of rainy window. It's when you've barely started writing a line of code and your ID already lets you know about three syntax errors, two runtime errors, errors a grammar mistake five misdemeanors contempt of court and treason

just getting started why is it so hard i love it that's really good anyway yeah i kind of felt that way today i was i was playing around with some stuff i'm just like what is good this is over the top this thing yeah so but no good i'm glad the editors can help us out but sometimes i feel like an idiot so yes anyway indeed well great being here with you good being here with you indeed thanks everybody for coming bye everyone thanks for being here bye see you next time