Python Bytes - #251 A 95% complete episode (wait for it)

Episode Date: September 22, 2021

Topics covered in this episode: auto-optional Making World-Class Docs Takes Effort Starship JMESPath pedalboard - audio effects library PEP 665 (and the journey so far) Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/251

Hey there, thanks for listening. Before we jump into this episode, I just want to remind you that this episode is brought to you by us over at TalkPython Training and Brian through his PyTest book. So if you want to get hands-on and learn something with Python, be sure to consider our courses over at TalkPython Training.

Visit them via pythonbytes.fm slash courses. And if you're looking to do testing and get better with PyTest, check out Brian's book at pythonbytes.fm slash PyTest. Enjoy the episode. Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 251, recorded September 22nd.

Wow, 2021. I am Brian Ocken. I'm Michael Kennedy. And I'm Brett Cannon. So do we need to introduce Brett? I mean, thanks for being on the show, Brett. Welcome back. We can introduce my cat, Gidget, for all those on the live stream, because she's in frame at the moment, just lounging around on a nice, comfy blanket.

Actually, from Google, from when I won an open source peer bonus. You won your cat. Way more information than anyone probably cared but there you go nice yeah my dog is around here somewhere but probably won't show up in screen well um let's just uh get started i think uh if people don't know brad they need to get out from under a rock so yeah absolutely uh live chat real quick uh brandon brainer thanks uh they gave you a cat no uh yeah there's a long story about how we got gidget but uh no cat did not come with the blanket so the first thing that i want to talk about comes to us uh from this is both recommended and created by dan lutik so dan nice work i love

these little tools that you can run against your code that will just reformat them to be better. You know, one of the, probably the most popular and well-known one is black, right? We've all heard of black, which is great. But there's other ones like flint, F-L-Y-N-T is one of my favorites that converts all the various string formatings to F strings, which is great. So here's another one around typing called auto optional and brett i think people maybe are using optional a little bit wrong in the typing space in python what do you think or just not using it when they should be yeah that's definitely a thing that was actually i think discussed early on uh about do we even need to have the concept of optional to

represent hey this thing can be whatever the type is or none. And what was the thinking there? Because, you know, languages like C sharp, Java, C++, you can have a thing and then it can be null and C sharp, there's value types, but generally most things are pointers and they can be set to null and you don't have to have some kind of a special type to say it's either a user or it's not. You just say it's a user and because it could be a reference type, it also could be null or nil or whatever. But Swift and Python have this concept of you must explicitly say that it could not have a value. So explicit is better than implicit, as we all know. The other thing is obviously having none accidentally sneak in when you don't want it. As you mentioned, Michael, Swift specifically does not let you have null.

You can't have null references. You have to either declare that it will accept null and you have to check or it won't take it at all. And it's a similar thing here. No one likes it when they actually find out, oh, the null type's not callable because you accidentally passed null. What's the most common exception type is attribute or type attribute uh none has no attribute whatever the thing is you're trying to access right that thing yeah i'm willing to bet

yeah that attribute error alone it probably makes up like a huge portion of people's exceptions so that's why you have to be explicit like you need if you're going to be running a type checker let's be very explicit about what you do and don't expect yeah cool uh i agree with that i think it's nice it did surprise me at first but then i'm like oh okay well that's a that's an interesting choice be very explicit about what you do and don't expect. Yeah, cool. I agree with that. I think it's nice. It did surprise me at first, but then I'm like, oh, okay, well, that's an interesting choice. But now that I know, I should use optional. So here's a case of a function, and it's just some foobar example. But they have a bar parameter, and they say colon str equals none.

So they're giving it a default value and saying it can be a string. Well, obviously that is like in three, three words, patently wrong, right? It can't both be a string and be set to be none because it, you know, strings and type system can't be none. They have to, that has to be optional. Right. And so that except for all of us know what that means. I know we all know that. Well, so funny enough, you know what it means, but if you didn't write that, do you know what they actually meant? Right.

Like if Michael typed that and you saw it, would you go Michael meant for that to actually be optional and thus nullable in terms of what how other languages write it? Or was that on purpose? And that's why that's not allowed is because you don't know who screwed up, if anyone. Was that accidentally written that way and it really was meant to be optional? Or was it on purpose and thus it's listed that way? Because you don't know intent of someone else who wrote it,

this is why we went with the explicit where you have to label something as optional. And this is why it's awesome to have core developers on the show. So the idea of this is it's a thing you install, but then you pip install it's a thing you install, but then you pip install it, but then you just run it on the CLI like Flint and you just point it at a directory

and it will go through and it'll find all of these default values that are none, but don't have optional in their type information. And then it'll just add optional, which is pretty straightforward, but it's so easy to use, right? It's like, oh, I have this 20,000 lines of Python code and I really wish they were F strings,

like flint that folder, it's done. So same thing here, right? It's, I've got this huge bunch of code and nobody really thought about optional. Let's just quick fix that. It doesn't fix it everywhere, right? Like I could say a thing takes a string without a default value for the parameter

and then maybe it's somewhere else it's getting a none. And it's not like that advanced. It just looks, if you're having default values that don't match the type, make them match. Yeah. So I like it. One of the questions I have is, so if I do optional string equals none,

does that mean I still can pass none to the function? Yes. Because it's no different than is normal okay but wouldn't that be the same as saying that i can pass stir or none yes yes and also that's what um on the live stream will mcgougan hey will says um do you need optional now that we can have foo pipe none as a an alternative to optional of foo no but this is sort of more backwards compatible, right? Brett, when did that come out? Is that 3.9?

3.10, I think. 3.10, yeah. So it's not even out yet officially. Next month. Yeah. Give it a little over a week. Right on. That's exciting.

But how much backwards compatibility do you want, right? So I think soon, but maybe I don't know if I would go yet. And then Chris May says, oops, opening Pycharm to add optional to my last project uh before you go to the work to write it just run this against it on the cli uh on the command line and just see how it works right so yeah should also work in vs code although zdocs does say back in 3.7 you can use from dunder future import annotations and then get that to work. I don't know about that personally, Brett. So what ZDocs is suggesting here is if you use the from dunder future import annotations,

what that does is it turns all your annotations technically into strings to Python. And so it's actually not executed because before that, what actually happened was, is that code in your annotation actually got run and the resulting object got bound to the code object. What that future import does is just says, no, no, we don't want to pay that overhead on your import. You might have performance reasons. Don't do that and it'll actually just become a string.

And then we have some stuff to try to resolve that at runtime. This is the whole thing that came up in Python 3.10 that you all covered in a previous episode. Pydantic going, oh God, there's a change coming in 3.10 that might break us and there's a whole kerfuffle. And anyway, that's what that's all resolving. So technically you're right that you could write it back,

but it would break complete expectations of everyone who ever tried to use your code when you tried to run that somewhere else so i would advise against it unless you know you're going to be working against 310 because for instance someone might write code for python 3.7 or 3.8 tell my pi is python 3.9 and then it would error out saying well that like that syntax makes no sense right okay interesting all right well yeah pretty good one a good one um super easy to use not a whole lot to it but i think it's because it's so easy to adopt it's kind of a nice one uh yeah well i want to talk about something that's not easy

it's not easy to write good documentation so there's a article uh from daniel stenberg and this doesn't apply to just Python stuff. This applies to every software, all software stuff. There's an article called Making World-Class Docs Takes Effort. And I think it's an understatement. It takes a lot of effort.

But this is a nice little doc, project or article, talks about he's got some gold, things that he looks for to get a gold star, you must have these six items in your documentation. So let's just talk about what he's looking for. First of all, the documentation should be in the repositories, the code.

I've got mixed feelings about this, but in general, I'm on the side of this is a good idea just to keep the code and the docs together. The only reason why I sometimes find it difficult is when is I'm updating my documentation, possibly way more than I'm updating my code. And it looks like there's a lot of turn on the code when there's not. So I don't know. But generally I think that's a good idea.

Most, most projects have the reverse, right? Like the documentations were edited that was edited two years ago, but there's been changes continuously, right? Like, I don't even know if they still apply, but. Yeah, yeah. But it is good to try to keep these together. Also, you can make a pull request or a merge request be with both the documentation and the code together.

And so that's a good reason for that. Next up is that your docs are not extracted from code. And I know there's a lot of tools out there that can try to do this. But the comment is, just think about your favorite project documentation. Is any of it generated from code? And odds are not. So, yeah, I agree with this. Yeah, you can usually spot it because it'll be like method login user.

And then the comment will be logs in user and you're like on it honestly i've seen the reverse too where people have doc strings that are extremely verbose and take up pages in your terminal just to scroll through and because they just use that to spit out your docs in which case it's not as convenient as when you just run help from a ripple to just go like i what does this do again what are the arguments like i need a really clear point pointed thing directly at a developer who's just trying to double check something versus a whole x exposition of here are all the examples of how to use this thing it's like i don't need that level of detail i just remember one little thing right right yeah it's it's a range is it inclusive or exclusive of the

bound like something like that right one thing's the, the one place where I'm working on, I've got a project to try to use auto-generated documentation is I've got a really large test suite on a project. I'm shocked. Absolutely shocked, Brian. That I would really like to have doc strings, they encourage developers to put doc strings in there

for the reason why the test exists and what feature it's really testing. And to be able to have an auto-generated like website with just all of our tests that would be cool but i'm not there yet um that makes sense the next thing is uh your docs feature examples and um i love this uh because that's what people want to know if i if i want to do this how do I do it? And so the comment is really just use examples. If you already have examples, add more. You probably can't have too many examples. And test them.

Oh, yeah, test the examples. We covered a tool called Make Doc Test that you could use to test your code within your documentation. Use that. Next is your document. Document every API call you provide. This one's a tough one to keep up on,

but it's really important. Even some great projects like Typer and stuff that I've used, not to bash Typer, but I think all projects have this problem. They'll add a new cool feature and it's cool. I want to use it and it's not in the docs yet. So that's a tough one to keep up on, but please do. One thing I'll say that is actually hard for Python

is because it's so dynamic, it's hard to actually get a static list of all the API points. But one thing I will say, because Paul Everett says I have to bring more Rust to this podcast, docs.rs, which is auto-generated documentation for anything uploaded to crates.io for Rust tools, will actually give you a percentage of covered public APIs.

So you can actually see how much coverage you have in your documentation. That's an interesting definition of code coverage. Yeah, it is, right? But I didn't even know about it. And then I found it when I was just double-checking checking everything built okay for a project i will mention later in my extras extra extras uh and i noticed it was 100 it's like i thought i covered everything and i went back it's like nope there were actually a couple um enum values that i didn't document and so i was able to go back do that and make sure i had full 100 coverage of my stuff so

it can be handy but yeah it's it's tricky in Python, right? Where it's like, I could auto-gen my whole API and no way to know whether I covered it or not. At runtime and never have it hit disk, yeah. Yeah. Last couple of things. Docs should be easily accessible and browsed and hopefully a search feature or something, even if you attach Google search, that's cool.

But there's a comment in the text also to say, a search feature or something, even if you attach Google search, that's cool. But there's a comment in the text also to say, preferably be able to have that online. So offline is what the comment is. So if it's, and I'm on the fence on that. I'm usually attached to the internet. So looking stuff up isn't a bad thing. And easy to contribute to, the last um so yeah uh yeah documentation stuff i would um uh there's a lot of stuff it's hard to get it right i would there's a lot of stuff i'd add if i was being more verbose but i want to uh

the last thing i really want to add to this is don't slam projects that don't have good good documentation because it's hard if you want to to have it be better than contribute, fix it yourself. Brian, you're kind of blowing up the audience out here with the comments. So I'll throw a few out that seem interesting. Sam out there says, pro tip, keep on top of your documentation, even if it's just a cursory doc string and some notes.

Otherwise it quickly gets out of hand. It's yeah, it's definitely one of those things that if you had to sit down and write it all at once, it would be dreadful. But if you can kind of do little bits as you go, then for sure. Right. Brandon Brainerd says your documentation update should be part of the ticket or even better your definition of done for the ticket.

Right. Like when you estimate how much work in your sprint or something. And then Chris has a great one. Bonus documentation. If a bonus, if your documentation includes animated gifts or emojis. Yes. Yeah, those are awesome. We'll probably leave it there for that one, but that's great.

All right. So one interesting thing you could actually do is, if you really want to push this, is if you keep your documentation in the same repository as your code, you could. I'm going to use this from GitHub. I'm sure GitHub has equivalents. But you could probably set up a GitHub action.

Actually, I know you can, because I've written the GitHub action to do this, to actually have your checks, your status checks on your PR fail if there are no touch docs and have to opt out of that check, right? I actually have a GitHub action I wrote called check for changed files, and you could write a requirement saying if you change any dot pi file an equivalent dot rst or md file must be changed somewhere in the pr and if it's not have to add like a skip docs label to make it pass and otherwise let people know very explicitly like hey we care about our docs you can't just opt out of it please make sure everything's up to date there are ways to really push this pretty far if you really want to go for it. Yeah. Have robots be mean and say, no,

you haven't finished your PR yet rather than the maintainers. Yeah. All right. Tell us about this next one, Brett. Yeah. So once again, more rust on this podcast due to Paul Everett demanding it. And I've noticed there's been a slight theme the last couple episodes of talking about ways to kind of improve your shell and your general development flow, as it were. And one thing I use here, and partially because Michael's mentioned Oh My Posh multiple times, I believe Brian's now mentioned Oh My Zish. I thought I'd just mention what I use for all my prompts, which is Starship. The thing I love about this is it's actually cross-shell. So what you do is basically you define... It comes with a lot of basically plugins for configuring built-in. And what you do is you basically turn them on and off,

configure them one way or the other, and then there's a one-line activation you just put in your code, and that's it. Configuration's done by a Toml file, starts up Toml, and then it just works. there's no mucking about it doesn't doesn't matter what shell you're on and it just does its thing and the reason i really appreciate that is i'm one of these people who likes to play with tools right which leads to me playing with different shells right probably by the time paul everett

tries out fish i'll be trying out new shell so So because of that, I want to make sure that I can bring my shell with me very easily. And Starship, for instance, has support for fish and Zish and Bash and all of them and new shell. So I can very easily try out a different shell and not feel like I'm in a foreign country where the prompting is gibberish. Right. Like I always hate it whenever- It seems like oh my Z shell go onto Z shell and oh my posh go onto posh and like you're stuck on that specific one, right? Exactly. And there's oh my fish,

like pretty much every shell has an oh my something or other to tweak it out. But with this, at least for my prompt, which is actually what I see the most, right? Like I don't know about the rest of you, but my prompt is what I interact with more than anything in my shell, not the extra fancy stuff I put in it's this and so this is why i

really appreciate this project i will also yeah go ahead i did go for it it's just well what i like is we're seeing on the screen here if you um go to the starship.rs site you can see like a little animated gif and props to them for putting that on there. That is fantastic. As we just learned, it's good documentation. Yes. There's a lot of stuff about if you're in a GitHub branch and what branch you're on and versioning and things. Does it have specific stuff for Python and Python virtual environments and versioning like that?

Yes, because I don't use it, but there's built in PyM support. But if you use the Python launcher, I actually have an entry on it in the FAQ for the Python launcher to tell how to use the Python launcher to automatically tell you what version of Python is being used. And then you can configure the Python configuration to say, what should I trigger on for it to be considered a Python project? And it has a default list that's pretty good, like pyproject.toml, setup.py, setup.cfg. The things you would expect any repo or workspace or whatever to have to be a clear marker that there is Python that I care about here. And then the other thing is there is also a way to, if you use py, you can also have it set up to auto trigger

as long as you just have a.vemv directory, for instance, that contains your virtual environment. And then if you happen to use the Python launcher to be the way to query for what version of Python it is, it all magically just tells you what version your virtual environment is in. And so I actually use this with the Python launcher because it will always tell me exactly what will happen if I just type pi. Oh, nice. Yeah. Yeah, that's awesome. And I know nerd fonts have been mentioned

previously on this podcast. You'll notice that it has nerd font support. So if you get that installed, there's extra little fancy bits to it. You don't have to have it installed. All those things will either turn into mojibake or you can just turn them off. Yeah.

But there's all sorts of stuff that this thing does. There's even real nice little subtle details. Like if use starship the in your if you're watching the live stream you'll notice that the prompt is green when the command last command succeeded but if it fails it turns red like there's real nice little subtle touches this tool that i really appreciate yeah i really like it too it looks I'm going to try to check this out because I'm, I'm always using, I got some tools that I have to run from the, uh, the command prompt and,

but most of the time I'm in bash or something else. So, yeah. Yeah. Very nice. Uh, Z docs says, uh, using multi-line shell prompts and F ZF are two of the better productivity

improvements I've ever done. Uh, I, I wanted, I've highlighted that specifically because I really like these multi-line prompts. It has all the details and then the prompt is below it, but I can't bring myself to use them yet.

I don't know why. I'd rather have it more densely packed on one line. Where do you two stand on this? I used to be that way. Actually, I used to be a Zeese user before I became a Phish user. And even before this, I used to be a zeus user before i became a fish user and even before this i used to use left prompt and right prompt and i had a consistent left prompt which was basically

just the uh greater than symbol and then i had my current working directory as my right prompt so that at least the cursor position never shifted like i don't know about the rest of you but i always find that that they're jumping nuts like where's the start of my prompt now after i changed my directory so having that as the right side was great but then i saw this and it's like okay i've never loved this but this is so nice and set up across the board for everything that i consistently use like rust python npm whatever i was like okay i'm gonna give it a shot and i did it and i've stuck with and i continue to like it so i totally hear where you're coming from in terms of the density level but it's just turned out to be way too nice with starship for me to care about going back now if i

do go back right and i don't have z i don't have it set up for this i actually have like my dot files set up so that my default bash or c and all that, if I end up on a random machine, at least has that kind of prompting. But as soon as I'm on any machine where I know I'm going to be a fully set up environment, I just kick on Starship. I just don't care. Basically, I get it. But this was just too many benefits and I just let it go. And now I just don't notice it. Yeah.

And now you're used to it. I think I'm going to try to embrace it just just run with it and honestly the resolution on all our screens are so high now compared to what it used to be back when i picked up that habit that the lack of vertical density maybe it's because i'm a black user doesn't bother me so much anymore yeah that's true i do have a 4k monitor i suppose that like a 32 inch 4k monitor, I could probably fit a terminal on it. I suspect so. It's actually kind of nice to have the, those like color stuff in your, in the multi-line prompt to be able to kind of scroll up and see where the

beginning of your line, your command started. Cause it's easier. It's like an HR, a horizontal rule. Yeah, exactly. Or yeah.

Final thing,eremiah page out there in live stream says starship is great but i think you need nerd fonts for some of the plugins hold that thought we'll be back to more nerd fonts later but that is correct some of the things like if you're watching the live stream the there's a nice little symbol for like a branch next to the get stuff if you have the power monitoring one that tells you like your battery is low it uses nerd fonts to show that but but Michael's foreshadowing something coming later. Yes, indeed. All right. Let's talk about James path. Now, Brian, one of the challenges we always have on the show is we have these acronyms for packages and then we have to speak them. And

we've never spoken to the person who creates it or named it. And so we always, I'm sure we wreck the names of so many packages and let's just do a blanket apology. But this one is called James path. And then I know because it says J M E S P A T H pronounced James path allows you to like, yes, this is another new thing I'm starting to really like, allows you to declaratively specify how to extract elements from a JSON document. So instead of going through and sort of parsing up something as JSON, turning to a dictionary, and then traversing it with indexes and keys and such, there's a whole lot of interesting things you can do to quickly get at elements. So for example, you can just if you had the dictionary that had key foo, and then inside there, there was another dictionary that had a key bar, and then that had

a value baz, you could just say foo dot bar, and it'll give you baz, which is kind of nice. You can also do that with with arrays, although this is actually not doing anything. It's just giving you back the list and then you're indexing into it. So whatever. But you can do things like star on it. So for example, if you add foo bar and then bar was an array, you can say like foo.bar, a bracket star, dot name,

and it'll give you all the names out of the sub thing, the sub dictionaries in the list that bar points at so there's like these cool like almost like sequel like things you can also do uh negative uh indexing into it which is pretty nice you can do star like foo dot star dot name for um hashes to get the values out um kind of like a set or something so that's pretty nice and yeah it's pretty interesting this one comes to us from josh thurston so thanks for that and he said he was working with all these sort of arbitrary nested json documents and it was really handy for that so you know you look at this code and i'm sure brett you're thinking that's not python right no actually i'm wondering if the query syntax is the same as JQ.

And do people know what JQ is? I don't know what JQ is. Yeah, interesting. So JQ is a command line tool. So literally it's just JQ, Juliet, Quebec, that is designed to actually take in JSON, more or less use the same kind of syntax as the query selector and the browser for

CSS selectors and query your JSON. I think it's meant a lot for like tools if you're using like HTTP or curl or whatever to query some like REST API that's going to give you back JSON and then just to try to find something out of it. And it looks pretty similar. So I'm thinking that this is maybe kind of just a python packaging approach to the same syntax which is which is neat because it means if you know one tool hopefully it'll work for you yeah right yeah it could be so it's it's you should think of it not as syntax that you write but more almost like a regular expression so you would import james path and then you say expression equals james path compile and then you give it a string which is these things i've been describing

and then you can tell it to like search or execute uh queries against uh things so a lot of a lot of neat stuff you can do there you can even add custom functions that can become part of this query syntax so if you create a class that derives from functions that comes out of that library you can have things like unique letters and what you could do in there say given a string i would like to do a set on the string and then turn it back into a string so it only shows you the letters involved not necessarily um you know with duplication right and so then if you create that then you can do things like when you do your search you can do foo.bar pipe unique letters and you pass the parameters over and it'll give you you know basically the

result of those values but then applying these transforms and stuff to it cool so yeah pretty neat um you got a lot of traversing dictionaries and stuff like that and you kind of want to treat them like sql i feel like this is sort of along those lines people can check that out all right brian over to you yeah um i was uh wanted to cover um this was announced last week i think uh there's a tool called pedal board um this is a library came from that comes from spotify um and it's it's a it's for manipulating audio files um but uh it in the introductory article it says the power and speed and sound of a DAW. I never knew how to pronounce that one either.

Digital, what is that? Digital audio something. Workstation. Yeah, digital audio workstation. A DAW, I think. Maybe a DAW. I don't know. So what do you use?

I use Logic Pro usually. So something like that would be a DAW. Yeah, I use Adobe Audition. So there's a lot of stuff you can do that are just common things that you're going to do to all files. So it'd be kind of neat to automate that. So things I'm thinking about

are things like a compressor, a high-pass filter, a low-pass filter. But this thing has a whole bunch of other transforms you can do like convolutions and chorus and some distortions and gain if you want to make it louder um some reverb high pass filter um and then stuff i don't even know what it is a ladder filter i don't know what that is phaser phaser that sounds neat um but uh scotty so um i definitely want to try that out And if you're working with audio files,

I think it would be kind of a fun thing. I'm also going, we've linked to the GitHub repo for it, which is nice. It's completely open source. It says it also uses some, has some cool things like plugins for VST3 and Audio Unit, which I have no idea what those are. But also there's some,

one of the things with audio is it takes a while. So there are speed enhancements in this. So it says it really doesn't take that long to use it. It takes advantage of your CPU cores, which is neat. Also linking to an article to introduce it. And I gotta warn people, there's a image at the top which totally messed with my head because i mean it's just it triggers me with all the different flashing colors so be

careful with that scroll past that quick but other than that it's a good article on on what it is some graphics and i don't quite get there's some graphics on how it works and i don't quite get, there's some graphics on how it works and I don't quite get what the graphics mean. Like the noise gate really doesn't tell me much. I love the noise gate. That's one of the biggest tricks to sounding good actually. Yeah. So important enough for me that I've got a hardware noise gate in my studio.

Nice. Anyway, cool library. Check it out. Yeah, so if you want to do audio Yes, if you want to do audio transformation. If you want to basically write Python code that does like Audition or the one you said, Brian. Well, I mean, and especially with automation. So if you've got like a whole bunch of files you're dealing with and you want to make sure that they're all kind of the same level, you definitely want to automate that and not try to. And I know there's ways to automate with some of your other tools too

but it's kind of cool to have this yeah definitely the only thing i have to contribute is going back to brian's previous uh link about good documentation uh while you two were talking i just submitted a pr to fix the trailing triple back ticks i accidentally left in there read me so just to give feedback to people trying to fix documentation is always appreciated and it's usually something you can do pretty quick especially with either the pencil um editor in github or just hitting the dot to use github.dev if you want if you need a little bit fancier support and just send that pr and i mean most most maintainers would appreciate it as we all know it's hard to keep going straight.

If you submit an issue or send an email and say this part of your docs needs this misspelling fixed or this link has an extra trailing slash that makes it not work, that's work. If you submit a PR, that's joy, right?

You just say, I accept. Okay, you fixed it. Thank you. I'll just carry on, right? Rather, oh, geez, I got another thing to do. So yeah, that's great. Nice catch. I'll just carry on, right? Rather, oh, geez, I got another thing to do. So yeah, that's great. Nice catch.

It took me a while to see that. Yeah, well, I mean, some people ask me like, how do you have so many contributions to so many repositories? That's how. Literally, if you just read docs, anytime you find anything a little off, just take the time to just submit that PR.

And most of them get in, not all of them do. But honestly, that's enough to just kind of help out. And honestly, you just suddenly end up contributing to, I think I'm up to 160 something repos I'll get up at this point. Wow, that's awesome. Okay, so Michael said it was nice sometimes to have core devs.

We'll see if this doesn't cause you to regret having core devs on the podcast. So I'm going to use this as story time and just kind of explain how processes work in terms of packaging peps. Starting about six months ago, I took it upon myself to try to come up with a lock file format for Python. It seemed like it was kind of a gap we had where every multiple tools were trying to solve the same problem, and that felt like something that we should try to avoid if possible. Coming from speaking as the dev manager for the Python extension for VS Code, having each bespoke tool such as pipen, poetry, requirements.txt files, even through pip tools,

right? Having to support each of those tools individually is a drag, right? Like it takes a lot of time and effort to figure out what to do for each tool if the tools update suddenly we break so it's a bit tricky making calls through cli is kind of an annoyance too because we got to make sure that they're you know how to use their api what they report out is consistent that they don't change it because not all tools consider their output with their cli part of their stable api that they maintain anyway there's a lot of drawbacks. So I decided to be silly and take it upon myself to see if I can come up with a way to get all the tools to kind of rally around something.

There's also needs for like cloud, right? Like if you upload something to Azure Functions, they want to do the install for you, but you need to be able to tell them what you want installed. So lock files, good thing, right? Just in general. So I spent six months uh talking behind the scenes with various people originally it was me um prediun and zuping

from pip uh we came up with a lock file spec and then we invited sebastian um and frost from um poetry and pdm uh respectively and they said that's not going to work for us and we said why it's like well your lock file was defined very specifically for a certain platform right like think of wheel tags right it says what python version what ab what abid you support and what os are you on right that's the platform from what i'm speaking from it's like that's great but we want platform agnostic lock files right like if you ever crack open the lock files that poetry spits out they actually work no matter what os you're on and it was by design sebastian wanted to make it so that it didn't matter and actually uh i've heard a good example

from prodoon where it's like if let's say you're doing pi week right and you're doing a game um if you're you might be developing on linux but if you want to give it to your teammate who you're working with during pi week and they're on mac you want a way to be developing on Linux, but if you want to give it to your teammate who you're working with during Pi week and they're on Mac, you want a way to be consistent on what you work on. And then if the judges are on Linux, you're on a whole nother place

or Windows or whatever, right? So there is a use case there for platform agnostic log files. So we tried to add that back in. We took this public to discuss.python.org where all the packaging discussions happened and it led to 152 comments on a thread starting july 29th so actually it's more like oh god eight months ago so this led to a whole nother discussion right about people going like okay the clarification

and it ranged right all the way from the title's not clear enough to real nitty-gritty details. And it's now led to us going back, having discussions with the same people, plus some other people who participated here who made good comments, to try to come back with another approach that's a bit more from the ground up, covering everything, versus what we had before. It was just a platform-specific log file, and then we kind of bolted on the agnostic bit and try and design it from scratch but the key point here i want to make is you hear people sometimes go on about oh i really want lock files right like i've talked about this publicly and i think i talked about this uh py cascades maybe i mentioned it during the steering council uh keynote or maybe it was pycon and my wife andrea who said she might actually be in the live stream uh said oh my god all the comments about when you mentioned lock

files were so so positive like okay cool it's surprisingly hard to make this stuff work right like everyone thinks it's a snap your finger someone has an idea do you just get it done but there's a lot of use cases here right and especially with packaging you constantly hear people go like oh packaging and packaging in Python is so bad. Well, first of all, most people are out of date. Like Brian's been really great about having me and other people come on test and code, for instance,

directly specifically to talk about this past packaging stuff to keep people up to date. So there's a lot going on, but you need to try to keep up. But on top of that, it's just hard to make any changes because there's so much legacy out there. Everyone's got their own workflow at this point, right? Like 30 year old language that's had some form of packaging for like over 20 of it. Everyone's got the way they

want to work and no one likes to have it changed on them, especially your workflow, right? Like try telling someone who's using VS Code, like, yeah, we don't think you're doing it the right way. Does not fly, right? It just does not not work everyone wants to work the way they want to work which is totally fine but it also makes trying to come up with a standard really hard so the point i'm trying to make here is try to be understanding when you work with packaging in python not only is python challenging it on its own due to the amount of c code for train code all the crazy stuff that people used thanks to us being the glue code, the glue language of the world. But on top of that, there's a lot of legacy.

Everyone has something they asked for. And people are working very diligently to continue to try to improve it. But do understand that there's a lot of work going on to try to make it better. So please try to be understanding on Twitter or wherever else you're going. Oh, my God. Packaging Python is so bad. The places where outreach must be expressed. Yeah, yeah.

So that's what really here. So A, lock files are hopefully coming. I am working on a V2 of this PEP. Probably go public with that. I'm going to guess sometime next month. And these will be independent of things like poetry and pip and just pip and stuff like that? If we're lucky, right?

Like the key point here is we can't met we can't require this but the hope is whatever solution we come up with will be good enough for those tools to rally behind right none of this is like an edict from on a high to say like all packaging tools must use this the idea with all this has to be what's the carrot for the tools right how are we making things better both for's the carrot for the tools, right? How are we making things better, both for the community, but for the tools themselves? Now it can be, we make it so great from the community that the tools feel pressure to support it. But when you're talking from this perspective, the first thing you're dealing with is the tools. So it's trying to make the tools happy because then eventually the users also become happy. So we'll have something. At worst,

my suspicion is at absolute worst, we'll have platform specific locked files come out with this PEP. But my hope is we can get platform agnostic and get all the other tools on board so that everyone gets what they want and need and we can service everyone's needs. Nice. But the key point here is it's being worked on. It's taken a long time. It's going to take even longer. So well, and so understand that when people say, oh, why can't packaging get better faster? It takes a long time, right?

Like literally I've been working on packaging as my focus for Python stuff for years now, right? It's not a fast process because there's a lot of people involved, a lot of projects, everyone has their needs. So just people just need to be patient, but do understand things are getting better consistently. That ends my rant too.

Hopefully not make you all regret having me on. No, that's great. I love the insight. The only thing with packaging that I want to bring up is i wish it didn't change all the time sorry i couldn't resist um no i i think things are getting better and i like the direction um so but cool yeah i mean the the grant my grand unified goal when I got involved with packaging really was to try to drive everything towards standards and specs, right? Like I have a project called mouse bender after the clerk from the cheese sketch, right? If you don't know the Monty Python sketch, it's when one of them walks in and just starts

to ask for every cheese they can think of and they don't have any in the in the cheesemonger shop so same after them anyway i created an outline of what does it take to go from i want to install django 3.2.7 down to actually getting files on desk and i figured out where there was a spec and where there was a library back in that spec and where the gaps were and i've just been trying to plug it and this is probably the last one right because more or less this takes care of the pip tools pip and poetry kind of give me a set of locked things and let me be able to reproduce my environment after this i think i'm more or less kind of i'm personally probably going to be done with packaging after this is moving on to web assembly which should hopefully make michael happy oh yeah and then probably that'll lead into is is there a way to potentially start distributing like a self-contained binary

for Python or just try to help with Python as a Python, see Python interpreter distribution kind of solution. So is the new pep going to be 666 or was that already taken? No, actually 666 is taken. Okay. It's been taken for over a decade. It was purposely done to be immediately rejected because it said you should be able to allow

mixed tabs and spaces. So that was written to make people stop asking for that and it got rejected immediately. But no, I'll just since this didn't get accepted, I'm just going to rewrite it in place. The goals are the same. There's no need to have to do yet another pep.

So since it's draft, I'm just going to basically just gut it and redo it. place the the goals are the same there's no need to have to do yet another pep um so since it's draft i'm just gonna basically just gut it and redo it and since the goal is the same i'm just gonna reduce the number okay nice well thanks for working on this because uh it definitely is important yeah welcome hopefully it'll lead to something otherwise i've had a lot of interesting conversations about how people do and don't want to lock things brian back to you back to me yeah for extras uh for extras you got any extras um yeah so i've got one i'll throw it up uh so um looks like um looks like python's popular i've heard that i bet i bet it would be popular enough to have a podcast about it with lots of or three even exactly so the there's an article on zd zd net uh python is on the verge of another big step forward and the big step forward apparently is um the tie i'm gonna get this wrong taiobi index uh list c is the number one language but uh python only

needs 0.16 percent more to become the number one language, according to this index. So are they arguing that if we got to the top of Tyobi, that would cause another spike in usage? I have no idea. No, another spike in usage would cause this to happen, maybe. I don't know. Oh, I don't know.

Yeah. Yeah. So I think historically, only C and Java have ever had that designation. And so it's kind of big news. JavaScript's never gotten up there? I don't think so. Oh, one more thing. Brett mentioned the cheese sketch.

And if you go to PyPI.org and type like anything that's silly, you'll probably get a 404. And the 404 has the cheese sketch embedded in it. Yes, the cheese shop sketch is at the bottom. And a piece of history, PyPI was originally supposed to be named the cheese shop. It was actually originally supposed to be, it was actually originally cheese shop.python.org. But people were too concerned

that managers wouldn't take Python seriously if the package index was named after a mighty Python sketch. So it got its name changed. Yeah. Cool. Brett, looks like you got some extras as well yeah well i mean this is python bytes so we couldn't go an episode without mentioning well although i think will's out in the live stream uh so this is also more of the the paul love fest uh on here so paul had will go on to to an episode to talk all about Textual and Rich and how it's all structured, mainly Textual. I believe Will tweeted that he got a haircut for this and everything.

I actually haven't watched it, but as I said, it's not Python by itself. Will doesn't get mentioned, so I just want to make sure Will got called out. I will also say that it has been 30 episodes since I was last on, which I thought was kind of a nice round number. But on that episode, I mentioned that I was working on the Python launcher for Unix. And since over the last 30 episodes, I've actually wanted to mention that I had launched version 1. And thanks to Brian and Michael for feedback,

because actually one of their feature requests got into it. The one thing I did want to call out that actually did happen recently is the launcher is now actually in Fedora. So one of the first things that Evergard contributed was someone actually put Arch. It was actually Burnout, Gabor of Tox fame, got it into Arch via AUR. I don't know how Arch does community versus not. There's a whole thing, but you can install it via Arch. But just the other week, managed some very nice folks from Fedora were nice enough to get it in.

So you can actually now use DNF to install the launcher on Fedora. And then all the previous tarballs are all there as well. So it's still available. But yeah, version one, no bugs so far. So it seems nice and stable and meeting people's needs pretty much. Awesome. Awesome. Congratulations. Thank you.

Good work. I'm pulling a Michael extra, extra, extra, extra at the end. I will say that my Syntactic Sugar series is still going on. When it's done, I'll probably have a wrap-up thing, and this is probably going to be what my next set of talks will be all about. But they're still happening. One interesting thing, I actually just did two just in the last two days. They were short. Actually, oh God, I didn't realize

I published three in a week. That's a lot for me. I did have one that didn't work. So if anyone's any curious what happens when I can't unravel a piece of syntax, you can read that blog post to see how I put in all the effort to do it and then realize the day after,

oh yeah, that's not going to work. I did enjoy that. Oh, good. Yeah, yeah. Assignment expressions continuously break me in various ways, so there's a bunch of syntax I can never undo because of assignment expressions. They're handy, by the way. I've used them since the month after we approved them, but they do make

unraveling our stuff hard. And then for Brian, I just wanted to call out that in our last release for the python system for vs code uh we completely redid our ui uh vs code now actually launches with um a built-in ui for testing and uh we integrated directly into it so now you have consistent ui for testing like a test tree result type of navigation thing. That's cool. Yeah, well, and we had that before actually,

but it was bespoke to our extension. And so there was some wonkiness to it and all that. But now that this is actually built into VS Code, if all the language, I suspect, will eventually move over to this. So by having that, learning it for one tool, one language means it'll work for another language. And there's some extra niceties to it as well

in terms of updates and all that. It's really snappy. We'll probably, we are also going to rewrite all the code behind this and how it runs to make it a bit more performant, more stable and all that. So if you've had UI problems before with testing with VS Code for Python, please give another try. If you've had ui problems before with testing with vs code for

python please give another try if you've had more discovery run problems we are going to be working on that uh so that's coming up as well and then when that's all done uh we'll be bugging brian about this to make sure he it gets the brian atkins stamp of approval um yeah you know i charge for that but i thought that was just coming on the podcast was the fee uh no i no i definitely um i i tried this out and i like the changes and i'm i'm excited to see the discovery changes as well yeah well we reached out to the pi test team directly actually and asked them like what what would you want us to do and they gave us some feedback of how they think it should be done and all that so we're hoping to incorporate that and use that as the basis of how we do pi tests and then unit test unit test is going to test

unfortunately we had to break we had to drop no support due to this but honestly it was extremely low used anyway um but we did have a lot of go but we still have unit test by this support people shouldn't use it anyway you heard it from brian um don't don't send me the hate mail uh but yeah that's it nice those are all great ones all right i now have banners to separate my sections oh that's awesome nice all right i'll be quick but i do have a couple things i just want to go back and sort of uh talk more about nerd fonts a little bit at nerdfonts.com people can check this out so it takes all these different things and puts them together like a bunch of font awesome fonts dev icons weather icons into a bunch of things and then the ones that i really like

is the developer fonts right so you go over and check these out they all a lot of them have font ligatures brett i don't know how you feel about font ligatures but i'm actually a fan although they freaked me out when i first saw them uh i'm definitely a fan uh i'm actually a um fear of code user exactly yep um although the new uh ligature support for infinite arrow length throws me a bit sometimes to the point that i actually have turned it off and i use cascadia code uh the nerdfob version for starship in my terminal so i actually have a separate font for terminals in here. Yeah, Cascadia, right there. Cascadia code here. It's Cascadia code on the official. So yeah, I use Cascadia for my terminal, and then I use

Fira code for my editing, although I'm contemplating actually buying a font called Monolisa, M-O-N-O-L-I-S-A, and using that, but I haven't pulled the trigger yet. Nice. I'm not against buying fonts if they're amazing. Yeah, neither am I. It's $100. Armin Ronenacker actually uses it. That's partially how I came across it. Someone else recommended it somewhere. Kuseldas paced on me, tweeted about it, bought it, and actually thanked me for across it. Someone else recommended it somewhere. Kushal Das paced on me,

tweeted about it, bought it, and actually thanked me for using it. So multiple people I know use it and actually appreciate it. And it does have literature support.

Good. Yeah. All right. So one more shell thing. So here is my power, my Microsoft terminal running Oh My Posh on my computer.

Look how dreadful that looks, right? That's because I'm not using nerd fonts. If people are just listening, there's just tons of squares where there should be symbols and stuff. It says like, you know, username square, square, folderosh, actually, I guess is what it is, is tested against all the nerd fonts to work. So if you install that, then you end up with beautiful terminal, right? So definitely something nice to look into if you're just a, yeah, just use whatever mono thing is built in. Yeah. The nerd fonts are cool. All right. Let's see what's next. Oh, did a article interview. This company in Russia interviewed me and people are interested, I want to give a quick shout out to that. It was kind of fun to talk about some of the history of the podcast and stuff over there. setup process for setting up a new Mac. And there's all these cool scripts and brew setups and all sorts of things, including all the fish shell and Vim setups that you might need. So if you want to follow up,

we talked a little bit about that last time people can follow up with that. I also say one convenient thing about committing all of your dot files, at least on GitHub is if you use code spaces, GitHub and you check a box code spaces will automatically of your dot files, at least on GitHub, is if you use Codespaces, GitHub, and you check a box, Codespaces will automatically clone your dot files repo and load them up into your Codespaces. Ooh, that's nice. Yeah. And so I've actually published my dot files and done somewhat of a similar thing. The other nice thing is obviously when your machine, I don't know if I ever told the story in this podcast, but when your laptop accidentally gets grabbed by someone else and ends up in dubai and it gets shipped to you to a core dev sprint in london later and you feel the need to wipe it for security

purposes it's very convenient to have all your stuff in the cloud and have your dot files in a repo so it's just a clone run and you're done yeah i would want to wipe it too someone had physical possession of your laptop for a while yeah all right a couple other things brian last time i said i was using um the editor to just jump between projects and stuff and i one thing i didn't point out is like i'm on the terminal all the time as well but i basically everything that i have some kind of terminal command for is i've almost entirely set up some kind of bash alias. So that actually uses the specific virtual environment as part of the command in the alias. So it doesn't matter what I have activated or where I am or anything like that.

So anyway, just a quick follow up on that. Cool. All right. Well, I think that's it for extras. Do you have a separator for our joke also? I have a laughs. I have a separator for our laughs also? I have a laughs. I have a separator for our laughs. Yes,

of course. Nice. I hope this is not as funny as the joke, but you all, you all will be the judge. Okay. Okay. All right.

So this is going back to a solid, solid place. Geek and poke over at geek dash and dash poke.com. This one is called the last 5%. And Brett, your comment was perfect about setting up a new computer or read, installing everything. Right. So this is the last 5%. And Brett, your comment was perfect about setting up a new computer or reinstalling everything, right? So this is the last 5%. I know where this is going.

Or corporate IT made easy. Make sure the new developers' notebooks or computers aren't only 95% provisioned. So there's a senior woman developer helping a new one who can't seem to compile the project or run it or something that says, I had the same problem, but I barely remember somewhere in a wiki or on a file server, there was a certificate or something like that. And I had to copy it to some folder on my machine. And it's just entitled the last 5%. It's like anytime someone onboards onto your team, right at work. And it's like, oh yeah, go read with the last person five years ago or two years ago how often you get a new person come on yeah did it and by the way the processes have all changed so yeah chances does not work yeah just scroll back two years in slack or teams and find that message where we talked about it yeah it's a little okay you just

have to hire people more often so they update the wiki at least every few years yeah but the question then becomes is it because your head count's growing or because your turnover is so high unless you're just a bad manager unless everyone just keeps quitting your team yeah i'd rather just have the stable number of people who by the way told me they were going to be on the live stream and i didn't see uh michael let any of the comments by so i wonder if they actually showed up so hello to my team if they showed up and my wife if she did yeah hopefully they did and chris may has a final parting thought for us this comic hits too close to home or work or

work from home yeah definitely definitely cool well uh that's it so thanks a lot brett for showing up of course it's a joy to have you thank you to my cat for taking a nap during the entire podcast and let's not distract me in the middle of it. We'll put it on our calendar to make sure to invite you about every 30 episodes then. Perfect. Yeah. It's great to have you here.

Yeah. Thanks. Thanks everyone. Thanks Brian. Thanks guys. Thanks for listening to Python bites. Follow the show on Twitter via at Python bites. That's Python bites as in B Y T E S.

Get the full show notes over at pythonbytes.fm. If you have a news item we should cover, just visit pythonbytes.fm and click submit in the nav bar. We're always on the lookout for sharing something cool. If you want to join us for the live recording, just visit the website and click live stream to get notified of when our next episode goes live. That's usually happening at noon Pacific on Wednesdays over at YouTube. On behalf of myself and Brian Ocken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.