Python Bytes - #353 Hatching Another Episode

Episode Date: September 19, 2023

Topics covered in this episode: OverflowAI Switching to Hatch Alpha release of the Ruff formatter What is wrong with TOML? Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/353

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 353, recorded September 18th, 2023. I am Brian Ocken. And I am Michael Kennedy. This episode is brought to you by us. So please check out Talk Python Training and the new Python Testing with PyTest course that's in progress, but already awesome. And also thank you to our Patreon supporters. And if you want to connect to any of us, you've got Python Bytes and Brian Ocken and M. Kennedy all on Fostodon. And the links are in the show notes. So let's kick it off with our first AI topic, Michael. Let's ask the AI what our first topic should be.

In fact, let's go to Stack Overflow and ask them what our first AI topic should be. No, seriously. So Stack Overflow has announced Overflow AI, which is not, I'm sure they have, you know, they've got Stack Exchange with all sorts of different things like stuff for Linux, stuff for Mac, DevOps, you know, you name it. So I don't think this is, this is not another AI overflow type thing in that sense. This is adding generative AI to stack overflow. All right. Okay. So the best way to kind of get a sense of what's going on here is there's a little video, a three minute video that seems

like it could be about 45 seconds given how much content is in it, but you know, spend three minutes with them and you'll get an example. So the most relevant portion, there's some stuff that we don't care about here, but the most relevant portion is when you go to Stack Overflow, you search for an answer. It gives you a list of results, right? Those results are based on traditional search, right? You know, keyword matches. And honestly, it does surprisingly well, right?

When you go search there. But they're changing their search. And this is, maybe it's out now. I have a hard time telling if it's out now. As a preview, you can sign up for. But you go to Stack Overflow and you search for something. Now you get like a chat GPT-like response, if you want. And it'll say, here are some of the possible answers.

But here's also just, you know, like if you search Google or a Kagi, you might get a little snippet that just shows you the answer from Stack Overflow without taking you there. It's kind of like that. It'll say, and here's a little bit of an answer. And then you can actually start a conversation about it. Like you would with ChatGPT, like that's cool, but it doesn't quite apply. How would you, you know, I don't, I'm not using Selenium for this. I'm using

Pyright. Could you give me an example of the same, the same process, but with Pyright, you know, it'll try to give you the answer. Right. So that's all pretty cool. And they're integrating generative AI into the public platform into Stack Overflow for teams. I didn't know Stack Overflow for teams existed, but if I had a team, it would look really cool to use, but not in a huge corporation with a ton of people working on stuff. So they're also planning to add at the moment, VS code integration for Stack Overflow and the Stack Overflow AI. So you can just like, you know, you've got your source control section, your file section, it's just got a Stack Overflow section where you can start talking to it

and get generative AI answers out of Stack Overflow. So I find there's a little bit of irony here, Brian, in that Stack Overflow was banning answers from ChatGPT. Now they're adding a feature that lets you have a conversation and query it using generative AI, which seems normal. However, one of the features they highlight in the video

is you can say, none of this applies to me. AI, help me draft a question. So the answers can't be generative AI, but the questions can be generative AI. Interesting. For the right source of AI, right origin, right? The overflow A, the chat GPT. So this is in private alpha, I guess,

is the current form. But I went to, the way I discovered is I went to search for something on Stack Overflow. I said, do you want to try out the new overflow AI search? I'm like, yes, what is this? This looks very interesting. And in my profile under settings, I have the ability to have that turned on it is turned on but when i go to do search stuff it doesn't seem to use it so i think it's like fading in and out as they're making changes to it but yeah people can check that out and then finally vs code extension looks cool where's our pie charm one come on bring it on bring it on i want this in pie charm too pie charm also added its own little ai assistant but i'm i'm waiting for it to get better yeah you can yeah

it's it's pretty cool but it's it's not something i'm totally embracing yet i just wanted to be i'd like it when it rolls out uh completely i'd like it to be able to be able to know did this come from a person or from an ai um or was it some mix of the two or something. I don't know. And I kind of agree with him for not allowing people to just throw the question into chat GBT and paste the answer into Stack Overflow. That's not what people are expecting. Right. And if you want that, just put your question straight into chat GBT. Leave out the Stack stack overflow middle aspect.

I'm sure the real concern was, you know, there's a lot of reputation stuff. There's a lot of like, this person is really good at answering questions and here's how much you've contributed. And if you just, you know, you could easily game that with AI results. And I'm sure it is being gamed AI results, just with a little more work. But such is the world. Yeah. Times are a-changing. They are.

That's what ChatGPT told me anyway. How about you? What's your next one? Well, I was going to talk about switching to Hatch. So for packaging, for Python packaging, and also setting up virtual environments and stuff i guess so hatch is um i'm okay so for packaging we had setup tools and then we had lots of stuff we had like flit and hatchling and poetry and all sorts of things um

so uh hatch there's two parts to hatch there There's hatchling, which is the backend that helps you build and unbuild packages and stuff. And then there's a, there's hatch, which is built. It has hatchling as part of it, but anyway, let's go through this. There's an article from Oliver and rich talking about switching to hatch. Um, and great graphic, by the way, uh, there's a graphic that says, I don't know where I'm going from here, but I promise it won't be boring. That's pretty cool. and great graphic by the way there's a graphic that says i don't know where i'm going from here but i promise it won't be boring that's pretty cool um anyway uh he was using poetry looked at pdm now he's thinking about hatch it's uh possibly his new love we don't know it's a

question mark so um there's there's a sentence here that bothers me and we can get back to it but it says some prominent projects are using it or using Hatch. I don't know if that's true, but let's go on. He does show some really cool things with Hatch, which I didn't know it could do. For instance, when you say Hatch new project, it creates a new project.toml file. One of the things it does in that is it creates these entry points to environment, like virtual environments, and they're separate ones for things like there's a default one, and then there's a test one that has like your PyTest and plugins and stuff. Then there's linting that has like PyWrite and black and rough and things in it.

His question really was, why wouldn't I have just a dev virtual environment? But we're using a lot of extra tools now, they might have dependencies that clash. So if you do have dependencies that clash, maybe a separate virtual environment for each type of tool chain might be interesting. You are speaking my language now. I didn't really think about that,

but there's about a 50-50 chance if I say update all the dependencies for TalkPython training, which I think is about 50. There's the runtime ones and then there's the reporting dev ones and that includes some things like Jupyter notebooks for graphs and stuff.

There's about a 50-50 chance that I'll get an error saying cannot find any solution to these requirements that you're asking for. Because something in the data science stack forces something to be less than or equal to, and something in the other stack is greater than or equal to, and those are non-intersecting. And I'm like, well, it still runs fine and I'm sure it'll be okay. But it's like, I'm constantly fighting that. So actually having a, a dev environment and a dev tools environment in a runtime environment is actually

pretty, pretty cool idea. Yeah. And one of the examples is docs, like they're using make docs material. So it's probably got a bunch of dependencies or you might be using some other document generator tool. And so, and you don't really care if that collides with your testing chain or something, cause you're not, they're not, they're different. So, okay. Anyway, kind of neat idea. I didn't know hatch did that. It's kind of cool. You can also do scripts. So each environment can have its own set of scripts. So you could have linting scripts and testing scripts and docs scripts and things like that like for instance one of the linting one is you could say run black and it has all of your black arguments or something you can

have that that's pretty it's kind of neat no need to use talks this surprised me apparently there's a matrix ability for hatch so you can do test matrix test matrices within hatch I don't know if you can do other why would you do a docs matrix though you know so test matrix test matrices within hatch I don't know if you can do other why would you do a docs matrix though you know so test matrix makes sense I don't know it makes sense anywhere else but that's kind of neat and then the last bit I thought was pretty cool was that well for what it has scripts but I don't know if it's very convenient. So like, say you have a cove script for coverage within your test environment, you would run it by saying hatch run test colon cove. It's kind of a mouthful still. So I'm not sure it is. Yeah. Optional dependencies are kind of neat. You there's all

most tools have optional dependencies, but the apparently hatch has a kind of a neat way to say, so let's say in the example, he had two different optional dependencies for MySQL or Postgres, depending on what database you want to install. A default could be just saying MySQL and it would pull in all of the requirements for MySQL. You just so there's an easy way to do sort of a transitive default set. So it's kind of cool.

Anyway, nice. The the thing I wanted to come back with was the comment of like, everybody's using it. I don't think everybody's using it. If you look at the Hatch website, it does say all these different projects are using Hatch, but some of them, it's obvious they're using Hatchling. Like most of my projects use Hatchling. That's not the same as using hatch yeah it could

just be the build back yeah just for building the wheels right exactly that build back in is rock solid and i recommend it the hatch is sort of the work the top hatch is that workflow tool is similar to poetry so but still cool i kind of like that um poetry kind of makes you at least last time i tried it it's sort of in how do you take all of the tools with it but hatch is more of a use the tools you want sort of a tool so anyway yeah kind of cool so nice i just checked while we were talking you did what did i say 48 or something for the number of dependencies yeah it's insane i don't know how i got to where i 232 packages to run talk python training with the reporting i i can't easily separate the dev versus runtime without more work.

But that's a lot of packages. And it's like, are you sure you're not using Go or something else that uses tons of dependencies? JavaScript? Yeah, yeah, yeah. I'm not saying it's a problem. It's just with 232, there's a chance, there's a reasonable chance that a bunch, you know, a good bunch of those come from the non-runtime stuff.

There's a good chance that there's a clash between them. So this idea of multiple virtual environments is cool. Yeah. Yeah. But I'm assuming most of those are transitive. You're using some tool that's using some other tool. Yeah, yeah, exactly. Exactly.

Cool. Okay. On to exactly. Exactly. Cool. Okay. Onto the next, uh, hold on. Comment from Mark in the audience says, yeah, I feel like hatchling, but not hatch is a fairly common pattern. Uh, when mentioned on doc Python to me. Yeah. Awesome.

Speaking of speaking of, I just want to point out that episode 408 earlier this year, I had OFEC on the creator of Hatch talk about Hatch and its benefits and all those different things. So people can check that if they care. But let's talk formatting code. Formatting code equals black, yes? Yeah, and others. And others. However, Ruff checks your code formatting for correctness and we've we've

discussed how fast rough is right to the point where it's like hmm did i actually check the code did i enter the wrong the wrong directory and there just found no contents right that kind of thing right well uh this was sent over to us from sky so thank you sky for sending in. Charlie Marsh, creator of rough has announced the rough formatter. So not just telling you what's wrong, but checking for errors, but formatting your code based on convention, similar, but not identical to black. So that's pretty cool. Let me read a few things that sky sent over here, because I think it's their experiences worthwhile. So Charlie says, first of all, the formatter is designed to be a drop-in replacement for black, but with an excessive focus on performance and

direct integration with rough. That's pretty cool. Right? Yeah. So Sky says I can't find any benchmarks that have been released yet, but I did some extremely unscientific testing caveat there and found the rough formatter to be five to 10 times faster than black when running on already formatted code or in a small code base so five to ten times

but 75 times faster when running on a large code base of unformatted code however they point out that the second outcome is not that relevant because how many times do you format huge projects that are not formatted now Normally it's incremental, right? So the smaller bits is maybe worth paying attention to more there. Yeah. Yeah. So I almost missed this announcement because rough already had some, go ahead and fix it if you can features.

Yeah. It did have a few fix it things. Yeah, exactly. And like I said, I think that was about there's a violation rather than a convention, right? It's not a violation necessarily to say, I write generally in single quotes or I write in double quotes for all my strings. Or I might sometimes have a single quote or sometimes I have a double quote where like

you might have a coding convention that says all of our strings are the same. They all use single quotes or they all use double quotes. And I don't, there's no reason for like in the same function to have two kinds of strings unless, unless you're in that situation where like I'm saying it's possible. So I don't want to use a single quote, so I don't have to escape the apostrophe. So I'll use a double quote for that one. Right. But other than those like sort of weird cases, you shouldn't mix. Right. I think that's more what the point of the rough formatter is addressing and black as well yeah well rough has a a few and a few differences from

uh from black as well yes and they they call out it says the formatter is intended to emit near identical output when run over a black formatted code this is interesting when run over extensively black formatted code projects like django Zulip, it was 99.9% the same. However, it says somewhere when run over non-black formatted code, it might make different decisions than black is made. Yeah. And I kind of like some of the decisions that they're making. I do too. I do actually like them quite a bit uh so i'm i'm a fan of some of these things it's not it doesn't have as many features yet as as black does in terms of like controlling certain things or you know but they're they're working on it i was talking to charlie just an hour ago by talking i meant submitting a github issue it was quickly we're having a back and

forth there so that's awesome yeah i like so like, so for instance, the line endings, uh, I think are a cool way to deal with it. Um, yeah. So the line feed versus carriage return line feed backslash N versus backslash R backslash N that's a windows versus non windows challenge. Right. And so I guess if you're on a windows, you don't want it to keep like unraveling that for you, I actually i thought uh one of the things i thought i read was uh things about um comments at the end of the line uh black would often put like the comment on a completely your

your comment might not match up with uh what it what you actually commented against but uh rough is trying to be a little bit better about that so yeah the other area where this is supposed to be different intentionally says um frequently black will suggest um this is the same or the different yeah let's see there's there's some places where it's specifically different um uh versus black and it talks about yeah i don't know i haven't read it i don't want to like mistake quote it here but it's it talks about yeah i i haven't read it i don't want to like mistake quoted here but it's it talks about there's a whole bunch of sections of the the variations and so on yeah anyway i'm excited to try it and um and uh i think it's cool that this is happening

in this space indeed uh jeff out there asks is there also a rough daemon like black has i had no idea about the black daemon and i definitely don't know about the rough daemon. But what I use is I just have the IDE integration for rough. So it will automatically be running in the background. You get that for PyCharm and VS Code. I imagine it makes sense to have it run and just constantly checking. So I don't know. Not sure. Right. This is still in alpha, so probably not. So also good to note, this isn't a separate, it is kind of a separate tool, but it's part of Ruff.

So it's if you say like Ruff format. Yes, exactly. So yeah, uh, rough format and I actually was thinking, I don't know, I can't find my, my little example that I was running earlier, but there's a couple of, yeah. So you can do things like, I'll put it. So you can say like rough format dash dash line length dash dash respect, get ignore. So it'll say if it's ignored and get don't don't format it, don't format it. Please. You know, things like packages with it. Don't format it. Please. You know, things like packages that are installed in a virtual environment.

Don't go messing with stuff that, like, I don't care about, right? If it's ignored in GitHub or in Git, it probably is only going to cause me trouble by messing with it. So just leave it alone, right? It's kind of the way I see it. I never considered that. Does Black change, change like virtual environments this is a question i don't know okay anyway probably not but i don't want it to if it does anyway cool that that the rough rough space format very cool um well uh we're kind of going into like the the inside baseball on this uh episode but that's all right um uh next up we've got a

suggestion from will mccougan um thought this might be good a good one discuss on the show uh what's wrong with toml um i'm like i don't think anything's wrong with toml so let's take a look um so there's an article uh oh i forget com so it's hard to find but calm o'connor um cool name uh what's wrong with toml and the gist of this really is toml is is great for smallish things and even considering pyproject.toml is smallish but interesting quote from uh apparently from martin vign, author of PyToml, said, Toml's a bad file format. It looks good at first glance, and for really, really trivial things, it's probably good. But once I started using it,

and the configuration schema became more complex, I found the syntax ugly and hard to read. Not sure what he was doing with it, but anyway. So apparently there's some funkinessiness with big things and i'm like well what is big things and what are they comparing it to their compare one of the comparisons is against a thing called strict yaml which i didn't know what that was and strict yaml is yaml compliant it's the yaml that won't let you go out at night your curvy is like 9 30 it's it's really it's oppressive um apparently it's yaml with uh uh some of the features taken away so i'm not it's not a standard yet but apparently it's in

the process so like what are they doing with it that toml is a problem and uh it's around strict yaml also is built for what is is it built for? Readable story tests. I'm like, what's a readable story test? I want to see. Here's some examples. We've got, this is a strict YAML readable story. Mappings with defined keys. Anyway, this is sort of readable, but there's a lot of keywords in here that I'm not going to say this is readable.

I don't think this is that great now compared to the Tomo version. Yeah, this is weird. You've got like these weird brackets with lots of, I don't think this is necessarily a problem. My take on it is I wouldn't use either of these for this purpose. I would use Python probably to describe describe stories but anyway um i don't want to bash on him i guess something to think about if uh toml is if you're using toml for

really large things uh maybe it's a problem uh i'd be curious to know uh will if will if you're listening what do you think is it uh are you using some wacky large toml files that are becoming a problem i don't know so that's just wanted to throw that out there awesome got any extras to throw out there uh my extras are totally self-serving um but um i see the beginning of the episode yeah so the the last uh so last last week i announced i I think that I had part one of the by test course, all buttoned down and ready. And that was a kind of a problem, because my video intro video was like, Hey, I'm starting this course, but I've already started it. So I re what I did is I did a few things, I redid the video, the intro video to just sort of describe

where this course fits in with everything else. And so it's like a few minutes to check it out. The other thing I did was I had some feedback from people that said, teachable sort of easy to use, but some people might not understand, maybe you should do a little intro video. So I did a little intro video for how to use how to use teachable. And it's a few minutes also. And one of the things I like about this is I learned some things that I didn't know. So my favorite is you can adjust the speed. So you can listen to me at like 1.25 or 1.5 speed.

And it'll go faster. The other thing is you can add notes. You can add like notes for different video places and say, and then when you click on it, so I'll even do it with a explainer video. You can grab a note, hi. And then later you can go back and click on the note and it takes you back to that part of the video.

So if you want to keep some notes, that's kind of neat. So that's really all my extras is that's going on. Yeah, that's a very useful feature. Excellent, excellent. What you got? Making progress on your course. Nice to see. Thanks.

I also have a course announcement. So Christopher Trudeau and I teamed up to create a Django version of the HTMX course at talk Python. Now we have HTMX plus Django modern Python web apps, hold the JavaScript. So this is a two hour course that shows Django developers how to work with htmx, how to build up like a pretty, pretty realistic, pretty complicated, but not overly complicated, but you know, not toy type of application that they get to build throughout the course. So check that out. It's still

on the early bird special. So if people get to it by the 23rd september 23rd got a few more days to save 10 if you're thinking about getting it might as well do that now and it has the sister flask course if you don't jango but you flask so those are our two sides of the the same coin there and htmx is just awesome so check that out if people are interested look it's looking forward to that yeah thanks And then if you happen to be coding in Rust, JetBrains just released a new IDE called Rust Rover. Funny name. I'm not really sure the origin of it, but it's based on the same foundation as PyCharm. So if you're already using PyCharm or something like it, WebStorm, whatever, and you have the muscle memory for those hotkeys and basically the way it looks and feels,

but you also want to do Rust, Rust Rover, Rust Rover, come on over. You know, let's do it. Nice. I haven't tried this out. I don't do any Rust. So people who do Rust can check it out and let us know what they think. Nice. I use the C++ version also.

The C-Lion? Yeah. It's sort of a funny name for it. It is. They got good names. Also, Skyler Costco, who is the sky from submitting the rough formatter, says it looks like dash dash respect get ignore is the default behavior in a rough formatter. You should only need to set a flag, pass this flag. Do you want to format get ignored files via dash dash, no respect.

No respect for the Git-ignore. Got no respect around here. Yeah, some Rodney Dangerfield programming right there. You dissing the Git-ignore, come on now. New TalkBython episode I just released like right before we jumped on here. Delightful machine learning apps with Gradio. If you want to take an ML machine learning model you've created and put it into an interactive UI on the web that you can share, super easy.

Check this out. Open source. Very cool. They even have some hosting options. Nice. Both free and paid. All right. So that is it.

Oh, I guess one more piece of follow-up. Since you asked, Will McGugan says, are we talking about more hypothetical issues? I think there were some good points, but haven't faced them yet. Okay. Thanks, Will. I figured you were listening. Yeah.

Brian, are you willing to face a joke? Well, yeah, I think. Yeah. Okay. So sometimes, sometimes. I'm worried now. Well, I'm sure in school, you probably studied like the five stages of grief or something like that.

Okay, yeah. You know, that's not something you really want. Why? Well, there's not open image. I can't zoom it. Anyway, maybe it's just best. No, it's not. But I got to try to read very small here.

Hold on. So this is the five stages of debugging. Okay. I thought we'd find a way to make this bigger. So there we go. is the five stages of debugging okay well let me find a way to make this bigger so there we go so the five stages okay number one denial this stage is often characterized by uh phrases such as what that's impossible or i know this is right and a strong sign of denial is recompiling without changing any code just in case. Funny. All right.

Stage two, bargaining and self-blame. Several programming errors are uncovered and the programmer feels stupid and guilty of having made them. Bargaining is common. If I fix this, will you please compile? Also, I have only 14 errors to go. Stage three is anger. Cryptic error messages send the programmer into rage. This stage is accompanied by hours long diatribes about the limitations of language directed to whom

ever will listen. Stage four, it's getting serious depression. Following the outburst, the programmer becomes aware that the hours have gone by unproductively and there's still no solution in sight. The programmer becomes a listless, the posture often deteriorates. And you could see like all the graphics are like screaming and banging on the computer or like staring at the sky. The depression one is just sunk in the chair and acceptance. The chair is the, like the wheelie chairs turned around there's gone there's no one at the computer anymore yeah the final stage is acceptance the programmer finally accepts the situation declares the bug a feature and goes to play some quake yeah so uh i just yeah

uh there's tons of stages missing yes this is funny i know it's supposed to be a joke but um but like get up go talk to somebody else don't do like leave the computer that should be one of the first things you do is go like take a shower or take a nap or something like that and come back yeah that's very productive i also think somewhere in there there should be searching stack overflow yeah and there should be another stage where you go to chat gpt and see if it can help you yeah or, or duck, what, rubber ducking? Yeah, exactly. So I never really could get into the rubber duck completely of explaining the problem to a rubber duck

or some inanimate object. But explaining it to a non-technical person, I'll like try to explain the problem to like one of my kids or something. And I'm often, while I'm explaining it, I'm like, wait, I think the problem's there. Anyway, so yeah, that's funny. I find time away often is the most important thing. and i'm often well i'm explaining and i'm like wait i think the problem's there um anyway so yeah that's funny i find time away often is the most important thing yeah go go record a podcast

or something exactly or go for a walk go for my motorcycle ride bicycle ride like just get away from the computer for a little bit yep yeah cool well all right well speaking of getting away from the computer exactly let's get away from this podcast thank you everyone thank you bye bye ryan