Python Bytes - #374 Climbing the Python Web Mountain

Episode Date: March 11, 2024

Topics covered in this episode: 6 ways to improve the architecture of your Python project (using import-linter) Mountaineer Why Python's Integer Division Floors Hatchet Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/374

Hello and welcome to Python Bytes where we deliver Python news and headlines directly to your earbuds. This is episode 374, recorded March 11th, 2024. I am Michael Kennedy. And I'm Brian Ocken. And this episode is brought to you by Scout APM. Check them out at pythonbytes.fm slash scout. I'll tell you more about them later. If you'd like to connect with us, Mastodon is the main place. We're also on ex-Twitter, but primarily on Mastodon.

And that's at mkennedy, at Brian Ocken, and at Pythonbytes, all at fostodon.org. And watch us live, usually Tuesdays at 10 a.m. Pacific time. However, different today, but check it out at Pythonbytes.fm slash live. You'll be able to like get notified about the next upcoming recording. I usually put that up right after we're done with this one. Brian, before we jump into your first topic, I want to just take a

moment and appreciate the beginning of summer, the first step towards summer coming. It is daylight savings switch over here in the U S and I know it is much maligned by people, but I was delighted to see the sun was up past 7 PM yesterday after living in dark and rain for months and months. I'm feeling, I'm feeling the summer. I know it's not quite there yet, but yeah, I didn't really notice it until after I woke up. Um, yeah. I woke up and I'm like, Oh gosh, I, I thought'd slept in until like 10, but it was really 11. So sleep the day away. Oh, beautiful.

Beautiful, beautiful. All right. Well, over to Python. What's your first thing? Okay. Well, I want to talk about spaghetti code a little bit. Actually, or how to fix it. So there's an article.

Is that what you get when you're real hungry? You're like, what are you going to have for dinner? Maybe some spaghetti code. The little carbonara. Is that how you pronounce it? I've never been able to pronounce it. I'm probably going to get emailed that it's not, but that's how I pronounce it.

Article from somebody that goes by Piggly. Six ways to improve the architecture of your Python project using import linter. And I kind of like this article, even if you don't want project using import linter. And I kind of like this article, even if you don't want to use import linter. So because of the six ways to improve the architecture, I love that. I love digging into that. So and I actually have a couple projects that I'm working on that I would like to maybe clean up the architecture a bit. So kind of a fun picture, not sure what's going on here, just a whole bunch of boxes pointing at each other, I guess, just indicating that there's circular dependencies

or something going on. And anyway, so a little bit of a discussion about why complex architecture might be bad, but I don't think any of us need convincing of that. Simplifying is a good thing. So the first part talks about maybe setting up an idea of like maybe you could you have an idea of like setting your architecture into layers. If that's what you want to do, like a user layer or an application layer plus utilities or something services. But there's a tool called the import linter that there's a discussion about how to set it up so that you so that you can use that to both configure what you want your layers to be and then test for it. It's a like a linting to make sure that you're not importing in the wrong direction. So the idea would be like you're not importing like if you want the top level one to be calling the lower level ones, but not the other way around. So the importing is a way to test that.

Because of that, I'm not sure what these arrows mean, because it doesn't quite make sense if it's not inheritance or calling. I don't know. Anyway, it's just. That looks pretty interesting. I think that means that something from that layer is importing something from the bottom layer, the lower layer. Yeah, but, yeah, okay. You can have stuff like import across within, but then it can only, I think it would be an error or a warning if maybe, say, like the data layer imported the UI layer sort of thing. I think that's what it's saying.

Yeah. Oh, and it makes more sense if I would have drawn the top layer at the top. I think it makes more sense to me if we do it the other way. Anyway, that's, that's a drawing thing. It doesn't really matter. And also it was by somebody in Australia because everything is upside down down there. Oh yeah. Yeah.

That makes sense. Yeah. Yeah. That was it. Okay. Okay. So what happens when you run this is you're probably, the recommendation is you're probably going to get a bunch of Lint warnings or errors. Or maybe not.

If not, awesome. But if you do, the recommendation here, which I kind of thought was cool, was to kind of exclude all of them. Like go through and add these ignore imports and just go through and ignore the ones that failed. And I'm like, okay, well, then you're going to pass and it's just going to ignore everything. But the idea behind it is to do it one at a time. So it's going to be overwhelming to get a bunch of errors. So ignore them all and then unignore one at a time and then go through and fix them. So that's where the reason why I picked out this article isn't really because import linear is cool.

It might be cool, but it's that these ways to fix these import dependencies I thought were really great to walk through. There's six of them. First one is merging and splitting modules. And there's an example where you've got resources that calls clusters, clusters calls resources dot cluster utils, and that's going the wrong direction. But to fix it, you maybe move those cluster utils down into the lower, like a lower library, like splitting them off. Kind of sort of straightforward, but

it's good to like think about this so i like that that part of it dependency injection and if you're the kind of person that's kind of afraid of dependency injection then don't even think about that term because it's not actually that scary just the term was scary to me for a while so uh anyway dependency injection might help to be able to pass uh a pass dependencies down into lower layers from upper layers. So that's a good way to get around it. The interesting, I'm glad they dug into this, is when you do the inversion of, what is it? I forget, the inversion of control.

Inversion of control, yeah. Yeah. Sometimes with Python, that works great except for type hints um sometimes you need to to get the type hints getting those right so talking about how to use the um protocol uh from typing to get that to work right that's that discussion's in here which is great i love protocol um then another discussion about using simpler dependency types and then delaying function implementation um and then uh and then even what was the the last one look at for six configuration driven that was interesting um i've never really thought about doing this the

the idea is you'd have a like a settings file or something and have that have and then have the string that you would import like marketing.send sms and then later on in your calling thing you use import string to import it so you're you still have the backwards dependency but you don't know it at at run at compile you know at write time you know at run time i don know. I don't know what I think about that, but anyway. Yeah. Yeah. That's interesting. It's, it's a way to do it. So, and then the last one is a replace function calls with event driven and purchase. So like callbacks and stuff like that. So these are a great architecture things to think about when you're cleaning up architecture, because it's not always trivial to just say, just don't do that. Don't do those imports. Well, how do you get around it if you've designed it that way? So I think a decent discussion about software architecture here. So I do think that's

pretty interesting. And also just more broadly, I like the idea of ignoring all the issues you might run into with lenders and then slowly turning the screws to make it a little tighter is good advice because if you have existing code of any significant size and you've never done linting on it and then you turn on a linter. Yeah. Like you feel bad about yourself.

You're like, I've done all that wrong. 593 errors. Great. Well, we'll turn that back off because I got work to do. You know what I mean? Yeah. Well, I'm kind of going through that

with the rough right now. So got some projects where default rough works fine but the rough has like 800 rules or something you can add and so i'm trying to turn on some of those extra features i have a just a few at a time see testing the waters see if how many failures i get and whether i want to clean them up. So, yeah. Yeah, interesting. Just interesting timing. I'm going to be speaking with Charlie Marsh on TalkPython tomorrow. Oh, awesome.

About 24 hours from now, whatever that makes it for you. Yeah. Out there listening. Yeah, and we're going to talk mostly about UV, but I'm sure we'll talk a bit about rough as well. Cool. Well, I'm going to release an episode

with Charlie Marsh today. Well, how about that? Awesome, awesome awesome do you guys talk about uv as well yeah we talked about uh yeah flick eight or not flick eight um talked about rough and astral and uh and uv most of the conversations around uv and some of the discussion around that so Nice. That'll be fun. All right. On to the next one here by Pierce Freeman. That's kind of a cool name, isn't it? It could be like a 00 sort of agent. Yeah. 009.

I don't know what that is. Not James, but something like that. Anyway. Pierce Freeman to the rescue. Exactly. To bring you React and FastAPI and Python. In fact, with this framework called Mountaineer. So Mountaineer is a batteries-included web framework for Python and React.

So this, it plays in a similar space as FastUI from Samuel Colvin and the Pydantic crew. I don't know enough about them to know how truly similar they are. I think this is coming from a different angle. I think you you're doing, you know, the, so fast UI is more about maybe you don't really have to do the react side unless you want. And it kind of brings that stuff together. Whereas this kind of like you're embracing TypeScript, you're embracing react, but you also get some really nice Python integration.

And so if you look down here what does it say it says it lets you easily build web apps in python and react and if you are familiar this it should sound familiar to you and should basically seem like what you're used to and if not then yeah maybe not but it says each framework like flask or fast api or django or whatever has its trade-offs and features. And for this one, it focuses on developer productivity above all else with production speed of close seconds. So type hints up and down the stack, front-end, back-end database, trivially easy client-server

communication. So you don't find yourself creating a bunch of APIs so that your React stuff can talk to the APIs so it can actually get its data and all that, which is pretty interesting. It comes, this is kind of cool. So one of the things you can run into, I'm sure you've seen this, Brian, it drives me absolutely bonkers. You'll go to a website and it'll have something on the screen. And then like half a second later, it'll shift around and stuff will like all come into existence. existence you know you'll be like it'll have like the footer will be touching the top and then it'll bump out it'll be a spinner and then stuff

you know it's like what is it what is it doing well especially if you start reading and then a picture pops in and your what you read pops off the screen and yeah another thing that drives me crazy is um if you paste something into a input but but you don't type it, sometimes that won't trigger the data binding. And, you know, like if he's a password, password manager type thing or something, or you just paste something, it'll say, oh, this is invalid. What is wrong with this? Oh, if you put a space and delete the space.

Oh, oh, it's valid now. It's like, oh, I see. So something that would be really nice is if you just could ship straight HTML, right? Yeah. Or the first view of that was. And so what this comes with is they actually bundle the V8 engine. So on the server side, it can act,

it can render what your browser would do for you as final HTML and it delivers it with. So optimize server side rendering for better accessibility and SEO. That's pretty cool. I think. that was cool. As a Python thing. Like normally you hear that as a node thing or something right yeah it also does static analysis for validations of like links and css and so on and like i said skip the api uh or node.js server just for front-end clients okay so all of these things are pretty cool if let me give you a warning and a disclaimer

one, if you don't know react real well, I don't think you'll really appreciate the benefits here that much. Like this is really for react people. 100%. Second, that's not me. So I'm going to do my best to like tell you why you might want this. I think I can get there. So first of all, it has a scaffolding type thing called create mountaineer app and they suggest pipx and i'm loving to see more pipx come along for these kinds of tools right like you run this once it's not part of your app you just pipx install it and everything gets going it's great also uses poetry pretty heavily um so what it does is it creates some python bits and then some TypeScript stuff for your

front end. And it comes with a CLI as well, which is nice. It has a built-in Docker containers for like managing Postgres databases and so on, if you want to use that, but you don't have to. And let's see. So another interesting integration is it uses SQL model. So that's the typing and the data layer aspect. So SQL model is Pydantic plus SQL alchemy, basically the typing and the data layer aspect. So SQL model is

PyDantic plus SQL alchemy, basically by Sebastian Ramirez. So that's cool. And then you go down here and you create controllers. So it's kind of a class-based type of thing. First, it seems a little unnecessary, but as you interact with it and expose more features to the React layer, you'll see that kind of relevant there. So you just say, I'm going to render, say, my database things and some other pieces of data. And then down somewhere, you've got your React code and your TSX, your React component there.

And if you've written React stuff, you should know pretty well how it works. But it manages passing all that data over like serverstate.todos. Then you can just work with that, which is interesting. But it gets more interesting later if you say, let's add an async function like add one of my to-dos. They have a to-do example, right?

So if you put this at side effect decorator on it, here's where the React integration comes in. That's pretty wild. So it automatically generates typed, typed, let's just say typed TypeScript because, I mean, you can't have non-typed TypeScript, but generally TypeScript is typed. It generates TypeScript versions of the functions

that you write in Python on the JavaScript side so you immediately can just start calling it and using those features, right? So it kind of has this really tight integration with React, like, as you would expect. What else? I think that's pretty much it, it and using those features right so it kind of has this really tight integration with react like as you would expect what else i think that's pretty much it but it's if you're a heavy react shop and you want a nice python back in and you want a tight data integration between those this is probably worth a look yeah neat yeah yeah it looks pretty neat to me too i'm not like i said

i'm not really a react person so i'm not sure that I'm necessarily going to use it. But if I were to start adopting React, I may well decide that that's what I want. Speaking of cool things, Brian, how about Scout APM? Let me tell you real quick about Scout APM. They're big supporters of Python Byte. So we appreciate that very much.

So if you are tired of spending hours trying to find the root cause of issues impacting your performance, then you owe it to yourself to check out Scout APM. They're a leading Python application performance monitoring tool, APM, that helps you identify and solve performance abnormalities faster and easier. Scout APM ties bottlenecks such as memory leaks, slow database queries, background jobs, and the dreaded N plus one queries that you can end up if you do lazy loading in your ORM and then you say, oh no, why is it so slow? Why are you doing 200 database queries for what should be one? So you can find out things like that. And it links it back directly to source code so you can spend less time in the debugger and healing logs and just finding the problems and moving on. And you'll love it because it's built for developers by developers.

It makes it easy to get set up. Seriously. You can do it in less than four minutes. So that's awesome. And the best part is the pricing is straight forward. You only pay for the data that you use with no hidden overage fees or per seat pricing. And I just learned this, Brian, they also have, they provide the pro version for free to all open source projects. So if you're an open source maintainer and you want to have Scout APM for that project, just shoot them a message or something on their pricing page about that. So you can start your free trial and get instant

insights today. Visit pythonbytes.fm scout the link is in your podcast player show notes as well and please use that link don't just search for them because otherwise they don't think you came from us and then they'd stop supporting the show so please use our link pythonbytes.fm scout check them out it really supports the show awesome awesome what's next brian well what's next is Guido van Rossum. Oh, yeah. So he's blogging a little bit lately, and he put up a post called

Why Python's Integer Division Floors. And I think this is just an interesting little bit of history. And this was a difference between Python 2 and 3. I guess, no, integer division always did flooring, but we did kind of a thing of what one slash meant.

If you do two slashes, it's always integer division. If you do one slash, if it was both integers in Python 2, it would be integer division, and it would possibly not be the floating point result. But in Python 3, it will generate a float if you have it. Like, for instance, one third will, one divided by three results in a float. But if you do integer division, it's not.

It's something else. Is that why the two to three conversion was so hard? I'm just kidding. What was that? It was strings. Everyone knows it's not it's something else is that why the two to three conversion was so hard i'm just kidding what was that it was strings everyone knows it's strings so is that why the integer the the python two to three uh story was so hard no i'm just joking yeah um so uh yeah so if you do five to two slashes so integer division of five divided by two, you get two. And which is normal. That's everywhere. But if you do it, like, say, negative five, you get you don't get negative two.

You get negative three. So that's the there's a question. So Guido said that he had a question about that of why it's different um why it's like that and so it's going it does a division and then it goes to the the closest integer to the negative closest to negative infinity so um and there's there's reasonings behind that and but it's different than c apparently i completely forgot what C did. C does it closer to zero. So you'd both negative five divided by two would, you'd end up with two negative two.

So the, the history there is that there was a, there was a choice to make and you could either with you, it works with the modulo operator as well. Modulo will create the remainder where integer division creates the integer quotient. So such that a divided by b equals q with remainder r such that r is between 0 and b. There's math here. But you have a choice as to what you want to do,

whether you want R to be possibly negative, positive or negative for the remainder, or if you want R to always be positive. And basically Guido chose the one that looks nicer in math. So that we go do a floor division instead. The interesting take on this, I thought, was why did C choose the other way? And that's the part that I thought was really interesting. And his answer, or his guess, is that, well, C was doing it way before Python was. And C was doing it way before Python was, and C was doing it on hardware that

it may have been easier to do division closer to zero instead of floor division. And part of that reason might be because some of the early hardware architectures were using sine plus magnitude rather than two's complement, which is kind of, I didn't know that. I must have, either if I did know it from CS, I forgot it. Definitely remember two's complement. But anyway, it's an interesting history there. Also, one of the things I brought up,

one of the reasons, it's kind of a silly little article to bring up, but one of the things I wanted to bring it up is because a lot of new python people actually don't remember um we'll forget about integer division and just assume that division is division and that but inner division is really handy for a lot of cases so don't forget about that's true actually yeah i find myself sometimes doing int of some float result. Maybe I could just double slash it and not have to. So comment from the chat, which I'm not sure how to take this.

I'm thinking that they're bored with this topic, but... I'm not sure either. Anyway. No comment. All right. You know what? We're going to bring the hatchet out on this one, Ryan. Bring in the hatchet.

So there's this thing I ran across called Hatchet, which is a distributed task queue for more resilient web apps. Now, I don't recall exactly what it is written in. I don't think it's written in Python. It's unclear. Its primary language is Python, but also has a Go. Anyway, it doesn't matter. It has a Python. Its first SDK,

its first integration is with Python. So here's the idea. You've got some work. It's going to take a little while to run. You know, your web app says, hey, I want you to ship this thing or I want you to run some analytics. And if those analytics take 10, 15 seconds to run, maybe they're computational and they should run out on another computer rather than on your main web front end, or they shouldn't block it for 15

seconds or whatever. When you were talking about your, how do you break up import dependencies across the wrong layers and stuff? Maybe one way to fix that is to move some of the compute work completely to its own place, right? This whole queuing mechanism is super fascinating for creating truly scalable multi-user thing, right? So if the majority of your work is in some place and you kind of don't really need the answer to give a response, you can just throw it onto one of these background cues and and let it go so it one of the problems though you run into though is what if something goes wrong or how do i see what is running something fails can i resume it maybe it was really important like ship this thing to the customer but when i put that on the queue and it finally got around to being run, the API at UPS was down for who knows, whatever reason, right?

Things like that. And you can't ship it. And they just go into the ether as an error or you resume it. Another problem you run into is fairness, right? If there's a ton of work coming in faster than the processor can handle, is there some thing where maybe only the new ones get worked on and the older ones get almost abandoned, right? There's all these interesting things. So this thing called Hatchet is in the realm of many different things that attempt to solve this problem, right? It's interesting. It's Y Combinator backed. It's a company that presumably will have a

price, but there's also just an open source self, you know, take it for yourself and run with it version over on GitHub. So 100% open source, 2,200 GitHub stars. Pretty interesting. But I think the business model is, it says request cloud access. Like they can run it for you

or you can run your own, right? Yeah. So the website has a bunch of these little animations, which are kind of cool talks about fairness uh batch processing and like as you click on them see how they like has these little fun fun animations like what does this mean oh i see oh i love it i do too i do too so talks about uh fairness batch processing workflow and event stuff. It's engineered for scaling challenges, which is pretty awesome. So low latency, 25 milliseconds on average, which means if you put something into the queue,

there's ways in which you can sort of make callbacks to check on the process of the work, and it shouldn't take all that long. A bunch of rules about rate limiting. also durable. So you can replay events. You can do cron jobs and say, hey, every morning, just drop this into the queue and run it at 7 a.m. or whatever. You can schedule one-time jobs. It has ability to avoid being destroyed by spikes. Like if for some reason,

a whole bunch of work comes in all at once, maybe you got IoT things that do a bunch of work when people come into the office and they very first sees a bunch of stuff and in all at once and maybe you got iot things that like do a bunch of work when people come into the office and you very first sees a bunch of stuff and then it chills out right you could smooth that out all kinds of stuff but like i said it supports three technologies python number one typescript number two and go number three and it's really easy uh to do you just go down here and you just put a you know when this event happens i want you to run this class and then it has a bunch of functions methods you say here's a step step one do this work step two so you just basically put some decorators on a class and then plug it in and off it goes super easy and to run it you just say push an event whatever the name it is with the

data and it'll go put it off in the background and run it so yeah it also has like nice visualizations like here you can see there's this um on hatchet.run you can see there's this like live view of how is the work running through the system and all kinds of stuff so it looks pretty neat to me open source people can check it out it's worth knowing about yeah it's pretty cool neat yep indeed indeed uh that's it for our items how are you feeling how extra are you feeling i have only got one extra that i almost that i pretty much mentioned already so i'll do it quickly so um python test is at 215 right now as we look but the most recent episode that will come out probably today is two 16, which will be, um, Charlie Marsh, uh,

talking about UV. So check that out. Also. Awesome. Yeah, you did mention that. That's, that's really good though.

How about you got any extras? I'm feeling somewhat extra. I got two exciting announcements. One, I have a free new course over at talk Python training that covers a bunch of awesome technologies, but the, the core idea, the title is Build an Audio App with AI, with Python and Assembly AI.

So the idea is, what would you do if you, say, had access to, I don't know, a podcast worth of data that's been going for many years? Like Python Bytes or TalkPy Python are honestly the thing we build lets you access a whole library of podcasts. And you go in there and do things like, hey, create me a transcript, which seems kind of straightforward. But once you have transcript data, you could get really cool search, like building your own custom search engine, not just over the title and the show notes and stuff, but also all the spoken words, which is kind of neat. Then you also bring more of the AI stuff in. You could create a summary. What are the key moments of this? And actually, what if I could

just have a Q&A with like you and me around what we said in the show? So kind of creating an LLM chat GPT type of thing, but where it knows about any given episode out on the internet. So really fun. People will learn FastAPI. They learn Pydantic. They learn HTMX. They learn Beanie. And they learn Assembly AI and build a cool thing. And the whole course is like a four-hour free course.

So they can check that out. Wow, neat. Sounds fun. Thanks. And next to it, another new course, Rock Solid Python with Python with python typing so this one is not a free one but it basically shows you not just the how but the why and when of python typing a bunch of different examples um you know obviously the language but you know things like fast api and pydantic how do they use it um what you talked about protocol before what is

protocol where does it fit um bunch of design patterns and guidance for python typing and how to think about how you should use it so people should also check this one out i'm really proud of this one and it is also around four hours 4.4 hours and uh pradeep asks out there are these courses cater free yes the build an Audio app course is 100% free. You just have to create an account. The Rock Solid Python course is 49 bucks. Yeah, but that's like 10 bucks an hour. That's almost free.

Yes. It's certainly not a lot of money, you know, compared to other ways you might go learn about things like this. So anyway, both of these courses are awesome. People should check them out. So those are the two big announcements. I also have a couple of interesting things I just want to give a quick shout out to. Previously, we spoke about Doku. I'm going to go with Doku because I think it's based on Docker. I don't know. Yes. Doku. And this is an open source platform as a service alternative to Heroku. We already spoke about that. But when we did, Ray out there on Macedon,

thank you, Ray, said, hey, you guys, love the episode on Daku. Haven't tried it myself. Big fan of Heroku. However, I set up Coolify. Coolify. I haven't talked about this, have I? I don't think so.

Yeah, I don't think so. Okay. So Coolify is kind of the same. So it's pretty similar, but it has a nice GUI to configure everything and keep an eye on the status and all those things. So Coolify is self-hosting with superpowers. It's a self-hosted alternative to not just Heroku, but also Netlify and Verisell. So Netlify for static sites.

You basically set this thing up, get it goinglify and Verisell. So Netlify for static sites, you basically set this thing up, get it going. It'll run any language on basically any server. You just push to some Git branch to deploy it. It does automatic SSL certificates. So if it just had a static site and I just want it to run over SSL on the internet, boom, done. It just lets encrypt automatically as part of creating the app up there and gets it going. So it's probably a little bit of setup to get it going and get it running on Docker and stuff. But once you do, it just becomes the substrate for all of your apps that you want to put out there. And you don't have to think about anything but Git basically. That's pretty cool actually yeah it looks really really nice so

people can check this out um yeah it has a paid cloud version and a self-hosted version with 17 000 or more people using it uh self-hosted so that's pretty cool it's interesting they give stats there but that's my last one i think this is really neat as well. So Ray, thanks for sharing extra details. Yeah, nice. Cool. Shall we close it off with a joke? Yeah, let's. Yeah, okay.

So speaking of, I want to run my stuff on production. How do I do it? This is a great, a great one. Back to workchronicles.com. And so I don't know how you feel about this, Brian, but I think it's pretty true. There's two engineers talking. It says, oh, no, I broke production.

Will I get fired? The Morrissey developer looks over. If we fire engineers who break production, we will need to fire everyone eventually. Yeah. Yeah. Also, if you fix it, you'll get a promotion or a raise or something. So there you go. Yeah. Yeah. Also, if you fix it, you'll get a promotion or a raise or something. There you go. Yeah. Maybe you just need to keep your fix rate above your destroy rate.

Yeah. Yeah, there you go. Well, that's what I brought for us today. Cool. I like it. Yeah. Excellent. Okay, well, nice to kick off the week with a little Python Bytes. Happy to do it early. Yeah, good to see you.

And thanks to everyone who joined. Bye.