Python Bytes - #123 Time to right the py-wrongs

Episode Date: March 29, 2019

Topics covered in this episode: [play:0:34] Deconstructing xkcd.com/1987/ [play:3:12] Python package as a CLI option [play:10:29] Refactoring Python Applications for Simplicity [play:14:15] FastAPI [play:17:48] Bleach: stepping down as maintainer Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/123

Hello and welcome to Python Bites, where we deliver Python news and headlines directly to your earbuds. This is episode 123, recorded March 27th, 2019. I'm Michael Kennedy. And I'm Brian Ocken. This episode is brought to you by Datadog. They're a big sponsor of the show and longtime supporter. Tell you more about them later. Brian, do you think it's a pretty cool episode number?

I mean, often people say like episode 100, 500, 1,000. These are big, but one, two, three, that's pretty cool too. I was just going to comment on it. One, two, three is really cool. I like that number. So I think we should start, maybe count it down. Three, two, one, XKCD, go. Yeah, okay, sure.

And the intent is to say like, oh my God, Python packaging and deployment and version management is a mess. Like the subtitle is, my Python environment has become so degraded. My laptop has been declared a super fun site. Right. So that's kind of the theme of it. Yeah. But I mean, it is interesting that it's that title doesn't really say that the, the environment is terrible, but his particular laptop environment has a whole bunch of stuff on it. And that's kind of where there's also now Brett Cannon wrote an article called Deconstructing XKCD 1987, where he goes through all of the different pieces. So if you're looking at this, a lot of people might, especially if they're new

to Python, not even know what some of these things are. So if you're not familiar with Anaconda or Homebrew or other things or why this comes in. So Brett zooms in on all the pieces and just talks about all the different environments. And he's honest to say, yes, some people's laptops kind of looks like this because if you were trying out different things, you're like, oh, I want to try Homebrew, how to Homebrew install and or this other stuff you can go down this route and do all these crazy different ways to install python and have it in your environment but you don't have to any one of these would be sufficient and you can delete them when you're done the you don't need you shouldn't have just the system python you should have system

python and the thing that you're using at the very least yeah you know i feel like this is a little bit like saying gui desktop paradigm suck because like look at my desktop it's a cluttered absolute mess of like stuff piled on here and that's not necessarily the problem of your os it's also one of the benefits is that you can have multiple python versions there's like you're not going to say i've got like seven versions of chrome on my desktop no you can only have one i think you can only have one version installed python is something you can have multiple different versions installed and it's that's one of the benefits especially for developers to be able to test against multiple versions on one machine.

But it does kind of get out of hand. He also, you know, so he describes what Homebrew is and what Anaconda is and all that. But I think actually it's kind of a fun article because it's a little bit of an educational history lesson into all of this stuff. It looks super interesting. I love it. Well done, Brett, and I'm glad you picked it. I'm definitely going to go through it. I just saw it here on the list last minute so i haven't read it yet but it looks really good yeah nice and so uh let me take you on a journey with this next one brian okay so

have i talked about google lighthouse on the show yeah i think so it sounds neat yeah like about speeding up perceived page speed and most importantly what search engines think your page speed is because google uses your page speed somewhat as a index on like how you rank so going down that path i did all sorts of work to make our sites much much faster pythonbytes.fm a training site all these things so that was great but what i realized is when i restart like i deploy a new version of the python app the very first time you hit a page that hasn't been loaded by that like worker process it has to parse up the templates and do a little bit of work and so maybe the first time you hit like a landing page that might not get hit that often maybe it's uh i don't know it takes half a second

70 750 milliseconds or something to load after that it's like okay 20 50 milliseconds it's, I don't know, it takes half a second, 750 milliseconds or something to load. After that, it's like 20, 50 milliseconds. It's super fast. But the first time you hit it, it doesn't. So I'm like, all right, well, what if I restart and I've done all this work? And then that's when the search indexing happens and everything still appears slow. So what can I do to make it faster? Well, why don't I just request every page and do that in parallel as many worker processes I have?

So it has to hit all the worker processes. It's not perfect, but it's a decent heuristic, right? So it works. You run it one time, and every request is kind of slow. The next time, it's incredibly fast. So I wrote this little app, and I'm like, well, I don't want to just type Python, my little script against the thing. What you do is you point it at a site map of any site, and it'll figure out what the URLs are,

and it'll warm them all up, basically. So I linked to this little GitHub thing. But what I wanted to do is I wanted to type, I called it warm up or wake up or something like that. So I wanted to just go to my website and say, wake up pythonbytes.fm, and that was it. And make that part of my CI, CD deployment pipeline. And I'm like, well, wait a minute.

How do I make a... I know there are things like pserv from Python and cookie cutter and pytest and all these things that I can run, but how do I make that in Python? I just had never thought about that. And I think even you had sent me a message. And so the idea is that if you create an entry point

in your setup for your package, those will become executable commands on your OS, right? Yeah. So that's pretty cool. So what I'm going to do is I'm going to link to a thing called Python package as a CLI option. And this just talks you through all the steps. I've set it up so that basically you create a package. In the package, you set up the entry points.

It talks about how to structure those. And then if you just pip install your package, then all of a sudden these commands are available to you globally as if they were like full-on XKubicles in the OS. Yeah, nice. That's actually also one of those things that, like, I'm glad you bring it up because it's not obvious how to do it.

But once you know how to do it, you're like, oh, yeah, that's how you do it. And then you don't to do it you're like oh yeah that's how you do it and then you don't think about it anymore exactly it's super easy everyone's like oh this is what you do i'm like i've just never even thought about that yeah it's not super obvious uh because you know in other environments the way you do that is you compile it to an executable binary and you put that in your path that's not how it works in python no yeah you have to have like this magic entry points thing for setup tools and then then the other ones have different versions. Yeah, exactly. So anyway, I linked to that little sitemap one thing that drove me down this path.

And then also to the article that talks about this, how do you do this? And it's not super new, but I think it's certainly relevant to people out there. And I think I was inspired to do this because I wanted to have it as part of pipx so you can pip x install your thing it says here's the executables that you got when you pip x install this thing i'm like i want mine to have one of those come on yeah well i'm glad you were actually i'm going to use your app because we've got a a new visualization tool that we're using at work and uh at one of the demos, we demoed it, and it was super slow. I was like, oh my God, what happened? Well, the intern just edited all the code

right before we did the demo. Yeah, exactly. And this will fix it. If you have a site map, it'll just go request every page, you tell it how many worker processes, it'll do it all in parallel with async and await. It's beautiful. It's worth mentioning, I guess,

that that little thing is not on PyPI, at least not yet, unless people can convince me to do it, because I'm happy to build it for myself and it's on GitHub and you can pip install it from the GitHub link, you know, with the GitHub option. But I haven't decided,

I haven't committed to owning that puppy as an open source thing. So I just, I don't want to put it necessarily on the pip just yet. I'm not sure. And that's that. Okay, cool. Well, next I want to talk about necessarily on the pip just yet. I'm not sure. Okay, cool. Well, next I want to talk about, we've talked about type checking and what do we call this stuff?

Type annotations, type hints, yeah. Yeah, yeah. We've talked about MyPi several times. And I actually think, I don't even feel bad for not remembering who specifically, because I think we got this suggestion from lots of people. There's another tool called Pyright, P-Y-r-i-g-h-t and it's a microsoft it's from the microsoft repo and it's a the microsoft static type checker for python and it's got a little bit of an interesting twist on it it's written in typescript what Is TypeScript a JavaScript thing? That's pretty meta. So TypeScript is basically like the idea of adding type hints or annotations of Python. Like TypeScript

is that to JavaScript, but to like the nth degree. So like TypeScript adds concrete typing and static typing and whatnot to JavaScript. And then it's its own language and then it compiles to native JavaScript basically. So, yeah, it's kind of meta that that's the way they did it. Okay, so it's written in TypeScript, and it runs within Node. And they're very open about it. It's intended for large codebases. So they're hinting at this isn't something for just everybody to just use on your small little project or your small open source project.

Probably means it's a pain to set up. Maybe, maybe. I don't know, I've never tried it. Once you have it set up, it's supposedly five times faster than MyPy. For a lot of people, and it has a watch feature, it can watch code bases, large code bases.

So that's for people with a big code base with a lot of people getting involved in it, that might be a really great idea then. Yeah, this is like a super idea. I think typefins are a good idea if not over applied. And this definitely seems useful. One of their comments, I just wanted, they're not slamming MyPy at all, but they say PyRite was created to address gaps in the existing Python type checkers like MyPy. Okay, that's cool. Yeah, it looks really great. It has a lot of stars on GitHub already. So happy to see them putting that out there.

All right, before we get to the next one, let me tell everyone about Datadog. So this episode, as many of them are, is sponsored by Datadog. And they're a monitoring and analytics service. And they bring all of your metrics and logs and distributed traces together in one place. They're client auto instruments,

things like AsyncIO for Async and Away and popular frameworks like Django and Tornado to help you visualize performance. So you can trace all your requests across service boundaries, identify bottlenecks. You've got a bunch of microservices. Like how do you correlate these into one sort of call stack performance statement? It'll do that for you. So pretty awesome stuff. Check them out at pythonbytes.fm slash datadog. Nice. Thanks, datadog. Thank you, datadog.

Now, it wouldn't be a show if we didn't talk about Anthony Shaw, would it? Oh, is this Anthony Shaw? And a real Python thing as well. So this next one that I want to talk about is something I'm super passionate about, and so I'm glad Anthony wrote about it and Dan Bader had it on

real Python, is refactoring especially for simplicity so he wrote an article that i think people who maybe haven't thought too much about this lately should check out called refactoring python applications for simplicity so pretty cool uh a lot of it turns out to be about answering the question of, is my code complex? That's interesting. Like, I don't know. I've been working. It doesn't seem bad.

Like, where is it bad? That part over there, I don't like to edit. I know that breaks a lot if I touch that. So we just don't mess with that too much. Things like that, right? But in general, like, how do you know? So he talks about different metrics for complexity. So if you haven't thought about that, that's pretty cool.

So like lines of code, super obvious, like it's 10,000 lines of code. I don't know, that can mean something or it could just mean you have a lot of stuff you got to do. But then- Yeah, but 100 is easier, right? Yeah, exactly.

100 is definitely easier in general. So we talked about cyclometric complexity, which is pretty awesome. And as a tester, I think that's a pretty interesting thing to think about as well. So if I've got like four in loop and within that four in loop, I have an if statement, you know, maybe depending on how you've structured it, that might be like three for the cyclometic complexity, because you could write some code that has an

empty list. So you don't iterate over it. So that's one branch of execution. Another one is maybe you are looping through stuff, but none of them hits that if statement or maybe it does, right? So how do you basically execute each path of all the potential conditional logic and like going in or not into loops and stuff like that. So that's cool. If you have a function that has 15 of those things, I don't know what it's doing, but it's wrong. It should not be doing that. It should not be doing that much. You have smaller functions, probably, you know, maybe that's a little bit of a harsh blanket statement, but like there is a It should not be doing that. It should not be doing that much. You have smaller functions, probably. Maybe that's a little bit of a harsh blanket statement, but there is a number where,

I don't know what you're doing, but it's too much. You know what I mean? Where's lines of code? You can't really say that. And there's also a couple other metrics that I'm not going to talk about that go into this thing that's like a globe, like sort of takes more of them into account called a maintainability index.

And he also talks about Wiley, which I think we've covered on the show, which is a tool he created to compute those numbers for your Python application. That's pretty cool. So all of that is to say, is my code complex and where? And then he talks about,

all right, how do we refactor it? What are the tools? We can use PyCharm because it has killer refactoring stuff built in. There's some plugins for things like Vim and stuff or packages you can get. Also VS Code stuff. And then I think the most important part is here are some anti-patterns, like highly nested code, for example.

And here are ways to refactor your way to better code. And I think that's actually the most valuable and actionable part of this article. Do I do this? Yes, I do. That's bad that's bad oh here's the fix let me do that i think that's great yeah actually this is incredible i think this should be like turn into like a chapter in in uh or a couple lessons in all computer science programs because there's a lot of information in here yeah it's super it's super good i mean the the complexity measures is really interesting as well as the anti-patterns and uh yeah i definitely like it and certainly i think it probably would resonate with you as well because it has this this testing angle right like how do you know it's safe to refactor your way out of

anti-patterns well if you have tests you're good yeah with some of the things like wiley and others you can test for this so Yeah. Yeah, super cool. Cool. All right. Well, check that one out if you want a refresher or on refactor or you want to see some of the anti-patterns. Speaking of things to learn and lessons, we had Colin Sullivan suggested that we cover FastAPI. So thanks, Colin. Yeah, and I hadn't heard of it, but it looks cool, doesn't it?

Yeah. My first reaction is, okay, I'll check it out. But it's yet another API generator stuff so that you can create like REST APIs fairly quickly and easily. But it is super cool. And they're building it as fast. So it's fast API, high performance, easy to learn, fast to code, ready for production. And yeah, I'm going to drop in their little sales pitch. It's fast fast fast to code fewer bugs supposedly more intuitive and i just this morning i was i just went through their quick tutorial on it installed it ran something because i was a one i had one question is there

it has both swagger and redock which are ways to document your apis like live you can just go to the web page and go to the docs and see what it is what your api looks like and i'm like that's just automatically there and sure enough yeah did the demo and it's right there you can walk along with the demo and see the hunt through it. And it only took like a couple minutes for me to try this out. And then it's built on top of Starlet, which I hadn't heard of before, which is a project for some of the web parts of it. Yeah, the most important part about Starlet, I think, is that it adds the async and await capabilities and the parallelism as well.

And then Pydantic which is uh for some of the data controlling the data structures and then at the bottom of the just the front page that only takes a few minutes to go through it says oh yeah we also have this tutorial and the tutorial looks like it goes through i think like some of the best practice, crash course of API best practices. And so I'm totally going to go through that. I think I might learn a whole bunch about schemas and a whole bunch of stuff just by trying that. Which I use this verb.

Yeah, this looks super cool. And one of the things I like about it, well, certainly one of the things I like is this async and await capability. You know, there's some talk every now and then you hear these things flare up like, oh, we're switching to Go because it's not fast enough or we're switching to Node.js because I don't know, right? You know, because it was hot and amazing. And they say we have super fast performance for this, like on par with Node.js and on

par with Go. And I think largely they say thanks to starlet and pydantic and it's also thanks to this native async io and uviacorn and all the all the asgi foundations which is super super nice yeah and uviacorn they have me using that with just the introduction demo that's cool and one of the things you can they have you doing is to try out the reload flag, which just means you can just type your code and change it, and it just changes on the fly, and you don't have to restart your application.

Right. Normally, when you run your web map, you type uvicorn or microSD or whatever, and it's going to just load your Python files. Until you restart it, it's not going to reload them. The uvicorn thing you're them. But it's like the UV accordant thing you're talking about is a watch the files. And if there's any change, it'll just automatically restart your process. So you can just type save request, type save request, and it's all good. Yeah. I feel smarter already just going through the little nature. Yeah, that's great. I'm glad you, you pulled that out because that's super cool. The API is really nice as well.

Okay. So I talked about at the beginning, like, I don't know if I want that puppy. I just, you know, there's a lot of folks out there that have probably open source projects and they're just like, ah, there's somebody angry at me on GitHub again. I can't go back there today. So there's a project called Bleach, which is a web server foundation type thing or web framework foundation type thing and i the reason it's called bleach is it will take like link text and stuff like that and make sure that it is safe for html because if you get it from an untrusted source there's all sorts of insanity that like with i don't know unicode escape codes and all sorts of bizarre stuff that

you can put into links to make bad stuff happen on servers. And so the idea of this is like, it's supposed to apply some bleach to this user input, right? And stop the problem. So this guy, Will Congreen, he had been maintaining this project. He picked it up from someone else. And he decided, you know what? I don't want to work on this anymore. I've been working on it for a while and it doesn't bring me joy. So I'm going to step down. And I thought I'd just highlight this because I think it tells an interesting story that probably resonates with a lot of folks said that, look, I picked up maintenance of this project because when I was familiar with it, the current maintainer wanted to step down. I guess he worked for Mozilla and Mozilla was using it on a bunch of sites and

he felt an obligation to make sure it didn't just drop to like nowhere. And he knew that he could do it. He didn't really like working on it because it's just you know really tedious to like sort of fight all these weird escape codes and stuff and um he did a bunch of work he didn't like using it but he felt obligated just uh make sure it kept going so he said um is he getting paid to work on it no does he like working on it no seems like he shouldn't be doing it so it's just basically he's stepping down but i thought it was just like an kind of an interesting journal entry of like that side of open source. Yeah. Yeah.

So, you know, people are out there. They can read this and maybe it'll resonate. Maybe it'll help them stay on the project. No, no, actually, I don't feel like this. You know, actually, I am getting joy from this or whatever. Or maybe they are. Is there somebody else taking it over? There's somebody else who's working on it.

I think who may be taking it over or do you know uh there's somebody else who's working on it i think who may be taking it over last line of the article said something to the effect of what happens to bleach i'm stepping down without working on what comes next i think greg is going to figure that out i'm afraid i don't know who greg is but he's one of the people working on so it's kind of like someone else is gonna have good luck greg yeah yeah good luck so anyway i uh not a super positive story but also i just thought it would be like kind of interesting to share because uh it's a interesting look into like the sort of life cycle of maintainers of open source projects yeah interesting nice yeah so uh brian you got any extras for us this week what things you want to start real quick so something that came up that I thought was funny, Tim Hopper sent this

out, and it's called Sleep Sort. And he found another implementation of Sleep Sort, and then he implemented it as his in Python. But I think it's just hilarious. So the idea is, can you make a sort algorithm by just

sleeping for the period of time what the number is, and then printing out the number when you're done. So if you're on it, just sort numbers. Oh, that's awesome. So if I have like one, seven and three, and I want to print one, three, seven, I just go to all of them and sleep as long as they are and then print them out. Yeah. With async, you can just like line them all up and sleep for the amount of time that it says, and then they'll all be sorted because that's time sorting. Time sorts for you.

How interesting. Yeah, I guess it does. I don't think it's useful, but it's interesting. Anyway. It's an interesting thought experiment. And, you know, if you're in college and you're in one of these algorithm courses

and they want to talk about quick sort, bubble sort, like here's a little interesting one that people might not see coming. Yeah. I have a couple I want to throw out there really quick. First of all, Python 3.7.3 is out now. So that's pretty cool.

There's a decent number of changes to it. I would say there's some decent number of changes. It's really hard sometimes I find on the change logs for Python to see what the point release changes are versus just the major ones. So you go to where they say what the changes are for Python 3.7. It says, well, these are the new pips in 3.7. Okay, well, what about this particular one? Anyway, so probably people can point me in a better place in the release logs, but that seems like it shouldn't include those.

Anyway, so if you want to stay in the latest official release of Python 3.7, you can go install Python 3.7.3 or however you do that, right? Don't end up like the XKCD at the beginning. Pick one and go with it. Yeah. Another one, this one I didn't really think is worth covering the whole episode, but Alexander Lorry, who is a medical doctor who's learning Python, a guy I know from the podcast and also from courses and stuff. Really, really great guy.

Sent us this thing called Stack RoboFlow. So we all know Stack Overflow. So, Brian, click on Stack RoboFlow and see what you get. It looks like Stack Overflow. So here, let me just read like real quick, summarize what I get. So it looks like Stack Overflow, sort sort of but there's obvious disclaimers and it says subversion branches related to local directory sometimes i need to rename a local file on my svn repository and in remote desktop i do this svn-l that it goes and it talks about it this is written by an ai

oh interesting so it's like an ai that's been trained on Stack Overflow, and it knows how to ask questions and answer questions as if it were a Stack Overflow movie. How funny. Yeah, and they have the code. So if you're into machine learning and stuff, you want to check that out. It's pretty amusing.

Or if you just want to laugh to see how close can an AI get to just random Stack Overflow. But I really also like the, uh, the, the, the logo. Yeah. It's funny. Yeah. It's cute.

All right. Last one really quick. I thought this one just might be useful. I can't remember where I ran across this. I don't think it's written in Python, but it doesn't actually matter. It could be useful for teams writing Python called Passbolt. Have you heard of Passbolt?

No. So Passbolt is a, uh, a password manager, like one password or a lastPass, something like that. But it's for teams, like for software teams and stuff. And it's free, open source. It's self-hosted.

It's based on OpenPGB and stuff like that. So it's like your own private, personal hosted stuff for things like server passwords. And how do I get into this GitHub? And whose password goes to the mail server? like all that kind of stuff meant for teams. So it looks pretty cool for software teams to like keep track of that stuff. That's actually pretty cool. Yeah, it's pretty awesome.

So maybe some folks out there can not put that on sticky notes or Excel or wherever it's right now. Yeah, or in a wiki or something. Yeah, or a wiki. Exactly. Okay. So that wraps it up for our serious topics. How about something not serious? You got a joke for us?

You got a pie joke for us? Yeah, pie joke. Thanks, pie joke. How many programmers does it take to kill a cockroach? I don't know. Two, one to hold it and the other to install Windows on it. It's pretty bad. All right, I got one for you as well.

Thanks, pie joke. Eight bites to walk into a bar. The bartender asked, can I get you anything? it's pretty bad alright I got one for you as well thanks to PyJoke 8bytes walk into a bar the bartender asks can I get you anything he said yeah replies to Bytes make us a double this is not really a joke

but it's just like serious advice from PyJoke friends don't let friends use Python 2.7 so maybe we'll just leave it at that yeah definitely

alright well Brian thanks for being here and finding all these things and sharing them with everyone. Thank you. You bet. Bye.

Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at PythonBytes.fm.

If you have a news item you want featured, just visit PythonBytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Ocken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.