Python Bytes - #163 Meditations on the Zen of Python

Episode Date: January 9, 2020

Topics covered in this episode: Meditations on the Zen of Python * nginx raided by Russian police* I'm not feeling the async pressure codetiming from Real Python Making Python Programs Blazingly Fast LocalStack Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/163

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 163, recorded live on location here at the Portland West Python Meetup. Hello, everyone. Hello. And it was recorded January 7th, 2020. Brian, it's 2020. Yeah, was that difficult to remember? It's, you know, I'm really not used to it.

I just got used to 2019, so we're having problems, but we'll get there. Yeah. Yeah, well, it's really great to be here with everyone. This is the first time we've done a live recording in a while. We do this at PyCon a lot, but PyCon's not that frequent. So here we are in Portland. It's great. Yeah, thanks.

And thanks for everybody for coming out. This is wonderful. Yeah, absolutely. I've got the first one, right? You've got the first item. So some Zen. I think it's a new year. It's a new decade. It's a new

decade. Let's kick it off with a little Zen. Yeah, a little Zen. So we're going to take 20 minutes and just meditate. Now, the first one I want to talk about, there's probably going to get this name wrong. Why did I pick this? Mosh Zadka. It's a pretty cool name though. I wrote an article called Meditations on the Zen of Python. And if you don't know about the Zen of Python, hopefully you do. We're going to put a link in the show notes, but there's also, in any REPL, you can say import this, and it will show you a whole bunch of little cones,

little snippets of the Zen of Python. One of the cooler things you can do in Python, second only to import anti-gravity. Yeah, that's good. So Moshe says that the Zen of Python is not the rules of Python or the guidelines of Python. It is full of contradictions and illusions. It is not intended to be followed. It is intended to be meditated upon. So you can read these and think about your code. And actually, I kind of like it, like that idea, because when I've read through them, there are some that contradict each other. I want to bring out a few of them that he points out. The first is beautiful is better than ugly. And one of his comments is that consistency helps.

So things like black and flake eight and pilot are very useful in making consistent looking code. Right. But those are only table stakes, right? It's just not ugly. It's not beautiful because of that. Right. So even more important, only humans can judge what humans find beautiful. So code reviews and collaboration and a collaborative approach to writing code are the only realistic way to build a beautiful code. So listening to other people is an important skill in software development and also just looking at code and seeing if you think it looks nice. I think it's good. So a couple more. Complex is better than complicated. And that one always like threw me. I wasn't sure why that was in there.

Moshe says, when solving a hard problem, it's often a case that no simple solution will do. In that case, the most Pythonic strategy is to go from the bottom up, build simple tools, and combine them to solve the problem. That's kind of a Unix style. So is that how you view the complex is better than complicated? I don't know if I've ever thought of it in these terms. And I do like this article because it did make me think like that. But I certainly think about software that way. I feel like so many people get caught trying to overthink it and they're like, oh, I can't get started. I'm trying to like get the right, the exact right thing before I can write my first line of code. And it's like, no, you're never going to know until you work on it for two days.

Then you're like, oh, I should have done this. But well, you didn't know what you knew now. So what are you going to do? Just get started. Yeah. It's my philosophy. And the last one I want to talk about is readability counts. So in the face of immense pressure to throw readability to the side and just solve a problem,

the Zen of Python reminds us that readability counts. Writing code so it can be read is a form of compassion for yourself and others. And one of the reasons why I highlighted this is we're going to talk about optimization and speed later on. And speed is great, but making sure that your program is readable and maintainable is very important. I really like this article. Well done, because we've all heard about the Zen of Python. We've probably all typed import this, but it's a little vague and this is not here's what it means this is here's here's me trying to think about it sort of philosophically and you know i've never seen anyone write that way about it before and i thought it was really cool i'd love other people

to like come up with their ideas about it so that'd be cool yeah absolutely so python and the web doesn't usually have like a james bond sort of places getting raided by the police secret service international angle to it but this next item does really yeah it does so um you know what the most popular web server is in the world used to be apache now it's nginx our stuff runs on nginx for example however there was some news a few weeks ago uh the nginx offices in russia were raided by the russian police and some of the core developers were detained and the reason is this is not as interesting as i made it out to be i don't really think but the reason was the guy who created it i have his name here um sisiov he created it and he at the time he was working for rambler.ru and rambler.ru is a like

a google yandex type of company search engine in russia and he worked on this in his spare time and he open sourced engine x and then later turned it into a company well rambler came along and said hey you know what? You worked on that while we were employing you. NGINX is ours. We're taking it over. Meanwhile, NGINX has been bought by F5, an American company, and they own it. And so there's all this intrigue around it. And yeah, so that happened. So a bunch of my friends from Spokane that when Agilent closed down there, went to work for F5. And I'd never even heard of them before.

And then they show up in this news article. It's pretty interesting. Yeah, they're all about the networks, but apparently servers. So I received an update, an email, from Nginx a few days later. And I'll just read it so I get it right. They said, promptly following this event that I just described, we took measures to ensure the security of our master software builds for Nginx, Nginx Plus,

Nginx WAF, and Nginx Unit, all of which are stored on servers outside of Russia. No other products are developed in Russia. F5 remains committed to innovating with Nginx, Nginx Plus, etc., etc., all the products, and we continue to provide the

best-in-class support. You can expect that to come. So, who knows. And we continue to provide the best in class support. You can expect that to come. So who knows where this is going to go? That's pretty interesting. If you use the Nginx, should you worry about this? That was why I brought it up. Because if it's the most popular web server in the world, some folks, their ears are going to perk up and say, wait a minute, what?

I mean, I don't think this is like Russia trying to impose its will on the world. I think there's just a dispute between a Russian company and some Russian folks who created Nginx, but it could have knock-on effects quite far. So yeah, it's pretty interesting. I think just keep an eye on it, really. This episode is brought to you by, well, us. Really?

Yeah, normally it's brought to you by other companies, but this time, you know, we both have interesting things to talk about. And we have a gap. So I just want to tell you about some of our stuff. So if you want to check out the courses that I'm creating or, you know, set up the business stuff, just visit PythonBytes.fm slash biz. And here there's something about testing over at PythonBytes.fm slash PyTest. And people can check that out there as well.

Cool. Did you set up a URL redirect from there? I don't want to say the whole domain name and URL. So yeah, that's yours. It points to Python testing with pytest, which was published in 2017, but I still think is the very best way to get up to speed very quickly on Python.

Yeah, absolutely. And we also have a Patreon that you set up at patreon.com slash pythonbytes. So, I have some thoughts on this next one, but why don't you kick it off? This one's from the creator of Flask, but not the current maintainer of Flask. Oh, that's true. Right. So, I brought this up

because I was curious what your thoughts were. So, the next one is from Armin... Armin Roeniker. Armin Roeniker. Thank you. He wrote an article called, I'm not Feeling the Async Pressure. The idea is like, async is all the rage. We've actually talked about async quite a bit on the podcast. And I think a lot of people are concerned about it.

And it's one of the reasons why it's going in place is because I think there's some pressure of people to leave Python to go to Go or other things. Node.js was an early example of that, yeah. Yeah, but before you go towards async, Armin is warning people to make sure you understand flow control and back pressure. And I had never heard of back pressure, but back pressure is the resistance that opposes the flow of data through a system. Back pressure sounds quite negative, but is here to save your day. If parts of your system are async, you have to make sure the entire flow through the system

doesn't have overflow points. And then Armin goes through an example with a reader and writer, and it seemed like simple code, but it really got hairy really kind of fast. And the example, yeah, got hairier than I expected. And he says, async brings you new foot guns. Async and Await are great ways to encourage writing stuff that will behave catastrophically when overloaded. For you developers of Async libraries,

here's a New Year's resolution for you. Give back pressure and flow control the importance they deserve in documentation and API. And there's just some hidden things within buffers and things like that. Yeah, well, this is a pretty nuanced article and it's pretty interesting. It comes from someone who knows a thing or two

about the network programming, Armin being the original creator of Flask. That said, my reaction to reading it, my reaction was there were a lot of examples of, and here's if you overpressure the system when you write an async system, it will fall down, right? Imagine you only allow 50 database connections and suddenly you get 10,000. My sort of reaction to this was, well, what is the response of the system going to be when it's single threaded and you give it 10,000 connections requests? They're all going to time out except for a call. It's just going to crash.

So is it crashing an async world? There's a crash in a non async world, you know, with enough traffic. Yes. But at the same time, if you can write basically the same code, put async in a way in front of a few bits and you can get 10 or a hundred

times the amount of performance out of a given piece of hardware before it crashes, that seems better to me so i mean i'm sympathetic to the the problem but at the same time it's always like well if we give it way too much pressure it's going to crash well if it wasn't parallel it would crash before then my thoughts were like can you can are there ways to throttle because i don't know enough about all the way to do network stuff so if if I'm setting up a web service, for instance, can I throttle the input to say to not allow 10,000 connections

and just allow 5,000 or something? Right, right, potentially. So maybe with something like UVA corn or something, you could set that up. I honestly don't know. It seems to me the danger that he's addressing is when the system itself is generating the input. Like we had this example of a guy who sent us a message and said, hey, I had this web

scraping thing. It was really slow. We turned async loose on it and it crashed the server because it ran out of memory because it processed it too quickly, right? Like there you need to find a way to slow it down. But when you're running a web server, you don't control how many people want to get to it. There's just people want to get to it and they either can or they can't.

And with async, more of them would be able to than otherwise. That's my thoughts. Okay, and I guess that my thoughts would be if you're going to throw async at a problem, make sure that you do capacity testing on it as well. Yeah, well, it's going to fail somewhere else, right? And so maybe your database isn't set up for it. Maybe your microservices can't handle all the traffic. Like there's going to be something, right? It's just going to show up somewhere else. But in general, I think you're going to get better

scalability with it than without it. So, you know, if you're not generating that pressure, if you're not generating the traffic, then I don't know what choice you have. Okay. But that's my thought. It was an interesting read. Yeah, it definitely was. It definitely was. Let's go for something a lot simpler than like a deep thing on streaming and buffers and async.

How about a new way to time your code? Yeah, that sounds good. Yeah. So this one comes to us from Doug Farrell, who works on the RealPython team. And as part of their work, they've got to time all sorts of things for their articles. You know, Dan Bader and crew over there. And so they came up with this thing. Either they came out of real

Python, I think, or possibly they were just using it a lot, but this thing called code timing. So if you've got some code and you want to know how long it takes to run, you could say, you know, create a time and like capture the daytime, do some work, capture dot now again, and do a Delta, or you could even use performance counters and other things but you can use this library you just there's a bunch of different things you can do you create a timer class you can call start do some work stop and out comes the time or you can put it in a context manager like a with block as soon as it goes through the with block when it's done that bit is timed or you can use it as a decorator and you can also wire up a logger which is kind of cool

so you can see like it'll just output standard python logging with time information of when it's doing bits of its thing give it a name and it'll tell you how long it took cool well they should add statistics too so i can get min max and average and standard deviation yeah that would actually be cool well it's you know open source i bet they accept prs yeah actually there's a bunch of features i want to add to it. And I started messing around with it. And I'm like, put it down. I have other things to do. I'm already too busy.

I don't need this. I'm going to leave it alone. But yeah, it's a pretty cool little timer class. And I'm going to probably use that. I like it. Yeah. I thought this nice follow on for this, the timer would be an article called making Python programs blazingly fast.

That sounds good. We all want that. You need to time stuff. You should never, I mean, hopefully we've all learned that premature optimization is one of the most horrible things you can do as a programmer. What I've learned is it's incredibly hard to guess where something is slow. Even if you know this takes a second, you look at the code, I bet it's there.

No, no, no. That's like 5%. It's over here. Yeah, because you're throwing the first version of the rough draft of your code down, and you write something down, and you go, I know I can do this better, but I'm going to just make it work here. And you know you're going to have to optimize that, and it turns out to not be the slow part.

Right, exactly. Yeah, so you need to find out where the slow part is. So this article called Making Python Programs Blazingly Fast by Martin Hines goes through a few things. He doesn't cover this timer, but there's a few other ways you can time it. You can use the command line time function to just time how long something runs. That might work. You might just go, I made a change.

It was five seconds. Is it more or less? Yeah, exactly. That'll tell you. Python-mcprofile can tell you a whole bunch about your program. Do you see profile much? I don't really. Yeah, I've used it some. It's pretty awesome, actually. Yeah. And then he goes through an example of writing a wrapper function for a timer, which is similar to what this last article was. It's

one facet of CodeTimer. One of the things that it doesn't cover is the time it function within that's built into Python, which allows you to just run a single function a whole bunch of times and then tells you how long that takes. Yeah, then you get your average and your deviation and all that. But then the article goes through how to make things faster.

So once you've found the slow parts, how do you make it faster? And these are some good suggestions. Use built-in types over custom types. Use caching and memoization through LRU cache, which is a built-in thing into Python. Local variables and local aliases when looping. This is something I didn't expect. This is something like if you've got a multiple dot dot dot dot something even to

a function call creating a local copy of that makes things run faster every traversal that dot is expensive in python whereas like c++ not so much yeah especially if it's in a loop so i use functions i don't understand why this was there kind of duh but you know well apparently the variable lookup in a function scope is faster than a global variable lookup or something like that that he was talking about so by forcing all the variables into the function scope they actually come out faster so there's all these like little weird edge cases yeah i don't have any code that's not in a function so don't repeatedly access attributes in loops okay there's a similar sort of thing one of the things i didn't realize is the f strings have been tuned to be very fast so if you're doing

string formatting use f strings over other things how many of you out there are using f strings these days right on like that's five ten times faster i don't know there's a thing by reman reman hendringer that's mentioned in that article yeah yeah so it's way way faster than the other ways that it was awesome. And then use generators because I added at least experiment with generators because generators are really about saving memory. But if you're really dealing with some large data, memory saving could result in speed up.

So I would say throw those in and see if it's faster. As soon as you start hitting that page file, you're done. I love generators. I throw them everywhere. I do too. Anyway, I think this was an interesting article on speeding up Python. And I warn people against premature optimization though. But it's fun.

Perfect. Yeah, this is a really good one. I like it. And it's a good follow on to the other ones we have. Brian, yeah, you're here. So you spoke about CDK, the cloud development kit from AWS. One of the big gives I have with working with the cloud is I work from home. I want to go work in a coffee shop.

Maybe I'm traveling. I want to work from the hotel and the internet's bad. I still want to be able to work on my code and know the internet is not available. Whoops. I guess my app won't run anymore, right? Well, that is a problem, which I mostly solved by avoiding the cloud directly. But there's another cool project called Local Stack. Talked about Modo before, which is a way to mock out AWS.

And this is actually built on Modo, but it actually does quite a bit more. This comes to us from Graham Williamson and Jan Gazda. So thank you both for sending this in. And basically what it is, is it's like a local AWS. Not just a little bit, like a lot of local AWS. It has S3, it has SES for simple email, it has EC2, it has DynamoDB, it has Lambda, it has Elasticsearch, all of that stuff locally. He showed us like tons of a huge list, though. Does it have all that stuff?

It has a bunch. I don't know how many it has, but I would say it's somewhere on the order of, I don't know, 20, 25 different services. Probably the most common ones. Yeah, probably the most common ones. And then apparently there's also some kind of pro thing I've not used, but then you get more services if you buy the pro version. But the lesser one, I guess, is open source, which is pretty cool.

That's neat. That's great. Like if you're on an airplane or something. Yeah. Or you just, you want to have like a little local dev environment. You don't have to pay for that, even though it's less than pennies,

but yeah, it depends what you're doing, I guess. But yeah. So yeah, it basically, it brings together some of these tools that bring together moto. It brings together this thing called Dynalite and put sort of a unifying

layer on top of it. So it's pretty cool. A lot of it runs in Docker. It helps to kind of get a repeatable experience there. That sounds neat. Yeah, absolutely. All right, well, that's it for our main items, everyone. Got any extras you want to share with folks?

Well, I just, I don't know if we've covered this before. I saw an advert for the Python job board on python.org. Yeah, yeah, I saw that. I hadn't seen it before, but yeah, apparently there's now... Yeah. We're joking around. We're laughing because the internet is not quite cooperating. That's fine. We don't need it. Who needs the internet? What did I say about the cloud? What did I just say, Ryan? You better hope you're not trying to test something right now. Anyway, carry on. Yeah. So the job board is cool, right? I hadn't noticed it either, but it's on python.org. I don't know if you have to pay for stuff, but you can just list jobs, so that's cool. Yeah, and Python's in demand. People want to have jobs writing code in Python, right?

Do you have any extras for us? I do have a couple. Let me pull these up here for the audience as well. So I have pictures because some of these are very, very graphical. So there's this really cool one. You've probably heard of the guy who created Python. So he was interviewed. He's Dutch. He was interviewed by this Dutch newspaper about programming.

And the title, my Dutch is a little off, but it's like Python is half my life. Right? So I worked on Python for half my life or something like this. And they said, this is like a developer thing. Let's put some code and show some Python. Yeah. what code do we see there i don't know is it javascript document dot get element by id yeah not so much not so much that was a pretty interesting little thing that actually happened the uh next one that's pretty interesting i don't have a picture for it's it would just be like a bar that's rusting or something but no it's pretty cool though microsoft you know they're all about c and c plus plus right

like windows is based on c and c plus plus they are actually been doing experimentations with rust and they're coming out with a rust based programming language for like rewriting things like windows and rust that's a pretty big jump for. And the reason is Rust is especially good at memory management and memory ownership. So things like buffer overflow vulnerabilities and stuff just go away in Rust, which is like, you know, every first Tuesday, here's the seven buffer overflows that are going to like lose all your data if you don't patch by the next two days that you get. And like, they're trying to avoid that, I'm guessing. So that sounds interesting. Do you know Rust yet? I've looked at Rust. It looked like C.

I said, I'm going to go back to Python. I mean, not exactly, but it looked like C-ish. Maybe I should take a look at it. Yeah, it's pretty interesting. Two more quick things. So I'm doing a webcast of Python for the.NET developer, kind of interactive one-hour thing at Crowdcast on the Crowdcast platform. I think that'll be fun.

So links in there. It's free to sign up. People can check that out. And then Reuven Lerner was talking to him today, and he has a new free course that he released called Ace Python Interviews. So people out there looking for a Python job, here's like 50 little exercises and questions answered live and live coding responses to 50 interview questions that are explained.

Ruben's a really cool guy. So I think that'd be cool to look at. Yeah, yeah, absolutely. It looks really good. And that was also free. So no harm,

no foul there. People want to check that out. And I've got a job opening. So if anybody's looking, I'm mostly last time I interviewed was for Python person. So I'm probably just going to take some of these things and convert them to C++. So if you want to pass Brian's interview, it may be a good idea to take this course.

Don't tell my boss. All right, so are we ready for a joke? We always end our podcasts with a joke. This one's very visual, so I'm going to put this up on the screen for you as well. And this is really like a sort of infographic. I'm a fan of infographics. And this one helps you understand

the different types of jobs in software development, which could be very tricky, right? Like what is a difference between a lead developer and a full stack developer and a coder? Well, here we go. So it says there's a person and pretty much it's the same looking person for every job description. And it says there's a coder. There's a little caption caption says he writes software engineer

he writes code lead developer he writes code devops well he writes code infrastructure is code right uh data engineer actually what does he do he writes code he writes code full stack developer he writes code alone computer programmer he writes code too s i mean he writes um this is actually a guy eating donuts with a big beard and looking very disheveled he says he writes uh in fact we don't really know what he does all right well that's the joke for this joke for today and i guess that's the podcast for today as well so thanks a lot yeah thanks a bunch and thank you everyone bye thank you for listening to python. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at pythonbytes.fm. If you have a

news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Ocken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.