Python Bytes - #28 The meaning of _ in Python

Episode Date: June 2, 2017

Topics covered in this episode: [more] pep8.org: PEP 8 — the Style Guide for Python Code Tokio: Asyncio event loop written in Rust language Python Boilerplate Instagram switching to Python 3 on one branch The Meaning of Underscores in Python The future is looking bright for Python Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/28

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. It's Tuesday, May 30th, 2017, and this is episode 28 of Python Bytes. And I'm Michael Kennedy. And I'm Brian Ocken. Brian, can you believe that we've done 28 episodes already? No, I can't. I was telling somebody about the show recently and they said, oh, we've only done like, oh my gosh, we've done like 27 episodes already. Yeah, we've done over half a year in the history of Python podcasts.

I think over half a year's worth, over 20 episodes, makes it like one of the longest running podcasts of all time, actually. All right. A lot of the other ones, they sort of cut her out early 20s. Yeah, things like that. Oh, yeah, that's true. Yeah. I mean, TalkPython and PodcastInit, they've been going for a while, but still.

One of the primary rules that, or documents or enhancements that tells us how we should write our code is PEP8, right? Yeah, definitely everybody that's really into Python knows about PEP8. But sometimes when you mention, hey, you should follow PEP8 to somebody new, it's not really that easy to get them up to speed. You know, it's not really that easy to get them up to speed. You know, it's not that hard to read. But I didn't know about this thing called pep8.org. And so I just stumbled across this recently. And it's from Kenneth Reitz. And it's just an easy to use. It is the just pep8, the style guide. It's a nice web page with styled nicely. It's got red for bad examples and green for good examples.

And it's got a nice index on the left that you can click through. Yeah, it's definitely, yeah, it's a really nice and navigable thing. And you're right, like the green do this, red don't do that. And, you know, even little touches like the code samples that demonstrate good and bad, just basically any of the code samples that demonstrate good and bad just basically any of the code samples that demonstrate anything are color-coded which is like surprisingly delightful yeah and i also like because there's the index there they're all anchors in the web page so you can take one of those and if you want to mention a particular guideline to somebody maybe a co-worker

that's not following it, you can email them that link and say, hey, could you read this? Exactly. Yeah. Put that in your issue response. Like I'm willing to take your PR, but hold on, you got to fix this first. Yeah. And, but that one of the things that I always want to mention when I talk about PEP 8 is don't be a PEP 8 bully. The first line of pep eight talks about hobgoblins or something like that. And yes, basically, blindly following rules is the hobgoblins of little minds or something to this effect. Yeah. Yeah. So somebody might have a decent reason for not

following pep eight. So tread carefully when you try to get somebody to follow it. If you haven't stumbled across PEP8 before, which is kind of hard to think is possible if you're listening to this podcast, it does describe itself as being the style guide for the standard library for Python. But it's a lot more than that. Most open source Python projects follow it to some degree, don't you think? Right, yeah. And you can set up lenders that will mention violations. So, for example, I think Cookie Cutter, if you do a PR back to them and you do something like, say, write an 81 character line, it will fail the automated build and say, we're not going to accept it because it fails Flake 8ake eight or something to this effect. Okay. Yeah. I, you know, I suspect everyone, pretty much everyone's heard of pep eight, but I'm not sure everyone has actually taken the time to read through pep eight. And I think that this thing is more readable. So it's probably, you know,

if you're in that group, then go check it out, pepe.org. Okay. So what's next? I want to talk about asynchronous stuff. Is it surprising I want to talk about asynchronous stuff. Is it surprising? I always talk about asynchronous stuff. So there's this new project called Tokoi, I think is how you might say it, T-O-K-I-O. And at the heart of AsyncIO and the Async and Await stuff are these things called Async event loops and whatnot. And this is one of those. We'd previously spoken about Sanic

and some of these faster web frameworks, and their magic was that they run on top of this thing called UV loop. And UV loop is, I think, implemented in C and is an alternate implementation of this internal async IO loop. Well, Tekoi is a similar concept, but implemented in Rust. So Rust is a compiled low level language such as, you know, like C, I mean, you can debate with this low level, I feel like it is, it's closer to the metal anyway, it's a compiled language. And it's supposed to be a little better for threading a little better for memory management and safety. So it's a little bit of a safer, more modern, low-level way to implement this. And I just thought it was kind of an interesting example that you could go check out and say, like, here's integration of Python

with Rust. Okay. So it's a Python thing? Yeah. So you use it from Python, but instead of having like a C extension, you have a Rust extension. Oh, okay. That makes sense. Took me a while to catch up. Yeah, yeah, yeah. So like the heart of the async IO event loop is like written in C when you're looking at a UV loop. Here it's written in Rust. And so here's like an interesting internal bit. But you basically plug it into your Python code and then you can, you know, work with the async event loop like you normally would. Okay. And I'm guessing it might be pronounced Tokyo, but just guessing. code and then you can you know work with the async event loop like you normally would okay and i'm guessing it might be pronounced tokyo but just guessing tokyo yeah yeah okay tokyo you're probably right you're probably right about that so um yeah cool and the project's still super early stage

development so if you're into rust or you want to play with rust and you would like a, a lot of people ask me like, hey, I want to do something open source. Is there an approachable small project? This thing's pretty early days. And so it's probably not too complicated to jump in and play around with if people are looking for some project to get involved with. That's great. Yeah, absolutely. Tell me about some boilerplate. Well, this is, I've got another project that is fairly early.

I guess I was just assuming it's fairly early in the project. But I stumbled across, stumbled, I'm saying that a lot today. I came across this website called Python Boilerplate. It's python-boilerplate.com. And it's addictively fun, at least right at first. I wouldn't try it on a phone. Actually, I haven't tried it on a phone, but it's got a whole bunch of push buttons and stuff, and you can say which sort of boilerplate you want to

have it spit out. And you can select Python 2 even, or Python 3. Luckily, it does default to Python 3. And you can say whether or not you want an executable, whether it's an executable script or, and whether or not you want arg parse stuff in there, which actually somebody that wants to add arg parse stuff, it's kind of a starter code, a starter skeleton. But it's more like that, but for command line apps, CLI apps, right? Yeah, smaller little projects. And it even puts together a preliminary. Like, for instance, if you add Flask or PyTest, for instance, it'll add your requirements.txt to it. So it's a similar vein as cookie cutter, but a little more interactive and it's fun.

Yeah, I know. I like it. People should check that out. So that one's fun. We recently went to PyCon. It was awesome, right? It was very awesome. Yeah.

Hopefully this year people will take us seriously when we say get your tickets before it sells out. Anyway, one of the things that's really cool that I really liked from PyCon was the things that that's really cool that i i really liked from pycon was the presentation that instagram the folks from instagram did and luckily this is all on youtube so people can check it out so instagram they're probably not the biggest but they're one of the biggest by traffic number of users whatnot consumers or users of Python for their web apps and their backend services. Like they seriously use Python, they use Django in many, many places.

And they had found themselves in a bit of a situation, like many, many of our listeners probably find themselves, but into a degree that is way, way higher, right? So they're, they've got hundreds of millions of users, many, many, who knows how many thousand lines of Python, and they were stuck on Python 2.7 and Django, I think they said 1.3, which is years out of date. It's quite out of date. Like the current one is 1.8.

The one people are working on is 2. The big news about Django 2 is it's only going to support Python 3, right? That was the big news there. And so they said, we somehow need to get onto Python 3, right? A lot of companies say we have this huge bit of code. And so, well, I'll throw my hands up in the air. We just couldn't possibly switch, right? It's just too much work. It's too much risk. If you watch this presentation, there's almost no way you can look at this and go,

oh, there's no way we can switch to Python 3 after you see what they went through. So it's really intense. So for example, they don't believe in branches in their Git repo. Wow, I must have missed that. That's interesting. Yeah, it's interesting yeah it's

interesting right because i think to some degree rightly feel that the farther you are away from master or or yeah from master what they're calling it the the more it's going to be hard to integrate that back in and the more you could introduce trouble by merging that back in so they've what they do is they use feature switches so if you're working on a new feature, there's a config on your dev server that says enable this feature, but in production, it might be turned off. So they're shipping, they said, they check into the master branch quite often, like many times a day. And with an hour of that check in is pushed to production, even when the features are not done, right. So now they have this huge, huge branch of Python code where lots of the libraries and modules and stuff are shared across different services and different servers.

It runs on our old version of Django and Python 2.7. And they're like, all right, what we're going to do is we're going to upgrade to Python 3 and we're not going to branch. We're going to keep shipping on this main branch and convert it to Python 3. We're going to upgrade like years worth of Django. I'm going to keep shipping on Python to production and so on. And so the process that they went through and the little steps that they took and even for you, the testing procedures they went through to make this work was really, really

interesting. So basically I'll give you guys the summary. If you want to watch the video, it's highly recommended. But it's really a concrete roadmap for every large project, every large company, there was lots of users and traffic to switch. So they said they got between 12 and 40% performance improvement, depending on where you look look and like say the web tier versus the async queue tier and so on. And I think like 30% better memory usage. So just like straight across the board, everything was better. And they haven't even started using the async IO stuff. And they're super happy now that they can switch to new version of Django. Something cool

comes out in terms of performance. Say like Python 3.7, some function call becomes much faster. They just get that. They don't have to like go, well, that'd be nice, right? They basically, so they get to be part of the forefront of Python once again, and they're happy about it. So then they like start converting their code base so that it could run on Python 3, but they were still deploying on Python 2.7. Yes. Yes, exactly. could run on python 3 but they're they were still deploying on python 2 7 yes yes exactly and then eventually one of the change one of the steps they made was we're going to make all the developers

run and develop on python 3 but still developed deploy to python 2 and so that was one of the main like checks and balances was the developers will start hitting these bugs locally and will never push it out and then they didn't push it out to production fully then they push it out to just facebook employees and then they push it out to 0.1 percent of their users yeah to see how that worked and then finally they started they switched to like 20 and then like they pushed it all the way out something like that yeah yeah so it was really interesting rollout and on their testing they had like at first there was a bunch of tests that just couldn't be fixed and so they set up a like these are the tests that are included in the automated build and slowly as they migrated

tests they added them to also include this so they would not regress back to be broken yeah and they kind of chopped their way through that and then eventually they said okay the default is new tests all have to pass on Python 3. And the old ones are going to go into this exclude list until we get the failing ones to zero. And I really liked the presentation. And I liked how they talked about actually a lot of the points they went through. And I think the example is a good example for any project. But I also really liked just the reality that it's not trivial.

Yeah. No, it's not trivial. Yeah. No, it wasn't trivial. It was like six months or four months or something huge. But very doable and while in production. That's pretty cool. Yeah, and they also kept shipping new features, right? That was one of their requirements. Right.

So they shipped a bunch of new features at the same time. I bet they used at least one underscore in their code when they did that. Probably several underscores, yeah. We've featured Dan Bader on the show a lot, but this is a pretty cool article. The meaning of underscores in Python. And again... I think this falls into the realm of like what you said with PEP8, right? Like a lot of people are seeing these, but maybe they haven't really looked specifically at what it's about, right? Yeah. And also he goes through pretty much all the different meanings. But one of the things that it took me a while to understand is that when people say dunder, they mean something with two underscores on both ends. Yes.

Shortened of double underscore. So it's a good list. He talks about the leading underscore meaning that it's sort of by convention private for or internal use only but um and i i added a note in art show notes that that doesn't apply to named tuple named tuple has its functions starting with leading underscore but you can still use them that's what they're for yeah nice nice yeah i was just pairing with somebody today and said uh what does class underscore mean? Like, why is the underscore there? Oh, yeah, right. So if that's the trailing underscore, it's because somebody doesn't want to collide with a Python keyword or something.

Yeah, exactly. Like, it's class, so class underscore. Yeah, and I do like, I actually like that convention a lot better than having, like, my class or this class or something like that. Yes, I know that really. And one of the things that a lot of people don't, that I was surprised by when I first

learned about it is a single underscore by itself. Yes. That's my, one of my favorites. That's a, I don't care variable that you can, like, if you have to assign a variable, something to a variable, but you're not going to use it anymore or use it at all. That's a decent one. And the warning, you don't get a warning for that if it's something assigned and never used. Exactly. It's meant to not be used.

Yeah. Yeah. I mean, the couple of places I see that, like a tuple unpacking when you don't care about all the values, it's kind of good. If you say, I just want to do a loop from one to 10, but I don't care about the number. I just want to do it 10 times. You could say four underscore in range, you know, zero, 10, whatever, things like that. That's a good use for that. Yeah. Yeah. The other place that I really like this is where I have function signatures. I have to match, but I don't care about the variables. So for example, in the web methods, a lot of them take request as a parameter. And if you don't put the request parameter there, like the framework will fail to like try to pass it and won't like match and it'll crash and all that. But I don't care about calling

the request variable or working with it. So you'll get like a flake eight violation. You're not using it. So underscore. In that case, can you use it multiple times? Like, can you have multiple parameter names that are underscore? I think so. Yeah. I mean, you just like blast away the previous value stored in underscore, but if you're really using it as like, I don't care, then it's fine. Cool, okay. Yeah, absolutely.

So the underscore is definitely good. If you're not familiar with that, check out Dan's article. It's a good one. So let's close this out with a little bit of a look towards the future. Okay. And we talked about recently the Stack Overflow tools, trends tool.

And when Stack Overflow released it, one of the things that they featured right up in the front was like language popularity for a few languages, including Python. And it looked pretty good, right? Yeah. Yeah. So I want to come back. There's a guy who wrote a Medium post that actually dug into this quite a bit and played around with it and said, the future is looking bright for Python. And basically took that tool and did a little better analysis with it than we did. And there's one chart that they put up as a standard chart at Stack Overflow, since we covered it, called the most popular languages trend chart.

And that's got like 15 languages or something silly. It's really completely overwhelming. But if you hover over the language names, it highlights them and dims the others so you can actually see it. What's really interesting is if you go through that chart and you hover over the languages, there are only like two languages that have upward trends

recently. And which one is the most steeply upward? Has the most positive derivative? Well, probably Python. Of course it does. Actually by quite a large margin. And there's also a trend chart on there for Python 2 versus Python 3.

And last year, Python 3 overtook Python 2. Oh, nice. So just a little, same kind of ideas with this trends, but just digging in a little more detail and it's kind of interesting. I thought it'd be fun to tie that together with the Instagram story. So you can go back to your company, your boss, whatever, and go into live 2020 for Python 2. Django is no longer supporting Python 2. Look, the Python 3 is growing here.

Just like all the good stuff, right? Bring it all together to help get a little more Python versus legacy Python in the world. Yeah. And I definitely don't hear, a couple of years ago, we still heard people coming up with good reasons to stay on 2.7. And I don't hear anybody talking about that anymore. Nope. It's definitely, we've crossed over some sort of boundary for sure.

Well, hopefully also in the future, it'll bring us at least another half year of Python Bytes. I'm pretty sure we can at least get another half year of this. It's fun. That's all of our topics, but any news for us, Michael? No real news for me. I'm just recording podcasts like crazy trying to get ahead for the summer. So I have, like I was just telling you the opening of the show before we hit record, like eight episodes of Talk Python to me I'm recording over the next six, seven days. So there's going to be a whole bunch of good stuff lining up, and I'm excited about what's coming. How about you? I'm kicking off some more recording also. Definitely the book is still going strong and doing a lot of work on that.

But I recorded an interview this morning, and I'll have at least a couple coming out in the next couple weeks. So test and code is not dead. It will come back. What was the topic? What did you record on today? Actually, I recorded somebody that does mobile testing. So mobile application and responsive web page testing.

And it's a company that offers a service that has, like you can like rent a device like so somebody like tells you that that your your website or your your web application or something doesn't work on this particular device you don't have to go out and buy it yeah android oreo whatever right yeah it's on this yeah you could just you can go and rent it for like a, I don't know. I think, I think their prices are like 10 cents a minute or something like that, but you can, yeah, I have no idea what their prices are actually. I can't remember, but, but it's a, it's a commercial product, but I, I want to, I want to make sure that test and code isn't just about open source stuff.

It's also companies that are helping out that area. So it was good. Yeah. It's cool to cover the whole, whole spectrum. And can you imagine what their data center must be like with a thousand different phones running? Yeah. And I, and I also asked him silly things like, okay, so I operate my phone with my finger. Do you have somebody,

are you paying somebody to move their finger or something? And they said, no, they've got that all figured out automated. So. Perfect. Perfect. All right. Well, no, they've got that all figured out, automated. Perfect, perfect. All right, well, Brian, thanks for sharing the news this week and meeting with me. And as always, you know. Yeah, thank you. Talk to you later. You bet.

Bye. Bye, everyone. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

We're always on the lookout for sharing something cool. On behalf of myself and Brian Auchin, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.