Python Bytes - #88 Python has brought computer programming to a vast new audience

Episode Date: July 27, 2018

Topics covered in this episode: Documenting Python Code: A Complete Guide * Security vulnerability alerts for Python at Github* How virtual environment libraries work in Python Learning (not) to Handle Exceptions Python has brought computer programming to a vast new audience Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/88

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 88, recorded July 26, 2018. I'm Michael Kennedy. And I'm Brian Hockett. Brian, how are you doing? I'm doing great. Yeah, it's a ton of good stuff that I think you found this week, and I'm looking forward to share what I got as well. It's never a boring week in the Python world, is it? No, it isn't.

No. Well, it's definitely not boring if your app has problems uh you need to track them down so if you're in that space you might want to check out datadog over at python bytes.fm slash datadog uh tell you more about them later right now i want to know about documenting code we do hear a lot about i think we hear a lot about about making sure that your code documentation is good. There's the read the docs and make the docs effort, or I guess it's write the docs. And there's quite a bit of that. And I think a lot of that is focused on the end user, user documentation. And that's definitely important.

But there was a there's an article on I think it was on real. Yep. Unreal Python called Documenting Python Code, a Complete Guide. It's an article that describes really why you should comment and document your code, talks about the differences between comments and doc strings and different separate documentation. But one of the things that I want to zoom in on is the code comments. And I don't think very many people get really how, if they think about their comments that they put in their code, how effective they can be.

So I want to kind of zoom in on some of the things that they cover in this article on code comments. So code comments are those things that start with a pound sign or hash, and they're not really part of your runtime. You can't get a hold of them at runtime. They're intended for you and other maintainers and other developers of the code. And some of the things that I've used them for, but I never really saw anybody point out before, is before you even start your code, you can use comments to plan what you're going to write. And you can also, once you've written your code, review to make sure the code really matches what you planned to do.

And then sometimes those kind of comments are best to be. If your code is clear enough, just go ahead and delete those when you're done. I like that. I end up doing that a lot myself. I'm like, OK, I'm not entirely sure how to do this. I'll sketch this out in comments like here's the five steps I think I got to do. And then I'll just take them one at a time. And then I always delete those those comments almost always. Yeah, the deleting part is important because if you don't do that, then you get the common

pitfall of a code comment that says, retrieve web page. And then the very next line is one line of code that's a request or something. And it's obvious. So if it's not obvious, using those to explain intent, don't explain really how it's done, but explain why you're doing it. And then sometimes you'll have like a complicated math algorithm or something that's fairly terse if you need to explain that better. But we are trying to get as few comments as possible, minimizing those because you want to make your code pretty clear to begin with. But one of the other things that I think

is I've just started using quite a bit is the to-do feature. So using code comments to the article mentions to-do bug and fix me type sections. Hopefully you're not putting too many bugs and fix me's in your code, but to-dos are, I think those are great for, because when we're in the middle of coding, it's often easy to get caught up in, well, I could make it more powerful if I added these other features. Just go ahead and write that as a comment. You don't really need to fall into the trap of making your code do too much at once.

And then the article also has some other tips on best practices for how to format your code comments and stuff. Of course, it goes on and talks about doc strings and how to utilize those. Also using type hints for commenting and documentation. And then goes on to do the larger, talks about the larger types of project documentation, like using Sphinx and MakeDocs and stuff.

Yeah, it's a really great article, and it's really in-depth. I always learn something from the real Python articles. It's cool. You know, one thing about the to-do stuff that I use a lot and I really like is the editor integration. So I can't speak to the other ones. I don't really know how exactly they handle it. But say in PyCharm, if you have hash to-do colon something, there's a special pane where you can say, show me all the to-dos in this project. And even when you try to commit, like, you know, I want to commit this stuff to GitHub, it'll say, warning, there's three to-dos in here. Are you sure you want to commit to GitHub with these three to-dos?

And then I think it actually even remembers those three. But if you add another one, it'll say, oh, there's like a new to-do you haven't like approved to commit, things like that. So it's pretty sweet. Yeah, I actually, I stumbled across that feature and without knowing it was there and I'm like, oh man, this is great. So I like it. Yeah, it's cool. Even colors the comment different if it's a to do comment so people should definitely do that so we talked a while ago about security vulnerabilities and other types of alerts and we even covered how github has this um like dependency thing

where you can see the dependencies but it it didn't cover Python, which was unfortunate, right? Right. Well, big news, it now supports Python. That was pretty fast. They got on that pretty quick. They did. So it's pretty sweet. If you go to any GitHub repository and you go to Insights,

and then there's a bunch of options on the left, and you pick Dependency Graph, it will actually show you all of the files it's either found in a pip file.lock or in a requirements.txt anywhere in the project. Wow, that's cool. And it'll list them out and it'll say it's pinned to this version, but warning, this version actually has a security vulnerability and is fixed by this other release of it. So if I'm using, say, Pyramid and there happened to have been like a security vulnerability in that, it would let me know and it would like actually highlight that and give me an alert. Okay, neat. Yeah, it's really cool. So it had previously only worked with Ruby and JavaScript,

but now they just announced it. They said for the Ruby and JavaScript people, they've identified millions of vulnerabilities and prompted many patches, which is pretty cool. Okay, I'm just checking my cards project to make sure it's clean. It looks like it's good. Yeah. And it's interesting. You can actually, it's like a tree structure that you get to. So you can go there and it'll say, suppose you're using requests, for example, right?

Then there's like a little Chevron expando thing by requests and it'll open up and say, well, this depends on six and char debt or whatever. It depends upon. I can't remember the exact dependency of request, but you can actually expand it out and see like the transitive closure or like what your dependencies, dependencies, dependencies depend upon. And like, if there's a vulnerability deep, deep down that you didn't directly install, but effectively you installed. So it's really nice. The dependency graph is cool. It even has, it'll even follow a

requirements underscore dev. Yeah, it'll get all of them, yeah. And then you can also look at dependents, so people depending on your project. Interesting. Right, right. So if you've got like a public package, you can see what depends upon it. I wonder if you go to request if that breaks the UI.

Yeah, I'll have to leave that exercise to the reader because i don't want my web browser to fail yeah exactly exactly anyway so we've seen other services like pi up.io which i use and still recommend but this is a nice way to just like walk up to a public github repository and say well what are its dependencies? Is it up to date? Things like that without actually using it or registering it. It's cool. Right. So PyUp, you would have to have it be on your own and you could do that. But this allows you to look at that for any project on GitHub. Right. Exactly. Does maybe this project have a security vulnerability in its dependency or not? If it doesn't, then I'm going to use it. If it does, maybe I'll wait or do a PR or something. Yeah. Nice. Well, speaking of dependencies, I depend on pip a lot and virtual environments. Yes. All the time. I, I've tried to like super optimize what I do on the command

line around pip and virtual environments so much so that I have a single alias that will create a virtual environment, make sure pip and setup tools up to date, and then activate it all in like a couple of characters. I love it. Yeah, nice. So the virtual environments are so useful. I was happy to see this article pop up that's called How Virtual Environment Libraries Work in Python.

And I'm just going to quote it. It says, have you ever wondered what happens when you activate a virtual environment and how it works internally? Here's a quick overview of the internals behind popular virtual environments. For example, virtualenv, virtualenwrapper, conda, and pipenv. And yeah, it's a really fairly quick look at how there's really not much. I didn't know this. So one of the things that it points out is that all of these sort of work on a little bit of

a hack. And these hacks are based on the fact that when the Python interpreter starts, when the interpreter starts, it searches, it's trying to find the site-specific directory to find all the modules it can import and all the packages. And so the search starts with the parent directory of the Python executable location and then continues backtracking up the path into parent directories until it reaches all the way to the root. But it's also looking for a lib directory with os.py in it. Apparently, it's looking for that magic module. And then if it finds that, that's the local site packages directory. So the hack is to let it copy the executable somewhere and then create a directory with that symbolically linked. And then you can have other stuff symbolically linked inside too.

And earlier versions wouldn't symbolically link everything in the site packages if you had everything or a whole bunch of stuff at least. I'm not sure if it's everything. In Python 3.3, it came up with a couple of tricks with a pyvenv config file that allows you to not have to copy the executable completely. You can have a symbolic link to the executable. And all of this is how the sausage is made, but I found it fairly interesting to figure out how all these virtual environment things work. Yeah, I think this is really helpful because certainly being effective with Python is being effective managing your dependencies and stuff,

and the way you do that is virtual environments. So this is a really good look at how that works. Yeah, and to me it did seem like after about 3.3 that things sort of went faster, and now it makes sense that it's faster because they're not copying the executable into a new location so it's a clean look and i'm with you i tried i've tried a whole bunch of off-the-shelf helper things and it's just my own handful of commands that i've come up with that i've just stuck them in little back in my actually just my

bash config file as little functions because they don't do much I just need it to create a virtual environment and then activate it and then like you said update pip and what else do you update set of tools okay yeah it's interesting there's a lot of things that try to help and manage this stuff and they're all pretty good but in reality I just like you say I just want to run these three commands and just get rolling and just keep it simple and i could just create that as an alias in a like a bash rc or a zsh rc and separate the three commands with a double ampersand and then boom it's one command you're good and for people like us that are doing a lot of like trying out a lot of new

things i create a lot of virtual environments. I do agree with Kenneth Reitz and the others that are promoting PIPenv that for, because of all of the hashes and security reasons, for full application development, I think using PIPenv is the right thing and keeps everybody using the project on the same page. But there's plenty of other uses

for raw virtual environment or V and V that I use. Yeah, pretty interesting. One final little tidbit of news that I don't think justifies its own place, but Pip, not too long ago, went from version 9 to version 10, right? Oh, yeah. Yeah, I'm glad you brought that up. That's big. So now in the last week or two, I haven't really been paying attention, but it went

from 10 to 18. Yeah, it completely skipped 10 to 18. Yeah. It completely skipped. A lot of features. But it switched from whatever that normal versioning was. Semantic versioning. Semantic versioning to date-based versioning. Yes.

So I don't understand how the date works. Maybe it's the year, but surely there's more than one release in a year. Who knows? Anyway, it's pretty cool. Maybe it is 2018. But yeah, so there's a brand new version of PIP, a new versioning scheme there. Yeah, cool. Nice. All right, before we get to the next one, let me tell you about Datadog, our sponsor.

So if you have infrastructure and you want to monitor it, you want to trace a request, across your web tier to some microservice series of services down at the database. And put the logs all together that Datadog provides end-to-end visibility for the health and performance of your Python apps. You can explore metrics, distribute request traces and logs, all that in one place. And even have a 14-day free trial. And if you check them out at pythonbytes.fm slash Datadog, set up a trial, you get a sweet, cute Datadog t-shirt.

So that makes it worth it right there. Yeah. So anyway, check them out. It really helps support the show. Brian, have we talked about GUIs? Yes, I think so. Maybe. Maybe. So one of the really nice things about having TkEnter built in to Python is it's really easy. It just comes along, you don't have to do stuff. Previously to use something like Qt, or I think the same may be true for WX Python. But for Qt, it was a serious pain to get that installed. So that you could use it. It's like, well, you run this funky package on Ubuntu, you download this installer for Mac, and then all of a sudden, you're like, here's my simple Python file, that's going to be your app becomes like this weird extra dependency installing problem right so the the qt company the people that make qt the framework and then they've taken over a

piside to the wrapper and they've rebranded it for qt. Well, they've made that much, much easier. So now if I have an application and it has like a requirements file or a pip file.lock or whatever, pip file that says I depend upon PySide2, which is the working package name for Qt for Python right now. If you depend upon that, all you have to do is pip install requirements

and you have it because it ships as a wheel right off Python. Okay. Nice. If you want to use Qt, instead of having like really painful dependencies that people have to set up in like a little step, you want to run our app, well, click here to go to this page to set up Qt. Now you just pip install your requirements. Boom, you have Qt. It's nice. That is nice.

That's a huge thing, actually. I was considering. Basically, I'm like, I'm not using Qt. boom you have qt it's nice that is nice that's a huge thing actually i was considering basically i'm like i'm not using qt if what is required is this separate go find and install qt so you can run my app like that just means there's no way that that's reasonable for distributing that to people but if all you got to do is like package the dependencies and you know presumably you could package it up with like pi installer at at this point, maybe. So really nice. Yeah.

And I want to shout out to all of the people that every time we cover this, contact me and tell me that I'm mispronouncing cute as QT. That this time it was Michael that pronounced it QT instead of cute. So I think I had it right before, didn't I? Oh, well, maybe you've infected me. Yeah, cute for Python. Yeah, it's cool. Well, I don't know. I think that's official. Yeah, yeah, I think I had it right before, didn't I? Oh, well. Maybe you've infected me. No, it's, yeah, cute. Cute for Python. Yeah, it's cool.

Yeah. Well, I don't know. I think that's official. Yeah, yeah, I think so too. And also for all the people that said, yes, you've covered 17 GUI frameworks, but have you heard of these other two? So that's, we're definitely learning a lot about the different frameworks out there.

So this is one more advance there. I don't think people should stop because I still don't, I think this more advance there. I don't think people should stop because I still don't, I think this is great, but I don't think it's completely solved. So people keep trying to solve it with new libraries. Go for it.

Absolutely, absolutely. All right, what's this next one you got about exceptions here, Brian? Okay, well, so exceptions, I think, and I misspelled it, but exceptions are part of Python. And even if they're kind of, I think they're handled sort of later when you're learning about how to program in Python. You sort of deal with telling people how to use exceptions later.

But you're going to have to deal with them because whether or not you write your own exceptions or utilize raise and try accept yourself, other libraries that you utilize are utilizing they're using it so if you don't want your program to crash you got to figure out do all this so this is um not only is it a tutorial on how to use exceptions it's um learning how not to handle exceptions because sometimes the right thing to do is not to catch an exception the right thing to do is to pass catch an exception. The right thing to do is to pass it, either ignore it completely, because it'll pass up to a higher level. And that's part of one of the things that's in this article, an article called Learning Not to Handle Exceptions.

It talks about not only how to use them, but how to not use them correctly. So first it does tell you how to use them correctly, like the try and accept and raise so try and accept are obvious so if you try a bunch of a bit of code and then you can use an accept clause to catch something right off the bat they tell you which is i think correct is usually you don't want to accept everything you want to accept a specific exception that you know how to deal with and then everything that you don't know how to deal with, just let it pass you up the stack. But sometimes even if you catch it, you want to keep raising it.

So you can, it does talk about re-raising an exception with just raise without any keywords out or any exception after it'll just raise the current exception. Yeah, I really like that is like you have a try, do your thing, accept. And then maybe you want to do some kind of compensating action, like roll back a transaction or delete a temporary file. But then you don't really want to handle it.

You just needed to say, if there's a problem, I got to fix this and then let somebody else actually handle it. So then you just say raise. Or you could even use the raise one exception from another exception and chain them. Yeah. That's really nice. And then there's other things like finally and else. So finally can allow you to run some code,

whether or not an exception runs. And then else lets you run some code only if there are no exceptions. And you can use both of them at the same time with the same exception. They both can be utilized. Yeah. The article goes on and covers dealing with tracebacks, custom exceptions, and then some best practices and then making your exceptions more readable by adding arguments and printing things out nicely and stuff. So this is good.

Yeah, it's really nice. Great little article. Definitely a nice find there. So I thought I'd put here at the end of our episode today, something that is kind of takes a step back and just looks at the overall community, the ecosystem, and a bit of an interesting, I don't know, high-level view of Python. So last week, we did our whole special episode on Guido van Rossum stepping down, right, with Brett Cannon, Kara Willing, and sort of cover that whole news. But another interesting thing that happened that is around that is, you know, sort of speaks to Python's popularity is, you know, the magazine, The Economist? Yeah, definitely.

They did a feature in The Economist on Python and Guido van Rossum. That's pretty, pretty mainstream and huge. That's pretty interesting, right? Yeah. So there's an article in The Economist called Python Has Brought Computer Programming to a Vast New Audience, which is pretty awesome. And it covers like the meteoric growth of Python,

how it's becoming so much more popular than many things, how, why, and where people are learning Python. So I thought that was all quite interesting. And then interspersed in there is sort of the history of Python with quotes from Guido himself. So it's pretty cool. Yeah, this is some nice publicity here. Yeah, not too shabby.

And it's not a super long read, but it's a pretty nice treatment. It talks about some interesting things like some really powerful things that people are doing with Python, some interesting ways that people are learning it. But it also says, you know, there are some dangers to Python growing so fast. It's a lot of people are coming into the industry, learning just enough, grabbing something like TensorFlow, you know, some AI framework and sort of understanding it. And then drawing conclusions from what the machine learning algorithms, they sort of understand, tell them, you know, and run off with like half-baked ideas. I thought that was an interesting comment.

Yeah. Well, I mean, yeah, that is an interesting comment. But half understanding what you're talking about and drawing conclusions, we see that every day with like statistics in newspapers and stuff also. Yeah, absolutely. I mean, even in like medicine and medical research, that stuff is rampant. It's a lack of understanding of statistics is a problem across the board. I haven't read the article yet. I'm excited to read it. But the interesting things is people learning to code isn't necessarily because you're going to end up being a professional

programmer. It's often one of the things that helps it grow is a lot of people using it as part of their other job, like a biologist or somebody else that is just using it to help out with some section of their job. Or kids, tinkerers just trying to play with a raspberry pie or something. Yeah, that's a really good point. They actually touch on that a little bit in the article. They say for professionals that have long relied upon trawling through spreadsheets, Python is especially valuable. And they point out that Citigroup, the American bank, has even put together a crash course for Python for its analysts. And the eFinancialCareers, like a website for financial jobs, found that there's like a four-time increase in the request for Python skills in the last three years among financial analysts, not programmers. Oh, nice.

Yeah, so there's a lot of interesting little tidbits of stuff coming out of this article. Yeah, cool. Yeah, definitely worth checking out. All right, well, I think that's it. That's our six items for the week. Brian, you got anything else you want to share with everyone while you're here? No. I've been having my head in antiques so if anybody's uh anybody's a listener in the portland area hit me up and i'll tell you where i'm going to be selling some antiques uh

this sunday so completely non-python job it's kind of nice every now and then to just take a step away put the keyboard down and do something really different so you can come back more refreshed right yeah you got any news no not this week although i'm working on something big that i away, put the keyboard down and do something really different so you can come back more refreshed, right? Yeah. You got any news? No, not this week. Although I'm working on something big that I will announce next week. So I'm close to having news, but I'm not quite there yet.

All right. Nice. I'll bring a little extra news next week. All right. Well, Brian, thanks for doing this and sharing with everyone. And thank you everyone for listening. Thanks. Bye.

Yep. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Auchin, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.