Python Bytes - #146 Slay the dragon, learn the Python

Episode Date: September 8, 2019

Topics covered in this episode: Positional-only arguments in Python django-stubs CodeCombat Four Use Cases for When to Use Celery in a Flask Application pytest-steps docassemble Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/146

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 146, recorded September 4th, 2019. I'm Michael Kennedy. And I'm Brian Eichen. And we have a special guest joining us this time. Welcome back, Trey Hunter. Thank you. It's nice to be here. Yeah, it's great to have you here.

And it's always nice to get a new perspective. And so it'll be fun to see what you're into. We've got some cool Django stuff lined up because of that, I know. But Brian, why don't you start off by telling us why there might be a forward slash in our Python function definition? I sometimes forget that 3.8 isn't out already because I've been using Python 3.8 for a while now. But it's coming soon.

And one of the features for 3.8 is positional only arguments. So positional only arguments are a way we have keyword only arguments that say you have to include the keyword in the argument. We also have normal arguments that you can include the keyword or not. In 3.8, they also add positional only, which means you can't provide the keyword and they have to just be in the correct order, kind of like C functions. So my first thought for this was that the reason for this is probably to, you know, when you're integrating with a DLL or something, it might be just better for integration. But we ran across this argument by a guy named Sanket. It's called positional-only arguments in Python. And it's just a nice little overview of what it is.

But I like the example. So the example, how you specify it is that you specify your normal arguments, and then you put a slash as one of the arguments, like a normal division slash. That isn't one of your variables. It just denotes the difference between where the positional only stop. And the example he gave was a power function. And this is like a math type function or something like that. It makes a lot of sense where the names of the arguments really don't matter.

And the order kind of really does matter because if you're thinking that the power function was x to the y, but it's really y to the x, that would be weird. So specifying them in the correct order is kind of necessary. So anyway, I just wanted to throw this out there because it was a good example of a place where positional only arguments makes a lot of sense. Yeah, it's interesting. When I first look at it, I thought, well, what is the value of this? But there are a couple of interesting things. One, functions that are called positional only without the keywords have some kind of significant speed up. I can't remember if it's in 3.7 or if it's coming in 3.8 or something like that, like 20% quicker.

So there's some kind of performance optimization for this i also heard that it might make other implementations like pi pi and other things easier to have compatibility with some of the lower level stuff in there if you can have these positional only arguments so it's interesting it's a feature coming in three eight i'm curious to see how it gets used right like how many people actually go and use it we've had like you said the keyword only arguments which is the star not, not star args, but star comma arguments, where it says you have to do the keyword argument style. And I don't see that very often, but I kind of like that as a feature for sure. So maybe this one will grow on me. Trey? This is an interesting one, because I'm pretty sure this was discussed when they were planning the feature that this is actually already possible at the C layer in Python.

And in fact, I brought this up while you were talking about palbryan. If you look at help on the built-in pal function Python, Python actually has the slash in the documentation because that slash is already there. It means something. You just can't use it as an actual Python syntax right now. So you already cannot call the pal function that's built in with anything but positional arguments. This just kind of reveals or allows this feature to be used by actual Python programmers outside of that C level. So I mean, as a teacher, I kind of feel like it's yet another thing to teach. But it's also kind of consistency, right? I can now explain the help documentation with an actual Python feature. Yeah, it makes it Yeah, like

brings it up for all the regular developers, not just the core feature. Yeah, it makes it Yeah, like brings it up for all the regular developers, not just the core developers. Okay, interesting. Trey, I promised something about Django. What you got for us? Yeah, Django stubs. So this is type checking in Django. So type annotations, which are, you know, a cool, I'd say new Python three thing, I guess they're not so new anymore, although they keep adding little edges to them. It's a pretty new library, I think. There's a blog post that I saw on this recently. Let me see when it was written. Yeah, it was just last week, I guess. And because of that, I guess it's in beta in the sense that without actual users giving you feedback, it's only going to work for your use case maybe, especially because Django is used so widely.

To be clear, I don't use type annotations in my Django code. But you know, this library might allow me to eventually and I figure more people knowing about it means more users, more people may be fixing little edges in there because there's so much magic kind of, I mean, as much as they say there was a big magic removal in Django, there's just a lot of corners in Django to kind of fix with annotations. Yeah, I think annotations are great. They can definitely overwhelm the code and go crazy. But judicious use here and there, where you're crossing boundaries and stuff like what the heck is that supposed to be? Or even something as simple as like putting a type annotation on a request object. So you can say request dot, how do I get to headers again? And what exactly is headers? Is it a dictionary? Is it a

multi dict is like, what is it right? So those kinds of things I think are super valuable. And I'm happy to see this, right? Well, in the edges between the code, you know, if Django or third party libraries expecting something that what you're sending it works 80% of the time, but there's that one edge case that technically you're not using yet. And no one's testing that thing you don't even know about, you know, it could could break but a type annotation might actually hint at a test you haven't written yet so so are the the django type checking is that going to extend to uh like apis as well oh that's a good question yeah i wonder about that because it says in the article it talks about django rest framework and i didn't even think about the fact that you could probably

piggyback on top of this for some kind of other kind of type checking on top of it but i don't know i think this is kind of more geared toward the um the actual programmer side of things but i'm not certain okay yeah that's a good one i'm sure a lot of django programmers out there will like especially if you're already using type annotations be cool so uh trey you talked about as a teacher earlier and uh I've done a lot of training myself. I also have kids who I would like to have some programming literacy, not necessarily make them into little programmers, but have them be able to use programming like they might in math or something or statistics, right? In some other aspects. So there's this

cool thing I found a while ago, but it's in the news again, called Code Combat. Have either of you heard of this? I don't think so. Yeah. So Code Combat is a place that you can go and it's aimed at teachers and educators. But honestly, anybody who wants to help somebody who's really early stage, get involved in coding with Python, this is a great opportunity. Let me tell you what it is. So it's basically like this Dungeons and Dragons game, or something along those lines. And you go into these different worlds, and you open them up, and each world has maybe 40 or 50 puzzles or challenges you've got across to get through visually, right? So you might go in there,

might be a dungeon, it's got like a maze, and then a door and then an enemy. And you might have to write some code that'll say, you know, take my hero and move them over here, have them go down, then have them pick up this thing, and then find the nearest enemy and attack them and then open the door and whatnot, right? Maybe you do that just imperatively, maybe write a while loop, because like, there's some kind of repeating pattern in the maze they've got to do. But what's really unique about this is it's very visual. It's got like all the gamification. You've won a badge.

You've got a new skill. Like you have to earn the while loop skill. So you get like a little, I don't know, special boots that you run really fast that they call like while loop. I don't remember exactly what it is. But it's really like quick feedback for little kids. What I like about it, though, is unlike many of these things, you write real Python code, there's a hero and you say hero dot move, right, or speed up or whatever it is, you actually type it. Now that sounds bad for like younger kids who are not great at typing or frustrating. But the autocomplete is insane for

this. So if you're in the editor and you want to say hero, find next enemy, you could just type E and hero, find next enemy comes up with the enemy variables. Like it's crazy helpful on how much autocomplete that it'll give you. And there's not that much that you can do. And so it's, it's super interesting. My daughter at the time, I think this was 10, she was all about it. She made me get her subscription and everything. And she just played it probably for a month. Then she kind of got tired of it, but she was really engaged. And I just thought I'd throw that out there. The reason it's back in the news now is they just got $6 million in VC funding to keep up this mission. Oh, nice.

That's awesome. Yeah. So just a free resource. Anyone out there who's a teacher or wants to help someone else learn, or maybe you want to learn yourself, right? Like it's really constrained and simple Python, but it's real Python code you write to like go through the journey. It's pretty cool. Right. And that style of learning really shouldn't be written off because I mean, you're anchoring, you know, some kind of thing in the game that's exciting and fun with the knowledge of that little tidbit of some python that you learned yeah and it lets you work in a like a real way you don't have to go well i know you draggy you did some draggy droppy stuff and that was sort of programming but now it's time to open up a blank text editor and

you have no idea what you're doing like you've been doing that the whole way so it's like it really introduces you to what professional programming is. It doesn't sugarcoat it too much for you, which is what I think is cool about it. Awesome. Yeah, worth checking out. All right, now before we get to the next one, I want to tell you quickly about our sponsor. So this episode is brought to you by DigitalOcean. Now, DigitalOcean has previously had Postgres as a service.

That's something they announced not too long ago. So if you're using Postgres, you can go sign up, say, hey, just take care of my database for me, back it up, make sure it's running, all those kinds of things. Well, now they're adding a few more things that go along with that. They just announced that they have MySQL and Redis hosting as well. So if you want to use Redis for a cache to make things go faster, check a box. You've got your own Redis cluster. Or if you want to use MySQL instead of Postgres, also an option now. So check them out, pythonbytes.fm slash digilution, $50 credit for new users and lots of cool services coming online. Speaking of stuff, services and background things like Redis, Celery is definitely

one that might land in that realm, Brian. I picked this topic, so there was an article by Nick Chanitakis called Four Use Cases for When to Use Celery in a Flask Application. However, I think it's just really any web framework. I'm not sure if this is specific to flask. But I like this sort of, okay, so this tool, and in this case it's Celery. When would you want to use it? I picked this because I'm not sure if he hit the nail on the head or not, and I wanted to pick you and Trey's brains on this. So Celery, people don't know, it's a module that you can run with your web application

that helps you run asynchronous or periodic schedules code in the background. So mostly just extra task multitasking, I guess. Sometimes it's super, it's very much the wrong thing to try to run that code right in the view handler, right? Like if I've got some code that is going to take 15 seconds, you don't want to just have the user sit there and spend for 15 seconds. You want to say, awesome, we're working on it. We kicked it off. Enjoy your day, right? The first example, I'm going to go through the four here. The first example is to send out an email. And this totally makes sense because all the work of actually sending it out, you don't

actually care as long as the central task collected enough information, you can push that onto some background task to actually get the email sent out. The user doesn't have to wait for that, right? That totally makes sense. The other three are connecting to third-party APIs, performing long running tasks, and running tasks on a schedule. And the third-party API one, I don't actually quite get. So I was hoping that one of you could explain if this is a good idea or a bad idea. Any thoughts? Yeah, I use it for all four of these things, actually. I mean, in Python

more so than I use it for all four, because sending emails, that's the thing. Every Monday, 7 a.m. Pacific time, I have to send hundreds of emails. And I actually was just wrestling with Selby the last couple of days to get the email sending down from 10 minutes to 23 seconds, which was only possible because I realized I could make it a little bit more concurrent because you kind of just throw more celery at the issue sometimes. And then third party APIs, if you've got a thing like maybe not a credit card processing, but something that could happen, it could happen in the background.

You just have to go notify some analytics thing or something. The user doesn't need to wait for that. So you can just spin that off in a Celery task on a schedule. Sending emails, in my case, was on a schedule as well. And long-running tasks, sending emails is actually a pretty long-running task for me. It used to be 10 minutes. Is there cases where the user's waiting to see something coming back, but you still want to use Celery? Well, I would say sometimes,

and I'm pretty much with Trey on all of these things. I don't use Celery. I just have a real simple background thread that I kick stuff off to because the work that I'm doing is 10, 20 seconds, whatever. That's good enough to just not have more servers talking to each other where there's more places they could fail.

But when you're talking to a third party API, if somebody comes and says, buys a course on my website, I absolutely want to wait until I get a response, say, yes, that was approved. No, it was denied or whatever. Right. So that one, I would never kick purchasing like a purchasing call or something like that off to a background queue. But what I may well do is, hey, I want to add this person to my mailing list. There's no reason that they don't want a response. I think the thing is, if they do not expect a response in the webpage, it's a totally good candidate for this background work, right? Like, if you say, I can't log in, I need to reset my

email. They don't expect that to happen. You can just say, hey, yeah, great, we emailed you. Go check your email, right? There's always like a five second delay anyway before it makes it through the email pipes. So it's totally easy to kick that to something like Celery, send off that email, but get right back. On my course side, if I put a new office hour event, there's a bunch of people,

like thousands of people have signed up to get an email when I put that in there. At first I tried to just send it out, but then it started timing out because it was taking so long. So now that's a background job and things like that. Okay, cool. I guess it's a great article then if all these are heard. I think the third party one needs a big caveat, right?

Do they expect a response from that third party, like a credit card purchase, or do they not care at all about that response, like they're now on your mailing list, right? Like, do they expect a response from that third party, like a credit card purchase? Or do they not care at all about that response? Like they're now on your mailing list, right? I think that's the dividing line there. Trey, what do you think? I'd agree with that. I've kind of in the same camp in the sense of when I add someone to my mailing list, that API call, there's no, they don't even know about that API call. They don't need to wait for it if that checkbox fails. Whereas charging your credit card, like that's got to be in the request response, in my opinion. Right, that's the primary thing they're trying to do.

It'll make them really nervous if they don't get some kind of feedback from that. Yeah, so you probably should test it, huh, Trey? Yes, you should. The next thing we're going to talk about is PyTest, which is one of Brian's favorite things, obviously. I feel like I stole this one from Brian because it's about PyTest.

So this is PyTest steps is what I'm sharing, which is I didn't really know what it was when I first looked at. Then I kind of scrolled through and saw the code. And personally, I would use this kind of coming from a Django perspective of my functional test. If I have a test where by necessity, I've got a lot of steps in it. You know, I have to go to the login page, log in, click here, fill out another form, do a thing. It's not a unit test. It's this multi-step process. It would be nice to put little checks in there and say, did it pass this far? Or if it didn't, don't show the rest of it as failed, show it as skipped because the failure was really at this part here. We don't even know if the rest

of it failed. And so this is a way to break up one longer test into steps. And there's a whole bunch of different syntaxes for it. I really like the generator syntax that's on this page. The other ones I find a little bit less readable, but you're going to really have to read through the page because it's, there's kind of a lot of things buried in it as far as different ways you can use it. Yeah. I think it's very interesting. Yeah. You're always told, well,

many people tell you that you should have one assert per test. You should be testing one thing. But also, a lot of times, you've kind of got to build it up, right? So this seems like a nice way to have the infrastructure break it apart. And doesn't it do reporting based on what step it's at? I feel like it kind of gives you better information when it fails. Yeah, I think Brian would know more about this. But I think it just integrates with kind of the way PyTest does its reporting.

And like it pretends as if it has run multiple tests in a sense, kind of like how parameterization works, I think. Yeah, it looks like the output just it looks just like a parameterization. So you have the test name is the same, but each step will be like listed in a bracket after the test name. Looks like it is piggybacking off the parameterization stuff. I think it's a nice implementation. I'm going to have to play with it. My fear in working with teams is it is helpful to try to focus in and have a test that really focuses on something.

And workflow tests are a red flag, but we do have workflow tests. Those are real things. And so this is a way to have workflow tests be in place. I would just say probably don't abuse it too much, but in some cases it might be the perfect hammer. Yeah. If you've got to have it, you might as well do it with a nice tool. Yeah. Right. Right. I feel like this is, you know, it's a smaller use case in terms of those very few workflow tests, but it's still useful. Yeah. Yeah. All right. Let me wrap us up with this last main topic here.

And now before I tell you all what it is, let me tell you the scenario which it makes sense to use because it came out of law and you might think, well, I'm not a lawyer, so this is irrelevant to me. And it may be, but it might not be. So the idea is with this thing I'm about to tell you about, if you want to conduct a survey, I think SurveyMonkey or something like that, or you want to create kind of a survey, think SurveyMonkey or something like that. Or you want to create kind

of a self guided interview process, but you want a lot of workflow and control. There's this really creative project called Doc Assemble. And it comes out of the legal space. There's a guy named Jonathan Pyle actually interviewed him for TalkPython on this. And that's coming out in a couple of weeks, something like that, probably about the same time this episode ships, actually. And the idea is he worked for a public defendant type place where it wasn't a big fancy law firm, but they needed to talk to lots of people and do legally valid interviews and gathering up information. So he wrote this open source thing called doc assemble. And it can be used anytime you kind of want to conduct interviews, or surveys that are way better than say,

SurveyMonkey, where you have lots of flow, the way you can create the workflow is wild, you write basic Python code. And then it evaluates the Python, the execution path. So I could say, if user.isUSCitizen, then something else, something else. And it will actually realize, okay, I have to first determine if they're a US citizen. And only if they say yes, do I have to ask them this other question that's inside that if statement.

And the way it does it is really wild. So this thing is packed with all sorts of features. It has since SMS, email does OCR. It has like plugins, it has background tasks, we just talked about celery, it does a bunch of background stuff, I think actually using celery, you define the interviews in YAML and you write the flow and basic questions in Python. And it's just a really interesting way to make like advanced interview workflow survey type things. So people might check that out. It's open source and it looks pretty cool. When I looked at this, I definitely want to try to come up with a reason to use it because it looks, it looks kind of fun. You can even go and there's like a demo, like try it or take a survey or

whatever. And when you do that, there's at the top, there's something that shows the source for each question. It's pretty cool. It'll show you the actual YAML and I think the Python code that's making it go. But it also shows you things like the reading level. There's a bunch of different measures for how hard or complex are these questions and terms. And so you can get it like, oh, this question is too complicated.

How do we rephrase it? There's like just tons of little edge cases that are super well polished like there's a plug-in for word docs so you can write them there instead of in eml all kinds of stuff i'm actually gonna have to use this this is or at least look at it because this is something i've needed and i'm asking like surveys for my python morsels customers if you're using it professionally versus if you're a hobbyist you've got a different set of concerns and different questions. In Python, it'd be like, if user.ishobbyist, colon, and like, that's how you write the workflow. It's really cool, this thing. Awesome. All right, cool. Well, that's our main topics. Do you guys have any extra stuff you want to just throw out there real quick? I don't. All right. Well, good thing I

brought some then. Yeah, actually. So really quickly, we are at 194,740 projects on PyPI, which means we are on the edge of 200,000 packages on PyPI, which is, I think, pretty awesome. So very exciting to have that growing. I remember when I was blown away that there were like 70,000 packages. That's awesome. Yeah, super cool. So I'm linking to a tweet that Raymond Hettinger put out around that.

Also, really quick, NumPy 1.17.0 was released. There's a bunch of mathematical stuff that if you care about, cool new features and improvements, probably too specific to go into. And also, Brian, you talked about Python 3.8. Python 3.8 Beta 4 is now available for those who want to test on the slightly closer to finished version. Yeah, I love 3.8. All right. I've got a joke for you.

We haven't told this one yet, have we, Brian? It seems familiar. Well, let's do it anyway. We're doing it anyway because it was fun to have it go through your office as well. All right. Knock, knock. Who's there? Recursive function.

Recursive function who? Knock, knock. Yeah, how long can we go for this? So it's a good joke, but you only understand it if you know recursion, right? So how do you learn about it? Do you Google it, Brian? Yeah, you can Google recursion and then you'll see a link that says, did you mean recursion? That's awesome.

All right, I don't know who put this next one here, but I really like it. You have to do it though. Okay, yeah, I put it down. So, hey, what's your address? 192.7.7.3 no your local address 127.0.0.1 no your physical address ac5ec yeah i love this yeah that's my mac address yeah yeah yeah so thanks that's a good one all right well those are some good laughs and uh trey and brian thank you both for being here today I love this. Yeah, that's my Mac address, yeah. Yeah, yeah. So, thanks. That's a good one. All right, well, those were some good laughs. And Trey and Brian, thank you both for being here today.

Thanks for having me. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at PythonBytes.fm. If you have a news item you want featured, just visit PythonBtes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Aukin, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.