Python Bytes - #121 python2 becomes self-aware, enters fifth stage of grief

Episode Date: March 16, 2019

Topics covered in this episode: [play:0:40] Futurize and Auto-Futurize [play:3:42] Tech blog writing live stream [play:8:50] bullet: Beautiful Python Prompts Made Simple [play:11:15] Hosting private pip packages using Azure Artifacts [play:13:15] Async/await for wxPython Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/121

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 121, recorded March 13th, 2019. I'm Michael Kennedy. And I'm Brian Ocken. And this episode is brought to you by Datadog. Check them out at pythonbytes.fm slash datadog. More on that later. Brian, how you been?

I'm, well, actually I had a rough week last week, but I think I'm getting back to it. Wait, you're getting a fight with a dentist? Yeah, I went to like two dentists and two doctors just to fix one tooth. But yeah, it's all done now. Well, glad to hear you're doing better at least. You know, I would say the future is looking bright. What do you think? Yeah, especially with Python 3 coming everywhere.

Yeah, absolutely. All right. So what's your first item here? Well, the first item is, oh yeah you were doing like a like a whole transition thing so yeah i get it now futurize and auto futurize so futurize is there's a website called python future.org and they have this thing called futurize that you can turn it on in stages so it has a whole bunch of different module or switches and the idea is to automate your conversion from python 2 to Python three.

So like one thing you can do is like a fix the print statement, but nothing else, things like that. Is that the idea? There's a whole bunch of different little modular things you can do. And I don't actually, I didn't dig into it enough to know if you can really, really go really small and granular and only change a few things at once.

But one of the things they do have is a staged conversion. So they have split up a bunch of their stuff into relatively safe fixes, things like changing the exception syntax using print function. So the print function is easy to change, making sure that your classes have an object-based class. And apparently the iterator syntax changed, but apparently I didn't realize that. And a handful of other things

that are really safe to just change. So there's a stage one that just changes all that stuff. And then the stage two, where there's some more risky items possibly, they do more kind of wrap some of the Python 2 style stuff with Python 3 style code. And, you know, of course, all of this would only be for large projects because small projects, just go do it yourself, man.

Anyway, if you're going to try to do all these modular things, it'd be kind of neat if you could test between hand and save all your changes. Timothy Hopper put together something called AutoFuturize that is a shell script that uses git to save all your changes and then talks to run your, is assuming you're using talks to check your code

and it goes and go ahead and does all of these changes to your code and tests it in between and if it does a little bit of a change and your tests pass, it does the rest of it. And then it doesn't check in things that don't work. So this is kind of fun and it'd be fun to try on a project. Yeah, I like the idea of both of these. I can totally envision like some

high paid consultants sitting down to do the Python 2 conversion for the companies that put their head in the sand until January. And then they come in and they just run this for the morning, go have a long lunch, come back in. Everyone will be super happy how quickly they did it. And it'll be great. The machine will do all the conversion. Yeah, there'll be like two files that they have to hand edit to fix. Exactly.

It's going to take a few weeks, but we'll see if we can get it done without too much of a problem. Hey, that's a new business idea. Maybe I should start another side business. It's a little shady, but you know, it's all right. So thanks, Tim, for Christmas money for next year. Yeah, thank you, Tim. And that's a cool, actually, you know, in all seriousness, it's a cool project and I think it's going to help a lot of folks.

Yeah. Speaking of helping folks, Anthony Shaw, he's putting a lot of content out there for everyone. And the reason we talk about him so often is often he's doing some kind of blog post or he's written some kind of cool article. And he went kind of meta recently and did a live stream. I think it was on YouTube of how he goes about writing these posts, how he does research, how he thinks about the different formats, all that kind of stuff. Yeah. So I was just, it's still on my to-do list to watch. So you watched it and it was pretty good? I watched it, yeah, about 20 minutes of it. Didn't have a chance to watch the whole thing,

but it was, it's good. Him just writing a post and you just watch him write it. He goes through and talks about the different posts he's written, some of the trade-offs and how he works and things like that. So he talks about one of his posts called modifying the Python language in six minutes, which is like a deep code focused article. He talks about listicles. And so he talks about five easy coding projects to do with kids. He did, he talks about what is popular and like why those are popular, which is pretty interesting. He says one type of article he likes to write is like question articles. So why is Python slow, for example? And of course, to disprove that or whatever. And then also the tourist, like I'm going to take you on a tour or something.

So tourist guide to the CPython source code. So if you're looking to up your writing game or get inspired, if you sit down, you're like, oh, I've got to write about something. What am I going to do? Well, here's a bunch of ideas like, do I want to write a listicle? Do I want to write a tour? Do you want to write a question? And so it really helps focus the mind and get you going, I think.

Yeah, definitely going to check this out because he's pretty much crushing it. And I'd like to kind of learn from that technique. Yeah, very, very well done. Nice job, Anthony. I feel like you've chosen a very controversial topic for the next one. Not so much, but pretty interesting. Yeah, I guess I just totally forgot that it was controversial. And I know it was like a big thing at the time,

but it's the Walrus operator or PEP572. And I kind of love the Walrus operator because it's the colon with the equal sign, and it's basically an exclamation expression that's coming in Python 3.8. And it's kind of hard to describe on audio, but it's a... So a lot of times if you were to say, I do a lot with dictionaries, or if I'm looking into some database or something,

I want to find out if an element is there. And if it is there, I want the value. And it's always this two-stage thing. And it'd be great. And pattern matching is another one, regular expressions. If there is a match, then search for it and get the stuff. Otherwise, don't do something. And being able to do this as one assignment expression is going to, I think, help out quite a bit.

Anyway, you can play with it now. So that's the whole point of this is Alexander Holtner. Yes. Wrote an article saying that now that the alphas some of the alpha releases for python 3.8 are out you can try these out and he wrote a post demonstrating exactly what i was talking about getting elements out of a dig iterating through dictionaries and grabbing keys out of their elements if they if they exist he used pi e used PyENV to grab the new releases. I don't know why, but I'm just one that I like to just download the stuff. So I've added the links to

the two 3.8 alphas available so far. People want to try it out. Yeah, the one that I really like here is the list expression, which could be a generator or it could be set expression or whatever, but a list comprehension, because those have to be one line. And if you want to assign a variable, and then test it and do other stuff with it, you can't really get really do that. And without the walrus operator, if you're going to like apply like a function to the elements that you're looping over, and then want to use those, you have to basically get access or compute that twice for each element. And the walrus lets you do it all in one.

So these little expressions are kind of like the lambda has the colon and then the body type of syntax. It's a little bit like that kind of in line, and that definitely seems nice to me. Yeah, that's a pretty cool use. I hadn't really seen that before this article. I guess I didn't pay much attention, but using it in a comprehension is pretty cool yeah cool yeah also your little nested for loop if test

it's pretty nice you know i guess i'm coming around to the walrus operator honestly yeah i mean once i start using 3.8 i'll be using it all the time it'll be like be like using f strings i think to say okay well now it's 3.8 and above because I don't want to use anything else. I'm not there yet. Awesome. All right. Well, before we get on to the next one, which I'm pretty thrilled about because it's simple and amazing, I just want to tell you quickly about Datadog. So they're sponsoring this episode.

Of course, Datadog is a monitoring and analytics service that brings all your metrics and logs and distributed traces together. And their client automatically instruments, things like asynchronous libraries, such as async IO and popular frameworks like Django or tornado to help you visualize your performance. If you want to trace your requests across service boundaries and figure out where your app is slow or find errors, you know, go over to pythonbytes.fm slash data dog, got a cool free trial there and you get a nice little Datadog t-shirt too.

So check them out. It helps support the show. Now, to the awesome thing. We talked about Qlik, and we talked recently about argpars and other cool ways to build command line applications, right? Yeah. And there's, of course, the tried and true input, right? You got input, what is your question, and course, the tried and true input, right?

You got input, what is your question? And you set the value and then you maybe have to test it, see whether it's an integer. And then like if there's an error, you say, no, there's an error. You can't say that. You got to say an integer. And there's all these challenges, right? So maybe I've got a list of things I'd like them to pick from, like what kind of, you know, maybe you're building a site and like through a generator says, well, what kind of data backend do you want?

I want Django ORM. I want SQL Alchemy. I want raw SQLite, whatever. You might give them a list and say, well, pick one, two, three, or four, right? Which box do you want? Well, with Bullet, have you seen Bullet? I'm just looking at it right now and it's pretty exciting. It's incredible, right?

So what you get is imagine like a dropdown combo box like you would have on the web that is the list and you can click, but in the command line, as a command line argument. So it says choose something, and you can arrow through it. It has like a little indicator of which one you're on. Oh, man, it is slick. It supports colors and emojis. It has a scroll bar. It has a scroll bar.

You can like even scroll through them if there's too many choices to fit on the screen. Yeah, it's pretty cool and it's easy to use. So that's nice. There's also other types of inputs. Like you can have a password. It says enter your password or what is the password. And it does the star marking, which is cool. It has yes, no questions, numbers.

It'll only, you know, a number input. You can only enter numbers. It won't accept, you know, junk that won't parse. So you don't have to go through all those steps. It's pretty incredible, right? Yeah. So we need somebody to write cookie cutter with bullet prompts. Exactly.

I was thinking of cookie cutter, exactly. Because cookie cutter asks all those types of questions all the time. Yeah. And it would be beautiful to have bullet just, you know, beautifying all these things. So there's one idea. People want to contribute to open source. Another one is right here on bullet. Uh,

you know, there's probably other types of input besides passwords. Yes, no. And numbers like maybe multi-select, I don't know, but also looking for window support. I think right now this does not work on windows because the way the terminal

works versus on, you know, a positive system. So pretty cool, though. Still digging it. Yeah. So go Bullet. Go Bullet.

Yeah. So we actually have a couple items to do with pip and packages and installing. You go first. Okay. Well, last week, I think it was last week, maybe it was the week before, we talked a little bit about maybe you want to try something different than Travis for your pipelines. For your continuous integration? The exodus after all the layoffs and all that business, huh? Yeah.

Well, I mean, or just maybe you just want to try Azure anyway or Azure just because it looks neat. There's an article that I thought was super cool that was hosting private pip packages using Azure artifacts. And the idea of the article is, let's say if I'm using Azure pipelines to do actually a data pipeline, doing a whole bunch of different stages of changing the data or whatever, whatever you do with data pipelines, some of the work can be packaged up

and used in later stages with just like a pip install. That'd be cool. And then how do you do that? There's a few gotchas that they get through. And they, for instance, in one part, you can't just use the normal pipelines just by itself, but you can use the CI pipelines from Azure DevOps tool to get the packages into an artifact form.

And basically, it's all the hacks that you need to do to make this work. I just thought that was cool. Yeah, I think that sounds really cool as well. And go ahead and leverage those artifacts to make your pipelines, which is cool. I don't think you can make these public. They're mostly, you're using it for your own stuff. So they're a little bit private. Like they're talking about using credentials,

hooking up your credentials so that pip install can get those correctly. And then I also wanted, one of the things we did talk about last week was Anthony Shaw's PyTest Azure Pipelines, PyTest plugin. We've already covered that last week, but one of the things that's cool that happened since then is that plugin is now part of the recommended setup from Microsoft itself. Yeah, that's pretty cool. Well done.

So it's definitely going to be sort of the de facto way over there now. Yeah. So I want to talk about something pretty unique. Maybe we haven't touched on this before. It has to do with GUIs. So last week we spoke about mixing in the async IO. I know this was two weeks ago. The async IO pipeline sort of event loop and mixing that in with Qt.

So Qt for Python and PyQt and things like that. So all GUIs have event loops, right? They're always just going around and around going, did something happen? Did they click a button? I'm looking for, you know, like mouse move. I'm looking for key down, resize events. And it just, you know, passes it on to like the various event handlers. Well, async IO is another loop that kind of goes around and around process things, but like, how do we put those together? So a guy named Annie Bolka sent us a

message after hearing the first conversation about what was it?? I think it was AsyncQt, was the name of the project. After hearing about that, he's like, hey, I work on WXPython. And there's a cool thing called WXAsync as well. And it does a really similar thing. And it basically merges those two event loops into one thing, which is pretty cool. So he wrote a really nice in-depth medium article about it called Async and Await for WX Python. Oh, very nice. That's cool. So one of the challenges with WX Python apparently is doing background work, right? So you try to

like call a service or you try to do something and you know, if you don't take extra measures, it's going to block up that event loop that is the gooey thread and it's just like your app's going to freeze and not respond and do all sorts of bad stuff you're not supposed to do in desktop apps so with this you can have async handlers that you know run in the background so if you're like waiting on a call to go to the service you just await that and like the event loop keeps on chugging so it's pretty cool obviously you can do threading and multiprocessing and that kind of stuff. But this is a little bit cleaner if you want to use async and await. Yeah, nice.

The other thing that he talks about in his article is about this sort of viral nature of using async and await. So if I define a function, I say async def something, the function that's going to call it may well have to itself be async so it can await it and then like the things that call that have to become asyncified and so on it creates this kind of like from the bottom up spread of async like invading all of your code which that can be a challenge and it talks a little bit about that but i would just wanted to throw a shout out which to something i think we've spoken about before. If I haven't, we should definitely make it a full featured item later. But Unsync, have I mentioned Unsync on the show?

I don't remember. All right, well, that's probably going to be a new thing. If we haven't, I'll double check. But Unsync is a super simple library that puts a unifying API on multiprocessing, threading, and asynchronous methods. And it manages its own little background queue running somewhere else on another thread. It allows you to do a lot of things like just blocking on those running sub-method calls

to get the result so that you don't actually have the problem. You can sort of use it to stop this async spread of these methods. So anyway, you know, it's an interesting article. It's interesting. It covers that. And I want to throw Unsync out there as one of the cool solutions for it.

Yeah. Cool. That's a good thing to bring up. I kind of didn't think about that. I mean, I'm used to it in C++ world that if you're running multi-threaded, everything's got to be multi-threadable. Yeah. It starts to go crazy.

And it's just like, yeah, similarly. Yeah. Yeah. Anyway yeah similarly yeah yeah anyway okay pretty cool all right well that's it for our main items i have a couple of follow-up ones and i have a set of jokes for you i have a joke generator for you even how's that oh that's great yeah all right you got anything you want to let people know about no so you just released a new uh test and code episode? Yeah, they released an episode talking about a fun alternative or version of TDD that's called TCR Test Commit Revert with the idea of

what happens if every time your tests fail you just throw away all your code that you've changed since the last good commit. And it seems absolutely crazy, but it's a fun thing to talk about. Yeah, it does seem a little crazy, but also yeah, it seems fun. Yeah it's a fun thing to talk about yeah it does seem a little crazy but also uh yeah it seems fun yeah and then there are a whole bunch of cool people lined up i've got like six interviews lined up so there's gonna be lots of good content coming up

soon yeah sweet sweet all right well i have a couple of things a correction some self-aware acknowledgments and then a joke so correction we spoke about the mongodb licensing and the aws kerfuffle where aws said well you're going to change your license so we can't use mongodb it is our own service well fine we'll rewrite a new implementation that has nothing to do with you but is you know wire protocol identical and things like that well mostly i got that right however will s sent me a message said hey actually posted a message on the comment section of that episode. I said, Hey, you said they switched to the AGPL. They actually had already been on the AGPL. And what they switched to doesn't even recognize SSPL as an open source license because the way some of its conditions. And so it's interesting in some sense that like MongoDB might not even be open source anymore based on its license. Not in the traditional what is its license mean sense anyway.

Right. I took the perspective of let's say i've got a service out there i'm just like a to have a side project where i've got a service out there where i just run my own instance of uh mongo and have uh i'm using that do i have to worry about this license change and the answer is no yeah that's what i i saw as well it's like but if you want to be a cloud provider and you want to offer the service to others then you do right. But if the service you're offering isn't Mongo, it's just some other service that happens to use Mongo. Yeah, you're good. Yeah, like businessinsider.com or whatever.

Yeah. Yep. Okay. So MongoDB license, sorry, it's SSPL. That's the change. It was already AGPL. Thank you, Will, for that. So, Brian, do you know that Legacy Python uh self-aware and has actually entered the fifth

stage of grief no so just people out there can try this so if you type python 2 dash m and you run any module so like python 2 dash m pip list like that's a meaningless command you'll get the output that says deprecation python 2 7 will will reach its end of life on January 1st 2020 please upgrade your Python as Python 2.7 won't be maintained after that date moreover my word a future version of pip will drop support

for Python 2.7 oh wow yeah well that makes sense I feel like that's acceptance it's like yeah I'm going away sorry there's a funny Twitter thread as well that I I mean, I feel like that's acceptance, right? It's like, yeah, I'm going away. Sorry. There's a funny Twitter thread as well that I threw out when I first saw that going by. So that's cool. And then finally, we talked about pip and packaging and stuff.

There's this thing that Dan Bader from RealPython shot me over and said, hey, have you seen this? This is kind of cool. Called PyDist. So this is a thing, a service, I guess guess that is in beta and the idea is it you can use it as your package source right for pip and whatnot and you can have public and private dependencies there it'll mirror the public ones and keep a copy of them and never ever delete them even if they were deleted from pip uh pipey i so that you always have them there stable. It's pretty cool. It's supposed to do a bunch of other stuff,

like show you, like if you use this package, here are all the packages that depend upon it and what you're going to get, which is not really available right now and things like that. But it looks like it may be at some point become a paid service. I don't know.

It's in free beta. They don't say one way or the other. So anyway, maybe that's helpful to people out there. Interesting though. Cool. All right. You ready for a joke definitely this is super hilarious all right let's do this pip install dash dash user pyjokes okay so you gotta do it or better i'm super loving pip x so pip x install pyjokes if you got that because that that is the business these days pip x is awesome it's like homebrew, but for Python executables.

Okay, this is going to take two then. All right, let me, I'll do a couple, then you can give me one at the end. So what this is, this is a package that you can install that gives you a command line access to developer jokes. Really? Yeah, okay. So once you install, I'm going to type PyJoke.

There are two ways to write error-free programs. Only the third one works. All right, let's see what else we got. This one, I think you'll like this one. A QA engineer walks into a bar, runs into a bar, crawls into a bar, dances into a bar, tiptoes into a bar, rams a bar, jumps into a bar. Okay. That's it. How many programmers do they take to change a light bulb?

None. They just make darkness a standard. Let's see. A good programmer is someone who always looks both ways before crossing a one-way street. See, these are like once you install PipX and sell pie jokes, these are at your hand. And you're like, I'm feeling a little down. What are we going to do? Why do Java programmers wear sunglasses?

Because they can't see sharp. Huh? So, like, it just doesn't stop apparently well we don't even have to have anybody submit any jokes for us anymore we just use these we have the fountain of the endless fountain of jokes although i'm sure there is some limit how many jokes are in there you can go to their website and actually submit a joke to be included in the app okay which is all good so i have to make sure I update it every once in a while. Yeah, that's why you need PipX because then you have to do PipX update all

and it does it. Okay, cool. Sweet. Anyway, that's our joke. If people can go out there and install PyJokes, the package, and run it. Although they might not need us anymore, Brian. Yeah, they will.

Because they'll need our wonderful transitions from one thing to another. That's right, that's right. All right, well, a lot of fun as always and I'll one thing to another. That's right. That's right. All right. Well, a lot of fun as always. And I'll chat with you later. Okay.

Bye. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at PythonBytes.fm. If you have a news item you want featured, just visit PythonBytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Ocken, this is Michael Kennedy.

Thank you for listening and sharing this podcast with your friends and colleagues.