Python Bytes - #188 Will there be a "switch" in Python the language?

Episode Date: July 3, 2020

Topics covered in this episode: Making a trading bot asynchronous using Python’s “unsync” library Fruit salad scrum estimation scale Math to Code PEP 622 -- Structural Pattern Matching CodeArtifact from AWS invoke Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/188

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 188, recorded June 24th, 2020. I'm Brian Ocken. And I'm Michael Kennedy. And this episode is brought to you by us, TalkPythonTraining and Brian's fabulous PyTest book. Brian, can you believe 188 contiguous episodes? I cannot. The number keeps going up. I guess that's what numbers do, but I guess so. It's awesome. I'm glad we're still doing it.

It's fun. What you got for us to start with? Well, you may have heard me talk about async and actually I didn't cover it on purpose. There's sort of a controversial async article going around. Don't really want to talk about it at the moment unless i have to but i do want to talk about my favorite async thing which is unsync u-n-s-y-n-c unsync okay i feel like the python core developers should look at this and say you know what we massively over complicated all the asynchronous parallel capabilities of python, let's create a unifying API that like is the one simple way that you can do things. And if you need to dig into the details of the other ones,

that would be great. They haven't done that yet. I mean, async and await, the keywords are great, but if you want to say work with like a thread and some async IO thing, the way you do it is like totally disjoint and unrelated. So this unsync library is like a unification library on top of threading, multiprocessing,

and async and await. So I've talked about that a bunch of times and it's sweet. I'd love to see Python just adopt something very similar to that API internally. Anyway, there's a cool article called Making a Trading bot asynchronous using python's unsync library by matt matt gosden oh nice yeah it's cool it just walks you through

a somewhat realistic example of creating something that does freighting and in order to do it has to go and you know it talks to a different web services and some databases and whatnot now all these are simulated with a function call to a time.sleep call right one of the things you'll see a lot of people when they do like these performance analyses and stuff is they'll have some async code and it's hitting up against some other thing and that other thing has a performance limit that is like near near what it can handle anyway. And so you can't get a big boost out of it. So this kind of like puts that to the side.

So it's like, we're not going to actually talk to another thing because maybe that thing is slower than we can handle or who knows. We're just going to go and simulate slowness by sleeping, right? So it actually walks you through some different ways of doing things. It has a synchronous version and the async and await version using unsync. I don't know if there's intermediate. I can't remember if there's intermediate versions in this tutorial about having just a pure threaded version, right? But it's mostly about taking the synchronous version and making it unsync. So it's nice because it's a somewhat realistic example. It's not as much of a realistic

example as some of the scenarios. It's not as ideal of an example as some of the other scenarios. Like, for example, if I need to web scrape 100 different websites, right, you can do that 100 times faster by just like kicking them all off and getting it back and not really waiting because all the work is happening distributed elsewhere. but it's still a pretty realistic example doing some different things it would be fun to hear it talk about scalability more like how much it's like we've got a couple i think it's doing three things it's like look we could do these three things quicker but you know like what is the limit like how far can you push it because i think you could push it quite far actually with what they're doing it's just hard to to know if sleeps are representative as well.

Yeah, it is hard to know if a sleep is representative. Basically, if it's a truly external system that has infinite scale, some cloud service, then a sleep is probably pretty representative. Whatever you're doing is not going to affect it. But if it's, say, a database, if I'm doing 10 requests against the database, versus one, and maybe the database can't handle it, and it slows down to what like two or three queries at a time would be like, there's things like that, right, where it hits a limit. But if the thing you're talking to totally scales, and sometimes it does, sometimes doesn't. One thing that I think is missing from this is it doesn't actually use async methods it just has regular methods so this is

really an interesting thing right so it's like okay we're going to put unsync on it there's a regular function that we put that unsync decorator which turns them into these things that are awaitable and parallel and so on and what you really probably want to do is actually leverage async and await create you know def async methods, you know, await async IO sleep and all that kind of stuff. So it didn't actually fully transition them to leveraging async IO. But what I think is interesting about that is it still got much better with unsync. And what's cool is like it said, okay, well, these are not async methods. So we're going to have to use threads, but let's just fire them off with threads anyway. And because the time.sleep releases the gill just like a network call would it still kind of works

so it's kind of neat that like it wasn't fully converted over but unsync still made it better in the same way that you would expect interesting yeah yeah yeah so anyway a cool example of someone talking about unsync was not me except for right now reviewing it there's a lot of cool stuff in there kind of like a like a fruit salad would you say just a lot of stuff to take and it's it's sweet and people generally enjoy it oh man i shouldn't have picked this story right before lunch also because i'm kind of hungry i know i know it's good so tell us about this i was going to put this as an extra thing but i put it as one of my topics because it's actually pretty darn cool. So this on Twitter, Lacey Henschel just asked the question of like there was this scrum estimation tool with fruit.

Does anybody remember it? And Kathleen Jones replied and said, is this it? Essentially. And it was. And so we're linking to this article. It's called the Fruit Salad Scrum Estimation Scale. And, you know, task planning and scrum estimation is, you know, it's a kind of an art form and a science together. It takes a while to get things right.

So maybe just really, really super quickly tell people, what is this scrum estimation about? Oh, okay. Maybe scientists and they're like we don't do that in our biology lab oh that's true lots of teams have picked up scrum or sort of a variant of scrum which is a way to plan what tasks what things that people are going to work on and the things that people work on we kind of want want to, you know, have a, like a size for them. And instead of doing like, oh, it's a one day task or a two day task or a five day task, or some people just use small, medium, large t-shirt sizes are popular. And then also for some reason, just

points are very popular. And the point system is often the Fibonacci sequence, which, but it's not really, it's just kind of Fibonacci. So I can't remember the numbers right off the top of my head. I think they're like 1, 2, 3. We usually skip 3, so 1, 2, either 4 or 5, and like 8 sometimes, and then like 13 and 40. So it isn't really quite fit normal Fibonacci, but I think that's where people took it from.

So we've been using them at work as well lately, but the numbers, it's kind of like t-shirt sizes, but I don't know, none of these things really fit. But the fruit salad estimation scale actually kind of fits. So they've mapped one, two, three, five, and eight to grape apple cherry pineapple and watermelon and i like these ideas because they're kind of like how easy they are to cut up something and how easy they are to eat them so like a grape is trivial you know you just pop it in your mouth it's no big deal you don't even have to cut it up for a fruit salad although i usually do an apple everybody knows how to cut up an apple but it's a little more food, so it takes a little bit more, and it might take some more time. A cherry. It's also easy, but there's some unknowns in it because of the pit. And a pineapple. Yeah,

you can't just eat that. You've got to actually put some work into it. Some people don't know how to cut it up, and it's a little messy. You're going to have to get your hands dirty for that one. And a watermelon, all bets are off. Nobody knows. You don't know what you're getting into until you cut open the watermelon. So I like that as the large size. There's more description on this article. And then they throw in some nice ones. Tomato and avocado avocado tomato and avocado do not map to points but tomato is unknown i mean are you a fruit or are you not and you need more information before you can estimate it and it really doesn't belong in the fruit salad until you change it into

something else and then one of my favorites is avocado that's something that you just get you can't really scope it very well. And it's probably urgent because it'll go bad quickly. These are great. I like this concept of thinking about them here. Yeah. So I think that somebody needs to put these fruits in an official product. That would be cool.

That would be cool. You know, another one that comes to mind here is a mango. Oh, yeah. You know, those are tricky because I like to cut them like in the orientation of the husk or whatever the thing in the middle is the giant seed but you got to hack into it a little bit before you can even figure out what that is so you kind of start out unknown and it's slippery you may hurt yourself on it but it's it's really i don't know it's a high value once you get it out of there i don't know mango and sometimes you can't tell until you get into the mango if it's even good or not.

Exactly. Yeah. I think mango is in this category somewhere. But this is a really cool idea. I was thinking about it because it's super hard to be very accurate when you're estimating stuff. And what I like about this is it just brings that together in a pretty clear way, right? They're not like, well, how many hours is it going to take? It's going to take three or four hours.

Like, no, no, no. We're we're not talking like i can't possibly give you that answer it's like all right fine it's four hours plus or minus 16 hours yeah and then that kind of stuff if you do points people always have like some conversion to hours anyway and it's annoying i think it'd be cool to if somebody said said, you know, I got a manager or somebody saying, hey, how much time we have left? I'd be like, well, we have three grapes and a cherry left to do. Yeah, just don't talk about it by lunchtime. Anyway, I am hungry now. So instead of doing another topic, we should just talk about how awesome we are,

how awesome your training courses are. Oh, thanks. Well, so this episode is brought to you by us and we both are doing stuff we would love for you to check out. So over at Talk By The On Training, we have a bunch of courses. I think we're up probably by the time that this goes out, we're up to almost 200 hours of courses and a couple hundred hours of exercises. So lots of stuff to be learned over there.

But I want to call out if you have a company and your company has a training budget or a training plan you know reach out to us we have special deals and offers and pilot programs to help get our courses started at your company so shoot me a message over the michael at talk python.fm or just check us out over there and uh if the writing code they should test it right definitely should test it, right? Definitely should test it. And tell everybody the Python testing with PyTest is the best way to get started with testing with Python. And even though it's like 180 pages,

the first two or three chapters will get you up and running like in a day, less than a day. So you can get started right away and then you just get faster and more awesome as you go along. Indeed. I take advantage of all those features in PyTest that people might not know about. So, you know, one thing I think people come into programming often feel like if they're going to be programmers, they have to be good at math. Like, oh, I was really good at calculus, so I'd be a good programmer.

Or I never actually got algebra very well, so I'd probably be a bad programmer. And I think that connection is often very much not true like i don't do math other than like basic arithmetic in my programming these days but some people do right some people come as engineers or they come as scientists and they actually do math frequently and they want to do it with python so vernon ther theramit thomaret sorry sent over this cool project called math to code have you seen this yeah i was checking this out the other day it's pretty fun yeah so imagine you wanted to learn something by doing uh flash cards right i'm going to show myself i'm going to flip through them and like see like a simple thing and then

what the answer to that is and this is kind of like that, like it starts out says, what I want you to do is take the square root of something using NumPy. And it's all based on NumPy, or just pure Python, like, raise something to the power is just x star star y, for example. But it just really quickly and simply takes you through that. So you type out the answer, hit enter, it's all hotkey driven, which is great for a web app. And it just kind of and simply takes you through that so you type out the answer hit enter it's all hotkey driven which is great for a web app and it just kind of guides you through like practice experience of here's a math problem solve it numpy yeah and it's so beautiful it is pretty right yeah cool yeah i honestly haven't made it to the end so i don't know how many answers or questions or whatever there are or flash cards as i'm calling them but it is open source on github which is pretty cool so you can go in there and you know check out the source code and play with it and yeah it looks like you know 13 questions but it's on github and the questions are just marked down so you could go add as many as you would like actually that's pretty cool yeah but what also is interesting just at a more higher meta level

is maybe you don't care about finding you know the square root of something in python is the building blocks so if you look at how this thing is built and like i said open source on github you can check it out it's built running python on the client side what really yeah so it's built using sculpt which is a javascript implementation of python and then it has sculpt numpy for the subset of numpy running on a client side that it wants you to experiment with right like numpy dot square root for example or mp dot square root it has ktex for rendering latex on the browser side it has next js for front end tailwind css which my friend mark just told me about which is a really interesting alternative way to like css front end frameworks like bootstrap

but different remark from markdown on the client side gray matter all sorts of stuff so there a lot of cool building blocks here, regardless of whether or not you're actually into the questions. Yeah, this is awesome. That might be more interesting, actually, than the thing, right? Because I honestly don't care about learning the math features generally. It's nice, but it's just not something I do. Yeah, but this sort of model of like have a, I mean, if you look at it, I don't know if you already said this, but if you look at it, you've got like the, some explanation and a question on the left, a place to type in and submit your answer on the right. And then there's even a place where you can like set up a hint and, and show, show the answer or hint or somebody. And having all this just, it's, it looks very nice.

And having like, this is an as an example for somebody else. I could totally see like a teacher running with this to help their students learn really pretty much anything. Yeah, it doesn't have to be exactly Python, as long as you can verify it with Python. Yeah, cool. Yeah. Tell us about the PEP. There's always a PEP to be discussed. I've learned about a couple of new PEPs in the 3.9 timeframe.

Right. So this is the 3.10 timeframe. So everybody should be using 3.8 now, and you should be testing 3.9, especially if you have a package that people depend on, so that when 3.9 is then official, it'll all work. But people are already working on 310 of course and one of the peps for 310 is pep 622 and it's not official yet it's in draft status but there's some cool people working on it including guido and i think it's super cool did you have you taken a look at this i have taken a look at it it starts out to me feeling like a switch statement. Yeah.

There's a lot more going on here. It's called structural pattern matching. And right off the bat, they note that there have been previous peps before that have tried to put switch case statements in Python. And they've been rejected. I don't know why they were rejected. It's kind of something I didn't pay attention to. But this, yes, instead of a switch case, it's a match case statement and a multiple statements. But the neat thing is there's all these different. So when we think of like a switch case statement in like from C, it matches by equality or value.

You switch on some variable name or some expression. And if the answer matches one of the case statements, then you run that part of the code. So that would be in the PEP 622 world, that sort of a use model would be like the literal pattern or the constant value pattern, basically with equality and stuff. But it does more than that. You've got name patterns so that if you just have a whatever, if these will always, if nothing else succeeds prior to it, you can just have a variable name pattern is only available in that it isn't assigned otherwise if that didn't get hit. So that's kind of some useful, neat things. Constant pattern is kind of the same, but then it gets interesting. So I think it'd be worth it just for that, but

you've got sequence patterns where you can do, it works like unpacking assignment stuff. You've got mapping patterns that are like similar to sequences, but for like dictionaries and class patterns where it can, you can have a, like a custom class or a class that it might be. And you have a match object. It's similar to an equality, but you could have, it could be different than equality. And then you can combine them with os, so you can match cases if

multiple things are true, like a Boolean expression in there. I don't think it's a full Boolean expression, but at least ors work. Or pipes. Guards, so you can say if this pattern matches, and then have an extra if expression so you can clarify it even more. And then even sub-patterns, and then have an extra if expression so you can clarify it even more. And then even sub patterns. And at that point, I kind of got lost. So just there's a lot here. I don't think it's going to clutter Python up. Actually, I've seen some examples of how this would dramatically simplify some Python. Yeah, it's interesting. What's surprising to me is

how many types of things it's trying to do at once right like it's not just like oh let's have a switch statement with like a slight variation like it's there's a ton of stuff going on including things like if you switch on an enumeration you can just say dot attribute and it'll try to like pull that attribute out of that type and check it and yeah it's it's pretty interesting one thing that's missing here that i would love to see is range matching right so i could say it's in the range of zero to ten match this case if it's a range of 11 to 100 do this other thing actually so i have my package the switch link package which adds switch to python and it has those types of things uh so

i actually sent that over to one of the people working on this pep and said hey this is really cool you know check out some of the ideas from this one like especially the the range matching and see if it makes sense here because it's so common that you you would want to say like this range is this case that range is that case and so on and we even talked about a package a while ago i can't remember exactly what it was called which is unfortunate we've covered too many things but it was basically you could give a like a number or a value and then this range and you could ask is this thing in that range in some real simple way so it'd be cool if like that was all combined you might be able to do with the if if guard though. So you could say like,

you know, match X, if X is within, you know, in this range. Yep. Yep. It's true. Yeah. We'll see. I mean, at that point, like if you're writing a lot of complicated if statements, you might as well just write if statements. Yeah. Yeah. Although, I mean, it's just always, I don't like it when I see like these, the if else ladders. So I think that this is more than, I mean, it's just always, I don't like it when I see the if-else ladders. So I think that this is more than, of course, more than just replacing the if-else ladders.

It's also doing things like unpacking and other sorts of cool stuff. But you're right, there's a lot going on here. There is, there is. But it's cool to see innovation around this. I do feel like that thing, that general idea is missing from Python. I know people say, you could just use a dictionary or stuff, but there's like a lot of cases

where if else is buggy, hard to maintain, or like these other cases are very, you know, verbose and so on. Anyway, I'm happy to see this, I think, depending on how it comes out. It's very complicated, but the idea is good. So one thing that we've talked about a lot brian is how do you host your own private pi pi right pip install a thing but i

want to be able to pip install and version like say you work at a large company you probably have libraries packages that you've shared across projects how do you like have the pip style package management but for you well we've got a custom PyPI in our network. That's right. And that's fine. That's fine. The more global you are, the harder it is to maybe do that well without people getting a hold of it. Obviously, you want to protect that code because that's internal private code, but at the same time, you want to share and version it.

So there's all sorts of options that you can set up and take care of that. But there's this new thing which Tormod McLeod has set over our way called Code Artifact from AWS. Neat. Yeah. So the idea is it is a general purpose, basically package management system for a variety of things. So right now it works with Java with Maven and

Gradle. For JavaScript, it has NPM and Yarn. And for Python, it has pip and twine. And basically, you can just set it up in AWS and tell it to go. And they'll take care of all the storage and the security and all that kind of stuff. And then you can just pip install all the things. And it also is backed by the real PyPI or the real NPM. And you can whitelist stuff. You say, okay, if somebody pip installs requests, just get the real request and stay in sync with that, but let them have it. But if they pip install the misspelled request,

I think you can say don't let them have the typo squatting hacker version. Yeah. So these sorts of solutions are pretty interesting to be able to have basically a cache of IPI stuff and then also your own stuff. So you push to it just like often you publish it. I don't know about this one, but often some of these you use Twine or something to push just like you normally would.

But it doesn't go to ipi it goes to your own thing yeah it's pretty neat you know it reminds me of artifactory which people might have heard of which is looks really cool however if you look at like the pricing it starts at you know just two thousand nine hundred dollars a year right for the base version the pro version is fourteen thousand and it goes up from there. So, you know, this looks like one of those cases where one of these cloud services from places like AWS and so on is coming along and going, you know what? We got you. And it's not $14,000 a year or whatever. Or a month. I don't know. Whatever the unit was I said there.

Per year. I think, to some degree, I believe GitHub is working on something like this as well. I can't remember what degree, I believe GitHub is working on something like this as well. I can't remember what it's called over GitHub, but it doesn't have Python yet, which is why I haven't jumped up and down about it. But I think it will. I think they're working on it. Well, supposedly, like, Warehouse was something you could just use that is used for, well, I guess they don't call it Warehouse anymore, but... PyPI.org, the new PyPI.

Yeah. So, I think that you can deploy that yourself, but I haven't seen very many people like, well, one, I'd like to see people write instructions on really how to get that up and running for yourself. And I'd also like to see some like, you know, warehouse as a service sorts of things. I mean, why not?

Some free money laying on the table there, people. Yeah, that's a good point. That's right. All right, what's this last one? I wanted to bring up Invoke, and it's at, what, piinvoke.org is where you can see the documentation

for this, and to tell you the truth, we've had it on our list for a while and I've tried several times and couldn't get it to work, but I had some confusion, so my confusion was the product, when you install it, you install it with pip install invoke, not with the Py part. But there is a Py invoke on PyPI, but it does something different.

So the invoke that I'm talking about is invoke without the Py in front, but the website is pyinvoke.org. Okay. If you're confused, I'm confused, but we've got it in the show notes. But why do we care about this? We care about it because it does make me feel, I like make files. I think make files are fine. There's parts of them I don't like. Like, I don't like the tab.

Like, it's the only thing in my life that I have to use tabs for because I usually use spaces in Python. But make files have tabs. And often things like cleaning out, you're building your documentation or kicking off your tests or deploying or a lot of those other book maintenance things for a project you put into a makefile. And some people use talks for that and you can, but invoke is intended to do that sort of thing,

but with Python code. So with invoke, you just create a little task.py file with these little types. You just have little functions that you decorate with the at task decorator. If you have it installed, you can just say on the command line invoke and then the name of the function. So if you have a build function, you can say invoke build and you can pass in command flags to it. And it's just pretty easy to run something. So something like you would use make files for, you can have it without any make files.

Yeah, that's cool. And you get to write Python for your scripting, right? Basically. Yeah. The project looks like, I'm not quite sure if this is recent or old. It says that it was intended for building documentation, but it does look like there's some examples on how to combine it. I don't see why you couldn't use it to run your tests

and run all sorts of other stuff too, because it's got things like you can run Python code, but you can also run command line. It can launch a command line tools and all sorts of stuff. Yeah, if you can run stuff off the command line, then you could do get pre-commit hook type stuff or all kinds of automation, or even like cycle the version,

like increment the version numbers and all kinds of stuff you want to put together. Oh, yeah, that'd be a good thing to put in there, version incrementing. And then also just like things like deploying, like check linting, that's a good thing. And then you can, so make fun linting. That's a good thing. And then you can make fun. One of the neat things about make is that different make targets can depend on each other.

And this one has dependencies as well. You can have, like, for instance, build can depend on clean. And it can have to run off and do a different target. And you can make a whole mess of stuff in there. Or you could keep it clean. But I like things like this that to have i mean one of the reasons i'm not even i don't mind make files but people that are not from a unix or don't come from a unix background have no idea what to do with a make file yeah it's good for the team to make it easy for them yeah yep indeed all right that's pretty

cool i'd like to use this and i just got to think of a reason a use case for it and i will it looks great though well that's all of our six items actually i've been like totally swamped at work i don't have a lot of extra things to share with people do you have anything extra for us i have a mystery that we can discuss you have a mystery yeah apparently you just talked you talked about python 3.9 and it's in beta which means it's getting no new features it's just getting tested and ironed out and uh there was python 3.9 beta 1 and then there keto announced oh hey everybody here's python 3.9.0 beta 3 is out for immediate testing and somebody uh disappeared beta 2 so the problem apparently with beta 2 was somehow the way that it looked up certificates was busted so if you like installed it from source and just ran it it would be fine but if you

installed it from the installer all of a sudden every web request that went to https or something along those lines would fail so that's not good to break the networking subsystem of Python because a lot of things depend on that. And so they quickly ditched that one and fixed it and rolled out beta 3. And just about the time that this is released, so on June 29th, we'll have beta 4, which presumably will still make web requests successfully and won't have to be disappeared.

So anyway, just some beta Python three, nine news, beta three is out. Beta four will be out days after this comes out or actually maybe even before. So there is no beta two before we'll be after, before we'll be, before,

before we'll be before I think, but it might be after, I'm not sure. So I've got a joke for you. I grabbed here off of Geek and Poke, which is like a cartoon strip. Okay. All right. So it's a mom and a son and you have kids.

I'm sure you know how this goes. How was your day? Good day? Yeah. What did you learn at school? Not much. You were there for eight hours, continuously getting instructed. Good day? Yeah. What did you learn at school? Not much. You were there for eight hours, continuously getting instructed,

like you didn't learn more than like half a set. Yeah. Right. But if this child happens to be a geek, there's another ploy in which you can employ, another technique you can employ to get them to tell you more about their day. So it goes like this. It says, hi, Darlene, how was your school day? And like sort of a blank stare. Hi, Darlene, how was your school day?

Dash, dash, verbose. Yeah, I totally wish I had verbose flags on my kids sometimes. Yeah. Also, sometimes a dash Q, like a quiet one, would be good sometimes too. Yeah, exactly. The dash, dash, quiet. If I got to pick one or the other, I'm going to go for the dash dash quiet as an option.

Yeah, especially for young kids. Yeah. Exactly. Exactly. Daddy needs a little peace, dash dash quiet. Yeah, definitely. All right. Well, that's it for the episode, I guess.

Yeah. Thanks a lot again. You bet. See you later. Bye. Thank you for listening to Python Bytes for the episode, I guess. Yeah. Yeah. Thanks a lot again. You bet. See you later. Bye. Thank you for listening to Python Bytes. Follow the show on Twitter at Python Bytes. That's Python Bytes as in B-Y-T-E-S.

And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. This is Brian Ocken, and on behalf of myself and Michael Kennedy, thank you for listening and sharing this podcast with your friends and colleagues.