Python Bytes - #21 Python has a new star framework for RESTful APIs

Episode Date: April 13, 2017

See the full show notes for this episode on the website at pythonbytes.fm/21

Hello and welcome to Python Bytes, Python news and headlines delivered directly to your earbuds. It's Wednesday, April 12th, 2017, and this is episode 21. Here, I'm Michael Kennedy. And I'm Brian Ocken. And we're going to tell you about all the cool things we found this week in the Python Ecospace. But before we get to that, Brian, let's say thanks. Well, thank you, Michael.

Yes, you're welcome. Thanks to Rollbar, actually. Oh, yeah, yeah, definitely. Thank you, Rollbar. Yeah, thank you, Michael. Yes, you're welcome. Thanks to Rollbar, actually. Oh, yeah, yeah, definitely. Thank you, Rollbar. Yeah, thank you, Rollbar. They're sponsoring this episode again. And if you want to check out Rollbar, get some special deals, it's rollbar.com slash Python Bytes. We'll talk more about them later.

Brian, let's talk about some profiling. Doug Hellman, he's the guy that did the Python module of the week. I don't even know when that started for Python 2. But he has, and I can't remember when he started the third one, but I don't know if we've mentioned it on the show yet. But it's a really good resource. It's pymotw.com, Python module of the week. And he goes through a lot of the standard library

and just does a little page on how to use different bits of the standard library. And this recently put up Profile and PStats. And it talks about Profile and CProfile for basically just for if you have a piece of your code that you think might be slow and how to profile that and figure it out. So I wanted to highlight that. Yeah, I definitely, I think this is awesome. You know, Doug, way to go. Very, very cool. And keep going with that.

And we're going to be checking out, of course, as you roll out new ones. I definitely want to encourage people to think about profiling. I've been doing a lot of work recently on certain parts of my web app where performance is somewhat important and things have changed, right? So for example, I switched out the database backend and depending on how that was working, like it used to be really fast, became slow in some situations. Maybe I needed to restructure the way my data was stored. Maybe I needed new indexes or whatever. And even just profiling the web application itself actually can tell you a lot about not just where your Python is slow, but where the web services you're connecting to are

slow, or the databases you're talking to are slow. Or you can see something like you made 600 requests to the database during this page load, like you can really bet there's probably some sort of lazy loading problem going on, right. And so sometimes just a cursory glance at profiling stats will like tell you like oh i didn't realize i was doing that wrong and you quickly fix it so i definitely check that out so um i the the the built-in profile stuff and uh c profile works great for a lot of the stuff i work on but i'm not often working on web stuff is is though are those the right tools for something like web applications, or are there other things that you search for? One of the things that can be challenging about profiling web apps

is there's a lot of startup code that runs, and that can be computational. So if you try to profile your app as it starts up, it could be swamped with actual, like just getting the thing going traffic. And then you hit a request for a page and it's small enough. It doesn't show up. So maybe you've got to hit it a hundred times or, you know,

something to get it to happen. So what I do, I don't know about the other frameworks, pretty sure they must have something like this, but in pyramid by default, there's a little red box on the right side in development mode. And you can click on it. It'll take you to, it's like called a Pyramid debug toolbar.

It takes you, you go to the back there. And one of the options is to turn on profiling per page request. So you just check a box and then request a page. And you can, it'll tell you how many milliseconds it took. And if you click on it, it'll give you the output from this just for that page automatically. So you don't have to do anything. Yeah, it's super cool. All right.

Well, thanks. Yeah, sure. Speaking of the web, so we have a new project coming out that I wanted to bring everyone's attention to. I don't know how old it is. It's like weeks, maybe months. I'm not sure. It's not very old.

It's by Tom Christie. So you may know Tom Christie from the Django REST framework. And he came up with something called API star. And this is an alternate REST framework. Okay. So Django REST framework has been around for like five years and it's well, very popular. It's well known. It's highly used,

but it's also very much wedded with the, the Python two prior web world, right? I mean, it's from five years ago, right? And it's based on stuff that was, you know, sort of put into motion before then. So it's not, if you want to explore some new and interesting ideas, you can't really do it. And that's what Tom was saying.

So he actually came up with this thing called API star, and it has got some slick features that that are really worth looking at. It's really new. So I don't know if it's quite ready for primetime. I just honestly don't know. I'm not saying it's not. But the idea is he calls it a smart web API framework designed specifically for Python three. And it's super easy to get start just started just pip install API star, you can say API star dash noon template dash minimal or dash dash template minimal, right, that'll get one, you can either run it and you can say api star dash new template dash minimal or dash template minimal right that'll get one you can either run it or you can run the test straight away so i thought you would like that little api star space test that's pretty cool yeah and nice interface too

it's looks really clean yeah so here here's where the magic comes in it's all about the type annotations so in python 3 you can say like i could define a function, show query parameters, and I could have a thing called query parameters. If I wanted it to be of type HTTP.queryparams, I would say colon HTTP type queryparams. And that tells Python, not at runtime, but at design time or development time, hey, this thing's supposed to be a queryparams instance, right? Yeah. Okay, that's cool. Okay, so now here, check this out. If you put a parameter on that API method and you say it is one of those and it's available

as part of the request, it will automatically come into your system regardless of what you call it and where you put it. Oh, nice. It looks at the type, hence the type parameters. So the thing you're saying, this thing called query params, you want the query params, I have one of those in the request. And similarly, if you return something from there, by default you return dictionaries like all these web frameworks in Python do.

But you can also use type annotations to say, no, this returns a response, and then return something else. Like actually using the type annotation to control how the processing of the request is done. Similarly, you can put things in like your URL. So right, like slash, slash users slash seven, right? And normally, that would all just come in as a string. But in this API, you can say like as a function on your rest method, you can say like user

ID colon int, and it will take that thing and convert it to an int out of the URL and pass it in. So there's all this really cool stuff around type hinting that I just type annotations. I really love. And not only that, it's faster than Sanic. Okay.

So very, very cool. Good job, Tom. Yeah. And now that I know you said that already, and I kind of went past me without me recognizing it, but the built-in testing with PyTest.

Yay. Yay. It's awesome, right? Yeah, definitely. Cool. Yeah, so it's a really new framework. Check it out. It's already got quite a few stars on GitHub, but yeah, API star.

There's a link in the notes. Nice. And there's some little example code as well. All right. But that's about fast web frameworks, but sometimes we don't care if things are fast, right? Yeah. So actually, we're kicking around the idea of talking about this last week and decided not to. But we're putting it in now.

It's an article called, Yes, Python is Slow and I Don't Care. And, you know, actually, I've never really noticed that being slow for what I have it for. basically that's kind of the point of the article, is that fast is great. Having Sanic and API star be super fast, of course, that's wonderful. But there's a lot of Python uses for Python and for other programming that the most expensive resource is not your CPU. Actually, it's very seldom your CPU anymore. So therefore, you usually don't really have to care about the speed of Python. And I'd have to agree. And a big chunk of this article is you should be optimizing for your most expensive resource. And that's your people. Your people are way more expensive than your computer time. And bugs. And yes, and. And so are bugs. I know that myself, we have at work, we use Python for testing.

And it's because we can write smaller test code. And the time our tests run is all in IO time. It's communicating with instruments. It's not sitting in our computer. And so I can have small, elegant tests that I can read well is way more important than how fast Python is. Yeah, you're talking about testing devices and hardware. Same thing applies to the web, right?

You're usually waiting on either a database, a web service, or a socket somewhere, and the timing usually doesn't matter, right? Yeah, well, I mean, of course you want it to be fast, but the slow part isn't the Python. So actually, I guess that ties in with the other articles we've had so far. Developing an API quickly and elegantly is going to eliminate mistakes better. If you can do that with less code, great. Yeah, absolutely. Well, and if things are slow, right, there's always options, right? You could write it in C, you could write in Rust, you could write in Cython. You could take that little bit and do something.

Or just like you started at the beginning, profile it and change your algorithm. It very well may be that that makes a huge difference. Yeah, and it might be that you think it's Python, but like you said, it might be your database or you're accessing some resource or something. Figure that out. And then if it really is Python, which I think there's rare cases where it really is, but if it is, there's options with Cython and such or taking a look at Intel's offering or PyPy or something. Yeah, exactly.

I just saw some stuff that PyPy is supposed to be getting quite a bit faster still. And, of course, they got that big grant from Mozilla to make Python 3 work much better. So all sorts of goodness there. Yeah, definitely. Now, before we get to the next item, let me tell you about Rollbar. You guys probably heard me talk about it before, but I've been using Rollbar for a long time. And the idea is you

basically put a few lines of code into your web application. And if ever there's an error, it'll automatically capture all the information that you need about it and ship that over to rollbar and then send you a notification, all sorts of different ways you can get notifications. And it comes with all almost all the context you need to actually solve the problem. So for example, it might have the stack trace and the local variables and the URL and all those sorts of things that you might not even have

to debug your code. You can just look at it and go, oh dear, I need to check for this or make sure that's not done or whatever, right? So if you want more reliable web apps, just be sure to check out rollbar.com slash pythonbytes. Wonderful. Thank you, Rollbar. And we have Rollbar to thank for why Python Bytes' website is always up. Yeah, that's right.

Absolutely. It's integrated in all. Yeah, that's right. Absolutely. It's integrated in all the sites. It's beautiful. So one of the things that I want, I kind of want to take us back to the basics for a minute, okay? So one of the things that I think people, many people know about, but a lot of people don't, is hashing. When to use hashing, how to use hashing, and things like that.

So there's a nice article called A Quick Introduction to Hashing by Gerald Nash, and this is specifically for Python developers and has some nice examples of, like, if I want to hash a string, you know, using, say, SHA-256 or something like that for the hash, how do I do that? What does that look like? Can we back up a second in what is a hash? Sure. So a hash is a hash? Sure. So a hash is a little bit like encryption in that you can look at the hash and you basically,

it's obfuscated, right? You don't really know what it is. But unlike encryption, it can't be reversed. It can't be unhashed, right? You might take a 10 megabyte set of data and hash that into 256 characters or something like that right the idea is if you have the same input and you run the hash over it over and over again you should get the same output okay but if even if it varies slightly you'll get a totally different output not a little bit different but dramatically like unrelated sort of type of thing and so it's used for verifying messages,

verifying like download files are not tampered with. But most importantly, it's used for passwords, right? Storing raw passwords in a database, not the best. So we typically hash them. But there's all sorts of other interesting uses you can use for hashing. So like in my websites, I have the static files, JavaScript, CSS, images, et cetera,

basically cachedached super hard. So they're cached for a year at least, maybe possibly longer. So if you go to the site and you come back later, it's not going to go back and look again. And so what I do, the technique to make sure that nothing ever shows up stale on the site, you don't have to wait a year for it to come back together, is I put a little hash of the raw files at the end of the URL of every bit of static content that I have. And so it caches it, but it caches it with the

fingerprint of it embedded. So if the file ever changes, it instantly generates a new file and it pulls it out. So hashing is used for all sorts of good stuff. So anyway, it's a great little article. I'll show you guys how to do that. So if you're not taking advantage of hashes, they're super easy.

Here's a nice way to see how to do it. Wonderful. This is a very fun article. I forgot to write this person's name, but there's an article called Wedding at Scale, How I Used Twilio, Python, and Google to Automate My Wedding. I feel like I missed this opportunity when I got married. Yeah. And this is just fun. And actually, the modern person's problem with like how to invite

people and, and I love this article, just, you know, the nerd in me likes it. There's no way I would have had time to figure this out when I was trying to plan my wedding. But if you got the time, it's great. So through all of all the potential guests and their phone numbers into a Google spreadsheet and then used Python and a Python package called G spread to access Google spreadsheets and pull that out. And with Python use Twilio and SMS people and invitation to the wedding. And then not only that, left URLs for people to get back to him.

I guess there's URLs and there's also like replies to have a set up a Flask app to collect everything of all the replies to that. And then even goes on to sending out reminder texts and having people select which food option they wanted. And it's pretty incredible, this story. Yeah. A real mashup of a bunch of cool technologies. That's awesome. Yeah. It's cool.

Nice. I have another fun one for us for our last item here today. And I've got to be careful about how I speak about this device because my understanding is speaking of it causes it to do things. So let's call it the Amazon virtual assistant thing that sits on your desk. All right? Okay. Okay. So there's a guy who created a framework for creating what are called skills on this assistant. Right?

So you can teach it to do new things. And the guy who wrote this article is Neil Stewart. And what I thought was really interesting was he was like visiting a friend or something like that. And they had one of these assistants and he thought it was really cool. He's like, I'm ordering one, but he decided like, I'm going to challenge myself to build a new skill for it before it even arrives. So as developing for hardware, that's a pretty interesting idea, like to develop for it before it exists,

you know, in your hands, right? Yeah, we do it all the time. Yeah, right. So the thing that he came up with is something called VoiceOps. So he has a bunch of AWS accounts and whatnot, and he can ask it things like, how's the status of his AWS servers and whatnot.

It'll tell him how many running and stopped instances there are, what RDS databases are on his account, and things like that. So that's a pretty fun skill to add to this assistant. Yeah, I didn't know you could do that, add skills to it. That's pretty cool. Yeah, and apparently you can do it in Python, which is even cooler. So he looked around and saw that there's a few things, but he wasn't really happy with it,

so he decided to create a new framework called Python, the name of the thing. And it's open source on GitHub. The article here that we have linked, this one actually has a bunch of cool examples on how to affectionate. He has a hello world type of thing. And he also linked to something called Echo Shim.

And Echo Shim allows you to create one of these skills, put it into AWS and run it on their Lambda system on Python, and then test it in the browser without even having hardware. Oh, cool. Yeah, so pretty fun project. So if people are into that system thing, then Python, check it out. I will check it out. That's neat. Yeah, I haven't gotten one.

Do you have one of those things? Any of those, like a Google Home or those types of things? No, actually that whole concept of letting some other company just listen to my house. I don't like the idea. Anyway. Yeah, I'm not there yet either. I'm not super crazy about privacy and whatnot. No, I'm not.

Locations on my smartphone and stuff. But that somehow crosses a little bit of a line to me. I don't know. I'm sure I'll be over it someday. Plus, I'm not. Locations on my smartphone and stuff. But that somehow crosses a little bit of a line to me. I don't know. I'm sure I'll be over it someday. Plus, I don't really, yeah, I don't trust my kids to not just order a bunch of stuff all the time just by talking to it. Yeah. After what happened the first time I got an iPhone that had Siri on it, I know what the kids did to that thing. This is even more accessible to them.

So I'm not sure I want that anywhere near my kids. Awesome. So that's it for the week. Brian, what do you got going on with you personally? I had an interview with Casey Rosenthal from Netflix on chaos engineering that I finally got edited and out last week on testing code. So that's out there. Oh, like chaos monkey, Chaos Gorilla, all that kind of stuff they do at Netflix. Yeah. And basically we tried to talk about not just what they're doing there,

but the concept of taking steady state measurements and doing experiments that might change the steady state on a system as opposed to traditional testing methods. It's pretty interesting. Yeah, nice. Oh, very cool. I'm definitely going to listen to that. It's pretty interesting. Yeah, nice. Very cool. I'm definitely going to listen to that. That's great.

How about you? So I spent the weekend reworking a bunch of my web stuff around the training courses, training site, and so on. So I rebuilt a new player because I wanted to surface the closed captioning a little bit better for people, things like that. So I did a bunch of JavaScript and whatnot. And the other thing I did is I added full text search to the whole site. So you go there and you can search like into the videos, even into the text of the videos.

So it's beautiful. And this is what I was actually, yeah, this is actually what I was thinking about when we were talking about profiling. So when I first wrote this, I wrote it like what you might think of as like na i wrote it and it's all backed by like a bunch of text data in mongodb it's not using full text in mongo it's using something else that i i've done but basically the back end is mongo and there's a bunch of data in there like 10 megs 20 megs i'm not sure exactly how much of text that i wanted to search and i ran this this, oh, this would be nice and fast. And it was like 600 milliseconds a page, which like is acceptable maybe for other people. But for me, this is not acceptable. I'm like, oh, no way. This is not going to stand. Like, what is wrong with this?

So I spent a lot of time in the profiler and I've gotten it down to 47 milliseconds. Oh, okay, cool. That's- So it's more than 10 times faster, right? That's like 10.5 times faster by using the profile. And it was just, you know, this part's slow. How can I do this differently? This part's slow. How can I do this differently? So just, you know, very neat.

If you want to see the fruits of that profiling, you can click the link at the bottom. Cool. Can we get a text search on Python bytes? Yes, we're getting to, yes, absolutely. We're getting full text search on Python bytes soon. Okay. Yes. I think we need it, though.

I mean, we have a bunch of transcripts that we have interesting stuff in. We've got pretty solid show notes for this podcast. I think we need full-text search. So, yeah, we'll be sure to announce it shortly. Okay, great. Cool. I need one more weekend, I guess. Yeah, like adding work to you on air.

Sorry about that. That's all right. It's good. No, no, it's on the roadmap. It's on the roadmap. All right, well, Brian, thanks so much for finding these things and sharing them with everyone. Thank you. You bet. That's all right. It's good. No, no, it's on the roadmap. It's on the roadmap. All right, well, Brian, thanks so much for finding these things and sharing them with everyone. Thank you.

You bet. Catch you next time. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool.

On behalf of myself and Brian Ocken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.