Python Bytes - #84 Vibora web framework: It's fast, async, and means viper

Episode Date: June 28, 2018

Topics covered in this episode: Correcting Documentation for a Deployed Python Package Flask Mega Tutorial pre-commit * Python 3.7 release and PSF board members* Vibora web framework Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/84

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 84, recorded June 27, 2018. I'm Michael Kennedy. And I'm Brian Ocken. And Brian, you brought a special guest along for this one. I'm so excited that we have yet another perspective here. Yeah. Welcome, Nina Zakarenko.

Hi, it's great to be here. Yeah, it's really great to have you on the show, and it's going to be fun to have your perspective on all these things. So let's get started. Before we get into the topics, I just want to say thank you to Digital Ocean. Digital Ocean is great. If you want to try some of their cloud hosting, just go to pythonbytes.fm slash digitalocean and for new users, you get a hundred dollar credit. That's pretty sweet. Hey, Brian, we've talked a lot about Python packaging, right? We have, and I'm going to talk about it more today. We were trying to talk about how to do it right, but sometimes it might go wrong, right? We have, and I'm going to talk about it more today. We were trying to talk about how to do it right,

but sometimes it might go wrong, right? Yeah, so one of the fun things is, well, I've tried to, I'm working on switching packages to use Markdown versions of Readme's, but a lot of them are in restructured text, and sometimes, really either any one of them, sometimes you can get the documentation.

It can be kind of a little mucked up. It might look fine on your computer. But you push everything up all the way up to PyPI, and it doesn't look good. Something's messed up. And so somebody wrote an article called Correcting Documentation for a Deployed PyCathon Package. And the answer, there's a couple answers. One of them is to try to prevent it first by checking your documentation first.

And there was a recommendation of a tool called RestView, which is a way to view your long description of your package before you push it up. But even after then, if you don't do that or if it's still messed up, to use post version numbers, which I knew about these things, but I didn't know what they were for. So let's say your package is on version 0.3.2 and it's messed up. Well, you can't just fix the documentation and push it back up again. PyPI doesn't let you push up the same version again.

Yeah, they're basically immutable once they're published, right? Yeah. So if you push up 0.3.3 with the fixed documentation, there's like nothing changed. Nobody really has to download the new one. But that's, I guess, what the post version numbers are. So you can say 0.3.2.post1 and post2 and post3. And these don't cause people to have to re-download if it's a PIP dependency or something like that.

But it allows you to push up new documentation, and I thought that was cool. That's interesting. So the post on the end means like if I do a dash U or dash dash upgrade in PIP, it's not going to do anything, but it'll still let you update it? Yeah, it still pushes up a new version and um i don't actually know if it prevents the pip install stuff but at least in your version your change log and stuff you can actually i don't know all the details but it's a cool trick yeah it's definitely a cool trick the other thing i wanted to bring up since i'm talking about packaging is i was pushing up a different going through the

release process of pushing up a new package just the other day, and I was trying out the test. For some reason, I was having trouble with the test PyPI. Before you push it to the real one, you can test things out on a test server. But for some reason,

even though the packaging instructions are really pretty clear, I couldn't get it right. So the Python Package Authority has a document called Using Test API that is a little more detailed, and it helped me, and it is nice. So check that out also. Yeah, that'll help all the package maintainers out there. Nina, do you have any packages you have to maintain or take care of? I'm very lucky I don't.

Yeah. I think it's probably a blessing and a curse to have a really popular project. You know, you just think of the people who create a Django or requests or something like that. And on one hand, it's like sort of geek stardom in a sense. But on the other, all these people are asking you for changes and maintenance and all kinds of stuff

that you're like, I've got another life to live. live you know this is not the only thing i do right so yeah maybe it maybe goes both ways it's definitely a blessing and a curse yeah for sure so uh nina what's the item that you want to talk about first i want to chat about something that's not particularly new it's been around for a long time but it's gotten an update in the past year. And that's the Flask Mega Tutorial by Miguel Grinberg. Have you guys heard of that? Yeah, that's a great one. You're right. He had that around for a while and he actually did a Kickstarter to revitalize it, which I thought was awesome. He used that Kickstarter money to put out a great ebook format of the tutorial as well. And for those of you who are listening and

don't know about the Flask Mega Tutorial, it's a, I have the page open right now, a 23 chapter Flask tutorial. And it's come up for me several times in the past few weeks because I get a lot of questions from beginners about how to learn Python, how to really learn Python. They maybe have a good grasp on the language, but they're stuck on what to do next, or they want to learn about web apps, and they kind of don't really know where to go. And I always point them at the Flask Mega tutorial. Oh, yeah, that's really great.

It's so comprehensive. I mean, you're right. It has three chapters on deployment, one for straight virtual machines, one for Heroku, and one for Docker. Yeah, Miguel did a great job on this. And I feel like, you know, you talked about new users or developers coming in and learning Python. There's just so many layers to what you have to do as a developer, right? Like, well, I learned Python really well.

Well, now you have to learn about databases. Oh, okay, great. I'll go learn about that. All right, now I know about databases. All right, I want to put it on the web. How do I do that? Well, you got to go learn about Flask and HTML. And okay, great. I did all that. Now how do I get it on the web? Well, now you got to learn about Linux or whatever, right? Just like so many layers and having this is definitely helpful. Now you have to figure out how to maintain and deploy the thing. Yeah, there's a chapter in this tutorial about databases, but there's also one

about dates and times, which is super important, and internationalization. One of the questions I had for you about this was, I'm looking at a couple of different ones, and Miguel does recommend some, as he's going through the different topics, some different Flask extensions. And have you found that his recommendations are fairly spot on for what you should be using? So I actually did go through this tutorial back in the day when I was learning Flask, but it was the version that was four or five years old. Okay. So I'm not fully up to date with what he recommends now.

What extensions does he mention? Oh, like, like for instance, Flask login for dealing with the user data or user login stuff. And Flask migrate for migrating databases, things like that. I haven't used either of those. Okay. Michael. I haven't used a ton of them.

I know he's doing some interesting stuff with some forms, extensions, and I've gone through his tutorial somewhat. It's really nice. Okay. I mostly write in Pyramid, so I haven't been looking too closely, but I definitely do a little flask, and it's good to definitely experience all the frameworks.

And I have another one for you guys later, a pretty awesome one. So there'll be more to learn. There's always more to learn. Yeah. Exciting. Yeah. So one of the things I think is cool about Python is it's not like a full-on compiled language that lets you sort of work directly with memory. And that means that there's no security vulnerabilities in Python, right?

There's some. There's a few. They come up. They do. So actually, so the thing that I want to talk about is a show by, an article by Anthony Shaw. Of course, it wouldn't be a show without mentioning Anthony Shaw doing something or other. It's called 10 Common Security Gotchas in Python

and How to Avoid Them. So he's done some really cool research here and talked about 10 things, maybe more than 10 things, 10 categories, let's say, of errors you can run into in Python that are super bad, can get your company in the headlines in the way you don't want it in the headlines. So I'll run through these and you guys let me know if they're familiar or if you've seen examples or anything like that. So the first one, probably the most common one is some form of injection attack. And when I think about injection attack, I think about little Bobby tables. Have you seen little Bobby tables? Absolutely. It's a classic. Oh, he's a classic. He's a troubled student. So little Bobby tables is like a cartoon XKCD about, about SQL injection attacks, but there's also process injection attacks in Python that

if you use process P open and things like that, that you can actually do all sorts of badness by escaping or getting out of the various commands and sending multiple commands to the shell, which might also be super bad. So there's that. Parsing XML, there's basically denial of service attacks you can do by having a self-referential XML entity that refers back to itself, that refers to itself, that'll just destroy your memory and a couple of things like that. But there's also ways to get around firewalls with linking in XML documents. So there's some interesting fixes, different libraries you can drop in as replacements

that get around those attacks. Assert. So sometimes people might think assert, assert that you are an admin before I let me, before I run the admin command, things like that. But in production, you can turn off the assert statements in Python. So that could be, well, your error checking just left, right? That'd be bad. Timing attack. Some really interesting ones that we've sort of touched on a little bit, Brian, previously is with polluted site packages or things like that, right? Like if I pip install requests and I like switch, I don't know, just misspell it somehow, somebody could publish an evil request, right? We've covered a few examples of that actually happening on PyPI.

Let's see. Temporary files, yaml.load, pickling. Pickling is evil. So I don't know. Do you guys use pickle for anything? No, but it's still surprising to me that it shows up in a lot of tutorials on how to learn Python. I know. Every time I see it, I'm just like, oh boy, there's got to be, couldn't we just do JSON? The short version of the pickle. Go ahead, Nina. I was going to say, I have not come across any tutorial with pickle in it. That's good. I don't know where you get your material, Brian. Really old books.

Yeah. It's like, oh, we can just save that. We'll just pickle that. There's so many things wrong with it. But security is one because there's basically a step where arbitrary code is run on deserialization. And that's not amazing. And then, of course, patching your runtime and patching your dependencies. So these are all just a handful of things to check your code for, to make sure you don't do, to check your infrastructure for, and so on. Yeah, these were obvious to me. I was expecting the obvious one of don't use the eval statement on user data. But maybe that's just too obvious. Yeah, maybe. I mean, that one definitely should be thrown in there, right? That's definitely one that's sort of along the same thing as Pickles,

maybe even worse than Pickles, to be honest. But yeah, I think there's a lot of non-obvious things here that people who have been doing Python for a long time, like I had no idea that YAML files could be evil, but they can be really evil. And one of the things that I really liked was his fixes are pretty easy. Just, just learn about these, use these fixes and it's good. I didn't know about the, that you can turn off assert statements in production. I tend not to use them in code outside of testing, but that was a surprise to me. And I've been doing Python for quite a long time. It would be bad to go.

Why is it not checking? I know the check is here. I could see it in the code. Yeah, no, it just doesn't do anything. Yeah. So if you're running any Python that is basically exposed to the world in any way, you probably want to skim through Anthony's article. It's a good one.

All right. Before we move on, let me tell you all about DigitalOcean. So DigitalOcean is a big supporter of the show. Our infrastructure runs on DigitalOcean as well. So we're both, they're sponsors, and we're both happy customers at the same time. So you can go from zero to a server up and running in 60 seconds, probably more like 30, 35 seconds. It's super easy. Just go pick your Linux distribution. If you want it pre-installed with something like Mongo or Ghost, you can check a box and it'll just do that. Up and running, SSH in, and you're off to the races.

Really super reliable, super bulletproof, lots of data centers. Big fan. If you go to pythonbytes.fm slash digitalocean, you can get $100 to get started if you're a new user. So check them out and tell them thanks for supporting the show. Once your server's up and running, go back and read Anthony's article to make sure that it's secure. First thing you do is you patch your Python. That's right. So one thing that I don't use at all that I feel like I probably should learn more about are pre-commit hooks in Git. I'm on the same boat. I want to try to, we're using some things like linters and other tools like PyLint and Flake8, things like

that at work now. But I want to make sure that they're going to get into a use model where they're just used all the time before it gets committed. And so I came across this, I'm not sure who referenced it, but a project called PreCommit, which is built in python but it's not python only it's a it's a tool that can start easily hook up a lot of these git pre-commit hooks for you so you can run things like uh like lint or or black or other things before you check in and uh it's got some nice nice features for it it's a yaml based so it's really easy to read the setup. It does all the hooking up into Git for you. And I'm not sure, I'm guessing everybody individually has to do it once they check out a repo, but I'm not sure about that. I'm just getting into this. But also,

one of the things I wanted to make sure I could do was to every step of my pre-commit, like if I wanted to run black or pylint or something, I could test that out. And it does, it has the ability to just run each individual hook by itself. So this is something fun that I'm looking into. Oh, yeah, that looks really fun. Pre-commit hooks are awesome. Oh, yeah? Are you using them? Yeah, I actually use them pretty heavily in my last project. And they've saved my butt multiple times.

One of my favorite pre-commit hooks looks for debugger statements. Okay. Yeah. So if you have a PDB. Yeah. Like triggering a breakpoint or something like that. Yeah, exactly. Oh, that's a really smart one. Yeah. I guess you could do all sorts of checks, right? Like check for to do or, you know, not implemented all kinds of stuff that the people might put there. Like actually you probably shouldn't check that in. Yeah. And it's nice having kind of global team-wide pre-commit hooks where everyone's on the same page with things like checking for debugger statements, linting, any other sort of, maybe you have a line length rule that you want to follow, and then breaking that down and having individual pre-commit hooks. So maybe checking for to-do and your initials or whatever scratch code that you tend to put in

and don't want to check into the greater project. Yeah, I see the real value of the pre-commit hooks being for team work. Obviously, it's valuable for individuals as well, right? Even individuals don't want to ship a web app that has a breakpoint in it. But you know, it seems like the value is amplified when you have a team and you can all agree upon the way it works. And that just gets automated. Absolutely. Yeah, I'm a big proponent of having a code style document where everyone's on the same page. And where if a team is working on a code base, as they commit code, it looks like one person is doing it. And you can't just point in a chunk and say, oh, Michael wrote that. It's got his style in it. It's got that weird triple list comprehension with an internal dictionary comprehension. We don't do that. Right. Yeah.

Yeah. No, that's awesome. So we have kind of a big, big piece of news theoretically coming today. Right. That's right. Python 3.7 should be released today. So by the time you're listening to this podcast, you'll have it. Fingers crossed. So if you go check out 5.3.7, this is the release schedule for Python 3.7. It says expected on the schedule is 3.7.0 finals, June 27th, 2018. That's today. So assuming everything lines up right, you should be able to go and download Python 3.7. If not, just pause the show for a day or two and come check again.

Yeah, what are some of the features that you're finding really awesome there? So now there's a breakpoint function. No more import pdb colon set trace. There's just a breakpoint function that you can drop in. is tied to the PDB debugger. But there are many other types of debuggers you may want to use in Python. And this breakpoint thing lets you configure your environment to, when you say breakpoint, trigger a breakpoint in that debugger,

which is kind of cool. I personally use and love IPDB. It's kind of more interactive version of PDB. But the limitation in both that I, maybe there's a way, but I never quite figured it out was you can only type in one line statements. You could put in semicolons and a few other things, but you were really kind of limited. You couldn't paste in functions or anything too complex. Now, with the new breakpoint function in Python 3.7, you can just open up an IPDB shell and do whatever you want. That's so exciting.

Yeah, that's really awesome. I think that's great. Certainly makes building better tools a plugin for that in place of PDB. It's good. Yeah, nice. We also have some new board members, right? That's right. Yeah. The PSF board member elections just finished two days ago, and there are now four new board members. I have not practiced saying their last names out loud. So if I butcher them, I apologize. But we have four new members. They are Anna Osiewski, Christopher Nugabauer. That's going to be my final guess. Jeff Triplett and then Katie McLaughlin. Oh, awesome. Yeah, it's great to hear some of those folks in there. Congratulations,

everybody on that. And I know a number of us have voted and it's great to see the community sort of putting in place these structures to keep it vibrant. Yeah, I voted. There was a 47% voter turnout for this election. That's pretty good, really. Yeah. I guess we could go for 100%. But 47, it seems like probably a lot of people don't check their mail and the announcements just go right by, right? Yeah. I'm super excited for them. Yeah, that's cool. Awesome. Congratulations. And I'm really looking forward to installing Python 3.7. That'll be fun. So we talked about Flask. And one of the big, I feel like one of the big blockades that's going to crack loose and really sort of change things in the Python web

space is this async stuff. And there have been a couple of attempts at it. We've had Sanic, we've had Gepronto, we've got Cort, which is basically Flask directly converted to be async awaitable, which can give it really good performance benefits. But there's a new framework that is just going like gangbusters called Vibora, the Vibora web framework. Have either of you heard of this? I have not. No. It's shiny new. So I went to the GitHub repository, and it's only 14 days old, but it already has 21 contributors and over 2000 github stars

whoa that's pretty intense right so it's basically flask like right it's super inspired by flask but it's from scratch re-implementation or implementation of something like a asynchronous version of flask all right so if you want to create like a function that is asynchronous, you just say app.route, you know, decorator app.route, give it the URL, async def index, and then return some kind of response. So really, really nice to create these, you know, async await enabled functions. And that, that has some interesting performance benefits. If you go look at vibora.io, they have some nice graphs. So they've got like Flask running around 30,000 requests per second, Pyramid running 35, Sanic 60, Vibora 150,000 requests per second on the same, doing the same processing on the same infrastructure.

And I suspect the scalability is even better in terms of heavily IO-bound things. Like I talk to a database, I talk to a web service, things like that. This is really impressive. Yeah. It looks like it's got WebSockets built into it also. Yeah, and because of the async stuff,

it's super easy for it to do without blocking and consuming threads and things like that. I think this is great. It's got a bunch of things that were written from scratch for performance considerations and to make sure that async is first class, like schema validation, the template engine, sessions, all sorts of cool stuff. It takes advantage of multiple CPU cores. It uses UV loop.

That's the same thing Sanic is based on and other C speedups. It also has a really interesting thing that I haven't seen in a lot of frameworks called virtual hosts. So in like Flask or Pyramid, I can say the URL is forward slash episode slash seven to get the episode seven or whatever. But in here, you can actually have different domains. So you can have like docs.pythonbytes.fm and episodes.pythonbytes.fm within the same web app. Interesting. Yeah, it's interesting, right? Pretty funky.

And deployment's pretty easy. It actually comes with its own server because WSGI is part of the problem. WSGI, like the foundational server bits in most of these things, is a synchronous interface, and so there's no way to squeeze async in between it. So it's pretty cool. Anyway, the docs need a little help, but it's only 14 days old. So I guess we should give them a little slack. Yeah, I just added a new star to their tally.

Oh, awesome. Yeah, they've just got a few more. It's cool though, right? I mean, it's very much like Flask, but it's kind of a modernized version. I was going to say, I was wondering about the name Vibora, but they explained that on their GitHub page. It means viper in portuguese oh viper okay i guess vipers are fast i don't know i mean they can strike fast i don't know about their actual

motion by five four okay cool and you forgot to mention the best part on their uh page the benchmark title has a rocket next to it and i'm a fan of anything with rockets. I know. A testing rocket for sure. People who are doing web stuff, this is a new one to keep your eye on in terms of the shiny new frameworks for Python. Oh, another interesting thing is this is a

Python, not just a Python 3 only framework. It's Python 3.6 or above. None of that legacy stuff. I'm on board with that. That's pretty sweet. Yeah, that's bold of that legacy stuff. I'm on board with that. Yeah, that's bold. That's awesome. All right. Well, that's it for our items this week. I did want to give a quick shout out to one other thing because Brian, we kind of, have we talked about GUIs on the show yet? I think so. I think we did. Maybe it was a while, a couple of times. So we've gone around and around. So one of the major things coming for Python

GUI space is Qt, the new version, the Qt for Python, that's like PySide 2 reborn. So they have a webinar coming up where they're introducing all the features and stuff as it launches. I think that's in August. And so I put the link to sign up for the webinar if you want. Yeah, I'm already signed up. Yeah, me too. I'll see you there. That's awesome. Yeah, Nina, you got anything else going on you want to tell people about? Yeah, if you are going to be doing the Flask mega tutorial, and you're not super happy with your IDE and you haven't checked out VS Code yet, there's a Python extension for VS Code that makes working in Flask really easy. And full disclosure, I work for Microsoft, but I also use it and enjoy it. So if you want to check it out, I can include a link in the show notes.

Yeah, sure. Drop it in there. I use VS Code periodically when I'm working on like individual files. And I know it does more than that. But, you know, I use PyCharm for some stuff and then definitely VS Code. And it is getting much better. It's getting like the auto-completion and. And it is getting much better. It's getting like the auto-completion and all sorts of the linting. It's getting quite nice.

If you think that stuff is nice, we have some really exciting announcements coming up just before EuroPython. Okay, awesome. You'll have to shoot us a note and we'll talk about them. It's something that no one else has. And that's about as much as I can say right now. Oh, that sounds exciting. I think I can guess what it's going to be, but I'll, I won't speculate. I'll let you do the announcement. Awesome. And then anything else? That's it for me. All right. Well, that's a good one. Brian, you? No, it's good. Although I do have a, it's a lot out. I've got a whole bunch of interviews stacked up. I'm finally getting some time to do some editing and pushing out for testing code.

Nina's one of the people. I've got an interview with Nina that who knows when it'll come out and maybe in a few weeks. Maybe if you all email and tweet at Brian, he can be to chop the audio up

a little sooner. Or better yet, write a bot that every 30 minutes and be inspired to chop the audio up a little sooner. There you go. Or better yet, write a bot that every 30 minutes just tweets at Brian. Yeah. Sorry, Brian. Can we call it the spark bot? Did I say that out loud?

It just tweets the number of days. No, just kidding. I'm going to stop right now. Days since last episode. That was what I was thinking. That's kidding. I'm going to stop right now. Days since last episode. That was what I was thinking. That's right. I'm a little bit slow on this week on mine as well. So I'm not going to throw stones. Okay. All right. Well, Nina, thank you so much for being on the show. It's been a pleasure to talk with you. And Brian, thanks as always. Thank you. Thank you both so much.

Yeah. Bye. Bye. Thank you for listening to Python Bites. Follow the show on Twitter via at Python Bites. That's Python Bites as in B-Y-T-E-S. And get the full show notes at PythonBites.fm. If you have a news item you want featured, just visit PythonBites.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Ocken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.