Python Bytes - #7 Python 3.6 is out, Sanic is a blazing web framework, and are failing our open source infrastructure?

Episode Date: January 4, 2017

See the full show notes for this episode on the website at pythonbytes.fm/7

This is Python Bytes, Python headlines and news delivered directly to your earbuds. It's episode 7, recorded Wednesday, January 4th, 2017. Hey Brian. Hey, happy new year. Yeah, happy new year. It's great to be back together. We've gone three weeks without discussing cool things that came out in the Python community, and I think it's time to get back together.

The Python news must flow. It must, and plus I miss this. This is fun. Yeah, it's definitely, definitely fun. So like many of our episodes, this one is sponsored by Rollbar. They help you take the pain out of errors. So thank you to Rollbar. We'll have more to say about them throughout the show. So Brian, you had a little warning. You wanted to just get it out of the way. Yeah, I've got, there's a couple, I want to try to list some of the names of the articles, of the people that wrote the articles that we are reading and talking about in this episode. And some of them have names that I am most definitely going to mispronounce.

I hope you don't take offense at it. I have a last name that everybody mispronounces. So I understand. You're used to it? Yeah. I get Oaken and mostly Oaken is what I get. But you're not a tree. So. Hey, so before we get to the first episode,

the first news item, I actually want to do something. I'm going to open up my terminal. I'm going to type, type something, okay? I'm going to type Python 3-V. Do you know what it says? Not Python Python three, five, two, not anymore. It says Python 3.6. That's right. Python 3.6.0. Yeah. So, uh, Python 3.6 was announced on December 23rd and, uh, it's, uh, now Python 3.6.0 is our official latest release, and this is really exciting.

I've already started using some of the new features in my writing. I'm definitely taking advantage of the F strings. I don't know how you would call those. Yeah, I think the other languages call it string interpolation, but it's the F character that indicates it. Swift and C sharp call it string interpolation. Oh, I'm going to call them the FN strings. I love it. But in celebration of this, I know we've already talked about 3.6, but I've got links to the, in our show notes for the

announcement and the what's new article from docspython.org. But there's also a couple other articles that are cool there's a python 3.6 is packed with goodness article from serdar yagelalp i know i got that one wrong that's on info world but it's a list of uh things that he thinks is cool in 3.6 including async await and use the use the use of that more uh the improvements of memory and speed, and I'm really excited about that. And something that I didn't know about is some of the improved API support so that tracers and debuggers and stuff can hook into Python easier. Yeah, it's definitely packed with goodness. I think that's great. Another article that you pointed out was Adopt Python 3. I really like that one as well. And I'll let you go to some of the stats and say the name as well, the author.

Yeah. So this is on Medium. And I'm going to try this one because it's a cool name, Dibya Chakravarti. I don't know. Cool name. But yeah, so this this other adopt Python three article talk, I thought it was interesting. So there's a, I have heard of a couple of sites called the wall of superpowers and Python three readiness that are two, two pages that list most popular Python packages and how much of them are Python 3 ready. And the numbers are higher than I expected, actually. 187 out of 200 on one of them and 341 out of 360 on the other. And I think this is important because a lot of people tried to switch to Python 3 a long time ago and one of their favorite packages wasn't supported. But it probably is now. So check those out. And then this person also wrote up an IPython notebook and had some statistics to pull out. Not just the most popular, but everything. He pulled everything on PyPy that was listed as stable and had at least one release in 2016.

That was 6,000 packages of those. So there must be a lot of packages on PyPi that are not, or PyPI, that are not updated very often. Yeah, I really like it when the statistics do some basic filtering. Like, nobody cares whether that package that almost no one's using hasn't been updated for 10 years supports Python 2 or Python 3. Yeah. Yeah. But of the, of, of the, what he calls stable and active, um, I was interesting. There's 14% of those don't even cover, uh, Python 2, their Python 3 only packages and a 28%, um%, which is higher than I would hope, but 28% are Python 2 only.

But the author estimates that a lot of those could easily be switched if they're really important to you. But he does list out, he has a list of all the ones that he calls sticky. The one 25% of 28. So I don't know. That's like, that's not very many, 14, 7% total. Yeah, math on the fly. Anyway, of those sticky packages that may be a problem for you. But you know, I was in one of those camps a year ago, I tried to switch our testing to Python 3. And there was a package that I needed that wasn't switched. And it is now so I don't have that excuse anymore. Yeah, that's really great. I love the wall of superpowers. This article is really interesting. And I think it makes it's a

good argument. And it's data driven, which is cool. In fact, like you said, you can get the Python notebook for it. My final thought on Python moving from legacy Python to Python three, six, the new Python is you get a lot of benefit in this release just by installing and running on it. If you don't touch your code at all, cause there's a lot of memory and performance speed up. So that's really cool. Yeah. Um, and it's the, instead of walls of, and the wall of superpowers, um, maybe we could look at maybe roads and bridges. Yeah, there's walls, there's roads, and there's bridges. And, you know, I don't know how it is in the rest of the world. I know in Germany, it's not quite this way. But in the US, we tend to

neglect our physical infrastructure, right? Like, I had to swerve the other day because there's a huge pothole on the road and so on. And, you know, it's a problem. But I think the real problem for us these days is actual digital infrastructure. So there's a really great article, not an article, it's more like a report or even a book, it's like 150 pages, called Roads and Bridges, the Unseen Labor Behind Our Digital Infrastructure. It's written by Nadia Egbal. And she did a really good job. It's a new, relatively new report. It came out in July. It basically talks about open source and mostly open source infrastructure, not open source

projects. So not necessarily things like Django, but more things like PyPI and PIP, right? The underlying foundation, things like OpenSSL that everybody builds on top of. And it turns out there's actually a really big problem across the board. Like if you think physical infrastructure

is neglected, you should see the digital infrastructure. All right. So I actually did a whole session with this. Some of the folks from the Python community, Eric Holscher from Read the Docs, Donald Stuff from PyPI on Talk Python in episode 84. And we talked about this a little bit. Basically, this report goes through and outlines all the challenges that they see for supporting things like PyPI,

supporting things like OpenSSL. And they said that until Heartbleed, you know, Heartbleed was that really serious bug that you could sort of send a bunch of requests to a server running OpenSSL that was vulnerable, and you could actually get it to leak information, unencrypted information back out. So it's still until then, it was like, that thing was just barely coming along. Like there's almost nobody that worked on it. There's like one guy, or a couple of guys that very, very short term were able to work on it. And two thirds of all the web servers use that for their communication. And think about that. Another example, they talked about Ruby gems, the website done by volunteers, somebody had found some kind of vulnerability in it. And the people who had supported it, they're like, all right, we'll fix this on the weekend

because we have jobs. We can't work on this infrastructure. And like, I mean, that's like pip to Ruby, right? It's really important. And it turned out it got hacked and taken down before the weekend. And so people had to like schedule vacation days so they could get it back running.

One more example is PyPI itself. And, you know, Donald Stuffed works works on that he was employed by hp hp went through a bunch of layoffs and not even really knowingly i think took out his position now there's donald's doing that in spare time who else supports pi pi they do 400 terabytes of traffic a month and it costs almost $40,000 to keep pip running. So you can say pip install a thing, that's like $40,000 worth of infrastructure. And there's only a few baskets that are supporting that thing. It's quite precarious. So I wanted to bring up roads and bridges, the unseen labor behind our digital infrastructure. This report, I think people care about this. We'll find this very well written, very interesting, and lots of neat stories. Well, even if they're depressing. Yeah, it is interesting. Thanks for

bringing that up. Yeah, absolutely. I threw a bunch of little excerpts that I got from the article. People can flip through and see if they're interested in that. Speaking of things that we take for granted, I know that there's some libraries out there that people use all the time. And if a library is used by a lot of people making changes to it can be a little scary. RC2 of, uh, map plot lib 2.0.0 came out in December and Thomas Caswell is, I don't know if he's the only person supporting it, but he's one of the main developers. And, uh, he was telling me that there's, it's been, uh, one of the changes in this is, um, some of the changes in the default style. So the default, if you just, um, if you're not styling your plots, uh, specifically, and you're just going with whatever Matt plot lib, uh, comes up with those are going to change.

And, uh, that took 10 years in the making, apparently, to change that. So this library has been around for a while. Anyway, so he would love to have feedback. So we've got a list of, in our show notes, there's a list of changes. There's a list of releases, list of changes, and 2.0.0 is coming out soon-ish. He's not sure when, but he'd like to have people try it out and see if any of the default styles or what they think of them. He wants feedback.

Well, I have a little feedback for him. One of the posts you linked to in the show notes is basically a set of pictures, old version, new version with default styling, almost universally. They look better to me. They look more modern. They don't look like here's a cheap unstyled sort of thing. I'm just going to drop in. They actually look like something that you'd be really excited to share, right? So thumbs up for

me. Yeah, I thought so too. And I thought I didn't even think about how hard it would be to change something under the, you know, change the rug out of a lot of people. And, you know, a lot of people don't follow him or follow Python. So I'm glad we're bringing this up to try to get people to go check it out. Yeah, I totally agree. So it's easy to look through it. If you care about Matplotlib, get out there and give them some feedback while you can. Hey, before we move on, let me tell you about Rollbar.

Oh, please tell me about Rollbar. I recently sent out an email, like a newsletter email to a bunch of people taking some of my courses and it turned out that i had typed in a url incorrectly like so it was like click here to launch this this video it was like player slash lecture slash one two three four well some like non-numerical number or a letter got in there my web app was trying to parse that as an integer. And so I hit send on this email to like thousands of people. And then all of a sudden my phone starts going beep, beep, beep, beep, beep, crash, crash, crash, error in your website.

And I'm like, what the heck is this? I pull it up and like, I immediately see what's going on. I see the stack trace. I open up my, my open up PyCharm, quickly change it, do a push to deploy. And within five minutes of sending the email, nobody ever sent me a message. I found the error and fixed it, and the rest of the people that opened the email didn't crash. And that was from Rollbar. And if you guys want that same type of behavior, that same type of safety net, you can get it at rollbar.com slash pythonbytes.

That's awesome. I wonder how long you would have figured out on their own without that. I would have never known until somebody complained. So it was really good. I felt bad about it, but it was good that at least I got it fixed right away. Yeah, awesome. One of the things I'm a huge fan of is MongoDB and document databases. I really think document databases are like this perfect sweet spot between performance, maintainability, ease of use, and relational integrity. There's a really great article I want to point out. If you're interested in MongoDB or document databases in general and Python and you want to get started,

the guys over at RealPython, Michael Herman, wrote an article called Introduction to MongoDB and Python. They talk about what's the difference between SQL and NoSQL. They talk about the primary way to talk to the database through PyMongo, like inserting and querying data. And they also talk about this thing called Mongo Engine. Mongo Engine's like the SQL alchemy. You create classes, you put in constraints and defaults, and then you map those classes to documents in your database.

Yeah, I'm really excited about this. I'm trying to use PyMongo on one of my projects. So this is good. Yeah, it's definitely good. PyMongo is pretty sweet. It's MongoDB is easy to set up and use. It's open source. So that's all good.

I actually did a one and a half hour presentation at this conference in London called Software Architect. And they recorded it and put it up. And it was basically the same stuff. So I put a link in the show notes as well to this thing I call Applied NoSQL with MongoDB and Python. So if people want a video version of mostly the same topics, they can check that out as well. And actually from you, I learned about TinyDB, I think, on one of your podcasts.

Oh, yeah. TinyDB is awesome. Yeah. So that was neat too. Yeah. Also Document Database. You know, I love things that are for humans. Yeah.

Next up, we've got an article introducing Maya, Date Times for Humans. And most of the for humans stuff comes from Kenneth Reitz. Do you know how to pronounce his name? I've been saying Reitz. I think that's right. It felt like a German reading, but if it's Reitz, I'm sorry, Kenneth. So Kenneth, if you're listening, please let us know. Or I could, I guess, look at any talk that he's given and try to get that. But the date times seem like something that's just easy until you actually try to do something that's not. It quickly goes from easy to deal with to hard to deal with very fast. And so I'm excited that Kenneth has taken this on and tried to come up with his own library for dealing with date times and date time math and parsing dates and human readable stuff.

It just sounds exciting, and I'm going to give it a try. Yeah, he has Request, which is the most popular Python try. Yeah. He has, um, request, which is the most popular Python package ever downloaded 7 million times a month, which is insane. He has records, which is like just right. Sequel for human sort of thing. And now dates are very, very cool stuff. Nice work, Kenneth. Yeah, definitely. And another package that I'm interested in is, um, that's coming up as an alternative to Flask or something Flask-like. There are more and more reasons to move away from legacy Python to Python,

especially Python 3.5 and above because of the cool concurrency stuff. One of the things that people talk about when they talk about Node.js, which I'm not a huge fan of Node.js, but it does have its benefits, is it's sort of asynchronous from the start programming model.

Right. Okay. Now that's, that's been traditionally hard in Python because the WSGI API doesn't really support it super well. Frameworks like Flask and Pyramid, they don't, you can't just write async methods that all of a sudden become async, right? There's like too much, it's too much in the built in stack that doesn't understand async. This guy, I don't know what his real name is. I can just know him as Channel Cat, created this thing called Sanic, which is called a Python 3.5 or higher web server written to go fast. And it's based on the API Flask. It leverages the async and await functionality of Python 3.5, which I think is really, really cool. It also uses the UV loop, which is a really fast asynchronous IO loop for Python 3. It's really cool.

So if I want to create some kind of method, like a view action method sort of thing, just like Flask, I'll say app.route and give it like a URL, so like slash. But instead of saying def some method, you say async def some method. And you can await throughout your method and all the cool benefits of letting the threads go on to do other work while you're waiting on databases, web services, files, things like that just happen magically. So you can get really good performance numbers. Wow. Neat. Yeah. So it's pretty cool. And the fact that it's a Flask, more or less like the Flask API means you

don't have to like start from scratch and learn a totally new thing. They actually have some performance numbers compared to other things, and they don't have Pyramid in here, which is too bad because Pyramid flies on Python 3. But when you think of asynchronous programming in Python, a Python web framework, typically you would probably come up first with Tornado. And they have some tests they did,

and they said, okay, Tornado did like 2000 requests per second with a 44 second millisecond latency response time, basically. I said, okay, well, what about Flask? Well, Flask did twice as good. So like 5000 requests a second, 20 millisecond response time, almost double good in both directions. The Sanic thing using Python 3 in the UV loop is 33,000 requests per second with a 3 millisecond response time. Wow, that's incredible. Yeah. See, Python's better than legacy Python.

Yeah, Python, definitely. That's really cool. So Python 3.5, the async await is a cool programming model and have this nice web framework to plug in there. This thing's going to go places. We're going to be talking about it for a while, I bet. Yeah, definitely. At some point, I need to talk with you about trying to figure out if I were going to start up a new application, would I care about asynchronous stuff right away? Yeah, that's a good question, right? Like how much do you really care? Because I mean, let's say on the TalkPython website, I think I have 10 processes farmed out to handle the request.

So let's just go with Flask. If I were running Flask, 5,000 requests a second. That's 50,000 requests a second. That's like a pretty insane amount that I'm going to have to hit before I care, right? But still, there are people who care and having that built in, that's pretty cool. Yeah, definitely. I can't believe we're wrapped it up. We've got all these done already.

I know, I know. I think we've got a bunch of cool topics. I love Sanic. I love that Python 3.6 is out. And I'm happy to be back and sharing it with everybody. It was a nice break, but it's even nicer to be back. Yeah, do you have any extra news you want to share with us? No, I just stepped away from the computer.

I went to California, put my toes in the sand, I just stepped away from the computer. I went to California, put my toes in the sand and just kind of forgot about things for a while and came back refreshed and excited to keep doing what I'm doing. How about you? You're still working on your book, right? Definitely. And I've ramped up the schedule. So I spent the Christmas break at the keyboard and I've got, I'm, I think about halfway done with the rough draft approximately. But going through the first level editor, I'm still shooting for at least a beta release by PyCon. My goal of getting a physical copy by PyCon seems actually insane right now. But that's where I'm up to. You can do it. We're all rooting for you, Brian. You can do it.

And a whole bunch of test and code episodes coming up. I've got one interview recorded and about four interviews scheduled. So these will keep going. I have not quit. All right. Awesome. Glad to hear those are coming. Yeah. All right. Well, thanks a lot for talking to me today. Yeah, it's been great. Thanks for sharing the news with me and we'll see you later. Thanks. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes.

That's Python Bytes as in B-Y-T-E-S. And get the full show notes at PythonBytes.fm. If you have a news item you want featured, just visit PythonBytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Ocken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.