Python Bytes - #13 Python making the move to GitHub and Dropbox is stepping back from Pyston

Episode Date: February 14, 2017

See the full show notes for this episode on the website at pythonbytes.fm/13

This is Python Bytes, Python headlines and news delivered directly to your earbud. It's episode 13 on Monday the 13th, oh my god, recorded February 13th, 2017. Hey Brian, I want to start by saying thanks and hello. I want to say thanks to a new sponsor of the show. We have a new sponsor, that's wonderful. We do, and I want to say thank you to Metis. They're a data science training company. They have some boot camp type stuff that's really excellent.

And I'll tell everyone about that later. But I kind of like to learn about alternate Python implementations and where they are these days. We had heard about Piston, which is spelled P-Y-S-T-O-N, developed by Dropbox. And an announcement came out at the end of last month saying that Piston is no longer to be supported by Dropbox and an announcement came out at the end of last month saying that Piston is no longer to be supported by Dropbox. Yeah. So this is a really interesting development. It's a little unfortunate. I don't really know how to interpret it as a positive or negative. I mean, it could be just, there was an experiment, we're moving on to the next experiment, but what it was is an alternate version implementation.

I have a hard time calling it an interpreter when it's actually JIT compiled, but alternate interpreter for Python. And the idea was it supported Python 2.7 and it did JIT compilation instead of interpreting. So its main goal was to be a very fast but highly compatible alternate implementation of Python 2.7. Yeah, and so there's a couple of blog posts, one the general flow of it is that the cost and benefit, it cost more than they were hoping, and it benefited them not as much as they were hoping. Yeah, so we look at the different implementations of Python.

We've got, obviously, CPython and its versions, but we've got Piston. We've got Unladen Swallow. We've got Piston, we've got Unladen Swallow, we've got Cython, we've got PyPy, PyPy, IronPython, Githon. There's a bunch of different ones, and they all have these trade-offs. And so they were actually able to make Piston pretty quick in the benchmark story. So they got it up to 95% faster than CPython 2.7. But they also ran into other drawbacks, and they found actually when it ran the Dropbox code itself,

it was only 10% faster, but it put a lot more memory pressure on it because it does GC instead of reference counting. I've got mixed feelings about it. I like that the project was out in the open, open source, and people could watch this experiment. I wasn't really thrilled that they were targeting 2.7 to start with. The outcome of them just, I guess, not working on it anymore.

It totally makes sense, but it is, I guess, it's just a thing. It's interesting. Yeah, I think it's a case of a couple of things. I mean, nobody said it, but I feel like, you know, moving to Python 3 is also causing like a sort of a rethinking of this as well. Like if they're going to be moving to Python 3, then they'd have to redo this work and so on. One of the problems or criticisms, mild criticisms, let's say, was that this was like a from scratch

thing. It didn't build on existing stuff. So it didn't build on PyPy because Py pypi wants to be fast at the cost of compatibility and to run a multi-million line python application that is dropbox compatibility is like super super critical so they're like well we can't have any incompatibility but it wasn't directly based on c python even though it took a bunch of pieces in and so i don't know it was like this sort of other experiment where i the thing that gives me the most hope right now is what Brett Cannon and Dino Veland are working on with Pigeon,

P-Y-J-I-O-N at Microsoft, where they're trying to add like JIT plugin capabilities to the existing CPython. Yeah. There is some more CPython news. There's big CPython news. CPython has just as of three days ago moved. It used to be long, long ago. In 2006, it moved to Subversion, SVN. And from 2006 to 2011,

it was on Subversion. Then it moved to Mercurial, which is kind of interesting because that's when Git came out and Mercurial, all this whole distributed version control system story kind of happened around then. And Mercurial is interesting because it's written in Python. Like, for example, Facebook does some really interesting things with Mercurial. And so it seemed like a natural home for Python to live on Mercurial. But, you know, GitHub has really been the place where people go to do

open source, not just a container for open source stuff like SourceForge used to be or something, but there's like a whole social ecosystem where everybody comes and collaborates in GitHub. And so the big news is as of Friday, Python, CPython and a few of its pieces have moved to GitHub. I think that's great. And I think that one of the great things about it is the culture around GitHub. Whether you're a pro-GitHub person or not, that's where a lot of people look for opportunities to help out in open source projects. It's a higher incentive for people to get involved with helping to maintain CPython. Yeah, absolutely.

Yeah, I think that's actually the main motivation, which is really interesting. So there's a link to the mailing list where they talk about it. We'll put that in the show notes. There's obviously the link to the repository in GitHub. There's a Reddit thread where people are talking about this. And a lot of people are bemoaning the fact

that it's not on something totally open source. It's not on something that is based on Python. Like it used to be on a source control system built on top of Python. And while that's cool, like you shouldn't, my opinion is you shouldn't do that at the cost of people loving to work on the project. So the real thing that you want to read to get the deep insight on this is at snarky.ca. So this is Brett Cannon's blog, and it's the history behind the decision to move Python to GitHub. And he talks about how he worked with a bunch of people to come to this decision. And he says, you know, back in 2007, it became really

obvious that working on the source code and doing code reviews and collaboration was actually a burden. There were people submitting, you know, patches and stuff, and they would sit there for maybe years before somebody actually reviewed it. Like how excited would you be to work on a project to contribute to open source if you submit a patch and it doesn't get looked at for a year, right? And so the workflow on GitHub is just so much easier for pull requests and reviewing it. And so basically, Brett said, I want it to be so easy that I could take my tablet and sit on the beach where I have some kind of internet connectivity and review pull requests

and chill out. That's just not how it works now. You've got to have a machine with SSH keys and all sorts of stuff. So the main driver behind this is trying to make it easier for core developers to be more responsive to people who contribute to the project, which I think is super positive. Yeah, I think I'm really excited about it.

And actually, I had never really considered contributing myself, but I definitely consider it more now. Yeah, I definitely consider it more accessible now. And I think this is absolutely the right move, even if it's breaking some conventions, like it's moving off of Python as the thing that supports it. It's moving off a fully, fully open source thing to like this somewhat closed thing, which is GitHub. But still, I think it's great. So remember last time when we talked about the top 10 Python articles of 2016?

Yes. Yeah. So I said, well, let's just take a survey of the kind of category of these things. And I would say like a full two thirds or more of those were data science. Data science is definitely important in Python. So I want to tell you about our sponsor, Metis. So Metis is a data science training company based in New York, San Francisco, Chicago, and Seattle. And they provide full-time immersive boot camps and evening courses that you can come. So if you have a full-time job, you can go like two nights a week for six weeks and take classes on all sorts of data science things, machine learning and things like that. If you are wanting to learn more about data science, especially in a hands-on way, you live in New

York or San Francisco, or you're willing to spend some time there, check out thisismetis.com slash talkpython. And maybe we'll feature your article next year. Yeah, that actually sounds pretty interesting. Yeah, I would love to go do more data science. So TensorFlow, they've got classes on that, all sorts of cool stuff. So check it out. This is metas.com slash talk Python. All right. You know what they do a lot of in data science, a lot of functional programming, a lot of expressions, things like that, right? Yeah.

Actually, I don't know. I'm not a data science person. The ones I've hung around seem to. There's an article called Using Functional Programming in Python Like a Boss, Generators, Iterators, and Decorators. And these are generators and iterators. Okay, all three of those generators, iterators, and decorators are things that trip up intermediate Python people. Beginner people just ignore them for now. But this article is a Jupyter notebook, which I like this trend of

people just publishing Jupyter notebooks as articles it actually is good and there's some some good diagrams in here although the use of really large images it's kind of annoying to me but it's okay anyway the the this article talks about it also talks about just basically what a function is and what things can act like functions and then talks about, it also talks about just basically what a function is and what things can act like functions. And then talks about generators, iterators, and containers and how they all work together to make your code more expressive. And it seems like kind of an esoteric sort of thing to think about, but I think it's good. And this is a good, I like this article about thinking about this stuff. Yeah, I really think this is one of those things that when you come to Python from other

languages, it takes you a while to get into these, right? Like the concept of a generator is quite different to somebody coming from C or I think Java. I don't think Java has this. Things like that. But once you get the zen of them, they're amazing. And they make such a big difference. But it's really a nice mental shift. Yeah, so I really think it's great that he's covering it. Also one of those topics that I don't think I remember when I finally grokked what people were talking about.

But it took like three or four different articles talking about it for it to sink in for me. So here's another one. Yeah, excellent. Yeah, so we'll link to that one. And by some strange coincidence, just so everybody knows, the way that we pick things for this show is I go find a couple of interesting things. Brian goes and finds a couple of interesting things. We throw them together and then we just talk about them. And it happens to be the very next one is the same person in a Jupiter notebook, but it's actually a totally

different topic, but it's presented in a similar way. So this one is called it's metaclasses all the way down. This is a quick, but fairly deep dive into what metaprogramming is. And it starts out by saying, look, metaprogramming is a technique for writing computer programs that can treat themselves as data. So you can introspect, generate and or modify them while they're running. And that's definitely Python, you can at runtime, create types, you can add functions or even remove features from classes and objects, right? But mostly metaprogramming comes down to metaclasses where you can actually control what the class does. So there's all sorts of interesting examples of how you might use them. Starts out with an interesting quote saying, look, metaclasses are deeper magic than 99% of the users should ever worry about.

If you wonder whether you need them, you don't. So if you don't, like that was a quote by Tim Peters. If you don't deeply understand like, yeah, this is totally, I'm going to solve this with a metaclass. Like you probably shouldn't mess with it. But they are the power that makes a lot of the magic inside Python happen. So a couple of examples. One is Django ORM. We could substitute SQLAlchemy ORM or PWE ORM models, right? So when you define like a model

in Django or SQL alchemy, you put what looks like column types with database constraints as the fields of the class. But then when you interact with them at runtime, they look like strings and integers and dates, right? And that transition, that sort of two face behavior of these things that they can be used in a database query in one way, but then act as normal objects as others. That's metaclasses. Another example, there's a cool example about using them to make abstract classes and abstract functions that have to be implemented by things that inherit it and so on. Again, this is a nice Jupyter notebook. It has some cool pictures, but it really does help you understand this stuff pretty well with nice examples. Yeah. Yeah. I like this too.

It's funny that we both picked the same person's article. It's a small world. Yeah. I think we're kind of going on the stuff that most people don't need train right now. But next up is lambdas. And speaking of another thing that I think newbies get a little confused by, but there are lambda functions. And really, lambda functions are just unnamed functions that you can put the entire expression on one line. And so they're convenient in a lot of places. So here's an article from Dan Bader titled Lambda Functions in Python.

What are they good for? And what I like about this article is it's just a fairly quick jump into it. I think everybody that thinks of themselves as a little bit more than just a beginner in Python should at least understand Lambda functions. And the thing I really like about this article is it talks about places to not use them. Because when you finally get your head around lambdas all of a sudden it's your hammer and everything looks like a nail he talks about a couple places like um creating class methods and uh other places where that maybe that you shouldn't use a lambda for that you can but don't and then also um there's some places where list comprehensions and dictionary comprehensions

are wonderful and they're way more expressive, but you could use lambdas in those cases. Yeah, absolutely. I think the trick with lambdas that make them so wonderful is like, there's little places in your code. You're like, if I could just pass a behavior here, I don't want to go out of where I am and write a function and then like give it a name and then say the name here. I just want to go, yeah, sort by this reversed or, you know, grab this value out and transform it or something. And Lambda functions are beautiful for that. They're not so beautiful for writing your whole program. Yeah, actually sort is one of those, one of those places where one of the parameters is you can pass in a key or how to find the key. And I've used lambdas a lot there because I don't really need a named function.

I just need a function to pass in and do the work. Yeah, I've got a list of customers. I want to sort by last name. So lambda, C goes to C dot, last name, done. Beautiful. Yeah, it's great. So you wanted to do a quick follow-up on japan though that's our super super fast web framework right theoretically yep yeah that's what we we were

talking about that last week um but i ran across um a reddit thread um that we'll post in the show notes uh just and it is titled uh sticking with flask versus switching, um, to one of the new faster frameworks. And, uh, the Gepronto author, um, actually posted on this and said, yeah, if you're looking at Gepronto, uh, please read the read me there because I've noted that it's not ready for, uh, production, um, basically. So, uh, there's, he's not even, he's something about, I'll just read it. He's, as stated in the readme, you should not build anything serious with Gibrantha now because it's going to probably eat your laundry. Remember, it's hand-coated in C and this needs a lot of testing. On top of that, I plan to do several iterations of API changes in a largely incompatible

way. I hope, though, it's going to make a serious player in the field one day. I applaud the author for putting that out there. He's basically developing in the open, but he's not promising to stick with the same model or the same API, and I think that's completely fair, and he's pretty open about it. Yeah, that's really great of him, and it's, you know, he seriously is optimizing this stuff in some really interesting, but out there ways. And so, and all that part is done

in C, right? And so it probably needs a little bit of careful validation before it becomes a thing that you really, you know, depend on for your e-commerce site or whatever, but it's definitely, it's got so much promise, that project. So I hope he gets there. Oh, we're already done. So there we go. That's it. That's the news we got for this week. Anything you want to share with everyone while we're wrapping things up, Brian?

No, nothing. Nothing right now. All right. Awesome. Well, how about you? I'm cranking out some classes. I just am pretty much finished up my Consuming Restful Services course, so hang tight for that. It's going to be

out in maybe a week or so. All right. Cool. Fun to be building new things for everyone. All right. Well, thank you everyone for listening. Brian, thanks for chatting with me today. Thank you. Yep. See you all later. Thank you for listening to Python Bytes. Follow the

show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Auchin, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.