Python Bytes - #22 PYTHONPATH considered harmful

Episode Date: April 18, 2017

See the full show notes for this episode on the website at pythonbytes.fm/22...

Transcript
Discussion (0)
Starting point is 00:00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. It's April 18th, 2017. I'm Michael Kennedy. And I'm Brian Ocken. And we have a bunch of cool stuff to share with you. But before we do, Brian, we have to say thank you to Advanced Digital. Oh, yeah. Thank you, Advanced Digital. They're a brand new sponsor of the show, and they have some really cool Python job opportunities. We'll talk about that later. Let's talk about the Python path now. I liked this.
Starting point is 00:00:27 Somebody came out with a, I should write names down. Let me look it up. Just a second. Okay. Moshe Zedka came out with an article called Python Path Considered Harmful. And this is a good article. It's very short. And I guess I had forgotten about the Python path because fairly early in my Python
Starting point is 00:00:46 career, realized it's not useful. It's only bad. Ignore that it's there. So what the Python path is, it's an environmental variable that you can put directories in so that you add to the places where Python will look for import modules or modules or packages to import. Every time I've ever thought maybe I had a good use for it, I've regretted it later. And one of the classic examples is that if you had a package, you would, I guess, maybe want to add the top directory for a local package to the path,
Starting point is 00:01:24 but there's a whole bunch of stuff in the top directory that's not package to the path, but there's a whole bunch of stuff in the top directory that's not appropriate to be imported, like the setup.py and so forth. But anyway, just I guess a public service announcement, don't use Python path. Is there a good use for it? Yeah, I don't know. I mean, we have virtual directories. Those work pretty well. You can Python and, you know, set a PY, develop if you want to kind of inject something into the path. Like, there's a lot of options that don't involve that necessarily. Yeah. And also, if you're, like, one of the reasons might be if you're developing a local package and you want to be able to import from it, like from your test directory. But that's still the best recommendation is to do a pip install dash e as install editable is the best way to do that.
Starting point is 00:02:13 Yep, exactly, exactly. So the first one I want to talk about are actually algorithms. There's a really interesting article about how I think we talked about this one and I'm not sure if I'm blending in my mind with another article I read, but basically that we can use languages like C and stuff to try to make our code faster by trying to get closer to the metal. But a lot of times a way to make your code faster is actually to have better understanding and visibility of the algorithms, right? And so you could try to make the bubble sort really, really fast in C, or you could just use Quicksort in pretty much any language, right?
Starting point is 00:02:51 And it's way, way faster. So I ran across this GitHub repo that is just a bunch of minimal examples of data structures and algorithms in Python. And it's keon, you know, github.com slash keon, K-E-O-N slash algorithms. And there's just tons of really cool examples. So there's, there are different topics like arrays and graphs and trees. So like in the arrays, there might be like the ability to flatten an array or to merge intervals on arrays or in graphs, you can clone it or traverse it or find paths through it. All those types of algorithms that are like, wow, that actually solves exactly what I need.
Starting point is 00:03:27 And here's a little example. I thought it was really cool. And I think it helps professional programmers. But if you're looking for a job, there's a lot of sort of interview type questions around algorithms and data structure. So it might also be worth studying there. That'd be a good place. Yeah. Yeah.
Starting point is 00:03:43 So I'm not sure how many there are, maybe 50 data structures and algorithms, but pretty cool. That's neat. Cool. Yeah. So if you want to brush up on your algorithms, then there's a place to do it. Well, speaking of data structures in episode 11, we talked about a package called Adders, A-T-T-R-S. Adders has a lot of fans. People love that library. Yeah. It's a cool package to be able to make classes easily and just the attributes of classes are easy to define. Right. Kind of make them complete as well, right? Give them the equivalency tests and all the various things, not just, oh, here are the fields, right? Right. And this is an older article from last year. I guess it's not that old. It's from Glyph, and it's just a really good
Starting point is 00:04:26 article about, the article is titled, The One Python Library Everyone Needs, and he talks about the annoying bit of having to create your own classes and make sure that the copies and less than all work and the sorting and just letting that actors work for you. He also discusses briefly some problems with named tuple, but I'm a huge fan of named tuple still. And I don't know if I buy is slamming on it too much, but it's been all good for me, I think, but no,
Starting point is 00:04:58 this is really cool. And I, I need to be trying out adders. I haven't done it yet, but it seems like, you know, it really does add a lot. I should just make it a habit. Anyway, it's good article. You know where I would like to try trying out adders. I haven't done it yet, but it seems like it really does add a lot. I should just make it a habit.
Starting point is 00:05:07 Anyway, it's a good article. You know where I would like to try out some new stuff? Where? At a new job, building cool stuff with Python. I would love that. Yeah, so our sponsor, Advanced Digital, they sponsored this episode of the podcast because they're looking to work with you.
Starting point is 00:05:21 Everyone out there who knows Python and wants to build cool stuff with Python. So Advanced Digital, I've never heard of them, but they actually run one of the 10 largest news sites in the US by traffic. So they're a really high scale Python shop. They're located in Jersey City, just across the river from Manhattan. So beautiful view, see the Manhattan at night and take the path over there. They fund employee development and conferences. So, you know, beautiful view, see the Manhattan at night and take the path over there. They fund employee development and conferences. So you go to PyCon, things like that. And they do mostly Python, but they also run other things. So if you want to work in an
Starting point is 00:05:55 environment like that and do Python for your job, check out python.advance.net and those guys will hook you up. That's great it sounds fun right cool so one of the things that makes high traffic websites run well is concurrency right and we we kind of beat this drum often but because it's an awesome drum and last week i had david beasley on talk by the enemy to talk about a project he created called curio you heard of curio i have i haven't played with the though so curio is it's an interesting project in that it's kind of like halfway between about a project he created called Curio. Have you heard of Curio? I have. I haven't played with it, though. So Curio is an interesting project in that it's kind of like halfway between a framework
Starting point is 00:06:29 that you can just grab and use and half, like, really low-level building blocks. But the idea is we have async I.O. in Python Sense 3.4, and it's got this event loop and allows a sort of asynchronous programming through callbacks but david said well what if we actually started from scratch and we had async in a weight and available we had these asynchronous coroutines as the primary concept and an async library for python what would that look like well that'sio. So it's a library for performing concurrent
Starting point is 00:07:05 IO operations and system programming tasks like launching sub-processes or threads or whatever. And it's solely concerned with the execution of async coroutines. It's cool, right? Yeah, very. And it's got all these really neat data structures. So it has this thing called a universal queue. And queues are one of the primary ways to communicate between threads without locking. So you're not sharing the data. You make a copy, put it on the queue. So you're not worried about race conditions and things like that.
Starting point is 00:07:33 But one of the problems is the threading model that works for, say, queues between threads is not the same that goes between async IO execution. And that's also not the same that goes between Curio. So he added this thing called a universal queue that spans all three of those worlds and lets them intercommunicate with each other. He has async threads for sort of managing computational execution and threads as if they were asynchronous coroutines
Starting point is 00:08:01 and all sorts of stuff like that. And you really learn a lot about async if you dig into this thing. Okay. And you said it is sort of halfway between something low-level and high-level? Yeah, exactly. So if you want to make, say, like a TCP IP game server with TCP or UDP or something, like it's actually got constructs to say launch TCP server that's async and plug in the callbacks and things like that.
Starting point is 00:08:24 Oh, wow. Yeah, but there's no HTTP layer plug in the callbacks and things like that oh wow yeah but there's no there's no http layer right so it's not like a web framework it's not like sanic or japronto or one of those things but it's not super super low level it has all these building blocks so it's like if i was going to build a a framework thing of some sort that did a bunch of asynchronous and i wanted it asynchronous at its core maybe using Curio as the core of your project to build that framework on top of might be perfect, but it's not itself a framework really. Not yet anyway. Okay. Interesting. I'll have to keep an eye on it. Yeah. It's the built-in concepts for tasks and threads and queues and whatnot. It's very, very neat. Neat. Yep. All right. You have some package sorcery.
Starting point is 00:09:06 Yeah, I was playing with the SRC and trying to figure out if this section should be package sorcery or use the source loop or to source or not to source. Anyway. These are all great questions. Well, speaking of great questions,
Starting point is 00:09:22 in episode 15, we talked about Python packaging. And one of the listeners, Christoph, had a question last week. And the question really was that he was reading some other articles that were in conflict. And actually starting with the PyTest documentation and pointing to an article by Yonel, I think it's Yonel, called Packaging in Python Library. And the conflict really is whether or not a distribution package for like, if you're going to push it up onto PyPI, should it have all of the module or the source packages, should they be top levellevel directories or should they be under a
Starting point is 00:10:06 SRC directory? And the recommendation, some people recommend, and PyTest did as well, of using this SRC directory instead. And the argument that Yonel puts forth is referenced in lots of places, and it actually all sounds good. Some of the problems are based on using talks and other testing tools to be able to install things a lot and uninstall easily and not muck with the namespace too much. So I went out and tried to find some examples that used this and there's actually had difficulty finding some. If this is such a good idea, I guess my question is, why isn't this not promoted more by the Python Package Authority and their documentation?
Starting point is 00:10:55 And PyTest itself, even though it recommends this, that package doesn't use it. So I guess that's my question to the Python community is, should we be using a SRC directory or not? Okay. Yeah. Well, this is a good place to point out at the bottom of every episode, we have a discuss section and people come in and they ask questions and give us feedback on the various episodes. And this one came out of one of those, right? Yes. Yeah. This conversation. So if you want to comment on one of these shows, like this is episode 22, pythonbytes.fm slash 22.
Starting point is 00:11:28 Boom. Go to the bottom and pick it up, right? So feel free to jump in. And that's probably on episode 21 where this conversation was happening. Yeah. And I get why a lot of existing projects like requests don't have this in it because they didn't before. But there's a lot of people like Christoph out there that are trying to come up with some new code to share with people,
Starting point is 00:11:50 and they want to do it the right way. And it's a legitimate question is, what is the right way? Yeah, absolutely. Yeah, yeah, great. Great topic. Now, this last one I have is a bit of a downer, but I wanted to cover it
Starting point is 00:12:02 because I think it's an important topic, and we've talked about it a few times and I'm going to say two companies names, but before you think anything negative about these companies, these are the two companies that were funding a thing that nobody else was funding. So thank you to them for doing it. But it turns out that Intel is pulling funding from its OpenStack effort, like a sort of an initiative that it started with Rackspace. So Intel and Rackspace were collaborating on a project called OpenStack Innovation Center.
Starting point is 00:12:31 And that started back in 2015. And it was supposed to be funded through 2018, but actually it got pulled out early, right? There was a lot of good things that got done there. They said the objective was to create the world's largest OpenStack developer cloud and developer enterprise capabilities with OpenStack, and it quickly accomplished the first goal and made a great progress towards the second. So that's all good that that was done. But it turns out there's 30 Rackspace employees who've
Starting point is 00:13:00 been working on this now have two weeks to find another job within the company. And so I guess the story here is just, you know, we all need to be vigilant and careful about how we fund our open source work. Maybe a little bit of diversification, not in terms of what people are doing, but in terms of the companies that we have supporting us. So the more companies contributing smaller amounts, I feel is probably a safer place to be than a few companies contributing huge amounts. Like we have the same problem with PyPA. Yeah, definitely. Or PyPI and the Packaging Authority and all those guys, right?
Starting point is 00:13:33 Like that whole set of projects. Yes. So I don't know. I'm going to try to talk about this a little bit at PyCon this year. I'll try to do an open session or something if I can pull it together. There's not really a good time of the year to try to find another job in two weeks. No. Anyway, sorry for you guys. Definitely not. Yeah, that's a real bummer. And OpenStack is awesome. And this is not, they're pulling their support from OpenStack. This is this initiative that was on top of it,
Starting point is 00:13:58 but still. Okay. So it's not all of OpenStack? It's just one part of it? No, this was an initiative specifically that they were doing, trying to bring people together around OpenStack and some other stuff. Okay. Yeah. All right. Well, that's our news for the week, Brian. You got anything in particular? No, I don't.
Starting point is 00:14:15 Yeah, no worries. We're still awaiting that book release. That's going to be a good day. Well, I'm frantically in the middle of edits, and there's some incredible – I'm finally working through a lot of the feedback I got from people, and I'm just still very humbled by the help I've received by the community. It's great. Yeah, that's awesome. So I have a quick piece of news for everyone. a little time in Europe, Europython at europython.eu. Be sure to check that out and get your tickets
Starting point is 00:14:47 because they've already sold out the early bird tickets and the main tickets are on sale. I've already heard from some people who were hoping to go to PyCon this year in the US. Not going. Tickets are sold out.
Starting point is 00:14:59 So if you're in Europe and you want to go, don't wait. These things sell out and then you'll be sad. Are you going to go, Michael? No, not this year out and then you'll be sad. Are you going to go, Michael? No, not this year. I was supposed to go last year.
Starting point is 00:15:08 I really wanted to, but we were moving back to the U.S. within like a few days of it running. So it just didn't work out. Yeah. Maybe next time. Maybe next time. Yeah. Maybe we could both go. That would be awesome.
Starting point is 00:15:17 All right. Well, Brian, thank you for sharing everything with us. Thank you. Yep. It was fun. See you all later. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes.
Starting point is 00:15:29 That's Python Bytes as in B-Y-T-E-S. And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Ocken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.