Python Bytes - #151 Certified! It works on my machine

Episode Date: October 10, 2019

Topics covered in this episode: Python alternative to Docker How to support open-source software and stay sane MATLAB vs Python: Why and How to Make the Switch Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/151

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 151, recorded October 2nd, 2019. I'm Michael Kennedy. And I'm Brian Ocken. And this episode is brought to you by DigitalOcean. Check them out at pythonbytes.fm slash digitalocean. Get $50 credit for new users. More on that later.

Brian, how are you doing? I'm really good. Super. We are on PDX Python Meetup West, PDX West Python Meetup, whatever you're calling it. It's Eve, aren't we? Yeah, it's like just right around the corner. Yeah. Although it'll probably already have happened by the time this comes out.

Yeah, it may. It's going to be tight. Well, I'm sure it went wonderfully. So we've got all the speakers lined up and everything. I'm really excited to have that as part of our community around here who knows maybe we can put it online somehow but uh no plans for that it sounds like you know i hear a lot about containers and when i first heard of docker and things like that i thought docker was i thought they had created containers right like they had come up

with this idea of this other kind of isolation relative to, say, a virtual machine. And looking into it, it turns out that's not the only option. And they actually didn't even come up with it, right? They just bundled it up and packaged it nicely. I think what Docker has done is very cool. But I have yet to find a use case where I needed it so far. And there are other options. And one of our friends of the show, Matt Lehman, has written an article called Python Alternative to Docker.

The article uses a tool called Shiv from LinkedIn, which is a hilarious name. We actually did mention it briefly on episode 114, but I didn't know much about it. And so this is kind of a neat article. It talks about that Shiv uses a zip app. And I think we've talked about that before. And I think there's a few tools that utilize that.

Right, things like Pex and stuff, yeah. Yeah, you can zip up a Python application with all its dependencies. And I don't think that at that point it includes Python, does it? I don't think so. And that was actually super disappointing to me that it didn't just deliver a thing that ran. You still have to have Python, but it's like you don't have to install the app.

As long as you have Python, you can run it, was my understanding. This article is talking about, and this totally makes sense, you've got a continuous integration build pipeline where you're making changes and maybe you're updating dependencies, but on the deployment machine, you're not deploying, you don't have to update Python all the time. You're

just updating the thing that you package. And that's, it's kind of a neat thing about Shiv is your, all your dependencies, maybe you're updating dependencies or maybe you're updating your own application or fixing bugs or whatever that can all be put together and tested as one deliverable unit. And then just that can be updated to a server machine. But then all the steps of like, okay, that sounds in concept, that sounds great. But what are all the steps I have to go through to actually get that done? And Matt has done that. He just basically walked through all the steps to try to get that done. And I thought it was really great. He does talk about packaging a little bit because you kind of have to talk about packaging when you're talking about deployment.

So this is kind of fun. He does do a shout out to if you didn't use Shiv or Docker, what would you do? What are all the little steps you'd have to do? Which just makes it sound like something that you'd either hire somebody else to do. I'm not doing that. Yeah. But he also does shout out to platform as a service, things like Heroku and Python Anywhere.

And I think that's cool that there's other options. Like if you want to just throw money at it, you just use one of these. Yeah. And these actually aren't that expensive. If you're trying to just save time, that's a fine trade-off. But if you want to do this, this is a neat idea. Also, Docker is a big hammer. I mean, a lot of people love it and that's great, but bring up with a lot of people that you want to try to put Docker, deploy with Docker. And if it's a thing like this, like you're just

deploying an application, that might be overkill. It seems like it's a middle ground, right? Like I'm packaging up not just my Python code, but my Python dependencies and a lot of the setup around that. You still have to have Python there. But beyond that, right, it's like not really much to it. Yeah. If you need to depend on other things, like I need Redis running on this port and I need Postgres like that. And I also want this other thing in linux doing something like all of a sudden then you're like you've outgrown you know bundling python more cleverly

but if you don't need that it sounds like a pretty cool way to do it yeah like in this example the example that matt gives is a a django application that uses uh gunicorn or g unicorn so that all those dependencies are bundled together. It's not like a special kind of Django app with a huge backend that's necessary. But although, I mean, Django uses a database, so I'm not sure. Yeah, yeah, interesting.

While you're talking about Docker, I'm a fan of Docker in general, but kind of like you, I don't do a ton with it day to day. I just, the stuff I do is just not that complicated. But I recently read an article in ZDNet. That site actually still exists apparently. And it actually says, this is surprising to me,

sort of kind of like I was surprised when GitHub got acquired, I realized afterwards, like, oh, they're actually financially in deep trouble. And maybe this is good rather than just going away. The article says, Docker is in deep trouble. A leaked internal memo, the Docker CEO reveals that the troubled company is looking for more money. And they talk about like pushing through with perseverance despite lack of clarity on the future and stuff like that. So anyway, just something to think about. Definitely. And I read that and I forgot that Docker was a company.

Yeah, exactly. And I don't even know if it matters if Docker as a company goes away. I mean, there are parts of Docker that are important, like the registry and stuff like that. But you could totally use Docker if you just had the tools. But still, it's, I don't know, it's just always interesting to see how this is playing out and something you want to consider if you're going to bet your project on it, right? Yeah, definitely. So I have a bit of a theme for this week as well, and it has to do with helping scientists do Python stuff a little bit better. So this first thing I want to talk about is an article that's kind of a how-to from this woman named Anna Nowogrodzki. I'm going to go with that. Sorry if I messed up your name, Anna. And it comes to us by a recommendation from a listener named Jason Thomas. Thanks, Jason.

And the title is How to Support Open Source Software and Stay Sane. Nice. And this is referring to scientists who develop libraries like NumPy or some form of Scikit-learn or something like that. And then they release it to the world. And then all of a sudden, they've got all this software development work and 100 GitHub issues. They got a review and they're like, I was just trying to give back a little bit.

What just happened, right? So the article has some interesting quotes. So I'm going to do a little bit of quoting from it because I think it's pretty revealing. Anyway, it says, the premise of the idol is that releasing lab- open source software involves mountains of unforeseen work for the developers, right? Like you think you're done when you release it and it kind of starts, but it opens up saying on the 10th of April, astrophysicists announced that they had captured

the first ever image of a black hole. This was exhilarating news, but none of the giddy headlines other than us mentioned that the image would not have been possible without open source software. And the image was created by Matplotlib, a Python library

for graphing data as well as other components of the open source Python ecosystem. Here's where it gets interesting. Just five days later, the U.S. National Science Foundation, the NSF,

rejected a grant proposal to support that ecosystem, saying that the software lacked sufficient impact. What even? What even, right? So, like I said, it's kind of a how-to. It talks about where you're, as a scientist, if you're creating software like Matplotlib or something,

where do you go to get funding? Where have people been turned down? What has worked? And so on. It also talks a lot about using some software development, where do you go to get funding where have people been turned down what has worked and so on it also talks a lot about using some software development software engineering techniques for having successful open source projects that grow without killing you it's pretty good it talks about the challenges of not knowing as a scientist how you you might not have fluency with git you might not have fluency with Git. You might not have a continuous integration. Unit testing might not be a thing that you are super skilled in.

And it mentions places like the eScience Institute, the University of Washington, the software carpentry workshops, places like that where folks can go get that training. And then also talks a little bit about when do you keep your software going and when do you decide it's obsolete, we're not going to work on it anymore, what else are we going to go build? So if you're a scientist out there or you work with scientists and you want to help them get rolling with Python and releasing their projects in a way that they'll stay sane, check that out. I think it's pretty good.

It's a continuous problem. Yeah, it is. And some of the recommendations are things like make sure you have continuous integration, make sure you have version control, make sure you have testing. And one that I really liked was that you should focus on code readability over peak performance. Oh, definitely. Always. Right?

Yeah, because you're going to have to fix bugs. And I had a friend who had this saying that was really interesting that debugging software is harder than writing software, generally speaking. So if you write software at the edge of the complexity that you can handle, you probably can't debug it. So you need to step it back a notch, take it a little bit simpler, and then when it goes wrong, you could still deal with it. Yeah. I like that saying. So before we get to the next one, though, let me tell you about DigitalOcean. So DigitalOcean, all of our infrastructure runs there.

When you get your MP3s that you hear our voices on, those came through a whole bunch of stuff that's happening over at DigitalOcean. And they have two new services that I've been telling you all about for a little while, but I'm going to keep telling you about it because they're good. If you want to use MySQL or Redis, they now have those as a service right there. Just go flip a switch and turn them on. That's in addition to Postgres as a service as well. So definitely coming along, speaking of the software as a service over the DigitalOcean side, I'm still a fan of fire me up a VM and install some stuff and let it roar. But you know,

that's all good. Check them out at pythonbytes.fm slash digital ocean, get $50 credit for a new user. And they're doing good work, so we can recommend them. Nice. So you found another license in addition to MIT or GNU? Yes. GPL? I don't know what to make of this, so I wanted to bring it up just to hear your thoughts as well. So there's a license, and I can't remember where I ran across it, but I think probably Twitter or something. I spend way too much time on Twitter. It's a license called the Hippocratic License, like the doctors have the Hippocratic oath. First promise to do no harm, things like that.

Yeah. I mean, I know it's way more than that, but that's all average people remember, like me. So this is interesting. It derives from the MIT license, and it adds a restriction. And the restriction reads, The software may not be used by individuals, corporations, governments, or other groups for systems or activities that actively or knowingly endanger, harm, or otherwise threaten the physical, mental, economic, or general well-being of individuals or groups in violation to the United Nation Universal Declaration of Human Rights. Interesting. Yeah.

That's an interesting thing, and I kind of like it. My first reaction is, wow, that's great. You can say, I'm going to let everybody use this software, but I don't want it to be used to harm people. Completely fair. But I don't know if anybody the people that are like intentionally harming people they don't care yeah do you really think people that are in this are really gonna go oh yeah we really can't use this software to guide our like you know does this cover military i don't know also a lot of this has to do with like human well-being that's up for debate and like a perspective right

like well we had to start the war to help people or you know but obviously if you're on the other side of that like that you're not helping us this is not helping thank you for not for blowing us up it's really hard to interpret and i think you're right that it is very challenging because the folks who are most likely to abuse it it's like the hackers are like yeah well i was going to steal that steal that 10 million dollars from the bank but It's like the hackers are like, yeah, well, I was going to steal that $10 million from the bank, but it's got the wrong license. Yeah. It does still send a message that the person who works on the project cares.

And it's not necessarily a negative, even if it's not super enforceable. I mean, I'm sure that things about like the MIT license, people break. I don't even know what you've got to do to break it, but you probably could. I actually think it's an interesting thing to have around i could definitely see people coming up with like different sorts of alternatives to the mit license also like like not necessarily maybe this plus other stuff like i know people that intentionally would never work for a military work for the military or work for a government or a military contractor because of they don't want to. And I think it'd be completely legitimate for somebody to say, to have a license that said my open source software doesn't want to, I don't want that to

work for these people either. So, but I haven't seen that. Yeah. Not putting any judgment on those things like working for Boeing or whatever. If you wanted a license like that, I think you'd have to be more explicit. It's a slippery slope, but the things that I kind of agree with, I'm like, oh yeah, well that's completely reasonable. Yes, I know.

What if people released software that said, this is okay as long as you don't live in Oregon because I don't like Oregonians. That would be lame, but people can do that. It would definitely be lame. Yeah. That's why you set up a server in California and you just remote in. I'm vacationing permanently in Oregon. Exactly, exactly. It's an interesting find. Yeah. So the next one, like I said, I'm on a bit of a theme here. And this next one is also about helping scientists do more but more computational

people so this one is a real python how to on why and how to make the switch from matlab to python oh this is great yeah it's really really detailed like standard real python stuff i'm not entirely sure how many pages it is but it's's quite a few. So it's a good thorough, it's almost like a mini book on this as Dan and his crew are used to doing. So it basically compares the philosophy of MATLAB to Python, closed source to open source, things like that. Then it walks people through how to set up Python. And then just you use these computational libraries in MATLAB. Here's your Python ones. You did these operations like graphing. Here's how you do it in Python. And so if you're out there working on a team that uses

MATLAB, and I know there are a ton of them out there. I've worked with some even around here, a great team who was moving from MATLAB to Python, who I worked with for a few days, like helping to speed that transition and things like that. There's a ton of these science-like groups, data science, not quite maybe data science, but on their way to that world, who I think would really benefit from that. Like I long ago even used MATLAB for a little bit of work at this research lab that spun off into a company and did like wavelet decomposition of real-time eye measurements. 250 hertz, 250 samples a second. Like wavelet decomposition is like a way harder Fourier transform, but it's funky stuff.

Anyway, to do that it was it was fine but the little toolbox i think it's called the wavelet toolbox that you got was two thousand dollars per user yeah that's a lot of money when you can pip install something that also does that so there's matlab used a lot in our our company as well or there used to be i don't know know what it is now. But similar sorts of things. There's engineering. I'm more familiar with communication algorithms using FFTs and other communication transforms being mocked up in MATLAB before they go to like ASICs or something. Yeah, this is very common. And I'm really glad to see more and more people moving to Python from MATLAB.

Yeah, absolutely. I think it's just another one of those slices of the technology ecosystem where tons of people are leaving that space and coming over to Python because the great computational support that's developing things like JupyterLab and all the NumPy, SciPy type things. Also in this article, I learned that there apparently is this thing called GNU Octave, which is a free and open source clone of MATLAB that looks pretty good, actually. Oh, interesting. Yeah. Not that we want to encourage people to just go do that. They should come do Python.

But this is still pretty interesting. I didn't know that exists. Yeah, definitely. Have we spoken about GUIs? Yeah, quite a bit. I feel like we should actually cover that sometime. Yeah. So I almost felt guilty about covering this, but it came up in episode 143 already.

But Nicholas Tollervey has got a project called PiperCard. I mean, I remember PiperCard. I never really used it. Back in the day, the Dark Ages. But PiperCard is kind of this neat little thing. And did you use PiperCard ever? No, I have some friends who did, but I've never used it. It was a Mac only thing.

And it was this thing where you kind of have a whole, it's designed as if there's a whole bunch of different cards and a GUI window was just a card idea. And then you could print whatever you wanted on the card. And then there would possibly be buttons or something on the card. And if you push one of the buttons, you go to a different card. And so you can think of an application as a deck of cards or a choose your own adventure

type game. And that's where a lot of really complicated choose your own adventure games were made on HyperCard. And so HyperCard is a similar thing just in Python. And it started out with Adafruit, but it's now gone. Nicholas has made it. You can just run it on your anywhere and you can just pip install it. And I was playing with it the other day and at first I dismissed it, but when I had him on testing code on episode 89 and we talked about it a little bit and I didn't realize this. So one of the ideas is you have basically a simple list of what's in the card and then what card to go to next with different buttons and inputs and stuff like that. But if you instead of saying which card to go to, you give it a function to go to, you can run any Python code as long as that function returns a string to tell you which card to go to next. So that makes it actually pretty powerful to build up some fairly complex logic very simply. Anyway, it's fun to work with.

I was even thinking of some work business uses that I could use this for and get it mocked up really quick. That's cool. It looks interesting. It feels a little bit like a visual workflow executor, right? Like I go down this step, then I do this step, then there's a decision, and then I do this step or that step, and it's all represented by these cards. It's pretty cool. He has an example of a Celsius to Fahrenheit converter, but also like, I mean, there's a lot of little simple conversion math stuff, applications in businesses that would be a good use for it. And it's also

uses that user interface thing that I always forget, but it was like, nevermind, I can't remember what it is. It's one that you don't see very many examples of. Cool. Yeah. And it looks like a nice project. Cool. You know, something we don't talk a whole lot about on this show is Node.js. I don't know if we ever have. Have we? You know, I think the extent to which we talked about Node.js was maybe some security issue in like NPM and how it is an analogous thing over at PyPI. Maybe. Like that's the extent of it.

Okay. So Node.js obviously is a pretty big server-side framework for running JavaScript code on the server, right? And, you know, the story is like, hey, if I got to write JavaScript on the front end, I might as well write it on the back end. That's not my philosophy. Mine is more like, how do I minimize the amount of JavaScript I write? Let's not also do that on the back end. I mean, not that I hate JavaScript, but I don't want to overdo it. Anyway, there's a cool project

that brings some integration between Python and Node. So imagine you're working on some project or you're on some team and they're like, we Python and Node. So imagine you're working on some project or you're on some team and they're like, we've chosen Node. You're like, sigh, sad face. But you still want to use some Python or maybe you want to use some kind of Python library like, I don't know, NumPy, TensorFlow, whatever,

something that doesn't have a JavaScript equivalent, right? Yeah. So you can use this thing called PyNode. And what it does is it it's a bridge between node.js and python that allows you to call python code directly from javascript oh wow so this is on the server right like if it was on the client side then who knows would have to use something like sculpt or brython or and then it would convert the python to javascript like that's a different thing. This is the CPython interpreter

with everything that Python can do on the server, but you can interact with it, call it, and get return values from those functions in Node.js. You call into Python, and you get your answer back. Nice. So that's pretty cool. And the API for working with it is not too bad. You have to basically, in Node.js,

you require stuff to load it up. So you require Py node and then you go to get your pi node you can say start the interpreter which will i'm guessing embed c python into the node process and then you just call functions the way you directly call them is kind of crummy you say like pi node.call and you would say quote add comma one comma two to call the function add by passing one and two to it and then you get a callback from that i think you could do some interesting stuff by wrapping this up into a javascript class so you just say you know create it and then you just start working you just it has a dot add that takes a one and a two and you could put like a facade that makes this much simpler,

do a little async and a wait to get rid of the callback potentially, things like that. But yeah, it's definitely got some potential there. And if you're working in Node.js space, you're like, I really wish I had Python. Well, PyNode. It's great. I mean, there's more and more projects

where there really aren't one language. You're trying to do lots of different things or you're trying to migrate from one thing to another. Use the language that works the best for wherever you're at. And similar to the last one that I covered, not trying to send people away to that thing, not to Node or to Octave or whatever, but just to acknowledge, right, like just what you said,

that there are these environments where you don't always get to pick Python. Here's another environment where you can now blend these together. Yeah, there's lots of languages that are good. None of them are as good as Python, but just saying. That's right. All right. Well, those are the major topics that we're covering for the week. Do you have anything extra to cover? No. Although like putting together a meetup is more work than I expected. I appreciate you doing that. I'm sure that it is. I did submit, there's like call for proposals out and I did submit to a couple of conferences. So we'll see. Awesome. Good luck

on that. The call for proposals for PyCascades, right? And also I think main PyCon, right? Yeah, PyCon's open. Tennessee's open. A couple others are open. Super. I actually have nothing extra to share except for I have a joke. Oh, good. I didn't bring a joke. It's not exactly a joke, but it's definitely something that's very funny. So have you ever heard the excuse that, man, this works on my machine? Yeah, it's closing a defect with cannot reproduce. Exactly. Like, dude, this is working.

Like, why does this not work for you? I just, who knows what it is. It could be the version of some library that's installed. It could be like permissions. Who knows? But there is a, it works on my machine certification program that I want to tell you about. Okay. Okay. This is a real program and it has like an honors badge. It says works on my machine. It's kind of golden. You can put that on like your wall when you get certified and there are like real steps. This is

not new. This is a pretty old certification, but I thought it'd be fun. We haven't talked about it before. So there's some specific steps that you got to follow. And the steps here are talking about like a compiled language. So maybe it slightly varies with Python, but still, I think it's funny. So step one is you have to compile your application. Getting the latest version or any recent changes from other developers is purely optional and not required for certification. Okay.

Step two is you launch the application or website that's just been compiled. Step three is you cause you're going to like this part as a person who's into PyTest and stuff. You cause one code path in the code that you're checking to be executed. The preferred way to do this is with ad hoc manual testing of the simplest possible case for the feature in question. You may omit this step if the code change was less than five lines or if in the developer's professional opinion, the code change could not possibly result in an error. Yeah. And finally, you now check your code into version control. Congratulations, you're certified. It's nice, right?

There's probably a lot of certified people. They just need to claim credit. They've done this, I'm sure. Yeah, they just don't know. Well, they can now get the star and put it on their wall. Yeah. All right, well,

thanks for being here as always, Brian, and sharing all this news with everyone. Well, thank you. Bye. You bet. Bye. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes

as in B-Y-T-E-S. And get the full show notes at PythonBytes.fm. If you have a news item you want featured, just visit PythonBytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Ocken, this is Michael Kennedy. Thank you for listening

and sharing this podcast with your friends and colleagues.