Python Bytes - #37 Rule over the shells with Sultan

Episode Date: August 2, 2017

Topics covered in this episode: [more] New URL for Python Developer’s Guide Sultan: Command and Rule Over Your Shell Flake8Lint Magic Wormhole Python Virtual Environments Primer How Rust can replace C, with Python's help Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/37

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 37, recorded July 31st, 2017. I'm Michael Kennedy. And I'm Brian Ocken. And we've got a bunch of sweet stuff lined up for you. Brian, let's talk about contributing to Python. If there's something in CPython itself we want to change, and we don't like how it works, maybe we just go change it. It is open source, right? go change it is open source right right it is open source and there is is and has been a developer's guide

for a long time for c python but the url has uh changed recently i think i saw that recently but it's a it's at dev guide dot python.org and um i bring it up mostly because the URL changed. And I like that URL. But also, it is how to contribute to Python. But I was looking over the initial page where it drops in. And there's some really cool links in there. And one of them is just for everybody. It's how to contribute to open source. And another one is building welcoming communities. And those are both at open source dot guide, but they're, um, they're pretty good articles. I also liked basically the structure of some of the getting started. They have a whole bunch of, uh, I think it's a good onboarding thing, even if you're not contributing to going to contribute to see Python. I think it's a nice, a nice outline of things to include

in any large open source project to try to get people onboarded fairly quickly. Yeah, for sure. We've got like, here's how you propose changes to Python itself. They've got some nice things that take you through the whole like get flow style, you know, the life cycle of a pull request,

running and writing tests, talking about coverage. You know, there's a lot of good stuff here. Yeah. And talking about the issue tracker. And apparently there's an experts index, which is kind of native, like who do you send issues to? And then even things like how to run the tests and stuff.

So it's nice. Yeah. Yeah. Very nice. Cool. So if you have an idea for CPython, the thing is still changing a lot. Like we just covered recently the 362 changes, and that was pretty extensive for just a little, you know, tiny release to like basically a maintenance release. So how would you like to rule over your shell, Brian?

Rule over my shell. Yeah. In fact, not just ruling over your shell, but over many of the shells, like even the shells that are in the outer kingdoms. So the thing that I want to talk about next is called Sultan. And it says command and rule over your shell. And so basically you can run shell commands using this library, this package called Sultan in a Pythonic manager. So the simplest thing, like let's suppose you want to install tree, so you can type tree and you get this cool like ASCII hierarchical view of your files. And you would just say, you know, pip install sultan and then you would say create a sultan object because a sultan.sudo, yum install tree is a string and then dot run. And it will actually go and it will run that command

as sudo on your system, right? That's pretty cool. And you can do things like work with aptitude or yum or ls or all the shell commands, right? And they also have a way to work with it inside of a with block, a context manager, so that it can clean up with itself. It runs a slightly different command

that I think has some sort of better cleanup. I don't actually know all the details with yum. So anyway, it's pretty cool. It even works remotely. So I could like say sultan.load and give it a host name and say to do that as sudo and then run that remotely on the server. And if there's any errors or whatever,

it brings all that information back and prints it out for you in the warnings, I think. I think it'd be useful. I'd take it right off the bat, some of the automation stuff that just large projects have, like build scripts and whatever. I'd like to have them in Python, but sometimes I don't really know how to do it

within Python itself. But this is kind of neat. whatever. I'd like to have them in Python, but sometimes I don't really know how to do it within Python itself. But this is kind of neat. I'll have to check this out. Yeah, it's pretty cool. I think it's pretty focused on the use case that people have running stuff over SSH, remote servers, a lot of shell stuff. But if you find yourself doing that a lot and you want to do more of it in Python, here's a pretty cool library. It's really simple. So what is Yum? Yum is a package manager. I don't remember which distribution it's for, but like Ubuntu has aptitude, you know, like apt-get.

Yum is for some of the other distributions. For some others. Okay. Got it. Yeah, I don't remember. I probably just haven't run into that yet. Yep. Okay.

It's tasty. Okay. Next, I just want to talk about a tool that helped me a lot recently and um i've been working on the book uh which i'm not working on it anymore because it's uh it's turned into my editor but the i've been using sublime to write it and write all the code for it but the uh one of the things i wanted to do was to i got harassed by not being PEP8 compliant in a couple places. So I wanted to quickly go through and just check everything. Of course, I can use Flake8 just on the command line,

but once I know that there's a bunch of files with errors, what's the fastest way to go fix them? And for me, it was a Sublime Text plugin called Flake8 Lint. And I know there's a lot of sublime text linters out there but this one like did everything i needed it to do just right off the bat and was super easy it just installed it and it just worked so highly recommend flake 8 lint it's got all the it's got fl, of course, and it's also got PyDocStyle and some naming conventions, import order and stuff. Yeah, I think having these automated tools makes tons of sense, right?

Your editor should do as much of the work for you as possible. So definitely a good recommendation there. So one example is I would bring up, if I knew that there was a directory with a bunch of files that had problems, I would bring up, if I knew that there was a directory with a bunch of files that had problems, I would bring it up, open up a file. And the default, I didn't ever change it. The default is just to check it on save. When I would save the file, it would check everything and then also pop up a pop-up window with all the errors. So if I could just hit enter, it would take me to the very first error and I could just walk through all the errors, so I could, if I could just hit enter, it would take me to, like, the very first error,

and I could just walk through all the errors. It was great. Nice. And it's not just purely following, like, the PEP8 rules. Like, for example, it has cyclometric complexity checking, which is pretty sweet. So that's like, hey, I've got a highly, highly nested 500-line method. What's wrong with that?

It only has 20 if blocks, loops, whatever. It would complain about that if it goes over 10. That's cool. All right. So how would you like to travel time and be sucked into, say, like a magic wormhole? I'm pretty excited to hear about magic wormhole. Magic wormhole is pretty cool. Maybe you guys have heard of this. So the problem magic wormhole is trying to solve is I want to get things, be this text, be this a small file, be this a huge file from one computer

to the other safely. So encrypted in a way that other people can't get to and things like that. So Magic Wormhole is basically a command line thing where you just type wormhole, and it will come up with a code that your computer has. So you say, I want to create a wormhole and send this file, and it'll come up with a code. If you give that code to someone else, they can run wormhole receive on their command line, enter that code, and then over an encrypted connection that will automatically connect

the two computers, even if they both have firewalls blocking it, there's like some sort of intermediary server that will link them together. And it'll just do, from what I understand, end-to-end encryption between these two things. So if you need to get something and you don't want to put it somewhere public like Dropbox or you don't want to put it somewhere that's hard to erase or is hard to actually physically transfer like a USB drive, you can wormhole it. And there's even a PyCon presentation on this at PyCon 2016. So you can check it out.

What do you think about that, Brian? That looks pretty cool. I don't think I've got a use for it, but I want to have a use for it. I know. Well, you know, like suppose you want to get something to some server you're maintaining, maybe, or say you're on vacation and you want to send something back. You don't want to send it over Dropbox. I can see some good uses for it.

They talked about things like here's a way that you might be able to share like SSH keys that you don't want to put anywhere, but you've got to get them somewhere, things like that. Yeah. Yeah, it's pretty cool. One of the things I thought was a nice touch was the codes are short and human pronounceable. Oh, wow.

Yeah, so they had some nice examples. Let me see if I can find it. Yeah, so like the one that they have an example on the website is 7-Crossover-Clockwork. So pretty long password, if you will, but also you can tell it to someone over the phone. So the idea is like this is when you have the ability to communicate with a person,

but your computer and their computer don't necessarily have a way to directly connect. Oh, great. So you don't have to like email them the secret code or something non-secure. Yeah. I emailed you the ISO. It was only three gigs.

I'm sure it'll be fine. Thanks for that. And finally, there's actually an API that's part of this. So it's not just a command line tool. You can actually make wormhole capabilities in your own applications by just by importing the library. Okay. Now that sounds even more fun. I don't know what to do with it, but it sure sounds fun. It sure does. Yeah. So take us back to some fundamentals. One of the things that I think

we really don't do enough with as developers is virtual environments. What do you think? It's one of those things that I forget that it's hard to figure out the first time. So virtual environments are something, if you're not using virtual environments, you need to be. And there's an older tutorial, but this is from realpython.com. And it's Python virtual environments, a primer. It covers what they are. Why do we need them? How do you use them?

Of course. And then it also covers a few extra tools that a lot of people use like pyvenv for that came with Python three, six and virtual and wrapper. I point to it because it's a pretty good, this article is a pretty good description of really the, why do we need them and how, and what they are for if you're not understanding what they're, why you need them. This is a good, good document to talk about it. The thing that I ran into

recently was I was, I was trying to help somebody and install something. And it turned out that the problem that they were, that they were having was they were not using virtual environments. And when they would pip install something, they weren't able to import it from within the tests. And the problem was that their pip was pointing to a different Python than the PyTest was using a different Python. So when you pip install something, it goes the virtual environment or the the site packages for that version of python not for all pythons so yeah he was pip installing it with python his pip was 2.7 and it goes into 2.7 folder then and for me that's the main reason why i one of the main reasons why i use it is because once you use virtual environments

your pip and your python and your piest and everything are all using the same Python, and they don't get confused. Yeah, it's great. I use it a lot to basically make it very, very clear what versions a particular web app of mine is running and what I need to deploy and things like that. I think your point about it's easy to forget why this should ever be hard, like it's just second nature for those of us who've been doing a long time. But your example is perfect, like helping someone out who is somewhat new. Like I thought Python was simple. Why is this not working? What is this? That happens all the time. And all the extra stuff like virtual env wrapper and PyEnvironment and all these extra things,

those are great for the people that work with them. But if you try these out and they don't work for you, it might not be your fault, or maybe it is. But I haven't been able to get any of these extra things to work. I've tried virtual env wrapper. It's never worked for me. So I don't know what I'm doing wrong, but just the straight virtual environments work great.

So that's what I use. Yeah, that's cool. Nice to have a little reference there. So last thing I want to touch on is an interesting article that actually highlights a couple of projects. I think there's four projects it highlights. Okay.

Yeah. And the idea is Rust, Rust the programming language from Mozilla, is a pretty interesting alternative to C. Okay. Yeah. And the idea is Rust, Rust the programming language from Mozilla, is a pretty interesting alternative to C, right? It's kind of a system-level programming language like C is, but it has a type system that helps eliminate memory leaks. It has proper interfaces. It has better type inference, better support for concurrency, better support for first-class functions, those kinds of things. And so compared to C, it's a pretty nice language. And so when you think of a lot of the important Python packages, many of their important internal low-level pieces are written as C extensions, right?

Yeah. So could those be written in a better language than something like C? How about Rust? So here's four projects that make it easier to implement what would be the C extensions, the native compiled hotspot type of code, but do it in Rust instead of in C.

So the first one is rust-cpython. So that's an interesting, just describes what it is. So it's a set of bindings for Rust for the C Python runtime. It allows Rust to connect to C Python using its application binary interface, which is a Rust thing. And you can run Python programs from Rust or work with Python objects directly, the representations of Python objects directly in Rust. So this is actually for if you want to, if you're a Rust programmer and you want to hook into CPython and control it from the outside, there's Py03, which is a way for people to write Rust software with Binance and Python in both directions. So a Rust program can interface with Python objects in the Python interpreter,

and it can expose Rust methods to the Python program the same way like a C module does. All right, so that's pretty cool. Then there's Snake, S-N-A-E-K, which is kind of an early stage project, but it lets developers create Rust libraries that are loaded dynamically into Python as needed, and they don't have to be statically linked against Python's runtime, which is kind of cool. And I think it pulls it off using CFFI, which is the C interoperable layer for PyPy, the alternate interpreter JIT compiler for Python, rather than C types.

So if you don't have some old library that is based on C types, like say it's C and you want to upgrade, then this might be a pretty sweet option. And then finally, there's actually a cookie cutter template and it says the goal of this project is to be able to produce a binary distribution a wheel which will not require the end user to actually compile rust code themselves which everyone would appreciate that right so here's a bunch of cool options if you are like I really don't want to write this and see, but it probably should be written,

you know, not in Python, but in some other way, like maybe try rust. If you're looking to check that out, that actually looks pretty cool. Have you done any arrest yet? No,

I've, it's one of the languages I want to learn that I'm kind of intrigued by. I looked at go a little while ago. I'm, I'm not a huge fan ago, but people sort of compare these a lot. So I want to check out Rust. I think it's pretty different than Go.

Yeah, it's definitely something on my to-do list. Yeah, cool. Check this out. Yeah. Do you guys have any uses for Rust with all the hardware stuff you're doing? I'm sure we could come up with, invent some reason to use Rust. There you go. Perfect.

All right. How about you? Any personal news? Here's the end of the show. Just that I'm not writing anymore. Congratulations. That's awesome. How's it feel?

It feels incredible. Yeah. To go from I'm writing a book to I wrote a book is great. Yeah. That's cool. Was your wife and kids like, hey, there's some random guy in our house. What's going on? Oh, wait.

No. It's just Brian. It's dad. Was your wife and kids like, Hey, there's some random guy in our house. What's going on? Oh wait, no, it's just Brian. Yeah. No, I've tried to, I've tried to limit their writing to, to things, obscene things like early in the morning and late at night.

But yeah. So punish yourself instead of your family. Yeah, exactly. Cool. All right. Well, I am packing my bags and I'm going on vacation. So either we might be a little bit intermittent or, you know, there's some talk that we might have some guest co-hosts coming in here.

We'll see. So if you don't see some podcasts in the next few weeks, don't worry, it'll come back. But you might see them anyway with some new faces. Yeah, that would be awesome. So either way, great chatting with you, Brian. Thanks, everyone, for listening. All right, thanks. Yep, bye. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S.

And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Auchin, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.