Python Bytes - #98 Python-Electron as a Python GUI

Episode Date: October 8, 2018

Topics covered in this episode: Making Etch-a-Sketch Art With Python Dropbox moves to Python 3 * Resources for PyCon that relate to really any talk venue* Electron as GUI of Python Applications pluggy: A minimalist production ready plugin system How China Used a Tiny Chip to Infiltrate U.S. Companies Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/98

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 98, recorded October 4th, 2018. I'm Michael Kennedy. And I'm Brian Hocken. Hey Brian, how you doing? I'm doing great. Excellent, excellent. Happy to be back with you for another week's worth of news. Yeah, it's good to talk to you every week. Absolutely. So, I also want to say thank you to DigitalOcean for making this possible.

They're one of the biggest supporters of the show, so thank you to DigitalOcean for making this possible. They're one of the biggest supporters of the show. So thank you to DigitalOcean. Check them out at pythonbytes.fm slash DigitalOcean. Get $100 credit for new users. More about that later. They have some really cool stuff. I was just listening to It's All About the Pentiums. Do you remember that song from Weird Al Yankovic?

No. Oh, my gosh. We might have to put it on the show. Oh, it's like this, this funny, you know, weird out Yankovic type song. It just popped up for some reason. And, uh,

from the nineties and it's all about like tough rappers and these rappers who are like, but they're like programmers. So they're like insulting each other, right? Like how slow their computer is. And one of the, one of the insults is my flat screen monitors, 40 inches wide and yours says Etch-a-Sketch on the side.

That's funny. That's awesome, right? So here we are 20 years past then or whatever, and we're still talking about Etch-a-Sketch. Yeah, definitely. So this article is about Etch-a-Sketch and it's called Make Etch-a-Sketch Art with Python and it's fabulous. So background, it showed up in Toy Story with like the, and it's shown up other places of like really, really fast Etch-a-Sketch drawings happening. And I'm sure a lot of people, other people have figured this out too, or tried to think, man, if I could have, I could hook up some motors to those, the knobs and somehow get my computer to do all of this for

me, that would be cool. So that's what this is. It's this, um, this great article about how this person put together some motors, a Raspberry Pi and some software and, and a little bit of woodworking to get, to keep everything all in one place. And got it to draw pictures. It's so cool because it seems like a fun project you could do with your kids or something. And also it discusses some things like how do you get a picture into line format? And he uses something called canny edge detection. And then if you've got a line drawing in pixels,

how do you translate that into like motor control? And you have to have a path. So he uses a library called networks. It's a Python library to do path coming up with nice paths. And then he has some results where he shows some pictures where the starting picture and then the end result of drawing. And it's just fun. You know, what I like about this is it's got this cool IoT aspect with the Raspberry Pis and the motors and it's not just a software project. It seems like it'd be

something really fun for people to explore if they're getting into programming. And then it also has like a really advanced thing. Like you could do a really cool like booth or something where you put up an Etch-A-Sketch and when anybody walks up, it could do like face detection and like draw them. Like you could do all sorts of fun stuff with like the, you know, PyTorch, machine learning and all kinds of fun stuff.

So yeah, this is a great one for people looking for cool projects. Yeah, if you could get that booth thing set up so that we have it ready for PyCon, that would be great. Yeah, I'm working on it. We're almost there. So one of the things I really like from PyCon 2017 was the keynote by Instagram, how they almost magically migrated their entire code to upgrade Django, completely switch from Python 2 to Python 3. And they did all of that without branching continuously in production,

which was awesome, right? Yeah, that was an incredible talk. Yeah. And so certainly that was one of the major users of Python. But if you look at desktop applications, it may be that Dropbox is actually one of the most popular and quite large desktop apps in Python that there is. I would venture to say that's true, yeah. Yeah, so the Python, they have over a million lines of code in Python, a million lines of Python. And it's installed, I don't know, hundreds of thousands of desktops at least, right? So the big news is Python has had a really big gravitational pull around Dropbox.

We've had some of the core developers working there. Obviously, Gita Van Rossum works there. That's a big deal. So the fact that they were on Python 2, it's kind of a weird juxtaposition of these two worlds, right? So what's really cool is they have now migrated to Python 3. And it's not just like they've rolled out a web app on Python 3 where they control the servers,

but they have done some really incredible engineering to allow them to ship Python 3 into these more modern operating systems, enabling interesting things like kernel level, SmartSync, and all sorts of crazy stuff that's happening. So that's what we got this week. That's awesome, huh? Yeah, that's really exciting, actually.

It is exciting. And what's great about their write up is it's, it talks a whole lot about the challenges that they faced, why they wanted to switch, all of those kinds of things. So they said they've relied on Python 2 for many years, most recently 2.7. And they actually began this transformation from 2 to 3 in 2015. But if you go through and read that article, you'll see like a lot of creation of technology had to be done in order to make this switch. So in their previous version, their Python 2 world, they were using freezers. So like Py2App, Py2EXE, I think it was BBFreeze for Linux. And they were also exploring PyInstaller, but they ended up not using PyInstaller for some reason.

But they were using those types of things to package up Python. And now what they're doing is they actually had to build native apps. So, you know, using Visual Studio for Windows, Xcode for macOS to build like.app files. And what they're actually doing is they're hosting the Python runtime

in their native apps to bootstrap up their apps in order to get the deployment story working better. That makes sense, but that does not sound easy. No, it doesn't. They actually built a way so that they could host either Python 2 or Python 3 and flip a flag so that they could actually do that concurrently, right? So they built the Python 2 version and they started shipping that out and got that working. And then they slowly, for different sets of users,

started turning on the Python 3 version on the same type of code there. Wow, that's cool. But they talk about why they had to do it. So, for example, SmartSync, which I actually just bought from Dropbox recently, is pretty awesome. So I've got almost a terabyte of stuff in Dropbox with all the core stuff and whatnot, the videos and so on. And SmartSync will like let you go to your finder or your explorer and show you all of your files. And some of them are on your hard drive,

and some of them are in the cloud. And you can just like double click them and they just open, you know, or open them some other way. And they like Dropbox hooks into that and says, oh, hold on. We got to download that. Now you can have it. Right. So it was really great for like saving drive space and whatnot, but you can't do that. You can't install that kernel type of stuff with Python. So they needed their native apps anyway. They also talked about why they might switch to Python three. They said, there's two major features that we really care about. And there's, there's some pains that we're trying to get away from.

The exciting features were they're really interested in type annotations. Remember, MyPy is coming out of Dropbox. Guido van Rossum is working on that. And MyPyC that takes type annotated Python code and compiles it to C extensions also is coming from Dropbox. So they're really doing a lot with these type annotations, which is interesting. And they're also really interested in async and await. Apparently that helps a lot for what they're doing. Yeah. Asynchronous copying of files, you'd think. You would think a lot of IO happens on the network and on the drive. Definitely. So that was the positive. The other problem is they said that a lot of the tool

chains they're using is really old and crusty like for example they had to have a separate fork of c python so they could compile it with a modern c compiler like visual studio 2013 so there's something super old and not supported on windows and they're like this is just not cool like why are we maintaining our own fork of the c python just so we can compile it anyway there's a lot of weird stuff they had to get away from with python too right and that's not getting better yeah so they talked about all this stuff a lot of details and they also said we're planning a bunch of future posts so we're going to talk about how they report crashes on windows and mac os and how they use that to debug

both native and python code that's one of their posts coming. That's cool. How they maintain a hybrid Python 2 and 3 syntax and what tools help them do that. Another post. And then it's sort of a war stories, the very best bugs and stories from their Python 3 migration. So I think we have three topics picked for the next couple of Python Bytes.

Yeah, if they come out in time. Yeah, sure. Yeah, for sure. Well, maybe they'll even do a presentation about this out in time. Yeah, sure. Yeah, for sure. Well, maybe they'll even do a presentation about this at PyCon. Yeah, let's hope. Let's hope. Are you speaking at PyCon? Are you going to submit something?

I probably am, yeah. Yeah. I haven't yet, but there's time. Sorry, I'm going to submit something and I'll just put your name on it. That'll help. Yeah, if anybody else wants to do that too, just submit something and put my name on it

and I'll just, you know, I'll wing it once I get there. Well, ours might be a joint one. Oh, that makes more sense. Yeah. Right. But where would we get the resources? You lined it up for me. Thanks. It's not just these are targeted at PyCon, but it really is really any conference, technical conference, I think. And a lot of the regional Python conferences have similar things. So we're putting a bunch of links in the show notes. They have a nice speaker page that's kind of a top level, what it's like to speak there and stuff.

Actually, last year being my first time talking at PyCon, I think that I would encourage really anybody to try it because it's a fun experience. So take a look at the speaking page and then there's a proposal tips. So the first thing you got to do is you submit a proposal and that can be nerve wracking, but it really shouldn't be. So there's a few resources I'd like to poke people to. There's a talk proposal resource page on the PyCon 2019 site. And in there, there's also some links to some external articles and stuff. And then including some people who have shared past proposals and even failed ones and why they think they failed. And then Marietta, I don't know her last name, but she's the core developer.

And she recently published her set of proposals, the ones that have been accepted. And I really like them, actually. I looked through those, and some of these are really nice, clean examples. And it makes me think that maybe it doesn't have to be as complicated as I thought it was. So there's those. It's nice. Yeah, that's really helpful. And like you said, this is helpful however you want to get get started and you don't have to get started at the top level

PyCon. You could always do something local. Yeah. And then one of the things I wanted to bring up also is before attending PyCon, I didn't know what poster session was. I'd seen it a lot, but I didn't really know what it was or I'd seen it mentioned. And a poster session is, don't dismiss this. This is a pretty cool thing. The Pie Bites guys did a poster. And what happens is in the expo area where we were and a lot of the companies are, that's really cool, but it isn't on Sunday. Those people get out of there and on sunday that read that area is there's you know there's a job

fair thing going on but there's also these uh poster sessions where people have these little tight these areas and they get a few feet of uh backspace and they get a their poster put up and a little table for them and so the people that are there to talk about it if you have like a project or something you want to talk about that's a good a good way to get your feet wet and not, um, you don't actually have to present to talk, but you can talk with people about whatever you're passionate about. Yeah. And you can meet some pretty interesting people that way as well. And you also give a link, a shout out to Jason, uh, Wadier's blog posts. You don't need to be a pro at Python to crack the code of PyCon, which is great. And that's a really good article about how somebody new to Python went to PyCon and still got a lot out of it, even though they

weren't an expert. One of the things he mentions is that the expo hall, everybody there is trying to sell you something. I just wanted to say that we're not. We hang out there and we just want to talk with people and be there so that people can come up and say, hey. Yeah, that's right. It's a great place for us to meet people. Yeah. We typically lose our voice by the end of the second day. Yeah. It was a ton of fun. Yeah. This is great. Yeah. So one of the reasons that we get to go there is because of people support our show and sponsor our show, right? Like digital ocean. And I just want to call out a cool feature that I saw them recently talking about. And I think people might find actually super useful. So when you create, normally you create a virtual machine or some form of infrastructure as a service in the cloud, you get like a little dropdown, like, well,

what do you want Ubuntu? You want, you maybe have it pre-configured for this type of service or that. But with DigitalOcean, they have this thing called bring your custom image to DigitalOcean. So you can, you know, spin up a virtual machine and get it exactly the way you like it to run, perfectly set up on your system with your company. And then you can just upload that and say, now when I create a new thing on DigitalOcean, use that.

Oh, wow. Isn't that cool? Yeah. So all you got to do is make the image, upload it, and then you can just log in and say, you know, I'd like to choose from my custom image and hit start, and off it goes. So that's pretty cool. There are a few

simple restrictions on what has to be installed so they can manage the virtual machine from the outside, but it doesn't seem very restrictive to me. Anyway, it looks like a great thing. You can check it out on their blog, but if you want to get started and get the credit, pythonby.fm slash digitalocean. Nice.

Yep. This may be a little bit repetitive, but Brian, have we talked about GUIs? Yeah. Yeah, we have. Actually, this one I want to share with you because I think it's actually super awesome. I think we started this whole path long ago when I said I really just would like, what did I call it? Like, I called it Proton. I said I I really just would like, what did I call it? Like proton, I called

it proton. I said, I want electron JS for Python. I said, we should call it proton as well because of the P, but someone has come up with a great way to use electron, which is electron JS. It's like a Chrome and node type thing that stuff like GitHub's Atom Editor, Visual Studio Code, Slack, all those things are built upon. Those are nice looking systems, right? You write HTML, you can use cool JavaScript stuff on the front end, but on the back end, you got to do node and JavaScript. Yeah, I don't want to do that. Well, there's things you miss out on. Like what if you want to do something with machine learning? Like JavaScript doesn't have good machine learning tools to run locally. But if you had Python, you could use like GPU accelerated local stuff.

That would be awesome. So it's not called Proton, but I will point you at something called Electron Python. It's basically the same thing without the name. So what it is, is it's someone has taken Electron. Well, it's not even a new project in that sense. It's more of a way of using two different projects. So take Electron.js and take Python. And on Python, there's a thing called 0RPC,

which is like a raw sockets communication layer instead of HTTP. Okay. So this really simple local communication layer. Use that as a server and then use Electron to just show your app with your JavaScript and HTML. And instead of talking through Electron to Node, you talk through Electron, Visual Studio Code with, you could do that, but have Python as the implementation on the backend. And if you need to pip install anything, you do it. It gives you like native access. So native menus, notifications, automatically update your app, all that kind of stuff. It's pretty awesome.

That is cool. Yeah. It's really cool, right? The question becomes though, still like, how do I get that to somebody? Right? Great. So I have this this Rube Goldbergian contraption that I've created of like Chrome, a JavaScript node and this Python code, which then also depends upon Python. Everything you pip installed, right?

Like deployment can be a challenge. So one of our listeners, Andy Bolka, sent us a really nice message and wrote a super informative blog post of building a deployable Python Electron app. Takes you through all the steps of how we package it up. Python, its dependencies, how we created it for Mac OS and Windows, all sorts of stuff. Yeah, that's fun. Actually, that's pretty cool. With the code available so you can try this out and follow along. Yeah, it's got the code and all sorts of stuff. So it's a pretty good working example if you would like to try to create this and use this.

And if people do create an app with this, please put a comment on the show notes. We'd love to see it or shoot us a message somehow. Yeah, there's a lot of projects I want to try to do with something like this. So, yeah. Neat. Yeah, so I'll leave do with something like this. So yeah. Neat. Yeah. So I'll leave you with one final thing.

Like right. He goes through this long process of here's how you run this script and then that script and then bundle this together, you know, use PyInstaller and then use NPM package or something to like suck in the PyInstaller. And there's like a lot of stuff going on. So he ends the deployment post saying, and somebody please wrap Electron Python into an IDE so that in the future, all we have to do is click a build button like we could 20 years ago. Yeah, nice.

Yeah. Yeah, it's pretty nice. I guess it's doable now, but it's still a question of, is this really easier than using like Qt or something like that now? Well, he talks a lot about that. And he says, look, if your choice is to use Qt or say WXPython or TKEnter, and all of those, maybe with slight exception of Qt, there's really like one or two people

who are working on it part-time. And when you use Electron, you're using Chrome and all the JavaScript front-end frameworks you might want to select, right? Vue.js. And you have Google and all the web developers in the world basically pushing that UI platform forward. Okay.

Well, there's that. So that was the argument of what the two guys who are building this, you can either take that as a good argument or not, but that's what they were saying. Yeah. And now that it's doable, it'll get easier as time goes on and people add to it. All right, come on, PyCharm folks. Give us that build button.

Give us the build button. Give us the build button. We don't want to know the internals, come on. All right, so probably enough on that, but that's actually one of the more promising ones. And I'm still, I might give it the nickname Proton. I don't know. Yeah. We should just tell them you can use that if you want. Yep. They can have it. Awesome. All right. So what's our next one? I just wanted to give a shout out to Pluggy. If anybody doesn't know that I kind of am a fan of

PyTest. Really? Yeah. And part of PyTest, so one of the things that PyTest, PyTest actually grew out of the PyProject. Another offshoot is Pluggy. So there's a ton of PyTest plugins that you can add on. And I use quite a few of them. And the way these plugins work is through this plugin system. And it's a plugin management and hook system, there's often a problem. If you've got an application that has a lot of possible customizations for other people to do, but you want to be able to provide plugin or hook capabilities, but you don't want to expose too much internals. And you also want to isolate this API, this interface, so that you know that you can do refactoring and rewriting stuff and you're not going to break plugins. And pluggy is an option. And I think it's a really good solid option. I have noticed that a few other projects have used this, the, I don't have a list of those, but it's a, I don't think we've covered it before. And if we have my apologies, but, uh, it's kind of nice. And my little side project of cards, a lot of people said, well, if we're doing like task management, there's going to be extra little things. People want maybe

a different list of things. You may be not an owner, but you maybe need priority and maybe time estimates should be in points versus hours or things like that. A plugin system might be good. So I'm actually thinking about trying to put pluggy into this little side project of mine. Oh, yeah. That's really cool. Yeah. This is used by PyTest, by Tox, by DevPy. So certainly it's getting some real world testing.

That's great. I like these plugins as an idea of saying I would like to let people change how this works without just letting them actually change the project. Yeah. Yeah. That's cool. All right. Are you ready for something scary?

Like actually truly scary? No. but let's talk about it anyway. All right. This was said over by Eduardo or Chena. This is on Bloomberg. So on, um, Bloomberg.com. So it gives it's Bloomberg business week. It's a pretty serious article and it's, it's pretty deep reporting for something that has happened it's called the big hack how china used a tiny chip to infiltrate u.s companies and other companies but they were thinking they were aiming them actually at u.s companies and defense contractors and stuff like that oh creepy so tell me more all right so first of all when they say a tiny chip, I don't know. I don't know what you guys are visualizing. But if you hold up, say, a US penny or like a Euro cent, and you look at the lettering on there, the chip was the size of one of the letters.

Like so, or it says the year the penny was printed. It's like the size of like the two. And they decorate it as like a different type of component that goes onto the chip. So this is, it's really a long article, like I said, quite deep reporting. It says, this was an attack by Chinese spies that reached almost 30 US companies. How they know that it was 30 and not 37 or something is actually really interesting. Including Amazon and Apple by compromising America's technology supply chain. So what happened was they found out that there was some spies in the Chinese government that threatened the subcontractor of the subcontractor of a company that made motherboards for many data center servers. Yeah.

And they said, well, you have a couple of choices. One, we can bribe you to install these little tiny chips, or we can threaten you with inspections of your factory if you don't install these chips and keep it quiet. And pretty much they found out that some of these subcontractors were like, well, we want to keep our business, so chips go in. Jeez. Yeah. And so this all started when Amazon went to acquire actually a Portland-based

company called Elemental Technologies, which basically helped with like streaming video compression. And as part of the purchase process, I said, well, let's do some due diligence on you. And they actually sent off the server hardware that they were using off to some other company. I think it was a Canada or somewhere. And they said there was a nested on the server hardware that they were using off to some other company. I think it was a candidate or somewhere. And they said there was a nested on the server's motherboards. The testers found a microchip not much bigger than a grain of rice that wasn't part of the original motherboard's design. Like, I don't even know how they found that, right?

Like, that's insane. And so then they found out that these motherboards were used by the Department of Defense and CIA drones and Navy warships and Apple and Amazon and all sorts of places. And they're like, whoa, what's going on here? So they did a bunch of like a spy on spy stuff. And they found out kind of like what I told you. And they realized that these chips were basically opening back doors in the network of all of these servers, which would then go download bigger bits of code, would actually take them over, and you could just log into them.

But they were interacting with these other servers somewhere on the internet. So they hacked those servers, and they figured out that 30 companies' machines were interacting with that server, and that's how the number was 30. Okay. Anyway, the whole supply side chain thing is pretty scary, but this is not technically a software topic, but I think it's something that we in the technology industry need to just be aware of because this is pretty crazy, right? And there's no software that will tell you whether or not this is a problem, right?

Because this is at the chip level. Yeah. Jeez. Anyway, there, let's leave it with that positive note. Have a nice day, everyone. Have a good day. Apparently, the 30 companies who had their servers interacting

back with the hacker command and control servers have been notified silently. Yeah. Yeah, but anyway, there's a bunch of inside reporting and stuff. it's quite interesting so check that out well that's it for our official items anything else you want to share brian no i just want to rave about your new course oh thank you so the new course is going really well so i

you know the async course that talks about async io async and await multi-processing all that stuff it needed to exist and i'm so happy i built it. Yeah. So what I wanted to bring up is I know async is hot because of a lot of people thinking about the new async stuff that came into the more recent Python 3s, but it's not just that. So one of the things I like about the course is you do talk about the new AO. Async IO and-O HTTP. Yeah, those. But you also, it isn't just that. It's also things like,

if you are concerned about that, you're probably concerned about things like, I just want to make stuff run faster. And so there's other options too, like multi-threading and multi-processing, when and how to use different ones and kind of how to decide that. And then even if you want to make it faster, there's things like, I think, do you cover Cython then?

I do cover Cython. Yeah. So it's not just the more recent things. It's a big full picture. And it's not even that long. It's just like five hours or something. Yeah, it is. Yeah.

Thank you so much for that. That's really nice of you to say so. Good job. Yeah. Thank you very much. Happy to do it. All right. And happy to meet up with you and talk about the news as always. Yeah. Thanks. Talk to you later.

You bet. Bye. Bye. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at PythonBytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Auchin, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.