Python Bytes - #29 Responsive Bar Charts with Bokeh, Flask, and Python 3

Episode Date: June 8, 2017

Topics covered in this episode: Responsive Bar Charts with Bokeh, Flask and Python 3 Zappa Serverless Python Web Services Using a local cache for pip packages Building game AI using ML: Working with TensorFlow, Keras, and the Intel MKL in Python Debug Test Failures With Pdb Monitoring my VOIP provider with Home Assistant Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/29

Hello and welcome to Python Bytes, Python news and headlines delivered directly to your earbuds. This time it's episode 29 and we recorded on June 6th, 2017. I'm Michael Kennedy. And I'm Brian Ockett. And we're here to share some really cool stuff that we found with you. But before we do, Brian, let's say thanks. Yes, let's. All right. Thank you to Rollbar. So Rollbar sponsoring this show as they have many other ones. Rollbar is a great friend and supporter of the show.

So be sure to check out what they're offering at rollbar.com slash Python Bytes. We'll talk about that more later. Right now, I want to talk about charts and stuff that are responsive. There's an article on the Fullstack Python website by Matt McKay called Responsive Bar Charts with Boca, Flask, and Python 3. And I thought that was a great excuse to play with it because I've used, what's that other one, the other chart thing, Matplotlib. I've used Matplotlib a lot. I haven't used Boca and I've wanted to. And this was a really great walkthrough. So I went

ahead and went through it and it walks, I did the little tutorial and it walks you through writing a flask app from the start and explains kind of what you're doing. Goes through a little bit of the Boca, just there's, you're just creating a bar chart and it fills it with random data, but it also shows you how to do like tool tips, um, and how to fill those in with a, with a function callback. And now I've got a little Boca Flask app running on my laptop. Yeah, that's really cool. Yeah, Matt is really good at writing these articles that are focused. They're not too short, so they're all fluffy, but they're also not like, they don't drag on, right? You can get down to it pretty quickly. Yeah, and there's a lot of

detail missing, I'm sure, to really get into it. But having like a top to bottom full example running very quickly, and then you can dive into playing with different bits and pieces of it. That was very helpful. I liked it. Yeah, it's pretty cool. So the way Bokeh works is it kind of integrates the Python backend with the visualization JavaScript front end, and lets you really wire data all the way from your Python app down to JavaScript. You don't have to become a JavaScript developer in an extreme way, which I think is cool. So it's using, I guess it's sort of like a template thing to fill out the data that ends up going in that JavaScript is drawing.

So the drawing happens on the JavaScript side. Is that right? Yeah, which makes it nice and interactive, right? As opposed to say matplotlib, which you can generate like a PNG, but good luck for hovering over it and interacting with it and stuff, right? That's what's cool about Bokeh. Yeah. We've got an application at work where I definitely want to, it's actually similar to this, the, you know, the bugs found over so many days, not exactly this, but similar enough to where we need to have a little small application at work to generate some data around our you know around the test data that

i'd like to be able to have an interactive chart with so yeah you should definitely check this out yeah that'd be really cool and flask is super simple and it sounds all all nice and easy yeah speaking of nice and easy you know one of the things people ask for a lot, it's easy to get started with, say, Pure Minute or Flask or something like that. And you get it running and playing around, then you're like, all right, I want to put this on a production server

and I want to set up load balancing and scaling and redundancy and SSL. I'm like, oh my God, this is a skill. I didn't realize I need it, but don't have. It's like huge amounts of work so there's a project that i want to highlight and i don't think we've covered it on the show let me know it's called zappa and the idea is to take aws lambda do you know aws lambda yes yeah so the idea is aws lambda is basically here's a function a python function function, run it, please. And then you can set that function to be

run on AWS events like, hey, this file in S3 changed. But there's also something called the API gateway, which lets you map your domain, SSL certificates, all that URL into a particular function, very much like flask routes or things like this so you can kind of set up web endpoints in aws right in lambda so what this is is these guys built this thing called zappa and the subtitle is serverless python web services which i love the uh the clash within the short name, their serverless services. Anyway, what it is, is basically it is a Python WSGI application, which could be Flask, could be Pyramid, could be Django, whatever. It could be its own thing. And you write this normal application that looks like a normal web app,

but then you can deploy it to AWS Lambda. So every request comes in, runs in its own little container on Lambda. That's pretty wild, right? Yeah. So what's the deal? Like, why is this a good thing? There's Drawbox as well,

which I'll maybe touch on. But Lambda, the way you pay for it is you pay in terms of CPU used, right? As opposed to if you get some virtual private server, like Ubuntu or something like this, that you have to pay for because it's turned on. Whereas here, you only pay for per request. So if you have something that only takes, you know, say 10,000 requests a day, like it could be a really small amount of time you're actually paying for. Cause how long does

it take to serve those requests assuming you know it's a pretty quick app i have no idea you're the web guy exactly but it's pretty cool so it you know you can handle a ton of traffic for you know just paying for a few milliseconds of server time it's pretty cool and you can even do async stuff which is pretty cool so you can say they've got an example in there, where you can call an API and the API can actually kick off another AWS Lambda function, but not in its own execution, but in another one. And if you need like 100 little Docker container type things to run this, like that's all transparent at the Lambda side, you don't even care, just say here, go run this. And so on. So it's a really interesting way to look at building web applications, mostly just deploying web applications. Yeah. The example they're deploying is a Flask app, by the way.

So every endpoint, like in my path, is going to end up calling a different function and be a different Lambda server? Yes. And actually probably be in its own container that only exists for 40 milliseconds and then goes away. So like there's sort of a new instance of the thing created for every single request. Different, right? Very, very different. It'd be kind of neat to see an application built like

traditionally and then like with this and to compare them and see if you can even tell the difference. Yeah, it sure would. So they actually have not a ton of like really popular ones, but they do have at actually have not a ton of like really popular ones but they do have at the bottom like a couple of apps and some of them are like the small business registration for virginia website or something like this like some sort of governmental app and the thing that i i've noticed about these is there's just a little bit more latency than

makes me happy because every request kind of like starts up the whole web server process. And there's a little bit of latency and API gateway and AWS Lambda, cause it starts up a new container, I think. Yeah. And so like you're sort of guaranteed a baseline of, you know, a couple hundred milliseconds response time. If I, if I understand this right, maybe it's not that low. I'm not sure. But yeah, not that high. But there's some extra latency that you pay for the way Lambda works here.

But it's still quite interesting. Yeah, actually, I was looking down at some of the bottom, and one of the examples was like a MailChimp signup utility, a little microservice. And actually, that totally makes sense to have. Like you have a static website, but there's something that's a little microservice. And that actually, that totally makes sense to have, like you have a static website, but there's something that's a little bit dynamic and has to run aside. Something like that would make sense.

Yeah, if you have mostly a static site and you just want a little interactivity, you can take that little interactivity and make it an API or make it a separate thing that runs. Yeah, it's really cool. Yeah, okay, cool. All right, speaking of serverless server stuff, what if PyPI wasn't accessible? Yeah, exactly. One of the things we covered in episode 24 was the notion that

of creating your own local package store and kind of doing that with the built-in pip. And I was contacted by, and I'm going to forget his name right now, sorry, somebody that wrote a little blog post, and actually this is an older one, about the same topic, called Using a Local Cache for PIP Packages. And the reason why I'm highlighting it today is that I like that it's a couple aliases that didn't occur to me before. So it's pretty much the same thing using pip install dash dash download to download packages into a specific place and then how to install locally, but aliasing them to a couple of names, one called pip cache and one called pip install. Of course you can call them whatever you want. Actually, that's a pretty clever. And I probably will just take his names because I'll remember it and anyway exactly he's

already written the documentation yeah it is a cool idea like you can basically say pip install dash dash download some location and then when you want to actually install from it you say pip install dash dash no index dash dash find and you like give it this link and it's a complete pain to remember that stuff but if you just type pip cache as if you had typed pip install and pip install as if you had typed pip space install. And instead it goes out of this local directory. That's pretty awesome. Yeah. And like you said, or we were talking about before the show, it need to just take if you had a whole bunch that you knew of, like you're teaching a class or you're just you're going to get on a plane or something.

You wanted to grab a bunch of stuff to work with, just having something like that. And he does highlight also this works fine for a requirements.txt file, so you could just have a big requirements.txt file and download them all. Yeah, that's cool. pipcache-rrequirements.txt. Boom, you've got them all. Yeah.

I think that's really cool. And my idea was it would be great if you could say somehow tie this together with the data source that says here's the top 500 packages you might ask for so you kind of preload your cache with a bunch of local stuff so very likely if you want requests if you want beautiful soup whatever you just got it right i think that would be pretty pretty sweet yeah definitely yeah yeah very cool i love this one thanks it's nice and simple, but you could start using it like today

without much effort. Yes. All right, you know what else is not much effort? Tracking down errors with Rollbar. So Rollbar has been helping me out with my website a lot. And actually, I've run into it like an issue or two here. The idea is you basically add just a couple of lines. And if you're doing pyramid,

you put it in your config file. Similarly, with flask or Django, you can add it in there and even works with other stuff like Node.js. But I don't do Node.js because I don't like it. Anyway, for my Python web apps, you can just put it in there super easy, and it will actually capture all the errors and send them back with details like what was the call stack? What were the local variables? And they added this other cool thing that I'm wanting to play with, I haven't got it working yet is people tracking. So like right now, everyone who listens to the podcast knows if they find a way to crash my site, I know about it. But I have no idea who did that. I have no idea how to like tell them, hey, sorry, the thing that broke you here, you could try it, I fixed it. So periodically, I'll get emails from people and they say, hey, sorry, the thing that broke, you here, you could try it.

I fixed it. So periodically I'll get emails from people and they'll say, hey, sorry, this didn't work. I'm like, yeah, I know. I just didn't know how to tell you that I know. I'm working on it. So there's a new thing called people tracking that you can actually add. Like if you have a logged in user, like who caused that error and it'll track that for you as well, which is pretty cool. So check them out at rollbar.com slash pythonbytes.

The logged in user, that would be very effective for like your courses then, since somebody's... Exactly, because they already have to have an account. They're already logged in. So if something crashed for like a registered user, then I can actually contact them back and say, sorry, here's what happened. Here's how I fixed it. Things like that.

Especially for something, a service like that, where people are paying for it, being able to tell them, I saw that you had trouble and I fixed it. That's cool. That's great. Anyway, thanks, Rollbar. Yeah, I think that takes the sting out of it. Yeah, thanks, Rollbar.

Very much appreciate the support for the show. So PyCon was fun, right? Oh, it was really fun, yeah. Yeah, and one of the things that was cool, I mean, there were all these booths and people had set up so many things. The talks were fun. The people I met was fun were fun i felt a little bit bad that i didn't actually get to escape from my booth very far our booth because people were excited to come talk to us and that was great but one of the things i did get to go see was the active python folks and they talked about you a little bit as well so i think that you

went over there and saw them you must have and what they had built is like a simple little game in python using pygame and it was like a scroller like imagine like 1980s 1990s like space invader type of standard 2d scroller game but the thing that was cool was they actually used tensorflow carer, Intel, the math kernel library, and ActivePython to actually derive the AI of the game. Wow. Isn't that cool? So you could go over there, and they had two screens. It was pretty sweet. And on one was the game, the Pygame game, scroller thing.

You'd sit there and play. But on the other was actually the neural network doing like visually doing its thinking while it was fighting you that's pretty cool i thought that was that was pretty trick and basically the the ai would figure out when to shoot you whether or not to shoot you where to aim things like that and that was all a neural network so the guys over there peter sorry peter i don't remember your last name peter wrote this up with some of the lessons they learned, how they basically use Pygame. It's going to be open source, they tell me, but it's not yet available.

They're still getting the code ready to be put up on GitHub. So maybe if we're lucky, we can link to the GitHub repo. But if it's really interesting to you, just send the guy a message on Twitter, and I'm sure he'll hook you up. Well, if you have a link to that blog post, I'm sure they'll link it up there, hopefully. Yeah, I'm sure he'll hook you up. Well, if you have a link to that blog post, I'm sure they'll link it up there, hopefully. Yeah, I'm sure they'll update it.

So they had a few lessons learned of writing a game that is driven by TensorFlow. So they said, choosing the right data to train your network is super important. Prepping your data is key. And just experiment with the network topology. And actually, the last thing I talked about, the looking at the neural network, think, do its magic. So visualization is really important for like understanding what it is you built. On a side note, yeah, so this is a great thing. Check it out.

Like they've got a pretty detailed write up on it. But on a side note, Brian, do you think we're going to get to a place with all this machine learning and AI and stuff where the world is powered by programs that nobody understands how they work do you think we're not already there i think we're that's a great uh great response i think we're there from a complexity and just like there's too much crap and too many layers but theoretically applying effort you'll be able to go through and like stack trace it. Right. But at some point when it's just like a bunch of neural networks

working in like orchestration, like why did it decide that? I don't know. It decided that, you know, it's crazy. Yeah, I think possibly, but at that point, I mean, there's also, I just don't know enough about AI and neural networks. And so it looks like magic to me. Yeah. Somebody that knows a lot about it possibly doesn't look so much like magic. And it's more obvious how answers could be derived. But I don't know.

Yeah, the craziest example I think of is where Google got two of their AIs to invent an encryption language and a third one to try to break the encryption language. And the AIs actually invented their own encryption that nobody had ever seen. And then the other AI would break it, then their encryption would get harder. And it just went on and on like that. And nobody knows what happened. Yeah, that's pretty cool. Yeah. And I think some of those things are definitely possible. And those are like you said they're complexity building on complexity but i i think this the i just want to like to me i don't think there's a fear of a singularity thing i'm kind of a skeptic on the computer's going to take over the world scenario i'm with you i'm certainly with you i'm not a skeptic on these types of things disrupting careers and jobs in society.

Yeah, definitely. I don't think we're going to see Skynet right away. Yeah, hopefully not. Or we won't see it coming, right? Exactly. What's that noise? Oh, yeah. So speaking of things that you want to try to understand with a debugger.

Yeah, this is an older article. It's from Rafael Pirzina. It's Debug Test Failures with PDB. And Rafael's works, he does a lot of stuff with PyTest, and that's how I know him. He also works with Cookie Cutter. Yeah, he's a busy guy. I'm linking this up because it's pretty clever. I had legitimate error in the demo code I use on the book. And I left it in there because I wanted to be able to use I wanted to show a legitimate error in some code. I also show the fixed version,

of course, I use the debugger to understand what was going on with my code. And the one of the cool things about pi test is PDB is not like the best. I mean, I grew up on like things like Visual Studio and PDB is definitely not Visual Studio like, but it's not exactly the same. No, no, but but you can do quite a bit of powerful things with it. And like, show, show different variables and show your listing and exactly where it is. But often you have to like stick a breakpoint in your code in order to use it. But PyTest has a dash dash PDB option, which just sticks you right in the debugger right at the assert. Oh, that's cool. So right where your test failed, you'll be right there and you can examine all the variables and whatever, even set up an interactive terminal with all the local variables there so you can

play with them. And that's listed in this page, but there's also some of the other features that are great for debugging with PyTest, like being able to run just the last fails, last failures, and things like that are listed in this article. So I wanted to highlight that from Raphael. Yeah, nice work, Raphael. So if you were sitting at home, Brian, let me put this in a different way. If you were at work and you wanted to know if your phone, your voiceover IP, Cisco phone, was not working at your house, how would you do that? Suppose you have one. Amazon Pod or whatever the latest thing is. Your HomePod. Your HomePod from Apple.

Apple. Yeah. thing is your home pod your home pod from uh from apple apple yeah you would say something like hey home pod is my phone working and it would say here's what i found on the internet for you oh my gosh hopefully that thing works someday anyway there's a really cool article called monitoring my voip provider with home assistant so i wanted to bring this up not because the programming is super intense or anything but it's just like here's a little fun thing that you can do without a lot of work. So this guy has a Cisco ATA sort of home voice over IP phone with two lines. And his home is set up with home automation to automate a bunch of things using something called Home Assistant.

Home-assistant.io. Have you seen this thing? I think we talked about it before. Yeah. Yeah, so this is like a full-on Python central hub for your home, which is really cool. And so what he did was he plugged in some new code, wrote some new modules for Home Assistant that would look at his Cisco phone

and then watch it and let him know if it ever went down for whatever reason. And he's got a new open source package, Pi Cisco PA, I guess. I'm not sure what the PA stands for. And you can use that now if you want to. But it also does cool push notifications to your mobile phone, which is pretty cool. So it's not so much that I think people necessarily need to monitor their home phone because I don't know about you, but I don't even have a home phone. I have a lot of phones in my home,

but they're not home phones. They all go when people take them. But this sort of integration story of here's a thing, it wasn't integrated with Home Assistant, here's a little bit of Python code and magic happens. And that's really what I wanted to cover here. That's pretty nice. And actually I'm about to get a home phone,

so I might be able to use this. You will totally know if it goes offline. Yeah, so it looks like the SPA is based on because it's an SPA, ATA device. So whatever that means. Yeah, Cisco SPA. Yeah, whatever the hardware is. Okay, cool.

So yeah, that's a fun little thing. It shows you how to basically extend Home Assistant, which if I had a smart home, I would be doing all sorts of stuff. Or I'll just get a HomePod and just ask it questions and it can tell me to go look on the internet. Yeah, but I wouldn't be home to ask it though. So you need to have two, I guess.

Like HomePod, ask my HomePod at home. Mobile HomePod, ask home home pod all right man how's the book are you still working on it yeah is it beta or is it is it like published it's been up we've had some good a lot of good a lot of sales so far and i'm really and a lot of feedback i'm i'm really glad that people are reading it and uh nobody's finding things too bad with it so far a A couple typos here and there. The chapter on configuration is coming out next week. And so we're doing the betas out.

It was out right before PyCon and we're doing a chapter every couple of weeks, but there's only a couple of chapters left and it'll be done. That's cool. That must feel great to be wrapping that up. Yeah, definitely. And having some early sales is good. It's a good motivator to finish it. Very that's cool. That must feel great to be wrapping that up. Yeah, definitely. And having some early sales is good. It's good motivator to finish it. Very, very good. Well, I'm just on a

podcast recording and course recording bender. Like I've recorded almost two months worth of TalkPython episodes in the last week and a half, and I'm getting the next month, the rest of this week. So things are going to be all laid out for the summer. And there's a ton of exciting things coming, including some stuff on serverless AWS Lambda. Oh really? Neat. Yeah. That'd be cool. You're going to probably try to get me to record like 18 episodes ahead of time on this or something. And I don't know how that's going to work being that it's news. Well, we're going to use our AI and our machine learning to predict the future and then we'll totally knock it out. It's going to be awesome. Hey, can we get like AIs to use our AI and our machine learning to predict the future. And then we'll totally knock it out. It's going to be awesome. Hey, can we get like AIs to mimic our voices and then we don't even have to do the podcast

anymore? Yeah, it can just like give us the cliff notes later. That'd be great. Yeah. No, probably not. All right. Until then, everybody, thank you for listening. Brian, thanks for sharing all the news, man.

Thank you. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at PythonBytes.fm. If you have a news item you want featured, just visit PythonBytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Auchin, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.