Python Bytes - #170 Visualize this: Visualizing Python's visualization ecosystem

Episode Date: February 25, 2020

Topics covered in this episode: Python visualization graph Awesome Zen of Python Jupytext Tour of Python Itertools justpy.io Modularity for Maintenance Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/170

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 170, recorded February 19th, 2020. I am Brian Ocken. And I'm Michael Kennedy. And this episode is brought to you by DigitalOcean. They've got something cool we're going to share with you a little later. But first, tell me about whatever you're going to tell me about first. I want to kick things off by building a picture for you.

A picture of something awesome and yet something fairly complex. The Python visualization space. I don't know how you feel about it, but whenever I think, oh my gosh, I gotta like go make some cool pictures and do neat things with Python. I'm like, uh, map plot lib, bokeh, uh, pi, pi plot. What else should I learn? All these other things. And then if I want to do this graph'm like, map plot lib, bokeh, pi plot. What else should I learn? All these other things. And then if I want to do this graph, like, just there's a lot of stuff out there. So, Price and Daniel sent across a website, a resource for anyone who feels like me and says, ah, there's like kind of an overwhelming sense of all the things happening. And this is called piviz at PyViz.org. And basically,

the idea with PyViz is they've categorized and mind mapped all the things around Python data visualization, and they've grouped it into different stuff. So the way they describe it, they say it's an open platform for helping users decide on the best open source Python data visualization tools for their purpose with links, overviews, comparisons, examples, and tutorials. That's cool. Yeah, sounds good, right? So they have different sections that you can go check out. Overview of all the packages, which is like a nice visual graph relationship thing. High-level overview of the tools.

A live table comparing maturity, popularity, and support. Also, a special focus on building dashboards that are like interactive web things that people can interact with. For scientific visualization, for rendering embedded three-dimensional objects. And like I said, some tutorials and examples. Also, some of the examples talking about like if you have this kind of data, here's you visualize it what do you think yeah i like it that the subset is still a lot of work if you know what you want to do and you still have to try to figure out what you want to do with it but at least it's broken it down into um just a handful of things that you have to figure out yeah absolutely well it doesn't entirely alleviate the challenge of this but like for

example if you go to the overview section you can see there's three main sections javascript matplotlib and opengl and that might sound like i thought this was python what's up like these are i'm gonna work with python and get the outcome as a javascript thing on a web page or something i can get into matplotlib or something i can get into like an OpenGL window for interaction and whatnot. So yeah, there's a bunch of stuff here that you can work with. But I think it's interesting to see the relationships as well.

Well, we had a visualization thing that we used at work that we had originally done a JavaScript side one. And I think that we're going to switch and go back to a Matlab plant plot lib side mostly because it's whether or not it's happening on the front end or the back end. That's a division that we have to make, made that decision.

But, you know, it's only software. You can change it. That's right. It's not concrete. Yeah, just get another intern. Indeed. Indeed. Alright, well, another intern. Indeed, indeed. All right, well, I definitely am going to make use of this site if I've got some visualization to do. And yeah, there's a couple of things I really would like to up the sort of reporting exploration game

for both the course side of things, understanding what users are up to, but also for the podcasts, right? So we can understand how our listeners are listening a little bit better. So I'll see what I can do around that for us. It might involve one of these. Yeah, that'd be cool. I'm intrigued by the breakdown of some of the dashboard ones.

Yeah. Would you say that that project's awesome that I was thinking about? I think so. Yeah. It's probably going to be an awesome project. What else is awesome? Oh, well, Xan is awesome. I actually like a lot of these.

I have to admit it. I like the awesome sites, the collections, awesome collections of different things, usually thrown up on GitHub. And this one's also on GitHub. They're usually more concrete, though. They're more like awesome async servers in Python or awesome web frameworks. Right. But we want to be a little reflective here.

We've got a rabbit hole of Zen with the awesome Zen of Python. Yeah, I was trying to make a bad pun of a rabbit hole lot of Zen. So anyway, so the Zen of Python is a thing. So you import this and you see it. And we've covered, I think, at least one article talking about that. But somebody put together a collection of articles and talks and some other things about the Zen of Python because people have thought about it and written about it. And I like it.

A couple of things I wanted to point out. Al Swigert has a nice one that's a Zen of Python explained. It's a really quick reference. If you're new and are curious about the Zen of Python and you read it and you don't quite get what the heck that is i think that he did a good start i'll probably get his name wrong but mosh zadka has a meditations on the xenopython which is probably a longer couple paragraphs per item a little bit longer but i like his take on it as well and one of the things i just learned recently is that there's a reference to the increment and decrement operator within the Xenopython.

Or not the increment and decrement, but post decrement and pre decrement from C is like something that trips up C people. There's actually a joke embedded in the Xenopython about that. How nice. Yeah, if it has to be an interview question about what incrementing a variable does, it might be too complicated. Yeah, yeah. What happens if you test if, you know, plus plus X of this thing or whatever, right?

Like that's, yeah, not good. So, although I do wish Python had a X plus plus, you know, sometimes I could go. Yeah, just an easy increment or decrement operator would be good. But we don't need two. No. That's right. I've got to admit, I read in... Now we're getting down the C world.

In a for loop, it's recommended that you do a pre-increment because there's a little bit of savings on complex objects to do pre-increment. But give me a break. It's i++ at the end of a for loop. I always do that. Yeah, it is.

It definitely is. I'm glad Python doesn't even have a for loop. We have for loops, just not the same. Yeah, not for I equals zero. Yeah, pretty sweet. Yeah, no, this is a cool find here. I like this. The Xenopython is something that's frequently spoken about,

but it's very terse and there's not a lot of explanation about it. So, you know, it's like one of those things I remember being new going, oh, the Xenopython, how cool. What does it mean? You know what I mean? Like, could I get an example for each line, please? It would be nice, you know? Yeah. I'm also including a link. I think it's important to read the Zen of Python as a joke, and here's why. And it talks about a few aspects of it as to, basically, don't take it too seriously. And

Python is a language that you're really not supposed to take too seriously. Yes, it's serious. We can get a lot of work done. But don't get freaked out on stuff. It's named after Monty Python, after all. on yeah exactly lighten up people you know what else is awesome kubernetes kubernetes and digital ocean yes yep so there is an enormous amount of material on the web about kubernetes available both online

and in print so the digital ocean community realized that it's a little, you know, you might have trouble figuring it all out, even though there's a lot of information. So they developed a Kubernetes for full-stack developers course to provide a thorough, well-maintained, and approachable reference set of tutorials for anyone who would like to start learning about Kubernetes. And if that wasn't cool enough, they also realized that some people don't want to take courses, they want to learning about Kubernetes. And if that wasn't cool enough, they also realized that

some people don't want to take courses, they want to read about it. So they compiled the course curriculum into an ebook to make it available for more people. So you can see that if you go to pythonbytes.fm slash do dash k u b e. And you can download the book and visit the course. And as always, you can visit pythonbytes.fm slash digital ocean to get $100 credit for new users to build something awesome. I think we should just keep rolling things that are awesome. Yeah, yeah. So Matt Harrison inadvertently recommended this next item to us. Really? I mean, like, very indirectly, but he did. So he posted on Twitter how he is absolutely loving Jupytext.

J-U-P-Y-T-E-X, Jupytext. And I didn't even see it on Twitter. No, Twitter emailed me a summary of popular tweets, which said Matt Harrison liked this. So thank you, Matt. Interesting. Okay. Okay, so Jupyter notebooks are awesome.

They're super cool for exploring data, Thank you, Matt. Interesting. Okay. Okay, so Jupyter notebooks are awesome. They're super cool for exploring data, and they're really powerful for people who are doing more exploration rather than writing software, right? You can run the first bits. They might take a while, but then you can slice and dice stuff to visualize it and explore it as much as you want. However, they're not all the way awesome because if I put one of those things into, say, GitHub,

and you were to check it out, and I were to be messing with running it, and that was getting live data, so non-deterministic results, and you did the same, and then we both put it back, it keeps those results and stuff. And it's not very friendly in Git, right? Another thing that would be nice would be sometimes you want to explore stuff in jupiter but then productize it turn it into like a script of its natural language right it supports a bunch of languages now it would be nice to be able to go and here's the one we run where the comment the text becomes comments or whatever so basically that's what jup text does it will take jupiter

notebooks and turn them into markdown documents or Python scripts. Wow. And then you put those in source control or whatever, right? You can work with those. So you could then export it out and work with it in like PyCharm or something like that. You also get meaningful diffs of what changed when you look at the differences of those. So this thing's really cool. You can export the Jupyter notebooks in all these different script languages. Currently it supports ready for this.

Julia, Python, our bash scheme, closure, MATLAB octave, which is the open source MATLAB. C plus plus you KDB plus IDL. I've script JavaScript,

Scala, rust, our shell, C sharp, F sharp hand. My favorite behind Python, Robot Framework. Nice. I just want to work in the Robot Framework.

What language did you know? I know, Robot. Oh, really? Cool. Yeah, anyway, people who are working in Jupyter Notebooks, this looks super helpful. And Matt, thanks for having a popular tweet that got emailed to me. Yeah.

So can you do Scheme in Jupyter? You must be able to. You must be able to. You must be able to. I did Scheme as my CS 101 course when I did a couple of CS courses. And I'm not a fan

of all the closed parentheses to be honest. I'm like, what a crazy language this is. But yeah, apparently you can.

Nice. I guess I would rather do that over like rather than through like an X term or whatever. But yeah, apparently you can.

Pretty cool. Anyway, so if you work on Jupyter notebooks and you want to be able to export them in these cool ways and do other stuff as well, check that out. Well, one of the things I want to talk about is stuff that we iterate over. Okay. This is a failed attempt as a transition. So a great article called a tour of python iter tools and iter tools is one of those things that you kind of it's so cool but you kind of blast past it when you learn it and you when you're learning python it's a little bit a little bit more advanced than you want but

this is cool stuff this article is by martin heinz and and he just kind of briefly describes what a handful of the different tools within IterTools and a library called More IterTools does. And all of these are around iterables and iterating over things. And you wouldn't think right off the bat that you could do so much, but I'm going to highlight a few of these, not all of them. For IterTools, he talks about compress, accumulate, cycle, and T. And I had not been familiar with cycle and T, but they're pretty cool. So, iterables, some of them that they, like, they're empty once you go over it once. And T allows you to make multiple references to one interval. That's pretty neat. And cycle is so cool. It allows you to like, so you, if you iterate over, like say a set or a list of items, you can just iterate through it

once, but cycle allows you to just keep going over and over and over again. It's like a never ending repeating iterable. And, uh, the little example in the document or in this article that shows you how to easily iterate through characters to make a never-ending spinner on a command line so it's kind of fun oh yeah that's cool yeah you just put the the different ones and a list and then you just cycle it yeah the note is that like for things like games and stuff like that if you iterate through through different players you might have to do that but there's a lot of cases where i want to like maybe zip through a couple lists but i it's really just a site one of the lists is a cycle you can kind of do that it's cool more iter tools has a whole bunch of

things like he's highlighting different things that are different ways to chop up your iterations like there's a divide that just divides it into different sub-iterables. Like paging almost, right? Like process them 25 at a time or something. Yeah, so there's a whole bunch of different ones. And partition is like if you have an expression that can be mapped to true or false on any of your elements,

it'll split the iterable into two iterables based on that predicate, which is pretty cool. A side effect that I never knew about. So if you want to add another function that gets called with each element during iteration, wow, this is cool. I think that's neat. I don't know what to use it for, but I'm going to come up with a use. There's bucketizing to be able to do a bunch of buckets, map reduce, just a whole bunch

of neat things. And I think this is a good, if you think you've sunk your teeth into Python and you want to take it to the next level, this is a good article to learn iter tools. This is super interesting. And iter tools is definitely one of those things where you feel like, I'm always surprised that there's extra stuff. Collections as well. It's just like, wow, there's also that thing. Why did I not ever use that thing?

There's some things about programming that I know some people say when they come to a problem and they need a solution for it, then they'll look up and that's how they learn new things. But you're right, with collections and iter tools, you don't know to look it up because you don't know there's a solution there. So these are good things to skim through so that you just have seen them once so that later when you run into a problem, you're like, oh, there's a thing in inner tools that I can use for this. Yeah. I know that that's true even before I look, but I don't know what it is. I just know if I go dive around there, it's probably something I can use at some point.

But yeah, that's a cool one. And it's nice to see it put together and highlighted like that. I just really want to talk about this next one, okay? Okay. All right, so this next one is called JustPy. And the idea is I want to create a web application that allows me to do single page app, cool interactive front end framework type stuff

that I would do with something like say Vue.js, but I don't want to write JavaScript. So it's pretty interesting. It's a web framework that lives on a server. So you put it on, you know, your standard, it actually supports async stuff. So you'd put it on like a async server like starlet or something like that or uvicorn and then you express how the pages are supposed to look in python and it creates the javascript that's going to make that happen and then i guess web sockets or some form of back and forth back to the server to run your python code as people interact with it okay using effectively like ajax type of interaction so yeah it lets you create these front end and you know sort of full stack web apps with no front end programming and the way it works is like

i said it has no front end back end distinction all the programming is done on the back end allowing a simpler more productive way a more pythonic way to create a web app, but it intercepts the relevant events on the front-end and sends them back to the back-end to be processed, either talking to a database or just like having the Python logic there. Neat. Yeah, it's a really, really different take and it's been getting a lot of traction. My personal thought on it, the verdict's a little bit out. I'm not sure how i feel about it but i definitely think it's interesting and people should check it out to see what they think about it it's really based on like classes and components so you can create

these components which then can be put onto the page like here's a login component or like a component that displays a list of podcast episodes or whatever you put those in different places and you can create these custom components and they can either be just straight python where you create kind of the dom in python that's part of the if it was more html based i'd be happier but it does also support html components that then work with it and even has like interactive charts and grids so that's pretty cool yeah i see what you're saying it's kind of neat but also if you don't want to learn html and javascript learn this instead yes exactly it's kind of like that it's like too much avoiding html i think it's awesome that the logic of it is in python and this this no distinction stuff is

really cool about the front and back end i just just wish I would design the site in HTML and CSS and then plug in the liveness through this, not as much of the content definition. That said, Quasar, I suspect many listeners are not familiar with the Quasar library, but the Quasar library is a cool library that's got all sorts of stuff built on top of Vue.js.

So their little slogan is build high-performance Vue.js user interfaces in record time. So you can see they've got like, think of kind of like what Bootstrap did for web design is kind of doing that for Vue.js, right? You get some like cool stuff going on there and the reason i randomly diverted sideways and started talking about vue.js and quasar is because just pi is built on libraries that you would know so starlet and uveicorn for the server side but vue.js for the front end that means the quasar library automatically plugs in its components and functionalities into just pi so that gives it like actually a huge boost. Like,

Oh, there's already this really cool library that just plugs into it. Oh, nice. Okay. That gives it a boost. But yeah, I wish you could define more stuff in HTML and then bring it.

Now you can say like, here's an HTML page. I'm going to import it and like work with it, but it's not, it doesn't seem to be my first reading that that's the natural way to go. Yeah. Okay. But nonetheless, I love that this exists.

It looks really interesting, and people should check it out. Nice. Nice. I wonder what the maintenance story is around it, though. Yeah, I'm curious. So we want to talk about maintenance, and I'm not sure why this article is titled Modularity for Maintenance. So Modularity for Maintenance,

it's a bunch of automation tools that you can use to help maintain your open source projects and it's really terse with links but it's got continuous integration and talks and linting tools and type checking and dependencies and security scanning and using code coverage and formatting help and releasing and all these automation tools that help you which is actually it's really cool to have all these listed down in one place and i like that if you've done all this for a bunch of projects and then you start a new project or maybe you split one project into two projects just the overhead of making setting all that stuff up again for another project is in itself a lot

of work and so he's hoping for some sort of cookie cutter like thing or something for somebody to make so that you can save some time but all of us also i never really thought about all these time saving things you actually end up having to set up again for each project are they actually saving your time yeah right it depends it depends how long you use the project probably they are but end up having to set up again for each project. Are they actually saving you time? Yeah, right. It depends how long you use the project. Probably they are, but yeah. Probably.

But I guess it is interesting to think about the extra overhead and you have to have some of these require like login credentials and stuff that you have to set up. But actually, I really like all this list and I kind of want to go take one of my projects and make sure I'm hitting all of these things yeah it's super cool and i just last night used cookie cutter rent the new python package template from cookie cutter to create a python package that i'm working on so yeah this is super real i'm like oh i bet there's a cookie cutter thing to set up like the setup file just right and integrate talks and whatnot and yeah let's hit it with that where was this a new cookie cutter thing it's not super new if you go to a

cookie cutter there's like a a place called a part called pantry full of cookies or something like that let me go i'll pull it up really quick. Under the documentation on cookiecutter.readthedocs, there's a pantry full of cookie cutters. And then the first four are for making packages. So there's Audrey's ultimate Python package project template creator thing. There's a minimal package for pip installable projects and a couple of others. The one I used was the second one i believe no third one i don't know i can't remember anymore and i'm it was on a different computer

so i can't get to the history of it okay but yeah anyway these are really cool because basically it sets it up you know i used the one that was the minimal pi package minimal that it already magically sets it up to work with pi tests and it has the setup py all generated based on what you tell it, and stuff like that. It's pretty cool. Okay. Well, if it's not using flit, I don't know if it's really minimal. But the thing is, it doesn't have all these things that Glyph's talking about, right?

It does have Travis CI integration in there, but there's a lot of things that it could be added, and it's pretty neat. Somebody could fork that and then make it. Yeah, I definitely think so. So, yeah, hopefully. Yeah, nice. All right, well, I guess that's it, huh?

What else we got? What do you want to let people know about? Here's a quick one, not really big enough to warrant a whole topic, but I noticed that Sentry is helping to fund some open source projects. They got some money from somebody else. They decided to match it and pass it along. And they're going to help fund Black, PyPI, PyTest, StructLog, and a Rust tool called Gimli.

Cool. And that's pretty neat. I like the, actually, I haven't played with StructLog too much. I've played with it a little bit, but all the others i use all the time oh yeah very nice yeah century a nice open source based company funding open source pretty nice the circle of life except for centuries not eating any of these but you know other than that nice okay thanks all right A couple of quick announcements on my end. Just launched a new course, Python for Absolute Beginners. So this is like, hey, I didn't take CS at college.

What the heck is all this stuff about when people talk about memory allocation and variables and data structures, but actually really getting started with Python. So that's a super fun one. People can check that out, build a lot lot of games but learn a lot of python and then also related on talk python training started streaming a couple of our latest courses where possible including this one i just announced is high dpi so almost 4k streams and it's it's like night and day it's it's so awesome to set the hand look at the screen and go that is like as if i'm looking at a real monitor just back in time.

So anyway, I'm trying that out. Hopefully, so far it's been working well for people, but if for some reason it doesn't, people can send me a note. Here's an interesting one that comes from Anthony Sotili. Amazon Web Services has decided to no longer publish AWS CLI, their package AWS CLI, to PyPI, pulling a 700 million download package off of PyPI. Think of all the bandwidth they're saving. That's weird. So this is not their Python library, potentially.

This is the CLI tool that people who don't even do Python would use, right? It's just implemented in there so it's not a huge deal it's not like a giant oh my gosh what's happening to the world sort of thing but it's also strange so uh what's more strange is i'm linking to anthony satili's twitter thread that talks about this but then that links over to github where there's more conversation on one hand it makes sense right like why use pip to distribute a library that is not specifically for python developers but the other bit is well the replacement's an unversioned zip file of scripts so i don't know which is better it's totally funky and then finally people might have noticed some weirdness with the podcast i don't

know what has gone on but something happened i think it was at apple on talk python it all of a sudden started showing four episodes some from three or four years ago as the latest and you might say well maybe something changed in the database and the dates got wrong or the number like we changed nothing didn't change the rss feed generator we didn't change the data it just all of a sudden started going crazy and uh my theory is they've put some kind of limit on the size because we were shipping like 2.5 megs of xml for each request for our especially for python bytes because the show notes are large and i think it just went like no but it's its response to no was just to do something really weird so

on python bites we now are showing just the last half year because the alternative was to have things massively broken in the rss in the podcast players so if you want the older ones they're still available on the website. We're not doing anything like that, but this was the fix that I had to put in to get things to work again on the teams. I don't know what happened. I think it's a fair trade-off.

Yeah, I do too. And also it means stuff refreshes faster for people. They're not pulling 2.5 megs all day long with their podcast players. I don't mind on the web server, but for them, they might mind. They don't pull it all unless they download every episode or something, right?

Well, it's constantly pulling that 2.5 megs over and over and over right? Like if you pull up the tail of the web server, it's just going against that over and over and over. Then it pulls the actual episodes it finds that are different. Oh, right. Okay.

But it's hitting the full on RSSs do now the rsf speed is like 250k which is still pretty big but it's not 2.5 megs oh yeah sounds smaller are you a fan of um physics einstein yes relatively i like gravity it's good yeah yeah and you're also a fan of software quality. Yep. As most software people should be, I suspect. But did you know that E equals MC squared is... I have heard that on t-shirts before. It's a statement about programming, really. I didn't realize Einstein was into this, but it's about testing and code quality

and bug rates and whatnot. How? Because E equals MC squared is errors equals more code squared. So as you add code, the errors increase at the square of the amount of code. That seems not totally untrue. Yeah, that's funny. But yeah, you're right. It's sort of kind of true.

Yeah. I'm not sure if it's the square or it's like 1.5 or maybe it's like... It's definitely nonlinear though. You know, some polynomial, but it's nonlinear and it's like 1.5 or maybe it's like it's definitely non-linear though you know some polynomial but it's it's it's non-linear and it grows yeah for sure yeah for sure anywhere e equals mc squared that's the joke for the week first law of software quality e equals mc squared nice i like it awesome all right yeah thanks bye bye thank you for listening to python bytes follow the show on twitter at python bytes that's python bytes as in B-Y-T-E-S. And get the full show notes at PythonBytes.fm. If you have a news item you want featured, just visit PythonBytes.fm and send it our way. We're always on the lookout for sharing something cool. This is Brian Ocken and on behalf of myself and Michael Kennedy, thank you for listening and sharing this podcast with your friends and colleagues.