Python Bytes - #315 Some Stickers!

Episode Date: December 20, 2022

Topics covered in this episode: Jupyter Server 2.0 is released! Converting to pyproject.toml aws-lambda-powertools-python How to create a self updating GitHub Readme Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/315

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 315, recorded December 20, 2022, almost the end of the year. I am Brian Ocken. I'm Michael Kennedy. It is almost the end of the year. Are you guys, you ready for Christmas and everything? Yeah, starting to think about Christmas, maybe try to do some skiing. You know, it's getting cold and it's snowing in the mountains around here, so yeah, getting there. Why don't you kick us off with our first story?'s go let's talk some jupiter server so this

is a an announcement by zach sailor and it says jupiter server 2.0 is released and we know what jupiter is we know what jupiter notebooks are we know what jupiter lab is well jupiter server is the core web server that powers those things right so? So when you pull up a JupyterLab or Jupyter Notebook, you're talking to a server. Oftentimes that's a local host little thing running, but other times it's a centralized server that's set up for like students or a research group, or it's set up on a supercomputer near a bunch of data that, you know, you log into it. It runs locally on that, right? So the core of Jupyter in a lot of ways. So 2.0, that's a big deal, right? I think so. It means new features, right?

Open source, when something goes from one major version to another, that's usually a pretty big deal. So let's hit some of the highlights. So one of them is a new identity API. One of the farther looking out features of Jupyter server that they're hoping to bring is like a Google Docs like experience. So real time collaboration experiences, there are some making them better. And around this, having multiple people working on single notebook, you need to have a good way to work with identity. So there's a new identity API in Jupyter server that makes it possible. It is configurable, pluggable. You can even swap out your own identity provider to authenticate users with any provider. So within your organization, you could do single sign on, all those types of

fun things. Yeah. Related to that, you have authorization. So once you know who you are, what can you do? Enabling collaboration on a notebook shouldn't just mean you have authorization. So once you know who you are, what can you do? Enabling collaboration on a notebook shouldn't just mean you have access to everything on the server or nothing on the server, but you maybe have access to these notebooks, but not those notebooks. Or maybe you can read that notebook and not write to it. So there's this new authorization API that allows you to share certain notebooks with certain people, even if they have access to the server. So that's pretty cool. Again, with the collaboration stuff, there's a new event system API coming along and they created this package called Jupyter Events that allows for

JSON based event driven system over here. So a lot of cool things that should be coming. So it's like a foundational piece so that they can add more, more events, more interactivity, and so on. There's a new kernel WebSocket API. It ships with its own kernel manager. I suspect most people are not messing with that, but it's down in the guts. In JupyterLab, you can open up a terminal. Now, this terminal service is a server extension, and it can be optionally disabled because it's an extension, which is cool. And Brian, the last one for you. They say, as we work to increase our test coverage to 80%,

major thank you to Steve Sylvester, we built a handy set of PyTest fixtures that can benefit Jupyter developers and extension authors. That's pretty cool. Yeah, we released PyTest Jupyter, a bunch of fixtures and other PyTest goodies for Jupyter core libraries and extensions. That's nice.

I'm actually going to check this out. And I think it'd be cool to get one of those folks on to Test and Code to talk about it. So I'll reach out to them. Yeah, I think that'd be great, too. All right, well, there you have it. The core of Jupyter got a little bit better. Nice. Well, what do we...

I'm next. So I guess... Well, you got next what do i got uh i i actually so last week we were it was just last week actually uh time flies we were talking about automatically converting stuff like having having other programs convert your python code modify your code and all the different ways you can do that. And one of the things I said was, it'd be cool if we had some ways to convert from setup.py to setup.cfg, or from setup systems to pyproject.toml. And so of course, lots of people reached out to me and said, there's already some ways that you should take a look at. So thank you, Christian Klaus and Brian Skin and others for reaching out and letting me know.

First one, actually, the main thing I wanted to talk about was a thing called Any to Tommel. So this is a project that takes, it converts, it attempts to convert Any files or CFG files to Tommel. They say it's experimental. So what is I thought I was looking at setup.py files? Well, actually, a lot of like large projects are not just setup.py, they've moved a lot of the settings from setup.py to setup.cfg. So this is still helpful. This isn't going to, I don't think I can grab your setup.py file if that's the only thing you're using.

But if you're using setup.cfg, maybe this will help. It converts to a tomo file. And it says it's experimental, but that's all right. So it's kind of fun. There's like a couple versions of it. One of the versions, it's light and it's got light, L-I-T-E and full. And the light version has less calories. Uh, no, um, the light version set creates a, uh, Tomo file, but the full version will try to get your comments over to, so convert your comments into Tomo compliant comments. That's pretty neat. So I

would, I definitely try that. Um, uh, what did I, I didn't, I don't think I tried this. I didn't try this, but it looks pretty fun. So a way to automatically change that. So be fun to grab that. And I've heard from people that, that they have used this to convert their project, at least the initial batch. But at least Christian mentioned that they needed to clean it up a bit afterwards. So one of the tools they used was a thing called PyProject-Format. And what this does is, and I did try this, it's pretty neat. It takes like if you got any PyProject.toml file and it converts it to a known way to set to set it up so uh it's like black

for project files kind of yeah but one of the things with like the pipe the toml syntax is you can have like you know like in any file you can have things in any order you want and it's still valid but are you going to be able to find it there i mean if you're working with lots of projects it'd be really nice if like the the the authors was in a similar place, uh, and the dependencies were in a similar place because they could be like all over the place and you, it is easier to find them if they're in the right place and formatted the same. And, and I mean, who really cares how much, how it's formatted as long as it looks good. Um, and it's proper.'s proper so um i think this is a neat idea i tried it on a couple of my projects and it and it worked okay so and i and i want to like put that in the asterisk

so you want to check what you get afterwards because one of the things it did which i didn't expect it to was it um it replaced it took the name of my project and replaced dashes with underscores and and i know that like a library, you can't have a dash in the thing you're importing, but the name of the project where it shows up on PyPI and everywhere else, it can have a dash in it. And traditionally,

and I was using it on a PyTest plugin, traditionally we do have dashes. We say PyTest dash something. So it needs to stay a dash doesn't don't translate it so that was a little to be pytest underscore check yeah no i don't want it to be that um so i did uh submit an issue i'm sure it's a minor thing but at least you know at least i know it's there um so to check it out so a neat tool to for auto format it and a couple other tools that were kind of neat. Um, pipe,

uh, validate pipe project. Um, Ooh, just a moment. You still see me. Okay. Yeah.

Sorry. It's fine. Something, something happened on my computer. Anyway, uh, the, it,

it's a, it just validates to make sure everything's fine. Uh, and I ran it on a couple of projects and nothing happened. I'm like, Oh, it must be good. So I,

I inserted some errors, uh, just to see what would happen. And it's pretty terse. It pretty much just says like there's an error on this line at this column or something. And so it was still helpful. You can go count and figure out where the project might be. But kind of a neat validator. But for validation, and I'm not sure what all it's checking, but it says it's JSON schema definition defined, driven. So you could probably check it

out. It probably converts it to JSON and then runs through JSON schema or something like that. Oh, you think so? Maybe. Yeah, probably. Might. One of the things that I wanted to bring up when validating toml files and projects is don't forget talks so um using talks to uh go ahead and in what what the default of talks you can do use it for lots of stuff but the default works great for me for this which is say to take your project build it put it put it in a virtual environment and install it there. And you'll catch things like you forgot to specify a dependency or, you know, missing dependencies or you forgot some code or something like that. You'll catch it if you test it. So test your projects.

Absolutely. Yeah, those are a bunch of great little tools. I didn't know about hardly any of them. I'm starting to do more with PyProject.toml. It's nice. Hatch and Hatchling and PyProject.toml. It's nice. Hatch and hatchling and pyproject.toml. Oh, yeah. Hatch is fun.

So hatch and hatchling. It sure is. All right. Well, quick word from our sponsor before we move on, do you think? Yeah, it sounds great. So this episode of Python Bytes is brought to you by Microsoft for Startups. Starting a business is hard. By some estimates, over 90% of startups will go out of business within the first year. With this in mind,

Microsoft for Startups set out to understand what startups need to be successful and create a digital platform to help overcome those challenges. And it's Microsoft for Startups Founders Hub. Founders Hub provides all founders at any stage with free resources to help solve startup challenges. The platform provides technology benefits, access to expert guidance, and skilling resources, mentorship and networking connections, and so much more. Unlike others in the industry, Microsoft for Startup Founders Hub doesn't require startups to be investor backed or third or third-party validated to participate. Founders Hub is truly open to all.

So what do you get? You get to speed up development with free access to GitHub and Microsoft Cloud with the ability to unlock credits over time. And to help your startup innovate, Founders Hub is partnering with innovative companies like OpenAI, a global leader in AI research, and deployment to provide exclusive benefits and discounts. Through Microsoft for Startup Founders Hub, becoming a founder is no longer about who you know. You'll have access to their mentorship network, giving you access to a pool of hundreds

of mentors across a range of disciplines, across areas like idea validation, fundraising, management, coaching, sales, marketing, as well as specific technical stress points. You'll be able to book a one-on-one meeting with the mentors, many of whom are former founders themselves. Make your ideas a reality today with the critical support you'll get from Microsoft for Startup Founders Hub.

To join the program, visit pythonbytes.fm slash foundershub2022, or of course, go to your show notes for this episode and click the link there. Indeed. Thanks to Microsoft for supporting our show. Very cool program. All right. Let's talk about the cloud. We're going to round this whole episode out with a little bit of talk about the cloud as well. Specifically, though, serverless and functions. Brian, have you done anything with functions like AWS Lambda or DigitalOcean or any of those Azure functions? Not for a while.

I played with it when I learned about them, but I haven't done anything lately. Yeah, they're neat. I've been thinking about this project. I'm not even close to committing to it and not going to give away too many details until I'm more willing to put time and energy into it. But something that requires interaction from many, many different geographical locations.

And one thing that's really cool about Lambda and serverless, these functions, is you only pay while they're running. And so if you want to say, I want to have 20 different locations in which I can, like geolocations throughout the world, I could do certain things, but I'm not going to do them very often. Well, having a bunch of functions, many of them on these different parts of the cloud would be really awesome. And if you're working with AWS Lambda, there's certain APIs for doing this, obviously that AWS puts out. But what if you were a Python developer and you wanted to use Python? So I present to you AWS Lambda Power Tools Python, a suite of utilities for AWS Lambda functions that makes distributed tracing, structured logging, custom metrics, and item potency,

and many leading best practices easier. And it is all based on Python. So you can just write your Python code and start working with this API. This one came to us from, let me make sure I get this right, via Mark Pender. So thanks Mark for sending this in.

It also happens to be available in Java, TypeScript and.NET. But you come down and look at the features, there's all sorts of different bits of Python code and classes or modules that you can use that do things like distributed tracing, right? Because if you're on a serverless thing and something you want to, you know, keep track of sort of logging or performance, you know, those things are super transient. They don't have

a hard drive where you append to the log or anything like that, right? So having that kind of setup would be great. Use pydantic for data parsing but there's just a bunch of built-in parts of this library that you can use within your functions your serverless functions to make it better so it's got tracing which includes decorators and utilities to trace lambda functions both synchronous and asynchronous there you go structured logging metrics that asynchronously go up to CloudWatch. Remember, this is AWS. Event handlers for putting stuff over into GraphQL.

You can build your own middleware, like something that runs before and after the request. That's really, really handy. Like only certain types of clients should be able to call this over the web. Batch processing, typing, validation, a bunch of other stuff, parsing, streaming. So you can actually set it up to sort of loading some massive data set into your function. You can just stream that data in and process it as it goes.

So there's some tutorials and there's some example applications. There's a serverless shopping cart, a serverless airline. That's a pretty big one. Serverless e-commerce platform and some graph serverless. Just let me just say that isn't the, it's not a joke. That's a pretty big one. Serverless e-commerce platform and some graph. Serverless, let me just say this. It's not a joke, it's serious. Serverless GraphQL nanny booking API. There it is. Sure, why not?

Anyway, if people are doing serverless, especially if they're doing AWS Lambda, this might be cool. John, the audience points out that Zappa works great for quickly and deploying Flask apps as, yeah. So if you look at AWS, Zappa, Lambda, Flask, that's probably a sufficient search, right? That's a pretty cool project.

I believe we covered that way, way back in the day. So basically it turns your Flask routes into serverless function calls as well. So anyway, here's in the day. So basically it turns your Flask routes into serverless function calls as well. So anyway, here's some nice tools. I suspect you could probably put those together as well, right? Like your tracing and logging, you could probably still put inside of Zappo, but I haven't tried that combination.

Cool, neat. Well, I'm looking forward to whatever secret project. There's like a 17 and a half percent chance that that actually comes to fruition because it's got a lot of operational overhead. I'm not sure I want to put myself on pager duty. Yeah. I do want to hear what you got next.

Well, I wanted to talk. My friend Bob is on the screen. Yeah. So I wanted to talk about Bob Elderboss and actually his GitHub profile. Several people have done things like this. So we're looking at Bob's GitHub profile, several people have done things like this. So we're looking at Bob's GitHub profile. And one of the things you see that's different than the default

is you see some stuff at the top, like the latest PyBytes articles and latest Python tips and even latest Fosted on toots. So that's kind of neat. But how did he do this? Well, luckily he's got a link at the bottom that says how to create a self-updating GitHub README.

And we did talk about this way back in 2020. Simon Willison had a blog post about that. And so we'll link to the episode and Simon's also. And Bob actually says he was inspired by both Simon and I think Brett Cannon as well has an updated readme. But I really like this article. So it's just a little short thing saying, hey, I started doing updating my profile,

GitHub profile with some tools. And then he just published all the tools. So that's pretty awesome. First of all, he's got a script. So the script that runs, of course he has it up on github as well um and it's it just goes through and collects all of this data for him and and the script's pretty easy to read um you know not terrible uh go through and it has links to where he's getting the information and then uh then all of it is used to fill out all that data is used to fill out a template,

which he also published the template. It's a Markdown file that, yeah, he uses for that. And then all of it's pulled together with a GitHub action. Wait, wait, go back for just a second there. It looks like it's a Markdown file that has Jinja template language in it. So hit it with Jinja, and it generates the final markdown, I think.

Yeah, so how's it supposed to look? I didn't look exactly where the template's being used in the code, but... Yeah, I don't see it either. But you don't have to have well-formed HTML for Jinja to run. You can just jam it against anything. For example, the RSS feed for Python Bytes

is generated using just like an XML template with markup language in it like this. Yeah, it does look like he's using Jinja template itself out of the Jinja 2 project to render it. Yeah, cool. Nice. And then the whole thing is run by a GitHub action and it auto commits.

So using another project that GitHub auto commit. But the whole thing's pulled together and runs whenever. So once a day. And when does it run? Well, it's a cron job. So I had to use my handy decoder because I can never remember that stuff. Cron tab dot guru. It's 845 in the morning.

It goes off and runs this. So kind of neat. I wanted to try this. So I highlighted it again because it's been on my to-do list since 2020. I haven't done it yet. One of the things that I'm hoping it will do is it'll highlight that I haven't posted

any articles recently and it'll get me off my rump and write some more because I want to write. Yeah, yeah, yeah. You'll have to take on the philosophy that I did as well. Like, you know what? Two paragraphs, that's a perfectly fine blog post.

Yeah, definitely. I learned something and also the today I learned sort of thing as well. So yeah, that was very popular. Well, that's quite a nice one. That's all of our normal items now. It is. You got any, well, let me, let me go through my extras first, since I got a follow-up for this one. I got a few extras. I know that we both do.

So you talked about crontab.guru, right? People haven't seen this before. I just learned about this, that there's, there's this funky syntax for cron jobs, but there's this site crontab.guru that lets you put in like, well, I want a five there. It says every minute past hour five. If you just say star five, star, star, star, for example. Or if you put like star over 20, every 20 minutes, you know past every minute past the 20th hour or like there's there's weird syntax here but there's this cool little site that you referenced uh oh wow yeah yeah i was curious if you could do like uh just day of the week and not and like no sun saturday and sunday how you do that yeah um probably probably yeah there's

20 star now anyway there's there's a 20 star now. Anyway, there's a whole bunch of different things and a little bit of a description on how this works. But way more interesting than I knew. All right, so let me tell you about a few things real quick. I've been playing with Canary Mail. Canary Mail is like a mail client, like Spike or Outlook or whatever. You know, one of the, like Thunderbird,

one of the client side rich email clients okay but this thing just added uh an ai capability called inbox copilot which is a lot like um github copilot and so you just go to it you just say hey there email write me a message that says this so for example you could just see like if you got a message asking if you're going to be able to make the, so meaning you could just say, you type CCC anywhere in your message, it pulls up this little separate window that you talk to the AI and you just say, tell them, you know, apologize.

I'll be late by 20 to 30 minutes traffic. And that's all you say. And it writes like a really nice, formal, like well-written, friendly, not terse message back. So I'm going to try to start playing with this a little bit more. We'll see how it goes. I already had the app. It just came with this Copilot thing.

I'm like, okay, that's pretty interesting. So people can check that out. I don't even know what it costs, just to be clear. Not an ad. This episode is sponsored by Microsoft. But yeah, it's like $20 a year. Not a terribly expensive thing to play with. All right, next one is, this is a tweet from the Python Package Index itself.

And the status is, or the message is, Python 3.11 delivers. Check this out, Brian. So they show some kind of analytics, real-time performance graph of the g-unicorn python worker processes that power pypi on python 3.10 and when they upgraded to 3.11 it's pretty striking huh yeah totally that just went down and in load if you look at it yeah i mean it dropped at least 50 of usage there and then if you um this is a pretty zoomed in picture someone else dustin ingram it was tweeted well you should see it over time more look at bam there's this huge jump in the middle this is when they restarted it you know whenever you restart the app it doesn't have like all of

its caches and all that stuff warmed up and potentially they even maybe had to throw away like redis cache data if they were pickling it because 3.10 and 3.11 didn't, you know, they might not be compatible in a binary way. So anyway, there's like this kind of like, ouch, you restarted us. It hurts type of thing. But then after that, it really, really drops. I mean, at least 50% or 2x more for 3.10 than 3.11. So that's really massive. That's pretty cool.

Anyway, so 3.11 delivers So that's really massive. That's pretty cool. Anyway, so 3.11 delivers. People can check that out. And this is, I mean, there's a lot of traffic on PyPI.org, right? Because it's not just the human browser traffic. It's every time you pip install something, every time you do a Docker build and it runs pip install something,

like all the robots of the world are just hammering this thing, you know? Yeah. Yeah. All right. So that's a good one um i was just in las vegas this weekend that was fun thanks for having it in such a good time went went to see the blue man group and we went to see cirque du soleil and we had breakfast at like a

beautiful brunch they even had a uh you know how some places have a like a little omelet factory where you can have some eggs or whatever? And you tell them they had a donut, one of those as well. So they'll make custom donuts for you on the spot. It was outrageous. Why do I bring this up? Not to rub in whether or not I was in Vegas. But it was my first chance to test out my iPad as my only dev device traveling experience.

Oh, okay. First trip since I'd tried all those tools. And it worked pretty fabulously, actually. I just took my iPad and my ergonomic keyboard so I don't give myself carpal tunnel again. And it worked great. So all those tools we talked about, people, maybe I'll find a way, if people want to know, I'll find a way to link back to that episode. Able to monitor the Michael Kennedy empire from remote. Indeed. Yeah. I even, I busted in and installed some Linux security updates real quick and then dropped back out. It was all good. Tailed some logs. Yeah, it was fine. Mostly I wrote email.

No, that's what my empire is about, is write email. All right. Here's another good one. Linux, Amazon, Meta, and Microsoft want to break the Google Maps monopoly. So apparently Google Maps owns a significant portion of the mapping data and the street view data and so on. But all those groups I mentioned are working on creating this thing called Overture Maps Foundation. And they're going to create an open source, open to everyone set of maps by pulling in some of existing open map data, but also really doubling down on trying to add more information. That's pretty cool, right? So if you want to do anything with mapping, that might be getting a lot better. Yeah. I mean, I haven't seen the word TomTom in a long time. I know. I actually had my hands on a TomTom device

this summer. It was insane. All right. Last one before we get to our joke. I decided my laptop that had zero stickers on it was boring. So I went completely bonkers and got a completely uh i went to redbubble ordered an insane number of stickers and put on my laptop what do you think of this thing brian i'll put the link in the picture in the show notes i like the laptop opening and closing tags um yeah it's got a bracket laptop at the top of the laptop and it's got a bracket slash laptop at the bottom yeah you got python bytes there it's nice i gotta get you a testing code sticker yeah get in there we'll find a spot i replaced the mac icon with a terminal icon so it's pretty good yeah anyway that was a lot of fun okay but that's that's it for my extras how about yours i had a few extras so i want to i just uh don't have a

lot to add to these but i'd noticed it's kind of interesting. So GitHub, I think this was on December 15th, announced that they're able to check for security leaks. And I'm linking to an article that they can check your GitHub repos for security leaks, like secrets and stuff. Apparently, that's how a lot of secrets get leaked is people like you know actually merge request a bunch of secrets or something um uh but the uh uh so it's kind of cool that it's there i went there's some it says you have to turn it on i went to check one of my repos and it looked like it was already on so um i may as well check it anyway so um i wanted to shout out to uh julia Julia Evans to get that right. Julia Evans.

She's written a whole bunch of great stuff to help programmers, but she just released Pocket Guide to Debugging. And I was pretty intrigued by this. One of the one of the things that highlighted was asking for help and writing a tiny program that reproduces the problem. Those are two things that I really encourage people to do before they waste too much time. So I might throw some money away because those are really great free tips. Yeah, that's cool. There's a, I found this, what, Python Easter eggs repository. Not all of these work anymore.

So I tried the first one, the import hello on Python 3.11. It doesn't seem to work. But a bunch of these others are really pretty fun. Import this, of course, is there. One of the things that I think I knew about at one point, but I forgot about was the from future import Barry as Fluffle. So this was a joke from 2009 that's still in Python 3.11

that it's Barry Wars. It's like a joke from barry warsaw and brett cannon um and if you do that it completely it removes the uh not equal like the pound like the the bang equal sign with um uh less than greater than for not equal so the joke being barry was the dictator for life, he would change it. Anyway, it's kind of funny. Fluffle stands for friendly language uncle for life. And there's even a pep for it.

So you can read about the pep 401 with this. Anyway. Nice. Just a few extras. It could have been our joke. That could have been our joke section. That could have been our joke section. But we have another joke, don't we?

We do. So I told you we'd wrap this up with some stuff about the cloud. So here's a nice little conversation a father and a daughter are having. They're out walking in a park and looking up at the sky and they see a cloud, mostly empty sky. And the kid says, Dad, what are clouds made of? The dad looks down and clearly says, Linux servers, mostly. That's good, right? Yeah, that's good.

Even when they're serverless, they're probably still Linux things running in Docker, running on a Kubernetes cluster, running on Linux. Serverless is a hilarious word. It just means you don't know where the server is. Yeah, it's like server's not my problem. Yeah. Well, this show is not a problem. It was a lot of fun. It was a lot of fun.

Thanks so much for joining us and showing up again. And thanks everybody in the audience. Yeah, absolutely. Hey, before we go, I have one quick more announcement really, really quickly. Next week, for those of you who are kind enough to drop in and join us live, we're not doing a live show. We're going to do a crossover episode with TalkPython and vice versa. Python by its own TalkPython. And Brian, we're going to be on vacation.

So I'll see you in January. All right. Talk to everybody in January. Yeah. So happy new year, everyone. See you later. Bye.