Python Bytes - #387 Heralding in a new era of database queries

Episode Date: June 11, 2024

Topics covered in this episode: Dataherald Python's many command-line utilities Distroless Python functools.cache, cachetools, and cachebox Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/387

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 387, recorded June 12th. And I'm Brian Hawken. And I'm Michael Kennedy. This episode is sponsored by Scout APM. Listen to their segment later in the show. Connect with your hosts, as always, through Fostodon, at Adam Kennedy, at Brian Hawken, and at Python Bytes. We'd love to hear from you.

Send in those topics too. You can also, there's a contact list on pythonbytes.fm. You can send things that way too. And if you'd like to join the show while we're recording, like some of the people right now, you can join us on YouTube at pythonbytes.fm slash live to be part of the audience. It's usually Tuesdays at 10 a.m. Pacific time. And you can also use that to find the link

to catch older episodes as well. And finally, before we get started, if you'd like to have all these topics and the links to everything we talk about sent directly to your inbox, you can go to Python Bytes to FM and become one of our friends of the show and join that list and we'll send those emails to you every week. Super useful. A lot of people are signing up to them. Brian, really quickly before we jump into the topics, you mentioned Vostodon. I got a comment on YouTube, I suppose is the right place to, uh, where it came from. Somebody said, this guy, me, has gone completely crazy because he joined Mastodon.

Then he joined invite only Mastodon because Fostadon went to invite only. I'm like, but, but you can join any, there's thousands, there's thousands of servers, join any one to any one. So you don't have to be on Fed on directly to talk with us it's just the same you know most people would talk to you or not on fasted on if you want an invite to that private server or invite only server which i think that's just uh trying to limit the scope of for the admins and the cost of it and stuff happy to send you one but yeah just any old mastodon will do it doesn't have to be fostered on in particular It doesn't have to be Fostodon in particular. It doesn't have to be. It's not Slack,

it's not Discord, it's something else. It's the Fediverse. But if you really want to get into Fostodon, go into the contact form on our site and send us an email and one of us will send you an invite. Exactly. Well, how should we start the show, Michael? Let's herald

in a new era of SQL interactions, shall we? With the data herald. So this is, it comes just under the organization data herald. And the idea is you have a SQL database like Postgres or Microsoft SQL server or whatever, but you are not, either you don't have SQL skills or you don't want to bother your SQL skills. This is kind of like for me with regular expressions, right? Like I can do regular expressions, but I will take a lot of steps to not actually do regular expressions.

You know what I mean? So this is like that for SQL. It's a natural language to SQL engine built for enterprise level question and answering on top of relational data. So maybe you've got a database with a bunch of analytics, bunch of customers, whatever contacts for like a CRM type thing. And you want to let your somewhat technical people just talk to the thing instead of making them learn SQL or use some kind of BI tool.

You just put this thing on top of you. Like, tell me all, you know, how much sales did we get from the Philippines last month? Something like that, right? Which you could write that as a SQL expression, but you don't have to. Yeah, so some of the features are you can allow business users to get insights into data warehouses without bothering the data analyst. You can enable Q& a chat on top of production databases inside your sas application that one sounds a little risky but you know

maybe internally it's fine you got like a self-hosted one or something like that or you could create a chat gpt plugin for your proprietary data even pretty pretty interesting stuff right this is super cool yeah yeah yeah I haven't tried it yet it's half Python half JavaScript 0.3% Docker but the 0.3% Docker

comes from the fact that if you want to just run it locally you can just Docker compose up basically so they actually have that all

put together in a script but you just Docker run off it goes yeah so it's made up of a bunch of different things, an engine, some enterprise rules like permissions and stuff, an admin console. And interestingly, one final thing to throw out here before we move on is a Slack bot. So you can add

the data herald for your database to your Slack channel and then talk to it as part of your Slack conversation. What's the deal with this? I don't know. Let's just ask the data herald, you know, and go and send it a message. And I think that's actually pretty interesting. I think it's, yeah, it's funny. It's interesting and kind of funny. I kind of forgot that a lot of people are still using Slack.

Are you slacking off? You're not paying attention anymore, right? I'm slacking off, yes. I still have a Slack. No, yeah. I have a Slack channel for the the course and the the book but it doesn't give a lot of activity so yeah i'm always torn about these things i want to set up something like that but discord seems to have a little more traction for public places like that the slack pricing has gotten gotten really weird is for free on open source stuff and big projects it's like it's got a 90-day limit on it on and it

throws stuff away which is a problem so yeah exactly I want to set up rocket chat which is pretty awesome it's like self hosted or you can buy their hosting but it's like open source a lot nicer white label etc etc etc the only thing is, if you want to get people to come in and use it and they, you know, do they want to have like another app open and remember to go to another place? I think that's the challenge with Discord and Slack. It's like, well, I already have that open. So now I get notifications for that versus here's one more thing to run. But yeah, this is something I'm thinking about setting up.

It could be fun. I switched the PyTest course recently from Teachable to Podia. run but yeah this is something i'm thinking about setting up it could be fun um i switched uh the pytest course recently from teachable to podia and podia has a has a community aspect where you can ask questions and answers and stuff like that i haven't turned it on yet but i'm planning on doing that soon just because it solves that login extra login problem they already have a login to get to the course right exactly and i can put it right in there. Anyway. Cool. Maybe someday. Alright, well, Data Herald, check it out, people.

If you want to talk to your SQL. Talk to your SQL. Talk to your database. Alright. If you'd like to talk to just random stuff in Python, I don't know how to transition to this. The topic I want to talk about is an article by Trey Hunter

called Python's Many Command Line Utilities. And I knew a lot of this stuff was around, but I always forget about it. So I love it when people bring this up. So the thing is, so if you, in Python, all the built-in stuff, there's standard library stuff. And usually you get at it from your Python code by saying import something, like import data classes or something like that but some of the stuff has

um has activities built into the dunder main of it uh in the the sub project so that if you write python dash m and then the module built-in something happens. And there's a bunch of them. So this is what this article is about. And the famous one that is Python-M anti-gravity, which pops open in the KCD comic, which is kind of awesome that we have that in the language. I never get tired of that one. Yeah.

But it does the import this thing. But so there's a whole bunch of cool tools that I was looking at. There's general purpose tools that I didn't even know was there. Calendar is one of them that is actually really handy. So if you just type Python dash M calendar,

it prints out the current year's calendar, which actually there's a, I used to have a desk calendar. I don't have a desk calendar anymore. So having something like that pop up really easily, really helpful. And you can also give it, you can also give it a specific date.

So if you're like, well, what's my anniversary going to fall on it a couple of years? You can, you can look that up. So really handy to have that around. There's the web HTTP server, which is built in, which of years. You can look that up. So really handy to have that around. There's the HTTP server, which is built in, which is fun. JSON tool is pretty awesome, actually. JSON tool lets you nicely format some JSON data.

Some great stuff there, general purpose. There's categories that Trey has put these into. And I love the especially handy on a Windows machine category. These are Unix-y utilities that I often miss when I'm on a random Windows setup, and I never really remember how to get them safely. And so having them built into Python, I didn't even know these were here, like gzip and tar.

I didn't know that I could use gzip and tar easily on Windows with just Python, built in Python. How cool is that? There's a FTP lib has a built-in FTP utility. It's kind of a quick FTP client. Nice.

I don't really need Telnet very much. But some of these are pretty nice. IMAPlib, I didn't know that you could curl to read email through Python, cool. Going down, analyzing, these are analyzing Python code. Huh, tokenize looks fun. AST, actually I did know that. Using AST to get an abstract syntax tree

for some code from the command line. A couple of the ones, I wanted to highlight is working with code. I don't know. There's just a whole bunch of fun stuff in here. You can get hello world as a command line argument or the command line tool. It prints hello world. That's funny. Sure, why not? Python-m this will uh do the zen and

python but and a turtle demo and yeah anyway um fun to check out and uh these are pretty good stuff good stuff here so yeah there's a lot more there than i imagine most people know that's pretty awesome pretty awesome and there's even a little demo of using C profile and P stats for getting some profiling of your code. Very cool. All right. What else is cool? I would say our sponsor. Let me tell you real quick about Scout APM.

They're big supporters of Python Byte, so we appreciate that very much. So if you are tired of spending hours trying to find the root cause of issues impacting your performance, then you owe it to yourself to check out Scout APM. They're a leading Python application performance monitoring tool, APM, that helps you identify and solve performance abnormalities faster and easier. Scout APM ties bottlenecks such as memory leaks, slow database queries, background jobs, and the dreaded N plus one queries that you can end up if you do lazy loading in your ORM

and then you say, oh no, why is it so slow? Why are you doing 200 database queries for what should be one? So you can find out things like that. And it links it back directly to source code so you can spend less time in the debugger and tailing logs and just finding the problems and moving on. And you'll love it because it's built for developers by developers. It makes it easy to get set up.

Seriously, you can do it in less than four minutes. So that's awesome. And the best part is the pricing is straightforward. You only pay for the data that you use with no hidden overage fees or per seat pricing. And I just learned this, Brian. They also have, they provide the pro version for free to all open source projects. So if you're an open source maintainer and you want to have Scout APM for that project, just shoot them a message or something on their pricing page about that. So you can start your free trial and get instant insights today. Visit buythumbites.fm slash Scout. The link is in your free trial and get instant insights today. Visit

pythonbytes.fm slash scout. The link is in your podcast player show notes as well. And please use that link. Don't just search for them because otherwise they don't think you came from us. And then they'd stop supporting the show. So please use our link pythonbytes.fm slash scout. Check them out. It really supports the show. Definitely. definitely. Thank you, Scout. Okay, how about we talk about Wolfie? Oh, neat. Do you know Wolfie? No, but I like the name. The name is good.

So Wolfie, this comes to us from Patrick Smith. And this is a project from Chainguard, which is a company that makes specialized Docker images. So the idea is, it's based on this idea of distro-less images. Had you heard of distro-less images? I never had. I'm like, what is this? So normally a Docker image, it shares the Linux runtime kernel,

but then you layer on whatever else you want to add. So, you know, you get the Ubuntu image, it comes with Apt and Bash and all those kinds of things. Systemd, which you're very unlikely to be using in Docker and so on. So those are helpful to have if you might want to run across them, but they're also, they have a couple of problems. One is they just make the images larger. So you might have a gigabyte image

instead of 50 megabyte image if you're going to push it to Docker Hub or something like that, or other people are going to download it. And you'd rather have that smaller, right? Yeah. And the other one is those things all open up security vulnerabilities and bugs and other issues, right?

If there's a bug in something that you weren't using, but it's there, you know, possibly somebody could use that to level up and hack your system, right? That would be less than good. So these often, these distro-less images often, or containers often come with, without even a package manager like Apt or a shell at all,

which makes them very small and very constrained in what people can do with them right so you can't like open up a shell and do something if literally the shell doesn't exist however debugging them is a challenge because you know one thing you can do with a running container that is misbehaving as you can say docker exec container name, bash or sh, and then get in there and sort of do exploratory stuff. Right. So that is a drawback, but they are way more secure, which is pretty cool. So this Wolfie thing, they make specialized distro list containers now that we know what those are. And they're just super stripped down, but so much so that in general, the distroless images are so stripped down that you can't run Python, right? For example, or other other things. So these come with, where is it down here somewhere? Anyway, it comes with, it's basically distroless images that are built specifically with just enough to run Python. And in the show notes, I've got a couple of examples of using like really restricted versions of these images,

as well as some developer ones. And the developer ones have pip and bash and APK and so on so that you can go in and get it working just right. And then once you're sure it's good, you just push it out to production and count on your logging, I guess. What do you think? Interesting? Yeah, I think so. Yeah. Much smaller, much more secure because you can't do anything.

And I found the answer. My thought was, why Wolfie? And it's inspired by the world's smallest octopus. Yes, of course. The wolf octopus. No, of course. The wolf octopus. No, wolfy. It's very cool. Octopuses are cool as well.

And, you know, more and more projects and even projects within companies are caring about software build materials or S-bombs. And so minimizing the things that go in your distribution and helps to make that smaller, your SBOMs smaller. So having that be part of Wolfie, including high quality build time SBOMs is a good addition there. That's cool.

Yeah, it's cool you caught on to that. That's another nice thing is they specifically call out all the software bill of materials and having so much less, well, it makes it easier to do so, I suppose. Yeah, yeah, definitely. Neat. Yeah, indeed. I ran across recently some, I really like caching.

So things like the idea of a memoization or caching of a um uh memoization or it's uh or caching of a function so is is where um if you run a function once it does it does the work but if you run it again with the same input it just memorize it remembers the answer but there's different ways to do that and uh i'm going to highlight a few that i thought were interesting um Functual's cache, of course, and I really like that Functual's cache has the cache shortcut because you can just say at cache for a function. And for a lot of stuff that doesn't have a lot of different input, it works great. So I've got some projects where it'll be a lot of work, but I call it several times,

and I could cache that externally, but just throwing cache over the top makes it work great. So for a lot of cases, just the built-in Functools cache works awesome. If you need more detail, cache is actually just a shortcut into LRU cache with the defaults filled in. But you can set the max size or set type

to also with LRU cache. However, you might need more control. And that's where some of these other things come in. So there's an extension or a third-party library called cache tools. And cache tools has like just more, you got more control over it you've got um you can use an lru cache or a ttl cache ttl caches are interesting because they have time to live

built into it so the cache can disappear after you know minutes or seconds or whatever makes sense for your application um to remember things for a while but then let them go. That's super handy. Yeah, really handy. Not just because it alleviates the cache management from it being too much, but one of the big problems in computer science is cache invalidation, right? Yeah. If it's doing something, I'd say like calling an API,

which is the example right on this page, is you don't want to just hit that. For example, there's a weather API at a location, like, given the location, the weather doesn't change that fast. So let's only refresh every 10 minutes. You put that on there and boom, and it's completely managed it, right? Yeah, and I like the example of weather

because we know weather doesn't, the weather forecast isn't going to change that much in the next, you know, I don't know what the 600 is, but... I think it's seconds, 10 minutes. Okay, so yeah, it's not going to change in 10, you next, you know, I don't know what the 600 is, but... I think it's seconds, 10 minutes. Okay, so yeah, it's not going to change in 10, you know, refresh it every 10 minutes. But for various other APIs, you might know about the API,

how often it's updated. And if it's only updated once an hour, you don't need to hit it every time your code hits it. You can cache that. That's pretty neat for TTL. I love that. So building on top of this is another tool called CacheBox. So the other one, first one we talked about was CacheTools.

Now CacheBox is a similar kind of thing. We've got lots of APIs that we can use for it, like LRU Cache and TTL Cache. It's a similar API as this other one but there's also like an lfu cache at least lf what's lfu least frequently used frequently used i bet yeah um and uh so some other things but the fun one of the fun things about this is um it's uh written in rust um so to for a of reasons to speed it up, um, a little bit.

And I mean, uh, but there's also, um, you could probably do larger objects in there because you're, it's a low memory usage. It's gonna, um,

it says it's a, a third of the size of a dictionary. So you're gonna, you're gonna use less memory than you would storing it as Python objects. Um, so that's pretty cool. Especially if you're, if you were're caching the output of an API that's a big JSON blob or something.

So really cool projects if you're trying to speed things up or save memory with the caching. So, nice. Yeah, that's very cool. I did not know about it. I'm excited. Yeah, that's very neat.

CacheBox just showed up a few days ago that I was aware of I thought oh this is interesting and yeah it looks like it's only a couple weeks old for the repo yeah very new excellent I will definitely give that a look alright any extras

I have I had one extra but then our conversation made me have two extras. So I'll just throw it out there. I'll just throw into the show notes, the rocket.chat. So people have that. We already talked about that. So Brian, I've been for many, many years,

a fan of bartender, not the drinking, but the Mac app. Yeah. You know, in Mac, you have this bar across the top. It's sort of equivalent to the task notification area on Windows. And for some reason, putting stuff up there on Mac is super popular. I just counted.

I have 28 icons in that thing and it's ridiculous. Being able to control that. It's not a little nice. Yeah. It's super nice. Especially if you're doing presentations like at a conference or for courses or live streams. You can just hide more stuff away and you don't have to have it there, you know, showing weird notifications and stuff. But it turns out like this, this bartender app is pretty cool.

There's some uncertainty about its current status though. So Bartender was recently sold to another company, but no one announced that it was sold and they were trying to kind of like, there's nothing to see here until App Monitoring Service said, you know that this is looking a little suspicious. Like maybe people are buying this to do more deeper tracking

and not illegally, but just sort of digging deeper into seeing what you're doing. This is bad because it has full screen recording access, access to hotkeys, all sorts. So it's, it's got a lot of permissions, this app. Yeah. That, so on Mac rumors, let's see, where's the, yeah. So the original article was PSA bartender Mac app under new ownership, but lack of transparency raises concerns.

Like the guy who sold it didn't even, wasn't willing to acknowledge it until months later. And the company that bought it still doesn't, won't talk about it. So much like a little sus. Anyway, so I want to link to a thing on Mac rumors. Sorry, Windows people. I know this is not that interesting but uh for you but six bartender alternatives to manage your max menu bar and a lot of them are open source so for example i'm using ice which is available on github with like 8 000 stars you know pretty nice guess what? It works almost exactly the same.

It's free. It's open source. You know, why didn't I just use this earlier? There's a few features coming that it doesn't quite have, but for the most part, I think it's really good already. That's pretty cool.

I should check that out. Yeah, so I've banished bartender just out of abundance of caution sort of thing. All right, that's it for my extras. How about you? Just a couple let's see um i just wanted to let people know uh the python 312.4 is out so um decent i haven't checked to see what is new in dot four probably book fixes so don't know um but uh it's there um and then another another recent release is um uh i don't usually keep up on these two tightly but vs code um the june this python for visual studio code

the june release uh includes um a pi test improvement so uh there is there's an ongoing that's been going on for a while they're they're rewriting some of the Python testing functionality. And there's a link in this, in the release note to, to how to, how to change that so that you can see it. But if you're using the updated testing rewrite, there's some changes that should, should help just help, help things work a little better. So very cool. Nice, nice, help things work a little better. So, um, very cool. Um, nice, nice additions.

That's all my extras. Yeah. I, um, want to say that when I saw you put into the show notes about an hour ago, or when I saw that they were in the show notes, Oh, that three,

uh, three 12.4 is out. It's quick updated a number in the Docker setup. Boom. Um, all the apps are upgraded to three, four and 13 four which is uh three thirteen four which are twelve four which is pretty awesome that all works right all right

uh how about a joke that sounds great so before i show this you know in css we've got different different settings you've got visibility importance opacity What if you made a series of superheroes or villains or cartoons out of CSS? Well, that's the joke. So I'll link to this and we won't go through all of them. You guys can check it out. But it's called CSS Cartoons by Alvaro Montoro. And we've got the cloud developer. And the cloud developer, this is a CSS class,.cloud developer.

Background is Azure, the color. Isolation is isolate. Rest is none. Stress is a 100. And content is none. They have cool little pictures of graphics of each, so check them out. We have a king. The position is absolute.

The richness is 100. The color is royal blue. And it says, you got to know your css it says ampersand greater than colon first child so stuff can you know the first child of all the things contained directly within it all all colon inherit isn't that awesome uh let's see this is great we've got phantom of Phantom of the Opera, Brian. There can only be one of these, so it's not a class. It's a ID. So hash Phantom of the Opera. Mask URL.

Hash face. Back face visibility hidden. Visibility hidden. Pitch is low. Volume is soft. Phantom of the Opera. All right. We have a winner.

Order is one. Running first. All right. We have a winner. Order is one. Running first, color gold. And then after, you know, colon, colon, after content is the, like the first place trophy. We have a dictator. See the dictator down here in the bottom, Ryan? See how he looks? He's so forbidding. A dictator is a class because there can be more than one, of course.

Position fixed, exclamation mark, important, cannot be overridden. Will change, order, important. Opacity, zero, important. Everything's important because of the dictator. You can't override it or change it. Border, double solid. And then.country has, not.supporter. Will change, orphans, widows, important.

And translate, 100 V v max to zero anyway they just goes on and on like the scroll bar goes forever we got a basketball player i i encourage people to check this out it's pretty awesome that's pretty good yeah yeah css jokes or cartoons very good all right another awesome episode thank you, Michael, for meeting me today to talk about Python. As always, thank you everyone for listening. See you later. Bye.