Python Bytes - #320 The Bug Is In The JavaScript

Episode Date: January 24, 2023

Topics covered in this episode: markdown-it-py Sketch Fixing Circular Imports in Python with Protocol unrepl Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/320

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 320, recorded January 24th, 2023. I'm Michael Kennedy. And I'm Brian Ocken. And this episode is sponsored by us over at TalkPython Training and Testing Code and Patreon supporters. We'll talk more about all that a little bit further into the show. And also, yes, for those of you listening live, you already know this,

but for most people, they may not know that we're live streaming this on YouTube. And if you just go to pythonbysetfm slash stream slash live, then you'll be able to follow over there. Usually on Tuesdays at 11 Pacific time, which it is right now, you can be part of the show.

That's how some of the folks are in the audience. So hopefully you go over there and subscribe on YouTube and press the notify so you know when we're posting live streams. And with that, let's keep going, Brian. Let's mark it down. I'm going to talk about markdown a little bit while you're queuing up your next item.

We do pay attention to Rich and Will McCougan in the show, and I'm not going to talk about them too much, except for that I noticed that Rich updated, got a release note that they updated their Markdown parser. And from something called CommonMark, which I was familiar with, to Markdown ItPi. And I was curious about this, i i went and took a look um so the common mark library if we go take a look at that um that's up on pi pi common mark no big deal but if we go look at the source um in github it says this repository has been archived by the owner on November of 2022. Oops.

And if we look down a little bit further, it says, warning, it's deprecated. We recommend using markdownit.py. And okay, I haven't used it. So I wanted to go take a look. So markdownit, I think I'm saying that right, is it's built on top of another, it's another markdownit tool.

So Markdownit is its own tool and the Markdownit Pi is a Python wrapper around it. So great. So it's, do we have to worry about this one going out of fashion as well? And I think we're probably safe because the project, this is a part of Executable Books and Executable books is a project to try to build more collaboration with Jupyter notebooks. So I think we're safe now to use Markdown for Pi for a while. And I just did a quick, there's a live demo thing. There's converting Markdown. It's kind of a nice package.

You can do it by itself. You can convert from Markdown to HTML on the command line, or you can use it within your code. So you can use, and it's got some nice features for in-code, pulling in different plugins and stuff if you have different specialties. And if we go back to the original release, one of the reasons why they did it wasn't just, well, the reasons why Rich started using it isn't just because it's been um uh the common mark's been deprecated but

they say it will allow us to implement a number of additional markdown features in the future so this is pretty exciting and yeah i'm i'm i don't know a lot about markdown it but i am gonna check it out. Well, if it's used by Executable Books and it's used by Rich, I think those are some pretty solid endorsements for it. I saw that it got installed as well, and I wasn't sure why, but now I see that it's from Rich. Yeah.

I use Markdown too, and I have no idea what the status of it is, but there is so much of our website that runs on Markdown. Like every episode page you visit, that's a Markdown page. There's a bunch on TalkPython, a whole bunch of TalkPython training. So anything that's better for Markdown, I'd definitely be wanting to give that a look. And I don't want everybody using the same thing because we do want some different tools to build on each other and stuff. Indeed.

All right. Should I sketch out the next idea for you? Yeah. Let's take a look. Let's take a look. Oh, my God. This is fantastic. So this was sent over by Jake Furman.

And Jake said, have you seen Sketch? So we've seen a lot of AI coding assistants. We've talked about the potential bordering on violating license agreements and stuff of GitHub Copilot. Everyone has surely heard about ChatGDP. I have friends who are not even a programmer. They come, Michael, I'm working on some programming thing because of chat GDP. I'm like, all right, well, interesting. We'll see where all that stuff goes. But this thing called Sketch is like a simple view, a simple way to ask natural language questions of your data that's contained in a panda's data frame. Ooh.

Yeah. Okay. So the way it works is you go and it's not built into the editor. It's part of monkey patches, I believe pandas, you know, so it adds a function to pandas called, I think, ask. Let's see. What is the way we, yeah, it's dot sketch dot ask or dot sketch dot how to, or sketch dot apply.

So these are the different kinds of prompts that you can give it. Okay. Okay. So you just pip install this. And once it's pip installed and you import it, that's all you got to do. And it lets you ask data cataloging questions like, is there any personally identifiable information PII within here? Like, is there a phone number or is there an email address

or is there a social security number? This could be some big, large data set, and you don't want to save necessarily that or share it. You can ask it, and it'll ask you how to, it'll basically tell you how to get rid of it as well. Also, give me descriptions of this data set, okay, which is nuts. It also does data cleaning and masking to, for example,

removing the PII. And it'll extract some features and just lets you visualize stuff. So the best way to do this is to watch this 1 and 1 1⁄2 minute video here. And so what it shows you is here's a Pandas data frame read from a CSV. And it's got an order ID and a price and a date and an address and they just say data frame dot sketch dot is there

any personalizing i uh information or any personal information this is yes these fields and says how can i remove it and so it writes out the code you just copy and paste that boom now it's removed and then it says give me a friendly see if i can pause this going quick um it says can you give me friendly names stop moving can you give me friendly names in a single sentence description of each column format output as an html list and boom it says index the numerical index product the product purchase in each order order id a unique identifier for each order, a purchase address, the address of the customer who placed the order and so on and so on. Isn't that neat? Yeah. Yeah. So it, and when it spits out the answer, it puts a little copy

tag. So you can copy, you can ask it, how do I do this? You know, how to extract city, state, zip from the address, which is a single string, and it writes the code to do the split statements to pull it apart. I'm telling you, this thing is, it's not going to be perfect, but it's going to be pretty good. It's pretty interesting to just jump in. What do you think of this, Brian? I think it's great. Hey, Sketch, how do I remove the data that disproves my hypothesis? Exactly. I'm going to need the data that shows a downward trend here. Can you extract just those rows?

So anyway, people can check this out. I think it's pretty neat. It says, in the future, we plan to update the prompts at this endpoint with our own custom foundation model built to answer questions

more accurately than GPT-3 can with its minimal data context, because it doesn't know a lot about your data. You can also directly call OpenAI directly, it says, and not use their endpoint by using your own API key. So there's that. Anyway, it talks about how it works and what it sends over and stuff. So this is pretty powerful, key. So there's that. And anyway, it talks about how it works and what it sends over and stuff.

So this is pretty powerful though. I'd really, because there's so much data for people are using and different things. We kind of expect it now. I was working on a project just recently where somebody, things like test result analysis and a question of like, can we just put natural language strings in there to query it?

Why do we have to? And so people are kind of expecting this all over the place now. Yeah, I just want to highlight just the last one there. It says, go to the data frame, sketch out how to. Using Plotly, plot a map of the total sales in each state. Boom, you get a perfect interactive map, a geographical map of the total sales in each state boom you get a perfectly in a perfect interactive map um a geographical map of the united states with that's pretty neat yeah to be able to just jump in and do that like sure you could go search and look that up but you know instead of just doing

df.head df.tail um and so on like how to i could think I'm ready to hang up a shingle for like a data science side job. I'm pretty sure that you and I could pass as at least junior data scientists now. Just with this. Yeah. Anyway, I think this is pretty cool. I'm pretty psyched about it.

It's pretty cool. Yeah. People should check it out. From what I can tell, it doesn't cost anything. So just play with it, which is really nice. And also, before we move on, I just want to tell people that this episode is brought to you by us, right? So there's a whole bunch of Python courses that if you get inspired over at TalkPythonTraining, we've got the Python 3.11 guided tour, which is a couple hours of hands-on

showing why that's awesome. Python data visualization. So if the sketch stuff was interesting, like there's a bunch of that there. And oh, there's also something on PyTest, isn't there? Yeah, highly recommend that course. It's great. I do as well.

All right, so if you want to support us, the best way you can do it is to support our other work as well as share the podcast with your friends. So thank you. Thank you. Thank you for that. Yes.

I also want to thank the Patreon people because they're great. We don't really talk about it much, but you can support us through Patreon as well. And we very much appreciate it. Absolutely. All right. What you got for our next one, Brian? Well, I want to go in circles a little bit. Actually, going back to the year 2021. No,

in 2021, we covered a article by Hinnick called subclassing in Python Redux. And it's a great article to talk about like just how to subclassing and dealing with classes in Python and just to get your head around it. So I did read it and I enjoyed it. And then this year I came across the problem and I'm like, I think I can solve it with something I read in this article. So I'm going to scroll down to the thing specifically. And it's called, so we have nominal subtyping with abstract base classes. I tried that, but I decided to go with structural subtyping.

And with structural subtyping, basically, my understanding is it's like duck typing, but it's like strict duck typing. And it says here, as you can see, if you've got something calling two things, two classes talking to each other, one of them doesn't have to know about the other. So in this example, we've got a reader and a foo reader. And, uh, you know, it's just a reader protocol. We just need to know that there's a reader protocol here. So I kind of like that. I just knew this was here and I'm like, how do I apply this? So I went ahead and wrote up an article called fixing circular imports with in Python with protocol.

And this is directly from this. So the it's just a zoom in on one special case. I really have like if I've got a class director and actor and they talk to each other. So director tells an actor to do an action. No big deal. It has to import the actor. Right. So the actor gets a director. So I'm going to it gets data from the director. And so it has to be able to call it. Well, I could just pass in self. And since we have duck typing, it all works fine that we just call director get data.

So far, we have no types that The problem comes in when we have types. If we add types to the whole mix, then when we add types to the director, it's no big deal. We just add types to everything. But to add types to the actor, suddenly the actor needs to know what type the director is. And that's where we get the problem. And that's exactly the kind of problem that I ran into is I wanted to add typing to this, to a similar problem, and it blows up. It actually blows up really bad. Python just says you can't do that. It's circular import. So the protocol solution is really slick. And I'm going to scroll down to the answer. The answer is just, I just need a little bit of a snippet of code that says I've got this class

that is derived from protocol, but it's just got a get data function. And that's all I know about it. I know what it returns and that's all I know. And now I can say that my director really is that type and that's it. The actor has to change to say what type it is and the rest of the code just works. And I the actor has to change to say what type it is um and the rest of

the code just works um and i don't have to change the director i don't have to change the test code it's just this uh it's just this one file that i'm changing to be and i like this because i i'm adding typing i'm not changing the code at all um and the only thing is that uh yeah anyway so it's a slick way to use it i think so too it's a little bit like interfaces but less you know in the traditional java or c sharp the i enumerable or whatever type of interfaces yeah the first uh and then uh hynek goes through uh uh abstract base classes also and that's one thing you could do is you could have a base class that has this get data function and both the director and the actor know

about the base class, but you don't really need that. Nobody else needs to know that except for this one file. So protocol's a nice hack, so. Interesting, yeah. Jonathan, the audience is asking why not just use abstract base classes. They both work. So it's just a personal preference thing.

So, yeah. And Kim in the audience is asking, as a question of style, do you prefer an ellipsis over the keyword pass to indicate an empty function? Well, I'm warming to the ellipsis a lot because I write a lot of test code. And I write a lot of test code and I write a lot of test example test code. And if I say an empty, an empty function that is a, is a test function and I use dot, dot, dot, it's clear that I'm, I'm not finishing it. Whereas if I say pass, I've actually had some people get confused and think that the pass keyword is how you pass a test it's not it can be uh if you've got no you're not going to raise an error in that that test

I mean it's sort of a way to pass it yeah um yeah yeah I kind of like it too to say look I don't I don't want anything here I don't intend to come back and fill it out. Just this is it. And it kind of matches, the ellipses match how we do type stubs as well. I think I'll use ellipses, but I don't know if they have to. I think they do as well. Like, if you go and look at typeshed and you look at the stubs in there, like, let me just grab one rando yeah they all have all those have dot dot dots dot dots for the default value dot dot dot for the implementation

kind of gnarly looking if you open them up but there they are so yeah to me i kind of feel like i'm matching the stub definition like i don't intend anything to be here. Exactly. Right. All right. Ready for the next one? Yes. This is a simple and I think will be useful to many, many folks. This one comes to us via Rudvanderham.

Rudvanderham, thank you. Sent it over and recommended, also created it. So we've all seen code samples that, I don't know why, but I guess it's a decent way to explore it. But I've always found it a little bit difficult to consume. Basically, I want to show you how this bit of code works in the REPL. Here's what happens. Copy this and use it, right? And when you see that here,

you have the triple arrow and you write some code, the triple, you know, right arrow, greater than, greater than, greater than, right? You might do a for loop, which then has the indents and it does a triple dot, dot, dot to show you that,

well, now you're still in the same command, you know, and then eventually a closing dot and you run it. And if you print out, you just say a variable or a response, then that gets printed without any of the dots or the greater thans or anything. It's just straight up. If you said print that, right. So if I do like copy this and I want to explore it, I've got it. Okay.

You know, remove all the triple dot, dot, dots, unindent it correctly, remove the arrows, a common, you know, it's just like dot, dot, dots, unindent it correctly, remove the arrows, uh, common, you know, it's just like, uh, yuck. Right. It's just a hassle. So enter un-reple, the, the inverse of repl, where you say, take this code and make it runnable for me. Isn't that cool? Yeah. Super simple. Like even if you're in the repl, you can't take what you're copying from someone else's example and put it in the REPL because then you'll have triple greater thans. And so you just take this code

and the way it works is you copy it to the clipboard, your code sample that you got from this REPL thing. And then you just call unREPL as a CLI and it replaces the clipboard with contents of exactly what you would want. And it even comments out the output. So what wasn't code is now a code comment. Oh, wow.

Nice. That's perfect. Yeah, really perfect, right? Yeah. So I can certainly see that this is a handy thing that people may want to install. So when you see the instructions here, it says you can just use unrepl.py.

And the way that you run it, it says up here somewhere. OK, what you do. So what you do is you copy it, and then you run. OK, he's updated. It just says unrepl. It used to say run unrepl.py. But it's also on PyPI, which means that you can pip install it. But I would not pip install it.

This is not the thing that goes into a virtual environment to be part of a program. This is a CLI tool that you just want to have available. So I would replace pip with pipx. And then you just globally have this command on your computer. It just happens to be implemented with Python. But you copy it, just type Unrupple, and then whatever was in the clipboard is now better. It doesn't say use pipx, but I think that's the way.

That's certainly how I would do it if I'm going to install and use this thing. There's a way to set up an auto hotkey on Windows only. I have no idea about this, so you all are on your own there. Okay, so how do you do how do you use it on the command line you're just um you just type the word unrupple once you've installed it in the path which pipx of course does and then you paste your code or use it to yeah so you copy your code type unrupple on the command line it looks at your clipboard oh okay unrupples it and then it replaces the clipboard, like, basically with a new copy command.

Oh, that's neat. Yeah. That's great. Exactly. Yeah, so really easy. It also says you can import it into a program and use it if for some reason you feel like that's the thing you want. And it has special handling of the underscore,

which has special meaning in the REPL. Like, it knows about underscore and treats it accordingly. Well, like, for instance, like, for instance, somebody could maybe change bpython or something like that to allow you to paste right in. Yep. Neat. People out there think it's nice. This is a nifty little tool, says Kim.

It uses tkinter, presumably for clipboard access. I believe so as well it should use paperclip i believe he paper paperclip rather piper clip if if i were saying it that's like a super small simple little thing cross-platform copy and paste but you know it doesn't matter uh really what it uses to accomplish copy and paste it's all good tk enters like built in now is just normally. Yeah, that's true. So I guess there's no dependency.

That's right. There's no dependency. So, okay, maybe I'll take that back. Maybe that is better. Anyway, I love paperclip. This is cool. If you're, I don't know, if you're working on a book or if you're learning a lot where you're grabbing stuff off of tutorials. Yeah.

I kind of sort of spoke a tiny bit disparagingly of this because the style of showing the REPL output. Because I'd rather show an example that has executable code and a print statement. I kind of already write this output like, here's the code, comment, here's the output. So I don't know. It seems a little more reasonable to me. But whatever. People can write them in any style they want. And this unREPL will roll it from one to the other.

Yeah, you can also use a doc test to check your REPL docs. Yes, exactly. And while they're giving a challenge to the community, I think someone should write a PyCharm and VS Code plugins using this so it becomes part of the editors as well, which would be pretty neat. Paste as. Paste from, right?

Paste from REPL. Yeah, that'd be cool. Yeah, but it's already really handy to have it just pop over the terminal and run this. Yeah, awesome. I guess that's it for all of our items, isn't it? I think it is. Yeah, we've come to the end of all of the main things.

What do you got for our extras today? Well, I was going to mention that I wrote an article, but it was my item. Correct. So that's it. How about you? There you go. I got a couple of quick ones here. So I think this happened since our last show, but pretty much right after it shipped.

If you have Git, you should update it. So there's remote code execution level vulnerabilities in the Git client that is sitting on your machine. If you have less than 2.39, maybe even less than 2.39.1, I can't remember the exact version, but until last week there was a problem. I'll link to this over here. But you don't really want that. Yeah, it affects 2.39 and older. Oh, no. So careful.

I mean, if you're only pulling stuff from your own repo and only you can put stuff in your repo, you're safe. But if you're checking out someone else's repo or PRs or things, you might not like it. Okay. Another one here. Over on Aquathon training, we, as much as I don't like to, had to add some kind of validation. It used to be that dreaded reCAPTCHA thing from Google. And I would say, find all of the stoplights. And you're like, well, is it the pole of the stoplight? Also the stoplight. And you're like, well, is it the pole of the stoplight?

Also the stoplight. I mean, technically it is, but it's not the light part. And it'll be like, try again. You're like, no, right? But because it's the internet, we can't have nice things. Unfortunately, I had to put something there to slow people down

because they were just pounding away on it in various ways that were not ideal. We had stuff to mitigate it. But once you start encountering botnets, then you kind of get yourself into a bad place. So I had to put the reCAPTCHA stuff there to keep people from messing with it. And I hate reCAPTCHA, but I did it because I had to. So then when Cloudflare came out with Turnstile, I'm like, yes, this is so much better. It doesn't ask you about chimneys or fire trucks.

It just does a little like run some code on your browser to prove that you're not just requesting and posting this from some bot, but it's a real browser. Off it goes, right? So that's great. Eli Cobbler said, hey, this is awesome that you got this working.

I can't get it working. How do you use it? So I ended up posting a gist, which has an implementation of this using like a faux web framework. I didn't want to tie it to Flask. It just says like, here's your thing that handles the forum posts. Do this. It shows the HTML bit, which also Cloudflare would show. But they don't have the Python version.

And then it has the Hydantic-based API validation stuff here, which is the gnarly bit. So he was like, I can't get it working. Could you post this? And apparently, he got it working as well. But I'm sure there are other people who equally hate find all the haystacks find all the fire trucks and would rather use turnstile but it's kind of

tricky to use so they can also use that gist if they so wish yeah that's funny yeah um david pool says google's google's captchas made me doubt my own humanity so i always thought like that'd be fun to do like trees instead or like plant identification like yeah like find the azaleas what uh let's see so on when i talked about getting a bunch of stickers on my computer the other day, remember that? Yeah. Well, one of the stickers I put is, I'm not a robot. And it's like a validated recap.

So I just thought that was fun too. But that's cool. Yeah. Anyway, so if people are interested and they want to try out Turnstile, which is way more privacy oriented, also nicer because you don't have to interact with it. It's just a little hard to work with. Here's some code that they can try that at least one other person has been successful using. So that's a tepid, but somewhat good testimonial.

How's that? That's nice. All right. Thank goodness it wasn't in the JavaScript, Brian. Yeah. Because if it's in the JavaScript, I don't want to have to go debug that. Do you want to debug it?

No, I don't want to debug it in the JavaScript. It might make you so crazy as if to go and like literally write a song about how much you don't want to debug it in the javascript it might make you so crazy as if to go and like literally write a song about how much you don't want to go into the javascript yeah now i am very unskilled at music but um dylan betty is actually quite skilled and has a whole bunch of really funny songs here like the big rewrite based on American Pie, an eight and a half minute long song, I'll point out. And he wrote a programming song against it.

Your API is a hall of shame. You give REST a bad name based on the Bon Jovi song. But neither of those are the one that I'm referring to. No, I'm referring to the song called Bug in the javascript based on piano man by billy joel this is good stuff right yeah now it's six and a half minutes so there's no way i can play it but it starts out like you've checked all of your database indexes you've tuned all your api hooks you're starting to think that you might need a drink because there's only one place left to look. There must be a bug in the JavaScript because everything else is built

properly, but the front end's a pile of crap. It's so fun. I like it. You're starting to think that I might need a drink because there's only one place left to look. It's really good. I recommend people put this on. And if you're with uh angular or or view or some friend and framework you can play it loud in the obvious in the office just to just to share a little bit of the feeling yeah anyway that's what i got for you this week people can go watch the youtube video thanks yeah you bet and uh thanks for being here thanks everyone for listening as always thank you bye bye y'all