Python Bytes - #127 That Python code is on fire!

Episode Date: April 25, 2019

Topics covered in this episode: inline_python (for rust) * Requests3: Under Way!* * 🔥 Pyflame: *A Ptracing Profiler For Python * flit + src* cheat.sh Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/127

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 127, recorded April 24th, 2019. I'm Michael Kennedy. And I'm Brian Ocken. And we have a special guest this week as well, Kenneth Reitz. Welcome to the show, man. Hi, thanks for having me. It's great to have you here.

I wonder if requests will come up as a topic during the show. What do you think? Request three might. The future of requests. How exciting. Awesome. Well, thanks for being here. It's great to have you here.

Before we get any farther, I want to say thank you to Datadog for sponsoring this show. Check them out at pythonbytes.fm slash Datadog. More on why they're cool later. Brian, I feel like WebAssembly, Rust, all these things are just, there's so many cool things going on around it. And that's something you found us first us to start off with, right? Yeah, I saw this from a cool, actually a Twitter post from Mara Boss. I think it's Boss, P-O-S.

And it's an extension to Rust that's a inline Python. So if you're a Rust person and you need a little Python in there, you can just inline it as a, I think it's as a macro or something. I'm not a Rust person, so I'm not sure how this works, but you can inline Python code, including things like embedding a matplotlib call

to pop up a window. So it's pretty neat. I don't really have much more to say about it other than neat. You can run Python within Rust. I think that's pretty awesome. Yeah, quite cool. I wrote something like this this week, actually, for Bash and Python. So you can do inline Bash and Python.

So it seems to be a trend. People want to embed other languages in other languages. That's cool. Can people get to your Bash thing? Is that a public thing? Yeah, it's just import Bash or pip install Bash. That's pretty cool. There's a lot of interesting interplays here.

Brian, have I talked about WASMR, as in WebAssembly-mer-er before on the show. I'm not sure. I think that's going to have to be a topic. So there's all sorts of cool interplays between Rust and Python these days. And like Rust obviously is big for generating WebAssembly. And so there's now a project called Wasmer, which lets you, I think I talked about last week. I did. Yeah. So lets you basically run WebAssembly in Python. And this is kind of like the opposite. It's an interesting way. Yeah. Yeah. So kind of like Node.js is the platform for running JavaScript, like outside of the browser,

like Wasmer kind of does similar stuff, lets you take any WebAssembly thing and run it in your Python code. You basically import the functions from it and then you call them in straight Python. So do you have to do reflecting, where you know what the structure of the code looks like when you import it? I don't know. I doubt it gives you any help, but the way you interact with it looks like Python.

You don't have to do anything special beyond an import statement. Okay, so you don't have to write C or anything? Like a load statement. No, nothing like that. It's pretty much straight Python. This is really cool, Brian. Yeah, I like this. Very, very nice find. nice find cool all right kenneth what's your next one

you want to tell us about the future with your back from the past uh microphone yeah sure i wanted to announce that request three is underway right now this we're doing a development cycle at the moment and i wanted to announce just some information about that basically so request two which is the thing that you know and love is going to go into CVE only mode, which is basically been for a long time. That's kind of the way things have been for a couple years now. Does that mean like security fixes only or security fixes and bug fixes? It means that if there's a CVE, like if there's a true security vulnerability, we'll fix that as soon as possible. And if there's a, you vulnerability, we'll fix that as soon as possible.

And if there's any security-conscious bugs, we'll address those as well. But people suggest UI changes a lot, and it's just not going to happen at this point. It's going to happen in Request 3. Awesome. So what's the story with Request 3? What are the major features? Yeah, so the major features are

there's going to have async and await keywords. So you're going to be able to do asynchronous calls. We're ripping out URLib 3, which is unfortunate because there's different approaches on how the future of the project could unfold. And this needs to start happening now. So we're just taking action and making our own library for making these low-level requests. And we'll see how far along we get with that. But we don't see any blockers that we can foresee at this point. So we'll have a new core, if you will. And it'll be as a new namespace, request3. So everything that you know that depends on requests,

this is going to have backwards incompatible changes, but it'll be under a new namespace. So you won't have to worry about that. You can have both installed at the same time. Nice. And it's modern Python only, right? Yeah, it's only Python 3.6. And it has type annotations and all the public functions. So your IDEs can get some nice code completion and stuff like that. I'm pretty excited about it.

So I don't want to take up too much time now. So I guess we got to keep moving. Well, we do. And I'm pretty excited. So I have one more question. So why 3.6 and not 3.5? Like 3.5 is fine for async and and await did you do like different type annotations or what was the three six maybe f strings okay f yes right on brian i still owe you a sticker they're really

useful when you're writing a low-level library or something that deals with a lot of urls it's really nice to have f strings yeah that's pretty cool i don't find it acceptable to use three five like if you have three six even three six doesn't seem to be acceptable i'd like to it's going to be documented as three seven only because like if you're going to be on latest python you might as well be on latest latest python yeah yeah right on and by the time that that comes out three seven will definitely be easy easy right what's the time frame yeah i think it'll secretly work at three six and secretly work at three seven but i'll just say it works at three eight i love it brian you excited yeah i actually i think i think that'll be fun yeah i'm thinking of also ripping out the uh the dot get keyword that's the thing i'm torn about at the moment

like the request dot get that everyone uses what would it what it's gonna force everyone to use session interesting okay so you create a session and then you call get on the session right yeah Like the request.get that everyone uses? What would it... To force everyone to use session. Ah, interesting. Okay. So you create a session and then you call get on the session, right? Yeah. So it would be... It's called HTTP session now. So it would be HTTP.request is what I want people to do.

But I'm still playing with the idea. So it might be not changed. We'll see. Yeah. You could face some backlash. But yeah, it sounds good. I'm willing to take it. You got some tough

skin you've been on the internet for a while right yeah let's not go down that path all right so maybe you could even say that you've been in some flame wars so the next item i have is called pyflame and pyflame is pretty cool like if you want to know how the performance of your program or certain library is going you can generate what's called a flame graph for Python. And what's really nice about this is it's super low overhead enough that you can even run it in production without hurting your program. So you could say, oh, there's something going wrong with the website. Let me log in and attach PyFlame to it and see what's happening. And then

just break loose and go analyze that rather than, you know, try to simulate what's happening. So I think that's pretty cool. We actually used this on the PIPM team at one point to determine what were our slow points were in like how fast our boot up time is. Yeah. And was it useful? It was very useful. It shows you every single call in your stack and just shows you a graph over time of how much time was spent on each call and all the sub calls. So it's great. I really like it. Yeah, the graphs are nice as well. They're pretty obvious, not just like a grid of numbers, like a spreadsheet, but you know, graphs. Yeah, usually when you hear someone talk about graphs that are generated from Python, it's like you have to work with them a lot to make them

useful. But these ones are useful out of the box, which I really like. Yeah, cool. Another thing that it said that it does, which is nice, has nothing to do with PipInf in this feature, but it's capable of profiling embedded Python interpreters.

So like, for example, in MicroWhiskey or GUnicorn or some of these, you know, backend worker processes for web servers. So you can get in there and get that and multi-threaded stuff. So I think it's pretty awesome. Brian, do you guys care about performance where you are?

I mean, you're doing testing of hardware. So it's not like users are directly interacting with it. Is it, I guess not do you care about it, but is it something you have to like worry about? In my production stuff, the Python is not the bottleneck. It's the communication. So yeah. That's the bottleneck. It's the communication. That's a great response.

It's on the wire. It's not our fault. Well, it usually is our fault, but it's usually an optimization of doing too many calls and reducing the communication latencies. Right, having a chatty API versus trying to bulk up the thing you want to ask it or whatever, right? Are you working with a lot of microservices? Working with test equipment.

So communicating commands and retrieving data. Yeah, that would take a lot of time. Yeah, Brian tests actual devices with Python. That's awesome. Oh, that's right. I knew about this. Yeah, you test like phones and stuff, don't you? Yep.

That's awesome. awesome yeah pretty cool all right before we get on to the next one which looks pretty interesting let me just tell you about datadog so they've been sponsoring python bytes for quite a while and we definitely appreciate them so they're a monitoring and analytics service that lets you take all of your metrics and logs and tracing and whatnot across your services like micros microservices, like Ken just said, and put them all in one place. And it automatically instruments async libraries like async IO and Django and Tornado and helps you visualize all that stuff

across the boundaries to find your bottlenecks. Super cool. So check them out. Get a free 14-day trial and a cool Datadog t-shirt over at pythonbytes.fm slash Datadog. All right. Thanks, Datadog. Brian, what do you got next?

This was going to be one of my extras, but I thought I'd just discuss it because I was curious what you guys thought. So Flit is, I really like it. It's a little package that allows you to quickly build a PyProject.tomo file and a license file. And basically, if you've got a module

that you want to share with somebody or a package that's just a source package, it's just a few commands to try to get it into a wheel form so you can share it. And it even does publishing within it. The one thing I didn't like about it is it didn't support source directories. And I like source directories in my packages. So I did what you can do with open source projects is I submitted a pull request. So just submitted this last night.

I've been playing with the corner cases today, and there's still some. I'm listing it as a work in progress because there's some corner cases that still need tested and probably ironed out. But I don't know. We'll see. We'll see where this goes. Did you make it so it's configurable to any directory or specifically SRC? Specifically SRC. Okay, cool.

It doesn't break any of the other. So really, right now, it used to have two kinds of projects, just a module or a package with a name. And now I'm just adding those two within a source directory. The old mechanism all works. It's just extending it. Yeah, Flit is a little bit like a combination between Twine and Cookie Cutter a little bit. Like it's far reduced in scope from Cookie Cutter. And in Twine, all it does is publish things,

publish wheels for you. And that's the tool I typically use in my workflow. But Flit is a great alternative if you don't already have your setup stuff built out. I think flit is like a great first tool to go for if especially if you don't have those files already written yeah that's awesome yeah well uh hopefully people like it i mean it's out there the pr right they could merge it that's pretty cool yeah so hopefully yeah hopefully they'll put it in otherwise you can't use it on your project. You'll have to succumb to our side where you don't use source. Well, I could fork it.

Yeah, you could. But I don't really want to support it. So I'm hoping that... No, it's split. Yeah, you don't want that puppy. You don't need another thing to take care of, right? Yeah, that's a good name though. Just going to do it.

Awesome. All right. So speaking of packaging and things that we've talked about and things we like, I'm a huge fan of PipX. I think PipX is really killer. Yeah. The way it works. I've run into some issues with homebrew in it, but I'm going to put those aside for a minute. Also, PipInf, tell us the story here.

My endorsement is also with this flag, which is I've run into some issues and I'm not sure what they are yet. So it appears as though PIPX is great. And what it does is it just creates a virtual environment and installs binaries for you. So it's like a PIPC replacement or a virtual amp wrapper. If you have like, sometimes people will have like a system virtual amp where they install their system utilities

or they'll just, they'll keep their system utilities in their in the system installation of python and then they'll isolate their different projects into virtual amps this makes it so every single tool has its own virtual which is really best practice so if you're using pip amp if you're using http ie if you're using you know anything pip itself anything that'sIP installable from the command line, you can install easily with PIPX. And it does appear as though it has Chimp. So I'm not sure that it's fantastic, but it appears as though it works pretty well so far. And I really like that it supports multiple binaries. Yeah, I do too. It's quite nice. And to me, it feels like homebrew,

but for Python executables. It's not, it feels like homebrew, but for Python executables, like it's not necessarily for managing stuff for your project, but for stuff that is Python based that you just want to run, right? Like I want to, I don't know, run glances or cookie cutter or something like that, right? Or pipemf. Exactly. And it gives you a nice isolated environment in which to install that. And that some of these like pip amp has like some like 50 dependencies or something crazy. So like, it's really good to have an isolated virtual environment for that. So that because because in Python, you can't install multiple versions of a dependency like you can in Ruby. So that's, that's why we need to do this, basically. Yeah, cool. Well, I definitely like it. Yeah, that's that's the end of my endorsement

with the asterisks afterwards. Yeah, I love it. The problem that i ran into the only problem i ran into was when i installed python 3 through homebrew and then python 3 got upgraded by homebrew it had lost like the sim link to where python was in all of the virtual environments that it had created oh interesting yeah yeah so when i basically when i upgraded python it like lost track of what it was up to so i just had to like re pip x install everything there's maybe another way to fix it but that that was what i did because all the virtual environments it manages seem to have gotten whacked yeah pip amp does that too where it'll break all your virtual just break

if you upgrade your python underneath it so yeah what do you do to fix it? You know better than I do. I just delete them all. Okay, so even you have the same fixes I do. Did you try turning it off and on how many times? Three times? I always do it three times. I would like to automate that. So it does it for you. Like, you know, it has a message like, Oh, you appear to have upgraded your Python, we're gonna clean it up for you. But yeah, you know, there's limited time. Exactly. Quite cool. All right. The last one, Kenneth, I think you'll like this one, because it has to do with requesting things on the internet. And I don't know how to say this, I'm guessing cheat sheet, cheat.sh as a domain. Okay. So this is a, an interesting, like cheat sheet for getting started with Python or bash or whatever. And the way it works is you send it curl requests or send it HTTP pie

requests on the command line to ask it questions. Okay. So you can go over there and say, like if you have HTTP pie installed, you could say HTTP space, CHT dotsh slash Python that tells you you're talking about Python, then you could say slash sort plus list. And if you run that, it will tell you, oh, here's a Python example of how you sort a list, you create the

list, you say my list dot sort, you set the key to like string dot lower, or you know, whatever you want to sort, if you'd like say want to sort alphabetically, but without concern of casing, things like that. So you can ask it arbitrary questions like cheat sheet slash Python slash connect to database or something like that with a pluses in there. And it'll like tell you, oh, here's some ways you can do it. Like with SQLite, let's see what it says for this one. Oh, yeah. Here's a little example of how I use PeeWee to open up a pretty little class and do ORM stuff to it. And just, it's pretty, pretty funky, pretty interesting. And then you can ask it questions like how to get started with a colon

learn on any project. Like how do I get started with Python? And it'll tell you in its little way. It's pretty sweet. I think funky is the right word for this. It's different, right? So it's like Google, but harder. Yeah, basically. It's slightly more deterministic. Yes, but it has integration into Visual Studio Code and Vim. Oh, nice. Which is interesting. And then you can also, like you can install a client and put it into stealth mode. So like if you're doing an interview, anything that you select in a text box it will start like on the side printing out its response to that okay so if like you're in a chat window i'm looking at the the read me and he really takes this very seriously like it is he

says ultra fast returns answers within 100 milliseconds as a rule so like he's definitely making scalability a concern. So I think this is a really fun production project. I think so too. And it doesn't use its own data sources. It uses other community-driven repos for different topics. And it tries to pull those in. So it's not like it tries to be the answer. It tries to be like a facade from an API to the answers.

So it has sources that it gets them from. I'm interested in seeing that part. Yeah, yeah, it's pretty interesting. So I just want to give a shout out to John Boltmeyer. He sent this in a while ago and, you know, I thought it was fun. So I thought I'd throw it out there. And he also has clipboard integration and tab completion support for Bash, Fish, and ZSH. I know.

I mean, this is in-depth, right? Like you can really go all out with this. Yes, you can. All right. Well, you can really go all out with this. Yes, you can. All right, well, that's it for our full major topics. But I'm sure we'll have a couple of extra things to throw out here real quick. Brian, you want to start us off? So you guys were giving me a bad time about being a VI person last week. So I thought I'd throw in an article about VI is good for beginners.

It's just a fun read for anybody that thinks that VI is terrible. Do you use Vim, not VI? Actually, I use PyCharm right now with IdeaVim. Yeah, this is an interesting contrarian sort of take on the simplicity is good for you. Yeah. Yeah, myself, I think it's a good thing to learn. I did a presentation last week with a Jupyter notebook. And one of the things I needed to do was to quickly make sure I cleaned out all the output

of a bunch of notebooks. And I found this tool called NB Stripout. And so I wanted to drop a link to that. It's pretty helpful. Yeah, it's just a command line thing. And then we had a bunch of people, a couple of people, including Cecil, mentioned Pyodide. But we did cover that in episode 93, but there was a recent article written up about Pyodide

that's kind of neat, so I'll drop a link to that in the show notes as well. I'm super excited about Pyodide. I actually just yesterday interviewed Michael Drotboom, who wrote that article you're referencing here and worked on the project on TalkPython going deep into WebAssembly and

what Mozilla's doing with Pyodide stuff. So it's, yeah, it's super cool. Kenneth, have you seen this Pyodide? I'm looking at it now. So it's like all of the scientific Python stack in WebAssembly. Yes. Including the libraries like NumPy and Matplotlib and stuff like that. That's awesome. Not just like CPython. That is not unimpressive. No, it's not. It's quite something. That's awesome. Let's see CPython. That is not unimpressive. No, it's not.

It's quite something. That's awesome. Let's see. I guess I'll start mine really quick. I got some quick conference announcements. Kenneth, I see you do as well. So PyCon Australia just opened their call for proposals. So Australia is a beautiful place to be.

Either maybe you're already there and you just want to go to the conference or you want an excuse to go to Australia, submit a talk. That's awesome. And then last time I had used about whether the gravitational waves of black holes colliding, that research was also done with Python. So remember we talked about the black hole picture and all that stuff, Brian? Yeah. Well, our listeners are awesome because they're like, oh, Michael, did you wonder about this? Yes. Here's the package they use to discover gravitational waves and do all this amazing science. So actually, there's some really cool projects out there. And I'll link to

a video of someone talking about it. So thanks to Dave Kirby and Matthew Fikert both for that. And I think it's gwpi. Yeah, gwpi.g. Dot. IO is a Python package for gravitational wave astrophysics. So detecting when like black holes collide and like the ripple and space time they make with Python is pretty awesome. Yeah. I was really surprised to find out that when that code was made available, that request was used for it. I don't know.

I have no idea how, but it doesn't matter. It's awesome. But it's in there. So I was like, like yes that is super cool congrats man and then finally another thing that's awesome about our listeners is anytime we mention something we think it's like unique they're like there's four other ones here's another one so bullet we talked about cooked input we talked about is awesome both super cool for like drop

downs inside terminal windows and like nice selections, but it doesn't work on windows. So someone else, uh, Sander Tunison sent over questionnaire, which is one that is also compatible for windows for our windows folks. Thanks for putting that in. Cause I need that. So see, you were like the first,

one of the very first people to benefit from this podcast. I feel the same way. Like people are like, Oh, you finally, I'm like, I, I benefit from this just like you guys do.

It's just, I have to get there first before we publish it. Awesome. Awesome. Okay. Kenneth, what do you got? Honestly, I have a few CFPs I wanted to announce. One of them is for Pi Colorado. It's a great conference. It's going to be their first year. It's run by a great team, at least. It should be a great conference. If they are actively looking for people to submit to their CFP, they have not had as many submissions as they would like to so far. They, you know, bad timing with the conference year, that type of thing. So if anyone has any talks that they think could be applicable to that event, please submit. Pi Ohio also has their CFP open at the moment and they're looking for people to submit, but they never have a problem with

getting too many submissions. So I wouldn't rush to that one if you're, you know, if you're feeling iffy about it. And then the last one is a PyCon remote, which is something that I'm going to be putting on, if all goes according to plan, with Calvin Spellingman, I believe is how you say his last name, who is from the PyTennessee organizers group. Him and I are co-chairing. That's cool. What's that about? It's basically a PyCon, but it'll be accessible to people who can't attend PyCon or who can't normally speak at PyCon. Is it also virtual or only virtual? Only virtual.

Okay, sweet. That's awesome. Yeah. We're still figuring out what the feasibility of different approaches are, but it'll most likely be live streamed and people will send in videos ahead of time of their talks. Okay. Unless they're like a keynote speaker or something like that so we're still figuring out the details but it'll be it should be good we have a mailing list with like 700 people who have signed up so far so it should

be um it should be something we can pull off so if you're interested and want to learn more you can go to do slash co slash pycon dash remote i know it's not the best url but that's the best i can give you right now awesome yeah we'll put the link in the show notes people can just click it in their their player that's awesome all right well let's round it out with some uh some jokes here brian you want to go first we're not doing all the jokes all right we gotta go fast we gotta go fast okay so yeah i'll just pick one I don't know how Tasteful this is We'll let it land anyway

So Python used to be Directed by the BDFL Guido Now it's directed by A steering council Guids 0 through 4 Love it

Kenneth you wanna do one You got any What's the difference Between a musician And a pizza I don't know You can eat a pizza Pizza can feed a family of four.

Oh, that hurts. I can say that because I've made $25 off of my music so far. Beautiful. All right. I got two, but they're related, so I'm going to put them together real quick. I think first to kick it off that Ubuntu users are apt to get this joke that I'm about to tell. Okay. And then the second one.

The second one. this is fun. I actually, I know Kenneth, you switched over to Windows recently and I'm also kind of a fan of it, even though I spend a lot of time on Mac. Still, I like the joke. How many programmers does it take to kill a cockroach? Two.

One holds, the other installs Windows on it. All right, beautiful. Well, I think that's probably about it for our show today. Kenneth, thank you for coming. Brian, thank you as always for being here. Thank you. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes

as in B-Y-T-E-S. And get the full show notes at PythonBytes.fm. If you have a news item you want featured, just visit PythonBytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Auchin, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.