Python Bytes - #141 Debugging with f-strings coming in Python 3.8

Episode Date: July 29, 2019

Topics covered in this episode: Debugging with f-strings in Python 3.8 Am I "real" software developer yet? Debugging with local variables and snoop New home for Humans The Backwards Commercial License Switching Python Parsers? Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/141

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to earbuds. This is episode 141, recorded July 24th, 2019. I'm Michael Kennedy. And I'm Brian Atkin. And this episode is brought to you by Datadog. Thank you, Datadog, for supporting the show. Tell you more about them later. Brian, how you been?

I'm good. Really good. Quite busy. How about you? Always busy. I feel like life always supplies a little bit more friction to me these days. You know, a little bit more email, a little bit more paperwork. But, you know, that's the world I inhabit these days. So it's all right. It's all good. Last time we spoke about Flint and Flint being a library or a little utility running as your Python code. And it will rewrite your expressions into F strings.

And F strings are pretty awesome. Actually, that was maybe two times ago but you know flint's been updated a lot there's been like five releases since we talked about it you're going to kick us off here with some more f string goodness not to do with flint but just f strings you and i are both fans of f strings i use them all the time now so now i consider 3.6 the bottom of the versions one of the things that's coming in that we i don't think we've talked about we've talked about 3.8 stuff but mostly that's coming in that I don't think we've talked about, we've talked about 3.8 stuff, but mostly there's been a lot of attention around the walrus operator.

But there's some extra debugging stuff that's coming along with fstrings that are pretty darn cool. The one I'll be using all the time is, man, I always like to throw, it's frequent for me to throw in a print statement for a variable just to see as it changes what the value is. And it's like, you know, it's print, the variable name equals, and then the variable name in curly brackets for the F string. So there's like a little equal sign decorator, or it's a little extra thing right before the last bracket that you can do. And it'll do that automatically in Python 3.8. Yeah, so in Python 3.8, if I type,

I have an F string and I type curly brace foo equals close curly brace, the output will be foo equals whatever the value of foo is. Yeah, huge time saver. It's cool. Yeah, that's super cool. I feel like you might have been doing some like low-grade debugging given all the items you're talking about this week i'm always doing low-grade debugging but along with that we're linking to the um some python docs but there's also a couple other things associated with that you can uh normally with f strings you you get the uh the repper value of an object but you can have an s decorator on it with a bang or bang s it'll do the string

operator instead and then for floating point numbers you're a lot you're going to be allowed to do modifiers so if you just want like two or three decimal places or something you can do that that's really cool so you can use this little equals thing and then like at the end stick like a formatting instruction like colon dot two f or colon comma or something like that yeah it's like finally i think uh python's getting to a point where it's the just printing debugging with printf i'm a c guy so debugging with print statements is usable now so great yeah that's really cool there was some comment on the f string episode the 139 that i talked about at the beginning here.

I can't remember the guy's name, but at the bottom of the show page, he put a comment saying, look, I'm an old C guy. So I had said, like, I love to use F strings, but I always find myself like halfway into the string and like, oh, the F goes at the front. I mean, this one I actually need to do some sort of format, so I'd have to go back and do it. And it's a hassle. He says, well, because I'm a C guy, and you just mentioned it as well, I think print F, parenthesis, so I just move the parenthesis up a little bit. That's a good mnemonic, yeah.

Exactly. I like it. Print F, yeah, exactly. So a lot of my team is jumping on board with the F strings as well, so much so that often strings will be F strings and there's nothing in it being formatted. It's just they're just using f by

default and i don't know if it's slower or not but the python part of our code isn't the slow part so that's all right it's sort of irrelevant and interesting super cool so i got a question for you do you feel like you are a real software developer uh yes when I'm not being a manager, yes. That's a different type of issue. Maybe like other end of the spectrum. So I guess to answer the question for myself, yeah, definitely. But you know, it's interesting because I did kind of like a smallish minor in computer science at best, right? So I don't have a CS degree. I was mostly self-taught. I studied math, right? What? You don't have a CS degree? What am I doing this podcast with you for? I know.

Well, it's really a fake one. I meant to tell you, like, we'll talk about that later. Okay. So do I feel like a real software developer? I remember getting my first software development job thinking, like, oh, I'm not sure I can do this. Like, what do they expect from me? Like, maybe the expectations will be really different from what I can do. And I don't feel that way anymore, obviously, doing programming for 20 years. But there's an interesting article called, Am I a Real Software Developer Yet?

by Sun Li Betye. And he's one of those folks kind of like me that changed careers along the way, didn't totally love what he was doing. So did something else and then came in through this sort of self-taught direction. And there's just all these layers of like, hey, do I belong? Like, can I call myself a software engineer or things like that, right? So it's a really interesting article, kind of a journaling and charting his path through there. So it's pretty interesting. It says, look, there's a lot of folks who don't have these CS degrees who feel like it's not quite appropriate to call themselves software developers or software engineers is the term he uses a lot in the article. And sometimes this comes up as imposter syndrome. And sometimes just people just think they're not a software developer. So it's pretty interesting. He talks through how he went through, like, um, really

started working on HTML and CSS and JavaScript. And I wasn't really sure how to get, you know, had done some stuff, but wasn't really sure how to basically have anything to show for it, right? Like, it how he built a portfolio site for his wife, who's a product designer. Oh, yeah. Because I know people often ask you like, hey, how do I get better at programming? I get this question all the time, but often enough. I don't get it enough. Yeah. Yeah. I mean, people are like, hey, I want to become a developer.

I've taken this course. I've read that book. But what I really need to do, I think one of the key things that people can do both to have something to show in job interviews and also to just build their skills is to just create something, even if it's kind of like not really that important for the world. Like there's this example I heard a long time ago, somebody who was friends with someone who built like did these pumpkin competitions, you know, where you grow like the thousand pound pumpkins and apparently that that community had no website no online place to live i mean maybe today it would just be facebook but there wasn't like really a place for

that so they decided hey i want to learn web development so i'm building a website and a community and a portal for pumpkin competitions they didn't even care about pumpkins but they did it so they could have a project right and? And it turned out to be really good. So suddenly he talks about basically immersing himself in podcasts and YouTube videos and all that stuff. And so, okay, after that release, can I call myself a software engineer? And then he found out on the web that people on Reddit and stuff said, well, web development isn't real programming. It's just JavaScript, CSS, and HTML. So then he spent 18 months studying software development full time and went so far as to quit his job and move in with his in-laws,

which is pretty brave. Wow. And then after a while, I said, okay, now I'm a software engineer after I got a job and I've been working for a couple of years. Internet says, not really. You can't really be an engineer after like a year or two. So, yeah, there's a bunch of mean people. He's hanging out in the wrong places.

Yeah, I think it's Reddit. So basically he said, look, I went and I talked to a bunch of co-workers and stuff and just finally said, look, I'm really insecure about this stuff. Like, how do you guys feel? How do you feel this way? How do you deal with it? And found out that it's, you know, not that rare. I think, you know, anyone who's kind of in this stage like asking themselves the question am i a real software developer yet they should check this out i think it's pretty uh it would probably help a lot yeah and brave of him to put it all together so congrats it's cool i gotta check this out i guess at the conclusion he says well am i a real software engineer yet and said i don't know i think so some people might say no they can call me whatever one i don't know. I think so. Some people might say no,

they can call me whatever one. I don't care. I write code today. I solve awesome problems with it and I'm really good at it. So I'm having fun and that's that, right? Like call me what you want. Oh, and by the way, he also is a software developer at DigitalOcean. So that's pretty cool. If you get paid for it, man, you're a software engineer. So the engineer part is, I never got the software part because i was you know i had a master's in cs but my first job it was um it was doing software but it was uh i was surrounded by electrical engineers and mechanical engineers and people that engineer was part of their degree title and i didn't have any engineer quote engineer in my in my education

it was science computer science and there wasn't much science in that either all right exactly And I didn't have any engineer, quote, engineer in my education. It was computer science. And there wasn't much science in that either. Right, exactly. So it's the engineer part that I felt like an imposter for a long time. Until it was a couple years in, I was sitting down with a mechanical engineer and told him about my, like, you know, this, I don't know if I'm comfortable with this engineer title. And he said, well, I'm a mechanical engineer. So I solve mechanical problems using the tool set that I learned. What do you do? You solve problems using software. You're an engineer. Get over it. So that's pretty interesting. Yeah. Yeah, you know, I also, the engineering term is cool. Like software engineer is definitely a pretty cool title, but I think,

I don't know if someone asked me to give myself a title at that point in my career, I wouldn't necessarily choose engineer. I don't think to me, like engineering is applying like these practices and techniques for building stuff. That's like really well tested most of the time, right? Like if I'm going to build like a power factory, like there's a bunch of like well-known examples of building factories and I could do that. Or if I'm going to build a bridge, the reason I build the bridge is because like the other bridges that were built are on that river and I need to put one on this river.

But for software, like that's not very often done. If there's already thing that exactly solves that problem, you just make a copy of it and solve it with that. You know what I mean? Like you just, because software is replicable, we don't recreate the same thing as often. So it's more of a creation process than it is applying the same steps. To me, that's kind of a distinction that I find in there, but that's also a bit of a diversion. With building, what you described was a contractor, not an engineer. Yeah, okay, fair enough.

Engineering in lots of different fields means lots of different things, like civil engineering and all sorts of other kinds of engineering. It isn't taking one thing and making seven of them. And even with bridges, for that matter, you don't build the same bridge in the same place. You build a similar bridge in a different place with different wind characteristics and all that. Yeah, all right. Fair point. Okay, let's go back to debugging. Yeah, I guess in a debugging kick. So I ran across this, a little package called Snoop. And plus, it's just got a fun name. But it's a set of debugging tools. But it's like the printf debugging sort of stuff. It does a lot of stuff actually, to help with the debugging.

But it's kind of like if you want to be debugging without opening the debugger. And one of the things that does really well is just you slap a decorator called Snoop on a function. And now when you run that function run your code whenever that function runs the lines that get run and the local variable values get printed to standard error so you can just sort of run and if you've got standard error pipe somewhere where you can see you can just kind of watch your code run with this and then there's a bunch of little extra things you can do. Like you can, if you don't want to see the whole function, you can focus in on just some values or a block of it with a with decorator. It is modifying your code to do that. But sometimes if the alternative was

you're going to throw a bunch of print statements in, this might save you some time. And it's kind of a neat tool. Yeah, this is super cool. I hadn't heard of this before. And I started looking through it. And yeah, it's really nice. It's, it's a little bit funky. And I think for people to get a good sense of it, they should just jump over to the GitHub repo, which we're linking to, obviously, and just look at some of the pictures, right? Like you have nice color coded bits of your your code. And then like, as it runs, the values are like, kind of like grade into where your code is. So it's like, you're looking at your code, but then actually the values of that execution are in there. And like, there's an example where it loops over

something a couple of times. And then like that loop is just replicated as if you wrote it a bunch of times, but different values. It's pretty cool. Yeah, and for instance, I think a cool thing would be if you've had, especially for situations where you actually have tried to debug it with a debugger and you're running with a multi-threaded system or something and you just can't capture the time

where you're seeing the error. So you can possibly turn this on and throw it into your continuous integration and pipe the output somewhere and be able to capture it. Yeah, exactly. And there's just times where attaching a debugger doesn't make a ton of sense, right?

Maybe it's embedded Python, like CircuitPython or something, and you can't reasonably attach a debugger there because it's running on some device. The best you get is like serial output or it's some kind of Docker thing where it's a little bit hard to set up or output or it's some kind of docker thing where it's a little bit hard to set up or maybe it's even some kind of production thing although i wouldn't put this in production actually but yeah but you could throw it in the staging environment though so yeah yeah yeah exactly places where you don't have super easy access to setting up

you know like you might have pycharm installed but it or visual studio code but it's not super easy to like get it all put together just throw this on there and see what happens. It's pretty cool. Also works in like Jupyter, which is kind of nice. Oh, cool. Yeah, this is definitely a good one. I'm happy you covered it. Keep in mind for the next thing I got to do. All right. So before we move on to this next one, which is very, very interesting, it's quite the controversial thing. I want to tell you all about Datadog. That's not nearly as controversial. They're supporting Python bytes, and they've done so for a long time. So they're a cloud scale monitoring platform built by engineers for engineers, Brian. They're tracing stuff automatically, instruments like Django, Flask, Postgres, AsyncIO, and lets you visualize your application architecture

and a whole bunch of other things like Hadoop and Redis to allow you to watch traces across your servers to kind of put it all together into like one view, not just what your web app is doing or what this background service is doing, things like that. Check them out at pythonbytes.fm slash datadog and you'll get a cool t-shirt. Nice. Yeah. So have you heard of this guy named Kenneth Wright? Yeah, we've covered him a few times. He's created a couple of libraries. Some people use them.

I heard they're interesting. Actually, Kenneth's been on the show, and he's done a ton of cool work, right? Like requests is amazing. He's got a bunch of other things going on as well, records, responder, and so on. So definitely, definitely cool. However, he recently posted this comment about a week ago. In the spirit of transparency, I'd like to publicly find a new home for all of my repositories. I wouldn't be able to make some contribution to them, but I'm kind of done with being the

owner and BDFL of these things. So that's kind of a pretty big statement. Like, hey, Kenneth is in kind of done with being the owner and BDFL of these things. So that's kind of a pretty big statement, like, hey, Kenneth, who's in charge of all these things, kind of just said, I'm done with them, which is, you know, not the end of them, obviously, but that's a pretty big deal that somebody should probably address, right? Yeah, definitely. Actually, a lot of these projects, it's interesting, if they were any, the communities around them, because they're not just one person, the communities around them have because they're not just one person, the communities around them have built up,

especially with things like requests. There's a lot of people working with it and working on it. Some of these are projects that have reached a size that most projects have already moved off of, moved into a group repo setting instead of staying with one owner setting. Like palettes and flask, for example. Yeah, exactly. I think it's a good thing um i don't see the controversy well there was like some of the

controversy a little bit was if you go and look at that you'll see like a bunch of people who may be super involved in python some people who are not very involved in python at all are like hey i'll take over this or i'll take requests oh give that to me it's like wait wait a minute shouldn't like some of these really important pieces go like to like have some careful thought about where they go so there was a lot of back and forth and some of these smaller ones people picked up and they're taken over and so on but uh earnest durbin jumped in and said hey you know the python software foundation would like to accept transfer of these repositories into the psf github organization apparently this is a new thing the organization recently acquired the python

software foundation the github or let me get this right the github organization was acquired by the psf so they can control it rather than just some other random thing oh yeah and the idea is that this is to provide an administrative backstop for projects in the ecosystem existing maintainers will still remain and the psf staff will be able to help take care of things okay nice that's a pretty good outcome right and i think that's a good thing i was just saying like i would i wouldn't want the want these projects to go into just somebody else's name. Yes, exactly. They should go into some group or something.

So, yeah. Yeah. And, you know, I think the biggest news here is not that request is moving somewhere, but that, hey, the PSF has a GitHub organization whose job it is to take over projects like this. That's pretty cool. Yeah. I think more stuff should go in there then. Yeah, exactly. Yeah.

We should let people know about it. So I get the question also every now and then, I have this project, it's open source, but I'd kind of like to make it more of my job. How can I somehow keep with the spirit of open source and yet somehow have a commercial license or commercial something for my project? And this one that you pulled up here, this is pretty darn interesting. I think it's interesting. My first, I didn't know what to

make of it at first. That's how I felt as well. I was like, what is like, is this even real? Is this a joke? What is this? So the idea, this is from Aaron Hammer and he's a, he's an open source developer and has created lots of things. I didn't know about him before this article, but the article is called the Backwards Commercial License. And he's saying that there's a lot of, well, some widely used projects will go kind of through three phases. First phase being it's got one project or one company using it, so that company or the individual person is using the use case.

They're the one developing it, of course. But as things grow, if it becomes popular, it'll go through a stage where you have active community members contributing features. There's a growing audience and a lot of people are using it. And then later, a lot of the people, the third stage is a lot of people kind of think it's done. And there's, you know, a few security fixes or bug fixes or minor features added. But for the most part, what's working is working, you know, like, you can use a nine year old version of K shell, and it works just fine. That sort of thing. At that point, that's when it's not really that exciting to work on it, but a lot of people depend on it. So how do you pay for that

maintenance? And Aaron's thought is at that done phase, basically don't support old versions, only support the latest and greatest, but have another name for something that's identical under a commercial license for maintain older versions. I think I got that right. It sounds about right. Yeah, I understood it the same way. I think it's an interesting idea. And I'm not sure how, how the licensing would work, because you essentially have code developed by lots of people under an open source model, you'd have to have a different open source license to allow this, because all of the code would then be transferred to a into a commercial license for a maintenance phase or something. Yeah, I mean, the logistics would be a little bit tricky, right?

Like, let's just say Django, right? Like, somebody has Django 0.5 and it's on 2 now. And they don't want to upgrade their code. But they want some change done to it or there's a security fix or something. And they're like, we'd rather pay you a thousand dollars a year so if a security problem is found in it it just gets fixed rather than us having to do that migration to Django 2 something like that right yeah you'd have to like somehow fork that into a private repo and then have some other channel for releasing that right like some private authenticated thing that people can log into i don't know exactly how

it worked i don't know how do you feel about it i think it seems like a reasonable way if you set up the license at a time so people for new projects people knew about it they know what's going to happen and they know if you want to keep the current one and make sure that it always works with your code and keep testing it you can do that free. But if you want to be able to set and forget and say, no, this is, it's doing enough for me. I'll just use it now. And I don't want to deal with new stuff. Yeah, maybe you should have to pay for that.

It's pretty interesting. My first thought was it was kind of weird, but the more I think about it, I'm not really super against it. This isn't his first idea though. Also in the article, he talks about how he used to, a lot of the projects were paid for by corporate sponsors, but it wasn't very many of them. It was a handful of companies paying for something and then hundreds of companies getting it for free. And then as more and more of those companies move into, it's good enough for us now, their support drops off.

Yeah, I definitely feel like companies out there just get way more benefit from these types of projects than they're willing to give back. Even if that company, say, hires and employs some core developer of a project, but they're like, let's just say a bank and this powers their trading engine that does a hundred billion dollars of revenue. Yeah. Right. Surely more than, you know, a hundred thousand dollars should go back to that product.

Like surely some kind of larger contribution is a reasonable to keep the ecosystem going. So I don't know. I feel like stuff that sets up a framework for companies to pay money legitimately to help keep open source going. I don't see a lot of great setups like that. So this is something down that path that's worth considering. Yeah, I think so.

Interesting. Cool. Well, also interesting is this next article that comes from us from Gibi on Twitter, I think it is. And did you know that Gita Van Rossum had a Medium account and he's blogging there now? No. Well, just with this one. Yeah. Well, now that it's here, right?

Like, so I don't know. Actually, I didn't look at those other posts to see how long he's been doing this, but this is the first I know about it. So he wrote an article called something. It's not the title I put here. It was PEG parsers, P-E-G parsers. And he talks about how the parser that actually parses syntax in Python and.py files, that code is some of the very first code

that he wrote for Python 30 years ago. Interesting. Okay. Yeah. And it uses this LL1 parsing mechanism, which is the right thing probably for 30 years, 30 years ago, but probably not the best thing now. And so the one moniker there implies that like, as you're going through the tokens and parsing them in the syntax, you're looking ahead only one of them at a time.

And so this actually limits the grammar rules that Python is allowed to have because the parsing is actually really limited. So he didn't say this in his article, but let me give you an example. Like in the C-sharp language, they have the exact same concept as we have with yield, but their mechanism, their syntax to say, I want to yield something is to say yield return the value so yield is still a valid word in c-sharp but yield return those two combinations of separate statements actually mean something different because like it can more complicate like in this context this is a keyword but in a different context it's not oh weird yeah so i think things like that are just not around in

python partly because the parsing just didn't deal with it that way. So anyway, he talks about the history of the parser and this idea of using this thing called a peg parser, which is more like a depth-first parsing thing that has an infinite look-ahead buffer. And it goes through and it basically parses the entire file. Well, it reads through the entire file understands it and then goes and does the parsing so you can have like infinite look ahead you can go back as far as you want and so

on using something called packrat parsing which is kind of interesting so basically before like when memory was really cheap like this was a really expensive really limited this was a problem like the decimal module like decimal decimal.py or underscore, decimal.py, whatever it's called, is like 220K. And actually loading that entire Python file to parse it turned out to be some kind of issue in the early days. Whereas like, who cares about loading 200K now? Yeah, so I'm basically saying like, look,

it might be time to replace this super old but really polished limited parser that uh they called pgen with something using peg and pack rack parsing and the way it works with abstract syntax tree might be able to have some interesting optimizations that actually make it use less memory anyway oh yeah that'd be interesting yeah so this article is kind of interesting it's it's interesting in a couple ways it's interesting just like hearing gita think about like the history and like where it's come. And you get a real good look at like his thought process of like, should we change the language? Should we not?

Why did it come this way? How did we get to where we are? And also, it's just it's cool that he's blogging. Yeah, definitely. One of the benefits of him not having to do everything. Yeah, exactly. For sure. All right. Well, that's it for our main items. What else we got? One of the things I wanted to bring up, sometimes we have a couple of quick things that we didn't have huge topics on, but Philip Bauer works on Plone and he contacted us and said,

hey, Plone 5.2 is out. And I'm like, okay, I don't use Plone, but yeah, maybe we cover it. No, this is a big deal. So 5.2 is a multi-year effort. It was a really huge amount of work. And Plone is a content management system built on top of Zope, which is a web application server framework.

And a lot of this was in the early days targeting newspapers and things like that. But there's still a lot of this was early in the early days targeting like newspapers and things like that. But there's still a lot of people using it. And 5.2 now supports all of the threes, at least 3.6, 7 and 8, which is super cool. And Zope 4 apparently supports Python 3. And it's all up to date now. And then if you want to read all about it, we've got a link to the release note, the release announcement, and also an interview with Philip about some of the transitions. Multi-year effort. That's pretty intense. So another major project comes along and now is Python 3 only. That's awesome. Yeah. Do you think it's interesting that so many of these CMSs came

from newspapers, right? Like Django also came from like a newspaper in Lawrence, Kansas, which by the way, it's where I went to college. So just, you know, who knows where that's coming from. Well, Lawrence, Kansas is not a big place, so not a big name. But yeah, I had no idea until recently that it was from there. But yeah, it's just interesting that like these Python web frameworks are coming from newspapers. Yeah, it is interesting. Well, I guess it makes sense.

You got a lot of people writing, adding content to stuff. So CMSs, probably all the CMSs came from newspaper stuff. Yeah, that's probably true. All right, well, I have another one for you. This one is way less serious. And about a year and a half ago,

this probably would have been all just super cool, hip stuff stuff now it's a little bit out a little bit dated in some of the the actions but there's this project called building dab and t-pose controlled light and dab is like this this is pretty interesting though i mean yeah dab might be the not by by the current dance move of the day, but controlling your lights with dance moves, that's pretty neat. Someone built this thing with Python, this thing called Make Art with Python, and you come in and you do a dab move, which if you don't know what it is, just click the link. There's like a little animated video, you'll totally see it right away. You do a dab, that turns off the lights. And then whenever you

want to turn them on, you do like a T, like just put your arm straight out and hold still for a second and boom, the lights are on. What I find super interesting about the article is the amount of effort that went into this. I mean, not by this person. He's utilizing a database of people movement and all that stuff. So this is like standing on the shoulders of giants

to change a light bulb. Yes, exactly. How many computer vision specialists does it take to change a light bulb? I don't know. It could be a joke of some sort. But anyway, this is pretty funny. People thinking of what they can do with Python and computer vision. And yeah, check it out.

Yeah, cool. All right. So I got a couple of jokes for you. These are quick ones. So I put two in here. And they're related. OK. All right.

So the first one, we talked a little bit about C and whatnot. What is a whale's favorite programming language? C, but you just gave away the answer at the beginning of the... Alright, well then, tell me, why do pythons live on land? Why? Because it's above

sea level. Yeah, okay. Yeah, well, the first one comes to you is from Eric Nelson, and the second one from jesper sorenson so thank you guys for sending those in uh yeah that's where nice let's say amusing i'm not sure i'm gonna go as far as saying funny but amusing yeah some levity cool yep exactly all right well good chat with you as always thanks bye yep see ya thank you for listening to python bytes follow the show on twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S.

And get the full show notes at PythonBytes.fm. If you have a news item you want featured, just visit PythonBytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Ocken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.