Python Bytes - #61 On Being a Senior Engineer

Episode Date: January 16, 2018

Topics covered in this episode: PEP 412's dict key sharing for classes Python Hunter Ten Things I Wish I’d Known About bash Snakefooding Python Code For Complexity Visualization On Being a Senior Engineer * Python UI frameworks* Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/61

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 61, recorded January 10th, 2018. I'm Michael Kennedy. And I'm Brian Ocken. And we have, again, some really cool stuff we want to share with you. The best of what we found this week in the Python space. Before we get to it, though, I want to say thank you to DigitalOcean. This podcast, these MP3 files, everything is coming to you directly out of DigitalOcean servers.

Tell you more about that later. Right now, I want to talk about dictionaries. Yeah, I want to talk about dictionaries too. So thanks to Raymond. I'm going to reference Brandon Rhodes actually for later on. But Raymond Hedinger is probably the person that I have to thank for telling me how cool dictionaries are in the first place, because he's talked about them at different PyCons in the past. And basically, I mean, tons of Python is built on top of dictionaries. But one of the things that I did not know was how awesome key sharing is. And we had one of our listeners on Twitter, Ned Letcher, keyed us in on this and

said, hey, if you haven't checked it out, you should go look at Brandon Rhodes, the dictionary even mightier talk from PyCon 2017. And so I did. And yeah, the key sharing thing is like something I didn't know about, which is it's totally cool. It's kind of funky to think that it's not that way until you really think about how it works, right? So when you create a class and you create an instance of it, an object, every one of those has a dunder dict, right? And what goes in the keys are the names of the fields of the class. And what goes for the values are, well, the values, right? So when you say in dunder init, you say self.name equals this, self.id equals that. Those are putting well, the values, right? So when you say in DunderNet, you say self.name equals this, self.id equals that.

Those are putting entries in that dictionary, right? Yeah. But the thing is, if I create a million of those, I have a million times the string name as the key and a million times the string id as the id. So basically every instance you create stores a separate copy of the names, not just the values, but the names of the fields, which can be super problematic, right?

The names of the fields are, and in Brandon's talk, he talks about since it's a dictionary, there can be hash collisions for figuring out where to put it. And so each of that, that happens for every single object has to figure that out. I think it was what Python 3.3, 3.4, where they changed it to where, so here's the trick. As long as you put all of the fields that you're going to use for your class objects, you initialize them with something in your init function, then all of your objects can share the same dictionary for the key lookup. Yeah, that's really cool. So I think that's quite awesome and should really help with performance as well, especially memory wise. Yeah, it's a little bit of a little weird thing of like you have to

kind of think about how this is implemented. But Brandon does a really good job of walking you through it and so that it's obvious how how the savings is and apparently from the pep 412 that this was started with for object-oriented programs uh the memory reduction is like 10 to 20 percent less using quite a bit less memory so that's definitely worth it oh yeah that's always a nice benefit there so i have a an interesting project recommended by a listener as well called Python Hunter. Have you heard of this? Well, just because it was recommended to us, but I haven't looked at it yet. Yeah, so it's pretty interesting.

The idea is it's a little bit like code coverage. And it's a little bit like a debugger in that you like enable it on your code. And then as you interact with the code, you get a report. But what you actually get here is it will give you basically as your program runs, it'll show you the internals of basically any module or package you want to point it at as it executes. So there's an example where they turn it on and they call os.path.join. And it shows you basically all the internal implementation running, like printed into the console intermixed with your output. Oh, interesting. Yeah. So if you were like, I would like to explore what this does when I call this function, and I don't want to root through all the source code and try to figure out what

piece this is referring to, you can just turn this on, call the function, and it'll show you the path through the external code it took. Okay. Yeah. So pretty interesting. It has some integration with debuggers and things like that. So you can also override it. And it doesn't happen for every bit of code you call.

You point it at a module. Like say I'm trying to work on, let's say, POSIX path. I want to understand my interaction with that module. Then you call os.path.join and off it goes. So yeah, pretty sweet actually. Like for instance, if I want to mock out my request interaction to external databases or external sites, I could point it at requests, for instance, to find out exactly what my application is doing with requests. Yes, exactly. And you would say, oh, this is the part where it goes to the network.

That's the one thing I have to mock out. The rest I can just ignore. That would be a great tool, especially for people that are trying to isolate bits of their application. Yeah, that's cool. So this was recommended to us by EvenPage. So thank you for that. And it's also kind of interesting that it is uh built on scython so it's nice high performance uh it's always cool to see scython making an appearance nice so you know

i always feel like when i'm working with bash i could do a better job and in fact i don't use bash that much these days i use oh my zsh but same thing there's all this stuff about the shell i could do better you use zsh use oh my zsh i love it oh my oh my oh okay yeah yeah it's really got some nice like integration with git the history and the search and the auto complete like a tabbed selection of various things that if you have like you type a few characters and hit tab it'll show you the list of all the things that you've done involving that it's kind of cool cool. Okay, I'll have to check that out sometime. But I think more people use Bash, actually. So I went through a phase where I started out learning KSH

in college and in my first job. And then I tried Bash and then I went to ZShell for a while. But it had support issues on whatever operating system that I was using at the time. So Bash is just everywhere. So we can just use it as is. The place where you have to jump through a few hoops is on Windows. So I wanted to bring up before we get into this article, there's an article called 10 Things I Wish I'd Known About Bash, which is a really great

little quick start on things that might speed up. It's just some fun things. There's somebody trying to sell a book also, but it's only like five bucks, so go for it, dude. But kind of things that I'd always wanted to know, like what the difference is between back ticks or the dollar sign per N replacing bits in your command. Just if it is, they're identical. It's just one's a lot easier to read

if you have a whole bunch of them. Yeah, okay, very cool. And then, you know, like globbing versus regular expressions, that's something that confused me at first, so that's a good thing. Dealing with exit codes. The number four is the one I really like the most because that one confused me is when you're using if statements, what's the difference between bracket and bracket bracket?

And I'm not going to talk about it here, but these are some good things. I also wanted to bring up, I spend a lot of time trying to hunt for a good bash application for Windows, and right now what I'm using the most is Git for Windows, and it has Git Bash is embedded in it. I think that's great,

even for people that are not using Git, because it's just a good bundle of all of your bash tools. And it just always works. So I'm pointing everybody at that now, even if they're not using Git. We did have a recommendation from one of our listeners a long time ago about a tool called Commander, I think. Yeah, I'd say Commander, yeah. Yeah, cmder.net. And I think there's a reference on here that I think it's related to Git for Windows somehow.

So not sure. But if you don't really use all the, if you don't want to try Git for Windows, but you still want some emulator working on Windows, check out Commander also. Yeah, and it has the Monokai theme. It actually looks really nice. Commander looks quite cool.

I saw it on my Windows 10 machine. One more thing I'll throw on your list here while you're talking about these is I don't remember what version of Windows 10 has this. It's not the original, but maybe the one last fall. I can't remember what came out, but they've added the Ubuntu subsystem for Windows, which yeah, it's pretty cool. But you have to turn that on in like the turn Windows features on and off. It's off by default.

And I'm glad you brought that up because I want to try to revisit that. I did try those and I had issues with being able to launch external applications. Yeah, they're like really isolated, like the Ubuntu part and the Windows part, they don't really communicate very well. And so it's not the same, I don't think.

It's not like you have to do that a lot, but I often just want to open up my text editor from the command line. Yeah, exactly. Things like that. Cool. All right. Yeah, those are all nice things to check out, especially on Windows. It needs the most help with the shell, actually. Cool. So before we get to the next one, which is a bunch of pretty pictures that I hope will help people understand the definition of micro frameworks versus, you know, batteries included frameworks, things like that. Just want to tell people about DigitalOcean. Those guys are doing awesome work over at do.co slash Python. So check that out there. This is where I have all my servers, I probably have eight servers for the various things all working together now,

failover and database servers and whatnot. But yeah, it's great, super easy, 30 seconds. You got a Ubuntu machine or what other, there's a bunch of different distributions and pre-configured servers. Go over there, set them up, check it out. They're doing a bunch of great stuff and it's definitely affordable and reliable and fast.

Awesome. Yeah, cool. So there's this thing called snake food, which is a tool written by Martin blaze and it creates Python dependency graphs. So I've got one package or one module or one class that depends upon some other thing. And it'll like put them in a graph and draw lines between them. So that's pretty cool. I haven't done a whole lot with it. You can combine that with graph viz and it'll like put them in a graph and draw lines between them so that's pretty cool i haven't done a whole lot with it you can combine that with graph viz and it'll create these nice visualizations of python code bases nice so that's pretty cool and this is not a brand new item but i had seen

it a while ago it's from the grok code guys i think i had them on talk python like episode 8 or something so it's been a while it wasn't this, but they took this snake food plus graph viz and they turned it on a variety of the frameworks that we know and love. And it's really interesting to pull up the pictures and just look at the relative complexity. So if somebody tells you, well, bottle is fast because it's simple, but it doesn't do very much for you. So you got to bring a lot of pieces in. Django is much more full featured and more complex, but it has a lot of things built in and flask is somewhere in the middle and pyramid just a little more than that. You can look at these pictures and get a sense.

Do you have the pictures pulled up? Yeah, definitely. That's a, you can totally see that that's the case. Bottle is like incredibly simple. I'm actually blown away that it is as simple as it is, but you can just tell like straight's the case. Bottle is like incredibly simple. Like I'm actually blown away that it is as simple as it is, but you can just tell like straight away by looking at it. It's so elegant. There's no cycles. Yeah. There's no cycle. It's an acyclic directed graph. That's quite interesting, isn't it? Yeah. Cool. So yeah, it's, and then Django,

how would you describe Django? It's like if you clean out a comb after it's been used for a year, that's what it looks like. Yeah. It's like a little bit out a comb after it's been used for a year, that's what it looks like. Yeah. It's like a little bit like when I was inkblot tests and flask and a pyramid, they're sort of on the same scale. Pyramid's got a little bit more going on, but they're definitely not simple, but they're not insane, right? They're not like Django level of interaction. Then they go and they do this for celery and uh rq as well and you could tell celery is way more complicated it's more complicated than pyramid or flask by the way but it's also more complicated by ways but than rq yeah let's see what else do they cover they

cover twisted and twisted is pretty insane it's just it's probably worth uh more tied together than django i would sayurial, quite interesting as well. Request is in here. Request has got some pretty nice look, actually. Yeah, it's oddly elegant and pleasing. It is. IPython, not so pleasing. No.

Anyway, I recommend people pull up this snakefooding link and just have a quick glance through the pictures. I think it's quite insightful. And, you know, it gives you the technique for applying this to other codeb bases as well. Yeah. These would be fun things to just print on your wall and tell people that you're using it as a reference. Yeah, absolutely. Pretty cool. Yeah. I just follow the graph when I need to know how things work. Yeah. Your next item is about being a good senior engineer. The new year, it's 2018 now. There was somebody that posted on Twitter

that I'm terrible about references said, hey, it's a new year. You should go read this again. So this is a 2012 article called On Being a Senior Engineer. It's from a website called Kitchen Soap, which is funny. But anyway, I think there's just a lot

of really good advice. And I just went through and pulled out a bunch of the headlines, but it's some things that I've been thinking about lately as I'm hiring people and thinking about always with evaluations and stuff like that. What does it mean? What does an engineer versus a senior engineer mean? And these, a lot of traits, I think fit well. I think these are all really, really good. I mean, maybe you don't know, that should cover them all, but like, don't practice C-Y-A-E. Yeah. Cover your ass, engineer.

Exactly. No, no, exactly. That's good. So, I mean, I think that's a really good point. Like, a lot of people who, when they're starting out starting out they're trying to like make sure they're never perceived as not knowing everything right right by the time you're a senior engineer you've been beaten with the fact that you don't know stuff and new stuff is coming along that you also don't know continuously right one of the top ones is seek out constructive criticism of your designs and yeah you're just trying to find other perspectives to make sure you you haven't missed something. One of the, I really like is to lift the skills and expertise of those around you. Yeah.

Well, you and I are definitely trying to do that. So that's good. I think a lot of people also try to like keep the recognition for themselves when they're new. All right. But yeah, I think one of the biggest skills someone can be providing on a team is actually anytime someone's get stuck, they kind of float over to that group and help them keep that part of the group and help that person keep moving. They can like almost write no code, but sort of float around and make sure nobody's stuck and everyone's

productive. And that could be a massive boost to the team. Some of the things like bringing up a couple more items, estimating how long something's going to take or how much work it is, that's hard. It's always hard. You know, as you grow in your career, you realize that you can't avoid it. So you just need to embrace it and be okay with giving ballpark estimates to people and then trying to get better at it. Keep track of what you did and try to improve. The other one that's difficult is trade offs. You know that you can't get all the information before you start working. So making good trade-offs between information and just risks and benefits and making good judgments, those are things you just have to practice. Another thing that I wanted to bring up is the, there's a list of cognitive biases. So just knowing that when you're making a decision,

there are biases in yourself that you need to be aware of and try to make sure those are there and compensate somehow. Yeah, these are all really good. I like to understand that not all projects are filled with rock star on the stage work, right? You got to become a finisher, and a lot of times there's a bunch of just little details nobody likes that are part of that. Probably the first half of my career, most of my job usually was I would be taking on the things that everybody else didn't want to work on and forgot about and just wrapping them up and finishing. But then it looked like I was getting off and getting credit for finishing projects that most of the work was done by other people. They just didn't finish it.

Yeah, become a finisher. It's good stuff. And then there's finally the 10 commandments of egoless programming. Want to take us quickly through them? Understand and accept that you will make mistakes. You are not your code. Each of these could be episodes in themselves. No matter how much karate you know, someone else always knows more. Don't rewrite code without consultation. Oh, I'm guilty of that.

Definitely. Treat people who know less than you with respect, deference, and patience. The only constant in the world is change. The only authority stems from knowledge, not from position. Fight for what you believe, but gracefully accept defeat. Don't be the coder in the corner. And critique code instead of people. Be kind to the coder, but not to the code.

These are good. Yeah, these are all great. So yeah, I really connect with a lot of these. These are great. So I'm glad you brought this up. And nowhere in here it says, don't make fun of PHP. No, you could totally make fun of PHP all day long. Yeah.

But you could also kind of make fun of Python, unfortunately, a little bit for its lack of UI frameworks. We talked about that last week, right? How that's kind of one of its last remaining places where it really quite a bit could be better. Yeah. One of the things that we didn't read above was no empty complaints. And one of my rules is I don't like criticizing stuff without offering a better alternative, or at least some kind of constructive thing. So here's my attempt to backfill my criticism of the

lack of UI frameworks with, well, here's where we are, and here's some things that seem like they're working, and a little bit of constructiveness. So I went through and I said, all right, well, what are the options for Python UI frameworks, right? Like, I'm not loving what's available right now in a lot of different angles, but what can we do? We talked TKinter, not super amazing. Like I have a link to an example. Brian, pull that up and tell me how you feel about that application. Would that like, does that look like a classy new app from 2018? No, it looks like it's running on Windows 95.

Yeah, it's like Battleship Gray. Looks like it could have easily come off. It looks like whoever built the original Minesweeper in Windows 3.1.1, they built this, right? And so that's kind of what the current UI looks like in Python, which is, it's a problem, right? So what are the options? So we have that, but we have PySide and Qt. And actually Qt is pretty nice in terms of how it looks like it really looks like it belongs on the platforms. There's a designer for it, a draggy droppy designer type thing, which is pretty good. But the license is a little weird.

It's like PyCute is GPL. PySide is like an LGPL way to try to go around it. But PySide, I think, is slightly out of version with Qt. There's just like real messiness around it. But it's still a pretty nice thing. There's Kivy, and there's PyGame. There's also PyOpenGL. But to me, all the, if you go to like Kivi and you look at the showcase or the examples of people, what they built, they all look like AR type things or games or other types of

simulation stuff. And I know you said people have created a few things that are not quite like that, but it's not really the focus of the platform, right? One that actually looks pretty good is WXPython. Click on the example for that. That's one that I've used before. Yeah, this looks pretty good, right? This could certainly be at least circa 2005 on Windows. It looks pretty different. And it has like a GUI, a couple of GUI designers.

It has these widgets. That's pretty cool. Yeah, and then there's all the Electron.js stuff. So Electron.js is basically Node.js plus hosted headless Chrome plus JavaScript for an HTML for the front end. So there's actually a couple of attempts to create a, what I really think should be called proton.py instead of electron.js, a Python equivalent, where the running back end is Python. And, you know, you could use like sculpt or something to actually make the JavaScript side also Python, like the Anvil guys are doing, which is pretty cool. So there's this thing called eel,

which is a library for making electron like offline HTML, JavaScript, GUI apps. And like I said, you could put Python in the front end with that sculpt bit. And that looks pretty interesting. So you get basically like a local app packaged up as a dot app or dot exe. But you know, it's more or less like a web front end that can do like local machine Python capabilities. So that's pretty cool. And then there's a bigger version called CEF Python. That name is not amazing, but it's like Chrome Embedded Framework Python. And it's the same type of thing. It's a little more advanced, but it's also a little less obvious how to get started with it, I think. But both of those look like interesting ElectronJS-like things. These are people trying to solve the problem we have. So that's good.

Yeah, it's good to see them doing it. And yeah, last week we talked about possibly having an open session at PyCon in Cleveland in May about this. So it'd be really fun for people to get together and join us there. I think Paul Hildebrand is kicking that off, but we can all go and be part of it. Definitely. Awesome. Well, anyway, here's my research bit I did on Python UI frameworks.

None of them are like, yes, this is the answer, but maybe they'll spark some thinking and get us going the right direction all right well that's all i got for you this week brian anything else from you no that was a fast six items it was it's because they're so fun i really like the one that you got about the senior developer stuff it's not totally from 2018 but it's that evergreen material that just lasts right we talked about quite a bit of those but every single item that we talked about has a paragraph or more of discussion on the article. So please go read that everybody. It'll make you a better person. Absolutely. Very cool. All right. Well, Brian, thanks for sharing this with everyone.

Thank you. Bye. Yep. Bye. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Ocken, this is Michael Kennedy. Thank you for listening and sharing this podcast

with your friends and colleagues.