Python Bytes - #49 Your technical skills are obsolete: now what?

Episode Date: October 25, 2017

Topics covered in this episode: * Conference videos for DjangoCon 2017 and PyGotham 2017* * Python 3.6.3 released on Tue. All machines at FB are already running it (3 days)* Your technical skills are obsolete: now what? Visualizing Garbage Collection Algorithms pathlib — Filesystem Paths as Objects LUMINOTH: Open source Computer Vision toolkit Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/49

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 49, recorded October 24th, 2017. I'm Michael Kennedy. And I'm Brian Ocken. And we have yet another six or more amazing news items, actually tons of really cool stuff to share with you. But before we get to them, I want to say thanks to DigitalOcean for creating Spaces and sponsoring this podcast to tell you about Spaces. Spaces are like S3, but way, way better. And check them out at do.co slash Python. Tell you more about that later.

Right now, I want to go watch some videos. Yeah, I know there's a lot of conferences that go on around the world. And actually, anybody that's doing a Python conference, I have no problem with trying to highlight those. So let us know when they're going on and also when the videos are up. And that's what I want to talk about right now is there's a couple conferences who have videos available now. DjangoCon and PyGotham both happened recently, and both of their videos are available. So we've got links to both. PyGotham is on, I don't know if it's on YouTube or not, but it is up on pyvideo.org. And the DjangoCon 2017 is up on YouTube.

And I've got links to both those playlists. I was excited to look at a lot of these. And one of the videos I started off with was a video from DjangoCon called Django vs. Flask. And it was by DB David Baumgold. That sounds fun. It actually was interesting because it's somebody that respects both Django and Flask, and it was probably one of the best Flask videos I've watched, talking about how to do some of the things you can do in Django right out of the box,

but how to do it in Flask and comparing those. It was cool. That sounds like a really cool video. It's one thing to go study Django. It's one thing to like learn Flask, but to see the features side by side sounds like a really nice way to, if you're new and you're trying to decide, should I go with Flask or should I go with Django? This is like half an hour, 45 minutes. You'll have probably a really good idea rather than you've got to like spend hours learning each and then decide, right? It sounds like he leans, he likes the flexibility of Flask. However, it's obvious that he has respect for both.

And he's talking at a Django conference, of course. There's some things I didn't even know about, which was cool. I didn't know about Flask blueprints and I didn't also definitely never heard of Flask Marshmallow for APIs before. Marshmallow. Yeah, that's great. And it covers things like SQL Alchemy, Mongo Engine, PeeWee, all the great ODM, ORM data access layers. It's cool. Yeah. But there's so many other great videos up from those two conferences, and I really appreciate that part of the Python culture that we try to make the conference videos available for people. Yeah, that's great. I love

that these conferences are so quick about getting the videos up. The videos are really high quality. Usually the sound is good. Everything's in focus. A lot of times it's picture in picture. I mean, well done. So that's nice. What do you got for us, Michael? Well, I want to pay a little attention to Facebook because Facebook does a lot with Python, actually. People will tell you Facebook is written in PHP and that it uses React, the JavaScript framework, which they invented on the front end. So it's like PHP plus React.

And that's interesting. But it turns out that I don't work there. I don't really know a lot of people there. From what I've been able to read, there's a decent amount of Python backend services sort of supporting that PHP front end. Okay. Wow. Yeah. And so that's pretty cool. And there was a tweet from one of the guys there, L Lenga. I think he works there. I'm pretty sure he works there from the reading of the tweet, but I was talking about Facebook and said, hey, Python three, six, three, was released on Tuesday. And I think last Tuesday

you covered that, which is pretty awesome. This was, maybe this was two weeks ago. When it was released though, they said this tweet was three days after that. And it said all the machines at Facebook are already running Python 363. That's cool. Three days. And they said that 36.3% of their Python apps are based on 3.6 already. How about that? That's great. Man, I love 3.6 now. I do too. The F strings are just so delightful.

Yeah. And so this is super interesting. And what's more interesting is I would say 20... Oh gosh, I'm forgetting the numbers. I'll have to just send you guys to the video and let you pull the numbers out. But a couple of years previously, Facebook was not friendly to Python 3. Basically, they had just gotten access to Python 2.7 from 2.6. Okay, that was the big upgrade.

And all the recommendations were used Python 2. The onboarding processes said, hey, when you get here, you're expected to write in Python 2, et cetera, et cetera. And this guy named Jason Freed, not of 37 Signals fame, but a different Jason Freed, took over the process of trying to shift the organizational culture to valuing Python 3 as the default and making the people that do Python 2 look kind of outdated and funky. So there's this really great presentation called Rules for radicals, changing the culture of Python at Facebook. And it's step by step. It's a step by step playbook basically of how he transformed the culture to where we're in this place where you say Python three, six, three was released on

Tuesday. Facebook machines are now running it. And that tweet comes out on Friday, right? So this is not from, this is not like, well, Facebook's a startup. It can always do like, no, it was this huge organization transformed through like clear, simple, repeatable steps. So I think this is just a really cool evidence of Jason Fried's work. And well done, Jason, and everyone at Facebook. Yeah, and I can't wait to watch this video.

Thanks for linking to it. Yeah, yeah, it's super cool. It was at PyOhio a couple years ago. And in more modern Python news, Ubuntu 17.10, which just came out, and I upgraded one of my servers recently to it, it says Python 2 is no longer installed by default, and Python 3.6 has been upgraded to 3.6. How about that? That's great. Another step forward. All right. So modern Python is on the march.

And 3.3 is end of life, but probably by the time this comes out. It's already end of life. So if you have something on 3.3, it is time to move up. And if you're going to move up, don't go to 3.4, go to 3.6. Speaking of new technologies, there was a pretty good article, and it's not Python specific, but called Your Technical Skills Are Obsolete, Now What? And this is, it's actually a pretty fun article about, you know, since you and I are both big proponents to people keeping their skills up to date and learning new things, I thought it fit in nice. And this article is focused on how to do that while at work to not try to eat into your home life to beef up your skills. I think there's a lot of people who feel kind of in a tough situation because they see all these

new technologies coming out and they hear presentations or people say, oh, the stuff that you do is so old and crappy, you should be doing whatever whizzabang new thing it is. These recommendations or this excitement, probably rightly so, but it comes from people who are young. Maybe they don't have kids or they're single and have a lot of free time. And to spend an extra two hours after work on some project is like no big deal if you're, you know, young and you got lots of free time and energy. But if, you know if you're getting dinner for the kids and you've been up early driving to school and you've got to work with homework, it's just like it's a whole other level, right? I think it's a great idea to try to fit that into your work if you can. One of the things is I really like that he starts off by talking about the reasons, sometimes good reasons, why companies stay with an old technology.

However, I mean, if Facebook can switch in a few days, what's your company's excuse? That's right. Seriously, though, sometimes there's good reasons to not jump on the bandwagon right away. So he goes through some steps of identifying obsolete and problematic technologies and trying to do some research on potential replacements and then trying to get buy-in from your management and hopefully to try to get some time for yourself or maybe you and some other people to do a pilot project exploring this process. Yeah, the main message of this was, look, if you want to find a way to learn new skills,

you need to fit it into work often, right? Unless you just want to really put in a lot of energy to learn new skills, you need to fit it into work often, right? Unless you just want to really put in a lot of energy to learn something outside of work. But if you're going to try to fit into work, go and identify a problematic area of your technology stack and see, like create a plan on how to solve that problem with the new technology by you, thereby you learn the new technology by solving the problem. So you're actually adding value, not just saying, please send me to training. You didn't send me in six weeks, six months, whatever, right? So then he does talk about how to pose those questions and talk to your

manager about making the change. But he kind of ends with, if your boss says no, then you at least have some extra skills that you can apply to your next job. And I'd like to shim in there and say, I think you should use it to start a conversation with your boss or your management chain about that you care about the company, you care about increasing the skills of both yourself and the others on your team. And how can you go about doing that? And why the no? Like, is it is it bad timing or you just have a bad proposal or too much time spent anyway i think that you should try to not throw your company under the bus right away and try to work a little harder so i think actually your the video you linked to with the rules for radicals probably fits right in here too of trying to change your company yeah

absolutely you don't just like jason didn't go well i said python too so i'm gonna go find a startup forget this right he he actually changed facebook which is even more impressive and then a book that a little tiny it's a little tiny read a book i read a couple years ago it's called team geek that has some great advice on this and it says that things like changing your process you probably ought to limit it to under 50% of your time I would say stronger I would say 10 to 20 percent of your time at most trying to fix things and then the other thing is there's a nice section on sometimes it's easier to ask for forgiveness than permission you can try to get your manager buy-in but also if you're not spending much time

on it, you might be able to just change some things and make it better. Like one of those disclaimers, my advice cannot be taken as legal advice and you should seek your own legal counsel. Yeah, your conditions may vary, but I think it's really good advice and this article is really practical, especially for people who are not trying to spend an extra hour a day outside of work learning something. So that's really cool. I feel like there's a lot of slack in big organizations already.

You get put into a lot of meetings. You have a lot of, I don't know, there's time between the actual work that seems to get done. And if you can just wall off like a little bit of that for learning and working on a project, it could be like a skunk work thing. I'm trying to prototype something different or I'm just learning something. But if you could like say find an extra half hour a day to be studying something in the context of your work that you're trying to learn, that actually could make a huge difference over a couple months.

So that's usually pretty good to do. I completely value in my employees, somebody that really wants to increase their skill set and also cares about the saving the company money and making things better. So at the very least, your your management chain should know that you care about that, too. They should care about that. People who care about this stuff and who are actively working on it. Those are the type of people you want. The ones that don't complain, that are just going to work on that old 20-year-old technology, that's fine that they're great employees and they're doing that, but that's not going to push you guys forward. So definitely, I think people should see this as a positive. Awesome.

Speaking of making and making things better, DigitalOcean has made online cloud storage for apps much, much better with their new thing called Spaces. So check out Spaces at do.co slash Python. And the idea is it's a similar service to Amazon S3, right? You have just a cloud storage. You can have either private or public things stored up there, huge, huge amounts of data. If your app has a process, a bunch of the pictures, and you got to store the picture somewhere instead of jamming them in the database, put the gigs of pictures up in spaces. And it's really easy to adopt because it uses the same API as S3. So tools like the transmit app I use on my Mac, I just pointed at the URL for spaces

and I tell it's an S3 thing and it just works. So that's super cool and easy to do. The pricing is so much better than S3. $5 a month flat for 250 gigs of storage, a terabyte of outbound bandwidth, which would be like $90 alone on S3. And then it's just one cent per gigabyte instead of nine. So definitely, definitely

check it out. I'm using it for this podcast and many other things. The audio you're listening to came over it. So how about that? It's super easy to use and it's a great deal and very predictable. So do.co slash Python and check out DigitalOcean Spaces. Let's talk about visualizing things as well here. So the last week you brought up that deep dive inside of garbage collection. That was a super good article. And I really appreciated getting a look at how the reference counting GC worked about how the actual mark and sweep garbage collector that would solve the cycle problem worked, how Python uses that combination of them and a lot of the really nice details about at this

number it behaves differently if it's bigger or smaller, right? These little cutoffs are always fun to know. There are many different garbage collection algorithms, of which reference counting and mark and sweep are just one. And so this guy named Ken Fox, it's not a brand new article or anything, but it's really a cool follow-up, wrote this thing called Visualizing Garbage Collection Algorithms. And yeah, I think this is, if you liked the last article, you should definitely check out this one.

The little videos on visualizing this are, they're kind of mesmerizing. I kind of want to like leave one up on my screen or something. They're great. Yeah, exactly. If you could have a screensaver of visualized GCs, it would be beautiful.

We need that company that made the toasters, the flying toasters. We need them to make that, right? Even though I covered it last week, I'm trying to remember. The Python is both a reference counting collector and a mark sweep collector, right? Right. So by default, Python tries to use reference counting. So every time you have a pointer to an object, it increments the counter. And then once that counter goes back down to zero, then it just deletes it.

The problem with all reference counting collectors are if you have any form of cycle, that's straight up a memory leak right there, right? Parent, child, child, parent, boom, never goes to zero, never deleted. That can be a huge problem. And so there's ways to solve that. Usually that's some other kind of managed garbage collector, right? And that's why they bring in the market sweep one, because it goes and finds these cycles and gets rid of them. One of the questions I have that I don't expect you to have the answer, I'm curious about it, is if fragmentation becomes a problem within Python or not. I think it does. I think it does. You can actually see it in the picture.

So let's go through the five visualized here. So there's actually five different types of GC. There's no GC, which just means we're just going to allocate and not clean up stuff. Hopefully we won't spend too much memory and that'll be fine. That's actually a super efficient way as long as you have enough memory. And it might sound outrageous, but Apache, Apache web server actually creates small pools of memory per request and then just throws them away and doesn't try to manage that memory. Dirty. So, you know, I guess in like small doses that could work, there's the

reference counting one, and that's Python's first pass, as we talked about. There's a mark and sweep collector, which goes and says, I'm going to start all the known pointers that I have, follow them, the things they point to, follow all that sort of transitive closure of pointers, and indicate or mark everything that is not garbage. So it's like a not garbage finder, and whatever it doesn't find, well, that's garbage. And then there's some stuff more like Java and.NET and those types of things that are mark and compact collectors. So Python, you asked about the fragmentation.

As far as I know, it doesn't move the memory around and change where the pointer's point. After it does a collection, it just cleans up the cycles. And so if you had a 10-byte object in between two things, it got cleaned up, that 10 bytes is just a hole. Maybe you can fill it, maybe you can't. But the compacting ones, those will squish the memory back together. And what's interesting about the compacting ones is allocation in a compacting collector is insanely fast.

Probably faster than C++ or C. Because you don't have to traverse that list of open slots where your new memory might fit. You just always allocate at the end because you've compacted it down, right? So allocate means increment a pointer by, you know, you want 10 bytes, you increment the pointer by 10. Done, right? So it's pretty interesting. There's a couple of kinds. And so this goes through and visualizes all of these. And you can actually see the effect, and you can see the fragmentation that you were talking about in the reference counting one. But it's pretty cool, this set of visualizations. Definitely. You know, I feel like I should be using Pathlib more than I do. And I just do os.path.join and

that sort of stuff. They're the older style of working with paths in Python. And that's worked with me. But this article by Doug Hellman that you pull out here, this is pretty cool, and it makes a solid case for Pathlib. Yeah, actually. And so I went back and I'm like, when did Pathlib come in? And supposedly it came in introduced in Python 3.4, so it's a Python 3 new thing. It's an object-oriented system to work with paths,

so paths as objects and the like i we're pointing to doug hellman's uh python module of the week the python 3 version and it's um pretty good it's a very quick read to skim over all the sort of stuff you can do yeah i learned things just the other day thinking oh my gosh i i've been using OS too much. I should use this more. Yeah, for sure. And I was really blown away at the way you use the division to hack the visual, right? The way you use the divider, like forward slash. So you can have like one path forward slash the other path and it just joins them as if

it was this thing slash that thing in the actual string, right? And you can use that division operator with either path objects or just some quote strings and characters and quotes to add things to your path. And I didn't know that it had glob in there, so that's pretty cool. And itergers, iterger for like iterating over files in a directory. That's pretty nice. Right. One of the things that's annoying about os.path is if you want to actually create directories

or delete them or stuff, there's like all these unrelated ways. Like you go out to other areas, you know, like OS.mkdir rather than OS.path.mkdir. Just funky inconsistencies. And this like brings it all together. You can like create and delete directories and things like that. And then there's some top level stuff for concrete paths that are shortcuts like path.home and path.cwd

that just work whether you're on Windows or a Linux-based POSIX system. Yep, that's really nice. Cool, yeah. So definitely speaking of learning new things, here's something people can learn. This is great. So we talked about visualization before, but can computers see? Yeah, sure. I think they can. I mean, cars,

we're surprisingly close to a world where cars can drive on their own. Are you ready? Yeah, I'm totally ready. Yeah. I mean, we've got a whole bunch of deep learning stuff happening. Nvidia made a car. I think we talked about this before where like they just had it observe humans and they made it go drive and like they don't know how it knows how to drive but it can things like this right they didn't teach it to drive it just like learned and so i want to highlight this thing called luminoth an open source computer vision toolkit so this is a new computer vision toolkit for python and it uses deep learning to understand computer vision. So basically its main job right now is to work on object detection and classification,

but they're trying to do a lot more of it like that was the first problem they solved. So it's built Python, it uses TensorFlow and Sonnet, which is Google's deep learning framework and DeepMind minds graph library. So these are, these are really cool. This is the same thing that Google is using very likely for things like it's AI that just beat go or it had beat go before, but they just announced that there's a new AI that beat go without anyone teaching it how to play go.

So that's the news this week as well, by the way, although I don't have a link to the article so you can super easily train this thing like it's like a couple of lines of code and then you can start asking it to find stuff in objects and they'll say that's a bike that's a hand that's a person the face is over here no that's three people things like that really really cool and it uses this um modern state-of-the-art algorithm called region-based convolutional neural networks r-cnns it even comes with gp gp gpu support and it's cool they even are going to send pre-trained checkpoints for objects that it is already pre-classified

with some like really large data sets from other challenges. Oh, wow, that's great. Isn't that cool? So if you want to go and create a project that does some sort of image classification or identifies objects in images, check this out

because if it comes with pre-trained algorithms already, like, and you have a similar type of data as what they already trained it on, you might be able to just pick this up and use it. Yeah, well, one of the examples it shows is detecting dinosaurs in a picture. Yeah, that's awesome. So you could build like a home security.

Yeah, it'd be useful so that you could find out if a dinosaur's at the door. Don't open it. No, it's a dinosaur. It'd be great. Like if you could come, for some reason I'm fixated on doorbells. If you come up with a doorbell that like rings differently

based on what it detects, if it detects a kid, like one of your kid's friends is at the door, it could make one kind of noise. If it's a UPS guy, it could like make a package noise. I mean, wouldn't that really be a cool product? Yeah, like a really big happy sound if it's a pizza delivery guy. Yes. Pizza.

You've got nails. Let's do that. Yeah, that sounds like a very good business. Let's do that. Yeah, that sounds like a very good business. Let's do that. All right. So anyway, if you guys are into image recognition, check this out. It's a cool project, Luminoth. It's fairly new.

You got a bonus for us, right? Yeah. I was going to put this as one of my picks, but it felt kind of... Anyway, there's a bonus article called The Cleaning Hand of PyTest. And it's somebody else's experience of what it was like to work in unit test and knows after using PyTest. And I feel a little, actually, I don't feel bad at all. But at the end, he links to my book, which is good. So check that out. That's nice.

That's a nice validation. That's cool. Yeah, The Cleaning Hand of PyTest. That sounds really fun. How about you? What's up with you? Well, not a whole lot going on. I wanted to highlight two things.

You mentioned the various PyCons going on, and PyTennessee 2018 is coming up. So if you're anywhere near Nashville and you want to go to a PyCon, you don't want to wait until May to go to the big, big PyCon in Cincinnati. I think it's somewhere in Ohio this year. You can go in February 10th and 11th

and go to one in Nashville, PyTN. So check that out. That's really awesome. That'd be fun. Yeah. I was thinking of trying to go, but it turns out that my wife's already out of town that week. And with children, we can't both be out of town the same week.

So that sort of squashed that. The other thing really quick is I wanted to point out a movie. It's not super new. It's like a couple of years old, but I think people in this audience will really appreciate it. Are you into gaming, Brian? No. Yeah, I'm not either. And so I'm kind of oblivious to this world and what has happened to it in the last five years. But there's this movie called All Work and All Play. It's available on netflix i linked to the trailer and it is a really interesting look inside what has been going on in the world of professional gaming so if you're into there's basically they had a the world championship of gaming they had in poland

and there's some big upsets and surprises and i think they said a hundred thousand people intended in person i mean it's like insane so if you haven't been paying attention to this, this movie is pretty eye-opening. So this is people playing video games? This is a competition of League of Legends players. I think there were six teams. Okay. Six or eight. No, eight teams, I think.

And then these are people watching the games being played live in person. So you're not a gamer either? We found two nerds talking to each other and both of them are not gamers? No, not really. I mean, my... What are the odds? I know. It's pretty low, I guess.

My pastime is to work on software projects. That's my game is like solving software problems, not, you know, made up game problems. Every now and then I'll play an iPad game or something, but nothing major. Yeah. Okay. Well, cool. All right. Well, thanks a lot. You bet. then I'll play an iPad game or something, but nothing major. Yeah. Okay. Well, cool. All right. Well, thanks a lot.

You bet. And I'll catch you next week. Thanks everyone for listening. Bye. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at PythonBytes.fm.

If you have a news item you want featured, just visit PythonBytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Ocken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.