Python Bytes - #204 Take the PSF survey and Will & Carlton drop by

Episode Date: October 23, 2020

Topics covered in this episode: nbQA: Quality Assurance for Jupyter Notebooks The PSF yearly survey is out, go take it now! From Prototype to Production in Django Deployment: Getting your app online All Contributors MovingPandas Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/204

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 204, recorded October 14th, 2020. I'm Brian Ocken. I'm Michael Kennedy. And we have a couple of guests, Will Vincent and Carlton Gibson. Hello. Hello. Hey guys.

Happy to have you here. No, thanks for having us on. It's really quite exciting. Before we move on to the first topic, people may already know you. You guys are Django famous, I hear. Tell people about your podcast real quick. Yeah, so I'm the Django fellow help maintain the framework there and with Will we run a Django chat podcast so and Will does some other things what do you do Will yeah well when we started the podcast I was just a book author but I'm a Django software foundation board member

now so I have a hand in coordinating with Carllton in official capacity but mainly i teach django through books and a learn django.com website oh and i have a django news newsletter as well so i keep piling things on we don't have two podcasts like both of you but maybe we'll get there we haven't got time for two podcasts you start with one and then you just get another one that's how it goes you guys okay like children brian's gonna start a third one i heard i'm gonna start a reamer for Brian's going to start a third one, I heard. I'm going to start a Rima for him. I'm going to start a third? Yeah, why not? I just should jump to four.

I mean, binary, right? Exactly. Hey, Brian, we talked about Jupiter a bunch the last couple of times. Don't let us down. Keep it rolling. It's a great thing to provide a tool for people, and then we get a whole bunch of people getting a hold of us and saying, hey, there's more stuff you should know about.

And this is the case this time. So Marco Gorelli, I think his last name is, sent us a, also I have to say, he said he was a longtime listener and he's a Patreon supporter. So thank you, Marco, for being a Patreon supporter. Thank you, Marco. Very cool. So he said, you guys should know about MBQA. So MBQA is quality assurance for Jupyter notebooks. So it also can run black so you can one of the things if you just want to run it like a black thing to run black one of the

benefits of using it to run black is that you can run it on an entire directory not just a single notebook but a whole directory full there's some modifications with its use of black so that it keeps diffs fairly minimal for the diff set. And then black will take off trailing semicolons because in Python, they don't really mean anything and they look ugly. But in Jupyter notebooks, apparently they mean something. They mean to suppress the output of the notebook or suppress output. So that's the black version or the NBQA version of black. We'll turn that, but leave those in place. And also supposedly supports standard magic commands and magics are kind of a big part of Jupyter thing.

So another thing I want to mention, it doesn't just run black. You can also use NBQA to run isort in MyPy and Flake8 and PyLint and even PyUpgrade and DockTest. So, it's pretty neat. Yeah, this is really cool. I think it brings so much of that code formatting and code analysis cleanup to notebooks, which I think

have been really lacking. Yeah, some of the standard practices that a lot of people are using now, as well as the configurations all in a Pyproject.toml file, and you can hook it up with pre-commit so that you can have all these things running when you check stuff in or whatever. Yeah, and you can even run it against a whole

directory, not just one notebook, which is sweet. I'm definitely going to check this out. It looks really fun. Oh, one thing I wanted to mention, I checked out the source code for it, and it's 100% covered, and covered by PyTest, of course. So, nice. Nice. Little chip. Will Carlton, what do you guys think about this? Well, I was thinking I use all those tools, but I don't use them in the notebook format. I have to

say I sort of bought it with notebooks, but I'm not a big user there. So, it sounds super, but I recommend all of those. Yeah, absolutely. Ditto,itto and i would whenever i think of jupiter notebooks i'm always reminded of i think the finest tech talk i've ever seen is i don't was i hate notebooks that joel gave a couple years back yeah joel gruce which is but i mean it is he's not just slamming out the whole time but it's a very educated talk and i think that's a high bar for sort of complimenting and pointing out issues on a framework or on a project that can be improved yeah well i think that actually a lot of the complaints are starting to get addressed with things like this right it's starting to get a little bit better there's also some other cool ones called jupiter lab dash lsp for language server protocol or provider or

something like that which also does a bunch of things that make it a little bit better so yeah it's getting there it's pretty cool just one thing the ultimate for general web developers is if you could take a jupyter notebook and just snap your fingers and have it be a django site you know you can't quite do that but if i take off my technical head and just what would change things like i'm surrounded by a lot of scientists to turn a Jupyter thing into a website with standard CRUD. I feel like it's possible one day, but we're not quite there. That would be really fantastic.

And we're going to hear like five ways we could do it that we didn't know about from listeners, which is great. The one thing I would really love to see in Jupyter Notebook and maybe someone out there knows about it is I would like to see collapsible sections. So I've got like a report and it's got like some markdown and then some code, then a graph it's generated. And then some more, like maybe a picture and then some more markdown. And the code in there is really

awesome to have. But if you're going through it as a report, you don't necessarily want to see the code unless you want to like dig into it. So it'd be great if you could say these columns or these cells are collapsed. I really would love to see that, but I don't know about that yet. That's opposite direction of what you're looking for. That's making the notebook more of a article, less of a website. All right. So the next thing I want to cover is the PSF yearly survey. Have you guys taken your yearly survey for 2020? I've done it. So I've done it. I already got my homework. Well done, Carlton. Well done. I have not, but I've done it past years, and I will. And we actually, in the Django world, we're inspired.

We had our own survey this year, the first time in five years, because I don't think Python does as well, but Django doesn't track anything, so we actually don't know. No installs or usage. Right, okay. And that's obviously helpful to fellows and technical team. Of course, and I think basically the extent to which they track that is the analytics that come

out of pip. Like pip was run on this operating system. This package was installed this many times. It was this version of Python that did it. Beyond that I don't think there's much tracking in the Python world either. The broader. We have PyPI but that's not completely accurate in terms of

popularity. All the Docker rebuild stuff that's happening all the time, that counts, but it's not legit and so on. So yeah, for sure. So if you haven't taken the PSF survey, I've put a link in here. It takes about 10 minutes. You should go do it. This is the fourth time they're doing this developer survey.

And it's the major, a major, the major, I'm not sure, a major source for sure about the current state of the Python community. So what editors do people use what web frameworks are people using are you a data scientist or are you a web developer etc etc are you just getting into python and if you haven't seen the analysis of this before i linked to the 2019 results which were put together by jet brains and they did a really nice job of like making a compelling story to be told out of it right yeah no it's really nicely done and presented

you're like oh wow yeah it's super that's why you know it's worth putting into because it's the production value at the end is great and so it's a valuable resource i felt bad for a second that we didn't have that on django but instead i just we We're not JetBrains. Yeah, we're not JetBrains. That is the gold standard. I was like, oh, it'd be nice to have that. And then I was like, or I could just make it a Google form. Well, you guys should reach out to the JetBrains team and see if they want to partner up. Yeah, well, they've probably got the resource in place.

You know, they've got the infrastructure. So maybe. Now, have they always done prizes? I don't remember that they have, but they have them now, right? Yeah. So that's cool. Yeah. So't remember that they have, but they have them now, right? Yeah, so that's cool. Yeah, so they announced that 100 winners, completely random, if you've completed the survey,

will receive the amazing Python surprise gift pack, which I have no idea what it is because that would ruin the surprise. I saw some good Python socks on Twitter today. I hope it's got Python socks in it. Oh, my God. I love socks. That's half the reason I go to conferences, let's be honest. I've got my Twilio socks. I've got all my different socks. I've got my I love socks. That's half the reason I go to conferences, let's be honest. I got my Twilio socks. I got all my different socks. I got my MongoDB socks.

I've used to go just for t-shirts, but now I kind of like the socks more. I do too. We've added an official Django merchandise store and there's some items on there and that's been helpful with the virtual conferences, but we don't have socks. So there's a lot of inspiration we can take for having better official gear out there. Absolutely. Get your sock game on yeah and then stickers right it goes t-shirts stock socks stickers i think in the hierarchy of swag that's right that's right okay i have to up my game i'm just giving out stickers usually i'm an enamel pin man myself oh enamel pins oh yeah

i should mention jet brains which is doing that survey, is a big sponsor of Django. We do a couple week long thing every year, and they're a major corporate sponsor of Django. So shout out to JetBrains. Very nice. Yeah, very nice. One of the things I'm really excited about is we have a new sponsor, and it's another podcast. Yeah, so that's pretty cool. So this episode is brought to you by Tech Meme Ride Home Podcast.

For more than two years and nearly 700 episodes, that's amazing. The Tech Meme Ride Home has been a Silicon Valley favorite tech news podcast. The Tech Meme Ride Home is a daily podcast, only 15 to 20 minutes long, and every day by 5 p.m. Eastern, it's all the latest tech news. But it's more than just headlines. You could get a robot to read you headlines, but the Tech Meme Ride Home is all the context around the latest news of the day. It's all the top stories, the top posts,

the tweets and conversations of those stories, as well as behind-the-scenes analysis. It's like a TLDR as a service. The folks at Tech Meme are online all day reading everything so they can catch everything up for you. Search your podcast app right now for Ride Home and subscribe to the TechMeme Ride Home

podcast or just visit pythonbytes.fm slash ride to subscribe. Yeah, it's like Python Bytes, but just for general tech. Every day though, these guys don't mess around. That's incredible. Yes. All of us who are podcast people are like oh my gosh that's insane yeah we all wince a little bit hearing 700 promised by 5 p.m

sounds like a burnout algorithm it's well done but yeah cool thank you guys for sponsoring the show will what's your item oh so my item is from prototype to production and django so this is a common thing where you get a little familiar with Django and you say, well, what's the, there's this big chasm basically from building a CRUD app locally and deploying it properly without being hackable. What do I do?

I run it as root. I leave the debug setting on. What else do I do? That's important. Yeah. Well, it's sort of like, you don't know what you don't know and then the older you get the more scared you get because you've seen it all go bad but when you're starting out like what could

go wrong yeah but as soon as it works it works yeah so specific to django i think like most web frameworks it has to it focuses on local production so when you run you run a start project command and it creates some scaffolding for you and then then specifically, it has a settings.py file. That's kind of the global config. And that's set for local development. So works great locally, but if you just dump it into production, you're going to be wildly insecure and easily hacked. And so it's a quick list of things you want to change.

And Carlton, please jump in here as the Django fellow if I missed something. But debug is a setting that is, you want to switch to false. That provides a very nice error message, but it also is a setting that is you want to switch to false that provides very nice error message. But it also is a roadmap to hacking your site if it's left on right, these are in settings.py, generally in settings.py. So it's all about settings.py. Basically, there's a secret key that Django that is 50 character long string randomly generated,

you want that to be secret, because use the hash throughout Django. And of course, what happens is you do one git commit and then it's out there. So you need to change that or really put it into an environment variable, which I'll get to in a sec. Are you familiar with shgit? No, I'm not. S-S-H-git.

So this is super scary. This is a, oh my gosh, it's live right now. I can't believe it. So I think it's at ssh get.com but there's also the open source version that you can get you can see it on github let me just just read you this title just to like point out how seriously this should be taken should get finds committed secrets and sensitive files across github gist get lab bitbucket and your local repos in real time it does this by subscribing to the commit stream on GitHub

and instantly posts the secrets like AWS secret keys and stuff. You can see if you go there like... Yeah, I see six for Django right now. And it's all the configuration. Yeah, I just got two more. Yeah, I just got two more, three more, five. I just got five more. I mean, it is insane.

Yeah. If you think my repo is not so popular, it will be fine. It may not be so fine. But this is the thing with security, right? Is that it doesn't matter how small you are because the people who are attacking you, they're using automated scripts. So they're checking every port on every addressable server

for every known weakness. It's not if you'll be hacked, it's when. Now setting the stage how frightening this is. Carry on while we shouldn't put that in. Well, I think it um reinforces that the settings that py file is where most of the action is in django and you want to be careful with it i mean i remember github back in the day you could just global search for aws keys and stripe and everything now at least you can't global search for that stuff and they'll even ping you so for me like

i have some secret i have some projects in my books that are on github and there's a secret key there and they bug me all the time saying, Hey, you have a secret key exposed. I'm like, Yeah, I know I do. I don't, it doesn't matter. So it's gotten better. But yeah, it's still all out there. So secret key, keep that secret. Allowed host is probably the last big one. These are the hosts that can come in. It Django will prompt you to change that. So if you're using Heroku, and it's my app.heroku.com, you want to set that host to be only that host, not all hosts can come in. Database is the next one. So by default, Django has a SQLite database, file based, really easy to use. Fantastic for production,

large scale, Facebook uses it. You know, it can be. No, I just do. If your workload is read only, so say you're running a content site, and it's you editing it, SQLite will go all the way with you. But as soon as there are more than one editor... It's incredibly fast. It's in memory, right? Yeah. You can have it in memory or you can have it on the file.

But on read-only workloads, it will go right out there. Sorry, I meant in process. In process, not in memory. But it's not like a separate server, right? Exactly. It's just a file next to all your other files and it can hold terabytes of data without a problem.

But as soon as you've got multiple users logging in at the same time or that kind of thing, then you need something like Postgres or MySQL that can handle that kind of concurrency. Concurrent runs. Yeah. So you probably want to git ignore your SQLite file. But also, you definitely want to use whatever you're using in production locally. So Postgres, MySQL, MariaDB, and Oracle are the supported databases. What else? Almost done here. You configure your static and

media files. So static would be images, JavaScript. Media refers to anything that's user uploaded. You definitely want to be careful whenever you have anything from users. You can't trust them. You want to use Django forms. You probably want to use bleach to sanitize. And you want to have that on a CDN, not on your server. Two more to finish up. So the admin, Django has a famous admin that's very powerful, which is at slash admin. You should change that away from dot admin because to Carlton's point, there are bots searching for Django sites at slash admin, and they will come in and hack away at your site.

There are a number of fun technical things you can do to honeypot it or this and that, but you should just change it away from dash admin. I'm tempted to note there's a very famous Django site that still has slash admin, but I won't mention it publicly. Carlton and I both use it though for our work. And then the last thing is user registration. Django comes with login, logout, reset, but it doesn't have a signup. So you can roll your own or most people would use a third party package called Django all off. That's fantastic that has social support. So Django has very robust third party ecosystem that over time, the most popular ones, or the strongest ones are rolled into Django. But there's also separation because it's too much for Django to maintain. Django all off is not

is a third party package, but it's, in my view, effectively part of Django for most people. So those are the big ones. The key things I mentioned there, environment variables, it used to be with the settings file that back five years ago, you'd have multiple settings files, you'd have a base settings file. Carlton's still doing that? I still have multiple settings files, go with those folks. I mean, you have environment variables too, but multiple settings files for the win. He's a Django fellow. He doesn't know what he's talking about. You need to use environment variables for this, because then you have one settings file, and you load it into

local staging or production. It's much easier. Spaces. And there's a number of third-party packages that will help you with that. I like environs, which will be linked there, which also has DJ database URL, which is a nice feature on environment variables for databases. It just means you have a single settings file, and you switch with environment variables. There's also Django has a handy deployment checklist, which I think

a lot of people don't know about. We have a link to that you can run Python check dash dash deploy. And it will check that all the things I mentioned plus a number of HTTPS things are actually configured properly. So you don't want to deploy your site if you don't pass most of those, if not all those. And that's, you know, there's testing, logging, performance, security.

We can go on and on and on. I wrote a whole book, Django for Professionals on this, but those are the highlights. And there's a check, there's a good doc on the Django docs for deployment checklist, which, you know,

you should open that every time you review. Yeah, the hard thing is there's like a couple of must haves, like the ones I listed there. And then there's a lot of it depends nice to haves. And that's where it's harder to make generalizations. This stuff is so rewarding when you get it right to see your site up and running, you know, 99% plus uptime and people using it so fantastic highly responsive but soon as you see like something go wrong it just your heart sinks and so most of those things are because like you both have hinted at there's some kind of bot that's looking for some vulnerability and like a known thing and just make sure you don't put those known things in front of the internet yeah and i should say actually there there used to be a site called Pony Checkup that you could put in your URL and would automatically

test a lot of this for you. It's actually someone has taken that over from the original maintainer. So it's now DJ Checkup slash Pony. So you can type in your URL and check. That's kind of a good way if you're a beginner, if you're not sure. There's nothing like going to a webpage and seeing security issues in your site or others to kind of scare you into doing something. Yeah, yeah, for sure. Awesome. Well, I'm glad you covered that. And Carlton, it sounds like the one that you got is a bit of a... Similar topic, actually. So I've been thinking about it, but I think it must be Django Chat. Every week, it seems, we have a guest on, and we end up talking about deployment, and it's massively complicated. And Will's just gone through a whole list of things and that's only the tip of the iceberg kind of thing. And the thing that I got thinking about was that there is this deployment gap.

So I imagine someone finishing the Django tutorial, finishing the REST framework tutorial, or finishing Will's Django for Beginners book. And then how on earth do they get their app online? And it's like, you know, unless you're going to dedicate a week or two weeks full-time researching and trying and following tutorials online, it's like this chasm. We can't do it. And so platforms as a service like Heroku or App Service or App Engine or DigitalOcean have got their new one. They look like a great starting point because they're kind of easy.

But in a way, for me, they're a kind of cul-de-sac. You go into them, you get to the end, and then you kind of have to go back out again when you want to do something more advanced. But then on the other hand, you've got this do-it-yourself option of provisioning servers and setting up firewalls and virtual private clouds. And it's just, it's way too much. It's scary, right? And then you read some blog posts saying, well, you've got to do it with microservices or you've got to do it with this container orchestration platform. And no, no, no, it's too much for me. So I've been thinking about this and trying to come up with a story of my own for it and which I'm launching next year. It's going to be called Button. It's going to be a little tool that just

wraps it up and tries to take some of the fuss out of deployment. So that's not ready to go yet, but I wanted to mention it because it ties into what Will was talking about. And you can sign up for early updates on btn.dev, button.dev, btn.dev. So that's my topic. That's a great topic. I think I personally struggled through this, right? I started out trying to run my websites in Python on some pass place that was very simple and easy to get started, it's just there's a ton of downtime and things weren't working the way that i was really exact you know hoping and i ended up having to do a lot of things anyway and so i finally bit the bullet and figured out how do you run micro whiskey safely how do you keep these things up how do you do zero downtime

deployment how do you continue so there's just so many and how do you keep updated right how is it so okay you get it set up and it's fine. But then six months later, there's a security patch, which you don't quite know how to apply without bringing your whole site down and rebuilding your server. It's like, how do you deal with those problems?

They're not something you can learn quickly or easily. Yeah, absolutely. Brian, do you got to deploy things? I have before. And that's why I don't anymore. I mean, Very wise. Yeah, last time we talked about DigitalOcean's new pass service, and you were like, I'm all about

this. Yeah. So you did your own Python Bytes and TalkPython. You did those applications, right? TalkPython, TalkPython and like 10 or other little APIs and stuff that I'm running. All those, yeah. Right. So I've done, I mean, I've built websites before, like in the way past, like decades

ago. And then when I wanted to do a podcast, I did a Python. The testing code started out as a, as just part of my blog. And it was like on WordPress or something like that. Now I will go with a Fireside thing. And I don't, I mean, Fireside is a good service, but it's not, I mean, it's not ideal. It doesn't, it isn't perfect, but I don't have to think about it. What I want to do is podcasting. I don't want to maintain a website. So there's a lot of things

where we do need to build these custom websites. And I'm glad that there's some attention given to this because yes, I can learn how to do a django website but going from there to a live site is horrifying so yeah i literally just spent an hour this morning maybe hour and a half before like up to 10 minutes before we started recording deploying my first fast api endpoint through g unicorn uvicorn behind nginx and all that stuff and a lot of it was the same but that's the first time i've done it with uvicorn and the settings are a little bit different than say micro whiskey so i can run some async stuff and i kind of lived in that world of like i gotta what is the config key to make sure the config settings that make sure that it runs as a

different user not root again in this server and just like you just go through all these things and having that like really automated would be great. Oh yeah. I mean, the other day, my site went down for half an hour and it turned out to be the DNS in the end, but I never thought the DNS would go down. How often does that happen?

So I log into the server and I'm checking the application server. That's fine. So I check the local engine X instance. That's fine. I check the load balance. So that's fine. And I'm like, it is, it's the DNS.

And then by the time I've worked this out, the DNS comes back up and the site's back up and it's ah and what i want to do is put all that fully automated you know so i just probably run the diagnostics it goes green green green green green red ah there's the problem okay fine that's awesome let me know when you got that i'm excited 21 btn.dev okay cool yeah well and carlton i think your cul-de-sac analogy which i haven't heard you say before that's exactly right yeah because it sounds you're like oh this will solve my problems and then you learn everything and then you kind of come back with a different problem i mean part of the problem with doing server stuff i think is that it's it feels good to an engineering mind right it feels good to be like you know drive manual and tweak things but then you've lost

weeks of time and your app looks the same and you know so there is a danger there in terms of i feel like you kind of need to do it once or twice and then be like okay okay i'll just have like i'll just have carlton handle it for me like i trust him more than this is the via media right this is between the the sort of the platform and service which perhaps doesn't cover all your need cases and then the over-engineered you know i'm using container orchestration for a Microsoft thing and I've only got one server and a thousand hits a month, right? It's, you know, there's a middle way between that.

AWS will never be bothered to make it friendly to small people because they don't care. I mean, they have this, you know, big client. So yeah. Yeah. Yeah. Very cool. I also like the cul-de-sac thing. So you think it's an on-ramp, but it's really a cul-de-sac. Yeah. You're just an infinite loop of...

But that's the thing, especially somebody who's in that deployment gap scenario that I talked about. They go down the platform and service route because that's the only thing they can do. But at the end, they're like trapped there. And I want to do more, but I've got to go all the way back down here to learn this other stuff, which is so hard and so scary and so overwhelming yeah well i want to talk about something that's easy and well it's sort of easy it's uh contributing to other people's open source projects that's not easy you're just petting carlton's here no i'm really excited about this topic so i ran across a thing called All Contributors. And actually, we talked about NBQA before. And that's where I got this idea because they use this tool called

or this service called All Contributors. And it's sort of a service, but also just a spec. So I'm just going to read it. It says this All Contributors is a specification for recognizing contributors to an open source project in a way that rewards each and every contribution, not just code. The basic idea is to use the project readme to recognize the contributors of members of a project community. The idea is like there's a lot more stuff going on than just code. There's things like documentation, design, writing examples, doing maintenance, writing plugins for things, doing podcasts, giving talks and all that stuff.

And it'd be cool to recognize all these people. So they've got this spec for kind of how to do that. But then there's even more. There's a cool emoji key, which I love this. I love the emoji key part. It's so friendly. Yeah. So it has recommended emojis to use for your contributors list that includes things like a little laptop computer for code, a little thing for documentation, design, examples, all the sort of things you'd want with like fairly good. You don't have to think them up. They thought them up for you. So it's nice yeah they also have like this bot that you can attach to your github repo so you can just add comments into somebody's pull request or something to say hey all contributors

please add this person to the contributors list or something and it just does it so okay that's super cool we're looking into that with jango i was a few weeks ago i've been reading i was reading will was reading other people were reading the working in Public book. For the life of me, I can't remember the author's name this moment. It's Nina, I think. Okay. Egbar. But it mentions this all contributors thing. And one thing we've got with Django, we've got a massive contributor base,

but we kind of only historically recognized the sort of 30 or so people that were in Django core. And then over the course of the last couple of years, we've kind of tried to restructure the governance and we've managed to do all of that. And now we're in a position where we want to try and recognize all the other people that we translate the docs into however many languages that translation team gets virtually no recognition. Let's recognize them. There's yes, there's code commits, but there's also all the people that help triage and review the tickets and review the pull requests. And, you know you know those people need recognition there's the people who organize all the django cons those people need recognition we really want to try and show that like django isn't just you know i committed to django django it's

the whole ecosystem so i think this all contributors thing is great tool you said you're reading the working in public uh book by uh would you recommend it is it good yeah it was amazing but like yeah just the first few chapters it's just like describing django to a t it's like oh yeah this is the challenge we face every single day like a couple of years ago at django con in san diego i gave a talk about your web framework need you and i put up a graph of contributors and in the first chapter this exact same graph i mean it's got different numbers and it's for a different repo but it's the same power law shape. It's the same problem.

It's like, this isn't just Django. It's every open source project out there. It's got the same issues and it's the same dynamic. Do you grow in on yourself and get smaller and more enclosed? Or do you open out to the community and welcome contributions and find a way of doing that? And if you can, you can survive and flourish. And if you can't, well, you're wither and die. Yeah. I mean, I literally took screenshots of the book

because I was like, Carlton, you should read this. He's like, and he's like, okay, okay. It's exactly my thing. And this all contributors is so relevant because the most Carlton mentioned at Django, we're changing around what Django core refers to. And I literally have a huge spreadsheet with all the various people we're trying to categorize that this would fit in perfectly for. So I'm going to potentially use this. And it has a GitHub bot, which is fantastic. Yeah.

What can be done manually and, you know, five minutes can be automated in an hour. So that's right. Well, Michael, what are you going to finish this up with? Well,

you know, I want to keep us on the move. Don't sit still, you know, rolling stone gathers no moss sort of thing. So pandas is a super popular library in data science, right? And there's a bunch of visualizations. One way to work with geospatial data is with geopandas, which is cool. So there's a library called moving pandas and the idea is if you give locations plus time you can

map all sorts of interesting things and analyze all sorts of interesting things it's cool right and this project as it should has a bunch of animated gifs so yeah yeah that's what we need to tell exactly what it's about right get in. So it provides trajectory data structures and functions for analysis and visualization. It started out as this QGIS plugin, but they decided it made more sense to just be its own thing. So it's its own thing. So basically it takes GeoPanda's geo data frames with timestamp points, and it converts them into moving pandas trajectory collections. And you have properties such as speed and direction. You can turn continuous observations into trips. Like I was here for a really long time and then I went to the store and I was there for a while.

And then I came back, right? That kind of stuff, I think. It'll aggregate them into flow maps. So instead of I went exactly from here to here, you can say, here are the nodes where I spend time and the paths I take between them. Almost like graph theory type of stuff. And work with it is super straightforward. So you can just go create a

pandas data frame, pass a geometry and a time, and you convert it to a geo data frame, and then you just say, give me the directory and you can plot it, and that's it. Like, incredibly simple incredibly simple that sounds super their website's really good as well it's just clicking on it because it's to see the animated gifs and whatnot well it seems like that would that would overlap with so i mean django is a big framework there's a whole geo django area which carlton i

have discussed i mean that and the orm are the two parts of django i kind of don't really know to be honest. But they're very powerful and people use them. You can also get graphs of kind of derived data. So like speed over time, rather than just position. You can get these other analysis in there. And I can see lots of interesting places. I had Ken Replical on TalkPython to talk about how they're using Python on the race teams

for simulations and stuff. And like, those types of analysis, this seems so perfect. Go spin a data track, collect a bunch of data, throw into these types of things and look at the curves

and whatnot. Yeah, and Pandas is almost like the kind of data transfer format now. I mean, you know. Yeah,

absolutely. So to be able to integrate this sounds super useful. Yeah, it looks cool. Awesome. All right, well, that's the last now. Yeah, absolutely. To be able to integrate this sounds super useful. Yeah, it looks cool. Awesome. All right, well, that's the last item. Brian, you got any extra stuff you want to share with people? No, just working and plugging along. How about you?

I do have a few things. First of all, I was talking to Hugo Brown Anderson from Coiled, and he asked, hey, when is the transcript from our research show going to be out i'm like don't really have transcripts at the moment the company that was working with to generate them stop generating stop doing that kind of stuff and i haven't figured out what to do he's like oh you should check out otter.ai like yeah but isn't that like for your phone and like you're gonna have conversation that's cool but but what i realized is i can upload files to that, our old episodes. It'll convert it to mostly correct transcripts.

Like pretty good, actually. You know, it'll get like AWS, right? And things like that. And we just wrote. Yeah, I do. I just wrote some automation to turn that into transcripts. So I added like half a year worth of transcripts back, which means that feeds our search engine. So search should be better as well and stuff like that.

Yeah, I think it's the top the top one. I think Wes Boss was also asking something and I was tweeting with him saying, yeah, Otter, I think, when I checked is by far the best one. It's not really designed for transcripts. It's designed, I think, for like meetings and groups. But yeah, it works. We've been using it for a year. Yeah, that's awesome. And you guys like it? It's been good. Yeah, as you say, it's, I mean, it's the most accurate out there. And usually it gets almost everything. You can kind of have custom things like AWS if it gets it wrong.

And yeah, I mean, usually I run it through and give it a quick scan. Maybe there's a couple of things to switch, but yeah, it's a no brainer. It's got like a nice editor that like plays and highlights the words if you were actually going to stop and edit them. I also have automation, like for my courses,

I have automation through AWS elastic transcribe or just transcribe, whatever it's called to generate those and then hand them off to people. But otters looking nice. I'm not sure if I'd switch the courses over, but anyway, maybe we have a bunch of transcripts, Brian. Yeah.

So I've never really done, I started doing, um, testing code transcripts, but I was, I was actually just paying somebody to do them and it was getting expensive so yeah let's check this out too yeah we're checking out it's the seo that really matters i think that's like the killer feature because exactly that's why i first created

them and i thought okay i'll make them searchable so people can also get some value out of but my original reason for doing it was like instead of having three paragraphs of content for an hour conversation, let's have the real conversation, right? Yeah. But then, you know, someone will find that and that will be useful. And, you know, they'll be like, ah, this is, you know,

even if it's badly transcribed, this is roughly what I'm looking for. I'll listen to the episode. Right, right. Yeah, let me listen. Here's the timestamp and they can get some value out of it. So hopefully, yeah, that's the idea.

Nice. All right. Also, I'd switch from Google to try to live in DuckDuckGo land, just using DuckDuckGo. it. So hopefully, yeah, that's the idea. Nice. All right. Also, I'd switch from Google to try to live in DuckDuckGo land, just using DuckDuckGo. Oh, join us. Yeah. Yeah.

Are you guys doing it? Are you liking it? I've been there for three years exclusively. Oh, God. I'm still on Google, but I did install a pie hole this week. So, you know. Oh, yeah, yeah.

Swings and roundabouts. Carlton's like, I see your DuckDuckGo usage and I got to... I'll raise you a pie hole. This is what i have to deal with guys so far i'm liking it i mean i've been using firefox with all sorts of privacy stuff for a long time but um i figured just one more thing and i just want to point out if you're trying to be like slightly less connected to google they have google takeout or you just want to back up right if you use google drive and you sync your google stuff

it'll give you just a link to the spreadsheet or whatever, right? Like if you use Google Drive and you sync your Google stuff, it'll give you just a link to the spreadsheet or whatever on Google. Excuse me. If you use Google Takeout, it'll actually convert, like say your spreadsheets to Excel. So you actually have them anyway.

So that's part of that. And like I said, I got to deploy my first fast API app today, basically. And I'm just, I'm really enjoying it. And I feel like it's bringing in

a lot of these ideas. I'm hoping maybe you guys could comment super quick on this it's bringing in a lot of these ideas. I'm hoping maybe you guys can just comment super quick on this. It brings in so many of these new ideas into the web space, like the async and await stuff feels super natural. You don't have to do anything to make it work. The type annotations mean things. I just feel like there's a lot of interesting sort of modern Python stuff coming together here.

What's your Django perspective? Well, so FastAPI is built on top of Starlet, which is by Tom Christie, who's Django REST framework creator. So from the async side, that's what we're trying to build into Django now. And we have async views in 3.1 and, you know, we're working on the ORM next. And then from there, it will spread out. So there's a PR came in this week about the cache layer. So there'll be third-party async cache backends for Django soonish. An async, fully async framework like Starlet is always going to be out there. It's going to be ahead of where Django is. But we'll wrap it up and give it that nice Django feel

where you define your class-based views and all these things. We're not there yet, but that's where we're aiming for. Then the other thing that FastAPI brings out, which is quite exciting, is Pydantic, which is the type hinting used for the serializers and for the validation. That's kind of really cool. And at the moment we don't have a story there with Django. We've got Django

Forms, we've got REST framework serializers doing the same kind of thing. But we've got our eye on that and we'll see how it goes. I know you guys are definitely thinking about these things. It's very exciting. Yeah, I mean, but what's really nice about the current, particularly the ASCII world, where all the kind of ASCII is the standard, is there's an amount of interoperability

in that you can kind of nest your ASCII apps inside each other and wrap Winnowares around them and it's just another ASCII app. And so, actually, there's a lot of interop things. So, it's a really rich and fertile time for Django web frameworks. Yeah, awesome. I don't have anything to add, Carlton. I defer to Carlton. Other than it's sort of wild

that, you know, I mean, from Tom, we've known about Starlet that FastAPI is better known than Starlet. It's a little strange to me, but it makes sense because Tom's been busy rebuilding everything in async the last couple of years, kind of on the side. Also, the thing is that people touch FastAPI. They just

live on top of it. Like they live on the shoulders of Starlet, but people touch FastAPI. They just live on top of it. Like, they live on the shoulders of Starlet, but they touch FastAPI, right? Yeah, no, exactly. Yeah. All right, Brian, I put in two jokes

that we can offer up today. Let's shout out to some of the stuff that Will and Carlton are doing first. Yeah, yeah, absolutely. Okay, well, we just, I'd say, listen to Django Chat,

which is at djangochat.com. That's our podcast. It's fortnightly now. That's a fancy British word there for you. Check out Will's tutorials and is at djangochat.com. That's our podcast. It's fortnightly now. That's a fancy British word there for you. Check out Will's tutorials and books at learndjango.com, and then, yeah, sign up for the early updates on Button, which is at btn.dev.

Just did. Super. Welcome aboard. You'll be subscriber number three, I think. Well, actually, I had to tell Carlton, I was like, get up a page before we go on the podcast, because he's been telling me about Button for a year. Yeah, it looks good. Well, it looks like a sign up for him right now. Well, yeah, that's what it is right now.

But yeah, 2021, it's coming. Nice. Yeah, great. Okay, now a joke. Thank you. Thank you for getting us back on track, Brian. So you've heard about give a person a fish versus teach them to fish. There's a programmer version.

Did you know that? No. Yeah. If you give a person a program you can frustrate them for a day but if you teach in a program you can frustrate for a lifetime yeah yes definitely unless what brian unless you teach them to test at the same time ah very good exactly and speaking of fast api here's a a joke that I just saw that's relevant, sort of similar to one put out by Sebastian Ramirez from FastAPI. So somebody just failed a job interview and the verdict was delivered like this.

I'm sorry, we're looking for someone aged 22 to 26 with 30 years of experience with Flask or Django. Well, didn't he tweet about someone was looking for five years of FastAPI and he was like, even I don't have that. He's like, well, as the creator of FastAPI, I would not qualify you for your job having done only 1.5 years of it.

Well, I don't think people realize that it's so new. I mean, it's kind of taken over very quickly, but it hasn't been around for very long. Yeah, it's kind of taken over very quickly, but it hasn't been around for very long. Yeah, it's pretty interesting. Alright, guys. Thanks for joining us. Yeah, no, thank you. It's been a really cool chat. I really enjoyed it.

Yeah, it's been fun to have you here, Carlton. Alright, bye. Thank you for listening to Python Bytes. Follow the show on Twitter at Python Bytes. That's Python Bytes, as in B-Y-T-E-S. And get the full show notes at PythonBytes.fm. If you have a news item you want featured,

just visit PythonBytes.fm and send it our way. We're always on the lookout for sharing something cool. This is Brian Ocken, and on behalf of myself and Michael Kennedy, thank you for listening and sharing this podcast with your friends and colleagues.