Python Bytes - #213 Uh oh, Vulcans have infiltrated Flask

Episode Date: December 23, 2020

Topics covered in this episode: Django Ledger Project Flask-Meld: simple JavaScript interactive features without all of the JavaScript. Bitwise operators in Python (RealPython) Why should you use an ORM (Object Relational Mapper)? sqlite-utils: a Python library and CLI tool for building SQLite databases Online conferences are not working for me. But this was a good talk, Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/213

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 213, recorded December 23rd. Or Anthony, Anthony Shaw here. Is this December 23rd or 24th? You tell me. It's the 24th. It's Christmas Eve. Yeah, of course. Yeah. In Australia. Awesome. In the future.

Yeah, 23rd for us, Brian and me, here in the US, 2020. And yeah, this episode is brought to you by us. So we'll talk more about that later. And I'm Michael Kennedy. And I am Brian Harkin. And Brian, we got a special guest here, friend of the show, Anthony Shaw. Welcome, Anthony. Hi there. Great to be on. Yeah, it's good to have you here. Thanks for taking time out of your holiday. Yeah, no, it's a pleasure. It started last night. So the Christmas holiday has now started. I'm off for two weeks. It should be lovely. Yeah. Yeah. Yeah.

It should be very lovely. So you've got, I, I think everyone on Twitter is jealous of all the pictures that you post when it's like cold and gray and you're like, oh yeah, there's this beautiful sunny beach here in the summer in Australia that I happen to be suffering through. Yeah, it is. Yes. Middle of summer here and it's, um, yeah, beautiful weather and five minute walk to the beach. So yeah. Yeah. Yeah. Yeah. Not, not too bad. Not too bad. Cool. Well, welcome to the show. Happy to have you here. I want to kick us off with this project called Django Ledger and Django Ledger is, have you guys heard of this? No.

Yeah. So Django Ledger, uh, you've probably heard of QuickBooks or FreshBooks or some of this accounting software, right? That you've got to work with. It allows you to do either, it's a desktop app or some kind of online thing where you create purchase orders, you keep track of who your customers are, when they owe you money, you accept payments, all that kind of stuff, right? So Django Ledger is something like that built in Django, which I think is really cool because it could serve multiple purposes. One, you could take it and just run it for yourself or for your company and then customize

it. Or you could actually use it to extend something, you know, build something on top of it, or maybe even offer services in it, right? If you're like Stripe, maybe it makes sense to integrate some sort of plugin here because then you get 3% of everything that company makes basically, the way credit cards go, right? So it's a bookkeeping and financial analysis engine for the Django framework, which is pretty sweet. It's open source. And if you look through its features, it has a chart of accounts and basically customers, financial statements, it has multi-tenancy support, it has stuff for operations, for investing, it has bills and invoices and bank accounts, all that kind of stuff.

And yeah, pretty awesome, right? Wait, multi-tenancy? So is there like North Tennessee and South Tennessee or? That would be Dakota. I think Tennessee, there's only just one tenant no yeah i think if you wanted to run this as like a platform as a service type of thing and you wanted to offer up if you wanted to basically create your version of fresh books i feel like that's what it would be if you're if you're an accountant um that serviced multiple clients you probably did that way i guess it looks pretty cool i thought

i'd add, there's a project called Ledger, which is open source, but it's not a, I don't think it's a Python project. I think it's written in something completely different. So yeah, this is really cool. And great, I think,

for businesses who maybe have someone who can set this up. I'm guessing it's not available as a service. You have to kind of... I don't think so. Especially, yeah, yeah. If you look at it, it says this project is under active development. It's not quite ready for production.

So this comes to us from Miguel Sanda. I believe that's his project. And I would love to see what the roadmap for stability is and when this is ready, when it's coming out. But it's basically all Python and HTML and just a tiny bit of TypeScript, like a salt level. Stuff that's coming is like inventory management, cashflow, taxes, all the fun stuff.

And Brian, even BDD is coming. I don't understand. Behavior-driven development tests. Yeah, yeah, yeah. And so they're also actively looking for contributors, especially anyone with financial or accounting experience. So if you're looking for some project to contribute to, you know, that'd be great. You could contribute to this one. It's not super well known yet, but I thought I would shine a little bit of a light on it because it seems like it's a cool

idea. Well, it's even got invoicing and stuff too. So like, I think this is pretty neat. Yeah. Yeah, absolutely absolutely so what's up next for you brian what you got for us uh next up oh um so another web sort of thing but uh uh this i ran across flask meld so flask and meld i've never heard of this so it's just like um is like a vulcan thing from star trek no the idea is like like melding the front end and the back end. Okay. And the, it's a pretty cool, there's a, you should click on the example article. But yeah, that, that first link. Right there. Oh, right here. Okay. Yeah. There's a, it's basically, it's fairly simple, interactive stuff where, where that flat, you know, JavaScript gives you, but it's it's super fast.

And, and it just like, you don't have to write the JavaScript code. So the, the article in this article talks about how this is from Michael Abraham, I think it's Abrahamson, but he, he wanted to avoid writing JavaScript. So he wrote a whole bunch of JavaScript to, uh, to make this plugin. So it's a flask, his first flask extension, but you, um, you kind of modify the, the flask templates to insert these elements. And then the elements just show up as like objects in, uh, in Python.

Um, and you can program them like that. So all these, all these elements are all implemented in Python. Uh, and the can program them like that. So all these, all these elements are all implemented in Python. Uh, and the, the, there's a little video on there, but there's a demo as well. So there's a, there's a working example and, uh, and then the, the code for the examples up on, on GitHub as well. So you can play with it. One of the, one of the fun parts, um, uh, in one of his working examples is the drop down for like a search. So if you start like predictive search sort of thing, so if you start, the example shows states.

So if you start typing states, it'll like start filling in this list of states that you might mean. And it's like super fast. Nice. Yeah, apparently it says right here that it utilizes WebSockets and Morphdom to create server-side rendered HTML and swap out DOM elements without refreshing the page. That's the killer, actually.

Yeah, and he's nice enough, I think this is cool, he's saying that he basically stole all the ideas from other people and put them together for this. Or standing on the shoulders of giants, so to speak. And there's a similar project called django unicorn for django people i played with that a little bit the um there's a there's the django unicorn is the the documentation site's amazing it's got a bunch of examples um but it it seemed i don't know if it's the the server that it's hosted on or what but the django example seemed a little bit slower. It seemed like there was this round-trip thing going on,

whereas the Flask example, Flask meld was zippy enough that that would be completely sufficient for a lot of the applications. I'm thinking of throwing in some interactive stuff. Yeah, this is really neat. This idea of components is pretty cool here. Anthony, what do you think? Yeah, it's interesting.

I don't really use Flask, but in the Django world, quite a lot of components and stuff that I've pulled in and plugins to do interactive elements. And each one is implemented completely differently, which is really frustrating, whether it's like a search box or a type ahead or like a multi-select field, just something that i need which is um not available in the standard forms model and each one tends to have its own

javascript and its own collection of bugs um and yeah it is quite frustrating to be honest because you kind of think oh i don't need to write this from scratch in javascript i'll use an extension that already exists out there and then you kind of get stuck in does it work in django 3 you know what state is the javascript in and then you read through the source code and you stumble across security issues and it's like okay i wasn't you know there's a there's a line between it's easier to write it myself and you end up i end up just maintaining forks of all these extensions and then trying to get pull requests back into them to kind of fix things up but um yeah it's interesting to see i

guess a different model for doing it uh which which you know could be applied to to django so this thing uh brings me back to like the year 2001 web with asp.net web forms which are like such a weird way to build web apps. They kind of like tried to mirror desktop apps, but you could put a little Ajax tag onto like a section and just that part of the page would just become like, it would automatically refresh and interact differently. This gives me like kind of that feel, but like not old school web, but more modern web, which is nice. Yeah. I mean, like I was thinking of it and now a lot of people that are comfortable with like jumping into some javascript this seems silly i'm sure but for

instance like me i never touch javascript so if i've got a little and i've got a like a little flask app that pulls up um like test result data and it'd be great to just have a be able to get a little form there that says hey here's here's the version i want to see the results for and be able to get a little form there that says, hey, here's the version I want to see the results for and be able to pull that up. And it doesn't have to be pretty, but this way I could implement it without having to go in and learn JavaScript. So I like it. Yeah, yeah.

And I think this is super neat. I'm with you, Anthony, on the trade-off of grabbing some of these cool plugins, Flask, Django, whatever, and go, okay, this is just now adding functionality. But then you've kind of got to understand its assumptions. When does it work?

Like, why does it not quite work for what you're doing? It's always a trade-off. I usually go for the vanilla version of the web and just build it myself until it's like really clear that there's a big benefit. Yeah, and you bring up an interesting point

with the security concerns, especially anytime you've got dealing with input fields and stuff. You've got to be careful with that. Yeah, because they normally run queries in a database. So you've got to kind of audit these things to make sure that they're not using raw queries and that they're not using Word templates and stuff like that. Right, they better be parameterized queries and not little Bobby Table type queries queries exactly yeah yeah all right uh wrong one this one so what about what's next um maybe they're using even bitwise operators in there anthony possibly yeah um so my next uh

link is bitwise operators in python uh by batash zachnisky um and this one's really cool actually Operators in Python by Bartosz Zacznicki. And this one's really cool. I think if you've ever used bitwise operators or seen them in the Python language. So this is where you'd use the pipe symbol, the less than, less than, which kind of looks like two arrows or right, right, right, which is greater than, greater than. There's a XOR is is there another way to

say x or i just use x or oh yeah that's what i say as well you didn't say gazar or something no um which is the the hat sign uh i can't what that's that symbol is called the tilde is for not um so yeah these are basically used for specific types in Python, which support bitwise operators. I like this article because they're rarely used in Python because you typically use types where you'd use a method to do a lot of these things. Most of the time, you wouldn't necessarily need to work with data which is binary. So you wouldn't necessarily need to do these things but if you are working with binary data they're super useful um and it takes a bit of time to get your head around so i've got a couple of examples but um yeah this i love this tutorial because actually

it doesn't assume you know anything about this topic uh explains what the binary system is and it uses these icons of uh hands like it uses emojis kind of like a mini sign language to explain the the the kind of the bytes and stuff like that um which is really cool um see i really like it for that reason it's really illustrative um and kind of takes you through a few concepts. And then how you can use these special operators for some of the built-in types, Python integers, bit strings. You can also use them for byte arrays, which is really helpful. And then it goes into things like bitmasks and stuff like that.

So if you're ever working with any lower level data, this is super helpful to understand. And also a little trick that i discovered a while ago was um if you've ever used the enum um built-ins so enum is a is a in the standard library and if you want to describe an enum you basically create a class and inherit from an enum type which is in the enum uh package um and you can do a whole bunch of things and just represent things as enums in python uh instead of other weird ways of doing it i don't know introduced in python 3.7 i think it was um but there's a little known uh feature in um in the enums which is there's an int flag and a flag type. An int flag

is an enum, which you can represent as a flag, which means you can combine multiple of them. So for example, if you had a color enum and you had red, blue, and green, you could represent white as being red, blue, and green by combining the colors together. Nice. Wow, that's cool. Yeah, you probably wouldn't make a paint mixer in Python, but there's lots of things. Flags, for example, like if you're representing system flags or compiler flags or any other kind of flags,

this is really useful because it actually implements all the binary operators. So yeah, flag and int flag are super helpful types, which are built into the standard library. So if you ever find yourself doing something like this or using a package, which has implemented its own weird version of this feature, which is built in and you can upgrade it to this new syntax and then you're done so yeah nice tell me what does this auto do oh uh so uh in enums um if you're going to say uh you know you need to make a representation of the enum name um and if it's an int enum or an int flag uh you'd say you know

one two three four five like so if i save it to a file or pickle it or something then it's this is the number that it represents so i can convert so if you converted the number one to your enum it would know that red was one so you know you could basically use it for storage or something it's also really helpful like if you're reading from xml basically use it for storage or something. It's also really helpful, like, if you're reading from XML files or JSON files or something, and there's, like, there's a field which is stored as an integer, but you know that it actually represents something a bit more logical, like enumeration, then you can do it that way.

However, for flags, then because they support bitwise operators, they should follow the binary sequence. So 1, 2, 4, 8, 16, 32, 64, et cetera. And once you get past 1,024, most people start to forget. So instead of working that all out in your head, you can just use auto, which is a function built into the enum, and it will basically just work out what that value should be for you so you don't have to work it out in your head it'll let

you like reorder stuff and not have to go oh now i want this one to go one two three and or i want to add another one but in the middle and it accidentally messed it up or whatever yeah i think i might use those anyway just to to indicate that the actual number isn't important. It's just that they're unique. Yeah, exactly. I think if you don't assign it, then in enums, if you don't assign it, then it works. But with flags, you need to assign it to something. Yeah.

Yeah. Just for people listening who don't necessarily see the code, the show notes, the ideas you create in a num class and you say like category equals just lowercase auto open close that's the way that you sort of invoke this behavior also a couple questions from the listeners uh who are in the live stream anthony lister says uh useful for micro python or hardware maybe what do you think yeah absolutely if you i mean if you're working with um like embedded systems or anything i mean brian you're the expert on this but uh yeah you have to work with binary a lot so um yeah this is super helpful because you can represent stuff, which is different states in the system.

Or if you want to read inputs and stuff from multiple channels, you're going to need to use bitwise operators. So both the bitwise operators are really helpful, as well as the flags enums. Yeah, and Brandon Rainer says, bitwise always confuses me. When I see it, I tend to roll my eyes back

and ask why. I'm sure there's a reason. But, well, I mean, one of the things that comes to mind for me is, you know, you're thinking about storing stuff in memory. You know, if I wanted to store,

say, a number or something like that, or, you know, you put that into a Python number, that's like 28 bytes. But if you create an array of bytes and you know the size of them are going to be, you know, packed into little into little bits there you know like one to ten or something

a whole bunch you could be way more efficient by you know creating little smaller containers and then bitwise boring them together and whatnot yeah like bitfields for instance are a really cool thing but even just just straight numbers with bitwise operators are are important for hardware because you're like a lot of times you just have register access to something or you have memory, memory mapped registers, and you, you know, just kind of read those out. And there may be a whole bunch of data, so each bit might represent completely wildly different things. So you can't really just check for equality. You have to check, is this bit on or off? Or I need to set this bit and leave all the rest alone,

things like that. Yeah, yeah, absolutely. Absolutely. All right. Before we get to our next item, let me just let you know, this episode is brought to you by us, things that Brian and I are doing. And so one of the things we just launched over at Talk By Thine Training is our brand new fast API course, which is a super cool new API framework that I think brings together a lot of the things that we've been proponents of, you know, things like Pydantic, TypeHints, Async, and Await, all those, they all come together really nice over there. So if you want to learn that, check that out. Brian has some book on PyTest. So yeah, you can check that as well. Links in the show notes. The next thing I want to talk about, though, is why you should use an ORM.

Here we go. So, Anthony, Brian, what do you guys think? Raw SQL ORMs, what are your thoughts here? Well, I've been using like document databases lately and I don't really need an ORM. So I'm with you. So over there, maybe the R is a D, at least if it's a document database, it's an object document mapper, maybe. But yeah, I find

being able to work with classes, like the way I think of it in Python to be, that's how I want my data to be, and just something else can figure out how the database has to break that apart into relationships and stuff. Super neat, you know. Like SQL Alchemy, go and say create, like if I had a user and the user had orders,

like they might have an orders list on the user class to create a new order, you can just go and the user had orders, they might have an orders list on the user class. To create a new order, you can just go to the user.orders.append that new thing and hit save, commit changes, and then boom. It figures out that has to be inserted and the relationship has to be set and all that. Anthony, what do you think?

I use the Django ORM quite a lot and really like it. Learning all the edge cases where it creates queries which are not super efficient but um so i guess there's pros and cons like if you were to write raw sql you know sometimes if you know sql really well then you can write more efficient queries however there are typically ways around that um the n plus one problem in jango the n plus one is the biggest problem for sure um you want to describe the n plus one problem in jango the n plus one is the biggest problem for sure um you want to describe the n plus one problem or take a shot at that oh yeah so if you um sort of have a

foreign key and you reference an entity which might be a many-to-many relationship um and then you reference a property of it in the query um not actually not in the query but actually in the view you you um you mention something or you look up a field which is part of a mapping to another table kind of like i described like if you gave a user to the view and the the view wanted to know about its orders yeah so more part of you gave a list of users and it wanted to know about the orders for each of them right that's the real bad case yeah so you had it for like um you know which group people are in or like which team they're in or something.

And it's just, it's the team ID. And then you actually want to show it on the table as the team name. So what ends up happening is that for every row in the table, it does another query to look up the team name, even though they're the same across the board. Whereas if you're writing that in SQL, I'd hope you'd do that as an outer join. But yeah, so an inner, sorry, not an outer join. So yeah, OOMs are really useful in that sort of thing,

but they do have edge cases. For N plus ones, pretty sure you covered this one before, but there is an awesome tool called N plus one, which you can run in your test suite. So when you do all your integration tests with Django, it fails the test if it detects an N plus one query, which I use quite extensively. I think Adam.

Oh, I had not heard of this. I mean, this is not such a problem for me because I work like Brian in document databases, but to the extent that I work with like SQL Alchemy and stuff, like this is really this is the one you're talking about the one yeah yeah that's it yeah t-l-u-s-o-n-e yeah that one yeah so i add it i only add it in the test suite uh so you don't you don't really need to put this into production uh so you just you're assuming that you have tests anthony yeah well that's

probably a starting point, isn't it? Or if you're developing an environment, you can just turn it on. But it would, yeah, basically, it would print a warning and stuff like that. However, there is a config option that says that it can raise an exception if it detects one, which is awesome because when you run all your tests,

it will fire off and fail the test if it detects this type of query. And the workaround in Django is actually really simple. You just add another function to the chain, the query command, basically, which indicates that I'm going to use this field in this other table, so it kind of pre-looks up. Right, right.

Please do the join or sub-query load or whatever it needs to do. Yeah. Yeah. And the other thing about rms that i really like is the is the migration ability like it's fine when you initially design a system you could yeah fine you can write your own custom sql queries and stuff but you know within a week or two you've added like five extra fields and actually that one field you added now needs to be a different type or you know databases are not static things like database structures change all the time um in a real application so that becomes an absolute nightmare if you've handcrafted all your sql um especially

if you don't have tests um so yeah i mean for different database engines there's different ways to do this um i mean sql server like has that uh dax so you can do dac packs and dac pack migrations but most of the other database systems don't have that um whereas if you're using the rm then um you know django and flask and stuff would create the migration for you um and you're kind of pretty sure that you can add fields as an as you wish uh without necessarily breaking things or you can add types and you can describe what to do with the old ones um so but i think it's saved so much time in just working with a live system where you're actually constantly making modifications to the to the table structures

and stuff and maybe you have different developers at different stages or you want to go back in the brand. You want to say, I'm going to go back here and then work on this version of the app at this stage. You want to be able to go to just run the migrations and get to the right structure. Yeah, I find it to be super, super helpful. I mean, there are times where, you know,

you want to get 100,000 records back from the database and an ORM doesn't make sense. To me, I feel like you should 80-90% of the time use an ORM and then there's that little edge case where something slightly different maybe needs to happen, but you shouldn't start there because there's that 10% or 5% chance.

Most ORMs give you the ability to write a raw query. You should also check those raw queries for SQL injection. And I do have a tool for that. If you want to, you can run it through the Python security thing that I've built,

which will scan your code and spot where you've used the raw SQL functions in the Flask ORM and the Django ORM. And it will see if there's anything in them which is likely to leave you subject to SQL injection. Yeah, is that built into your PyCharm security plugin? Yes, yeah, it is. Yeah, I want to just mention this conversation,

although not exactly, we didn't pull that much out of it, but it comes from an article I wanted to highlight called Why Should You Use an ORM? Object Relational Mapper, written by Kareem? I'll leave that in the show notes. There's a bunch of details that walks you through it. So I wanted to cover the cover this. So people,

if they're having this debate on their team or they're not sure which way to go, or they're, they're new and they're like, what are all these funky acronyms about? You know, they can check that out. There's, there's the obligatory little Bobby tables joke in there, right? You gotta, gotta go with that. This is what your plugin is supposed to detect is this cool XKCD. Cool. You're not on the receiving end.

One of the things I'd like to bring up around ORMs is that like a lot of the ORM tutorials assume that you know SQL. And I'd like to see more tutorials on how to use an ORM and how to use it correctly without any SQL examples. Yeah. Yeah. You know classes in Python. Yeah. Here's your way.

You don't have to know SQL. I mean, I thought that was the point is that you can think in objects instead of in SQL. So why do I got to learn SQL in order to understand the tool? Sure. Why do I got to learn SQL in order to understand the tool? Sure, sure. Another thing that you can turn on that's helpful for the N plus one problem, at least with SQL Alchemy, you can go to the engine when you create it and you can say echo. Echo equals true. And it'll echo every single underlying SQL command sent to the database. You can do something in SQL Alchemy and it'll say, here's what the actual thing is and if you've written your code well and you've done the right join stuff you'll just see like a couple of entries for each page or interaction you're having you've done it wrong you'll see your your output just scream by

full of these things like oh there's one of these problems so that's that's a easy way as well is there like a test way to like like for instance um to check to see how many for a certain test sequence how many uh database interactions you've done yeah there is yeah um there is in there is in um django you can kind of hack a bit of middleware which um kind of catches sql queries and stuff like that um it's not built in but you can write it i think i've got code sample somewhere that does that um and just say how many queries this page executed obviously it depends on that's why it's important to see the database first with test data because you know if you just run it on an empty database then typically or you've got like one row or something then it's it's going to be like yeah it's fine

but then when you deploy it to production like it's actually running thousands of queries for every page yeah did you end up with what you got in the United States when we tried to roll out the healthcare.gov and the entire system went down? And yeah. Like, yeah, whatever. They knew it was going to be the entire country using it and they didn't load test it?

Exactly. Exactly. One more thing, a comment from Matej. It's also useful when using Django to add the Django debug toolbar or Django Silk to be aware of what queries are going on. And also I would add for Pyramid, there's a Pyramid debug toolbar. And you can actually see the underlying queries and how many of them there are and the timing. It's really nice, those things. All right.

All right. Well, let's move on to the next one. Sticking with the database stuff, Brian. Definitely sticking with databases and SQL. So this one's from Simon Wilson and he's got a tool called, or a library called SQLite Utils. It's developed as part of the Datas set project but it's um it's usable by anybody that uses uh sqlite sqlite and uh it's a couple things it's a command line utility which um a lot of databases do have a command line way to enter to query the database but i don't know if sqlite does um but this is a pretty cool command line you can interrogate analyze

tables and dump things and do all sorts of stuff um and even search it's pretty useful in pretty simple examples um the one of the things i really like though is the um the api that he has so there's a python api to i mean you can can use a like SQL alchemy, for instance, to interact with SQLite. But another way to do it is to use the SQLite utils as a, as an API for, for SQLite. And it's a pretty clean, simple interaction. And I think that's, it's something I'm on. I want to try because it looks like a... I do need to get back into some SQL database work, and I think this would help a lot. Yeah.

This comes from the Dataset... It's one of the foundational tools from Dataset, this project that he's working on, which is really interesting. It's like empowering data-driven journalism and data exploration across all these different data sources by converting them all down to SQLite databases. And then once they're there, you can explore them in interesting ways. So there's like Twitter to SQLite, various other things to SQLite, Gmail to SQLite. And then

you can like explore all these different things like GeoJSON to SQLite. What else have we got here? There's just tons and tons of these things that plug into other stuff and then gets in this common format. And then he also has this cool tool called DogSheep, which builds on top of those databases that creates like a personal search engine for your life. So you could like plug in Twitter,

you could plug in your iPhotos library, you could plug in your Gmail. And then there's a search engine that just says search that. So it's like one of the foundational building blocks of like how do i get everything from its original source into sqlite pretty cool yeah yeah so i need anthony what do you think uh i'm actually really interested about the next um link on his blog which says i commissioned an oil painting of barbara streisand's cloned dogs it's really nice actually i kind of like it for gazing the tombstone

of the dog that they are yeah okay interesting yeah i'm not sure what it has to do with sequel like but now it's pretty cool i think um yeah more tools for and journalists and stuff as they start to work with data and data becomes more readily available or at least there's more of it so um yeah and people aren't necessarily yeah and don't necessarily have the technical skills to work with massive data sets and stuff like that so yeah it'd be really interesting to um yeah use some some of these tools for sure uh simon willison was one of the co-founders of dango i had had him on TalkPython recently, but it's not published yet. And we talked about Dataset and DogSheep. And there's just a lot of interesting stuff.

And one of the stories that he tells, so people can listen to the episode, but he has a dog, which may be this commissioned thing. He's like really got this dog that has a Twitter. The dog has its own Twitter account. So for example, some of the interesting stuff that he did was his dog in the Twitter account will tweet when it goes to the veterinarian, like how much it weighs and stuff. And then it will tweet pictures when it's on a walk. And he was able to do things like we create a graph over time of his dog's weight by just doing a SQL query against the dog's Twitter account that got pulled

in. Like all sorts of weird connections of like pulling data together that you just couldn't imagine so uh yeah anyway really really neat stuff there okay yeah all right people would check that out um and then you want to wrap it up with a a talk that is very uh work from home ish uh conference is very work from home i should talk from there maybe yeah yeah so um my next one is from the pyjamas conference um which happened a couple of weeks ago um and this is yeah lots of online conferences happening at the moment um which i kind of sign up for and look at the you know the talk list and think oh great that'll be really interesting and then never actually get around to watching any of them exactly um yeah i'm just kind of struggling with this online conferencing thing i've like the ideas are really cool the concept is cool but the practicalities of it um yeah especially like i don't know if it's different in our house but um the the conference ran over a weekend and i

just wanted you know maybe an hour or so to watch some of the talks and things but just sitting down on the couch and turning the tv on putting it on youtube and then sitting and watching talks and it was like the kids isn't making so much noise my wife's like why are you watching tv there's loads of stuff to do and i'm like no but i'm this isn't and then is this work it actually is work i always sure it is yeah yeah sure looks like work yeah um so yeah i see pajamas there's pajamas up there this doesn't make this doesn't feel like work yeah and i definitely wasn't in my pajamas i had to get dressed because it was 10 in the morning um so yeah i don't know i know i'm kind of talked about this and a couple of other people said they're having similar challenges with these

online conferences just like where do you fit them in um yeah and if you actually go to a conference it's like you've got you've got an excuse that this is more this is like my time is now set out whereas if i if i was to say i'm going away for a few days to a conference that's fine but if i try to attend that conference at home it's like that's not yeah yeah that's completely i'm gonna lock myself in my room and watch tv for two days i'll be back yeah yeah exactly it's not the same story yeah someone at work actually suggested that we have like a a work hotel sort of thing where people can attend virtual conferences um where you know especially if you're on if if they're late at night or they're in weird time zones and stuff so even if it's just in your local city you can go and stay there

so you can at least dedicate the time. Anyway, the Pyjamas, which I really wanted to attend, I managed to get through half a talk. And then I watched the other half like a week later. So that was not very successful. However, the talk was brilliant. So I wanted to share a link. It was called What Dest struct um by zachary

anglin and it's talking about the uh the struct library which is built into the standard library i didn't really realize there was a struct library right like there is an array library that says this will hold floats contiguously same thing for structs right yeah exactly and i think um you know brian was talking about data classes before we went live actually but um you know data classes are really cool like if you wanted to represent uh interesting structures and stuff like that um orms are really important for this sort of thing but if you're working with binary data um reading the binary data and then converting into native python types um often know, you do C for that sort of thing.

And in C, you just declare a struct and just say there's these fields and you can just read and write that into binary structures, which is kind of built in. But sometimes you need to do that in Python if you're working with... Actually, I've got some examples. So this explains what the struct library is. And it also has this macro language for describing what the underlying type is. Um, and then you can kind of like pack and unpack it basically.

So you can say, here's my, uh, here's my class or whatever. And here's the fields and the underlying binary structure is a, um, you know, a float, a 32 bit integer or binary, um, Boolean or, you know, it may be an ASCII string or something. So, yeah, there's basically all these, like, little characters for packing and unpacking data. So, yeah, it is really helpful. If you're getting into this topic of working with binary data structures or it's something that you've wanted to do,

I recommend if you're on a mac using a tool called signalize it um or spy synalysis um which is basically a gui for working with binary data and it has um a database of grammars um so you can basically like open a compiled executable or something like that. And it would be like, oh, I know what this format is. And it would then just represent it in actually something more understandable. And you can edit fields and hack around with stuff. It's great if you're doing capture the flags

or if you're doing some hacking for good. It's also really cool if you want to edit files, which are not in a human readable format, if they're in a binary format and in the database, they've got a whole bunch of examples, including a lot of save game formats. So this is actually how I got into this topic. Originally it was when I was a lot younger playing games and stuff.

And I'm like, what is this file and then trying to echo it on the shell and it would be like oh that's all gobbled and then realizing it's all in binary and then trying to figure out if you open it up in a hex editor or something what it is and then oh if I change this

field then I can give myself more gold or I can make my character invincible or so basically like hacking the game by modifying the boss would be so much easier if i had a hundred thousand hit points or whatever so let's just increment that number yeah exactly i think modern games are a bit they kind of put protections around the save game because there's a lot more online aspect but definitely older games you can you can hack the save games and most of them give yourself whatever it is that you feel like um so yeah understanding binary structures is really helpful on the struct

um you can even automate it using the struct much yeah nice yeah there's also some good comparisons between like the values of structs and then tuples and name tuples and data classes and pydantic so there's sort of a spectrum that gets covered there as well yeah interesting goes in line well with the uh the bit manipulation topic you were talking about it's like for instance um uh very structured uh structs are um are very important for like message uh, like communication systems. We have very defined fields within each, within the, you know,

we've got 128 bit sequence or a thousand bits in there. Each bit is specifically assigned to different things. And those are, yeah, you definitely don't want to just like hope that it's right. You have to put it exactly where you need it. Yeah. Yeah, exactly. Like little network packet headers where you need it. Yeah, exactly.

Like little network packet headers and things like that. Yeah, I'm using this at the moment because I'm writing a disassembler. It's quite helpful. Yeah, awesome. Yeah, you're definitely doing some low-level stuff these days. So I guess that's our main items for today. Brian, you got some extras you want to share?

Yeah, we got poked by the Python Software Foundation that we should probably plan some events because we've got a meetup, the Python PDX West meetup, but I can't access the building that we normally held it in and nobody would come anyway, right? So we're doing virtual. So we've got, uh, January 14th, we're going to do, uh, we're going to start seeing what it's like to do a virtual thing. And I thought, you know, normally when we did the meetups before

it was in the evening, uh, I don't really want to hang out in the, in the evening doing this, but at lunchtime it'd be fine. So I thought like a lunch, a lunch and learn thing. So we're going to try that. Yeah, it's perfect. Being virtual, you could just do it during lunch, right? You don't have to go, okay, well, I'm going to drive out, you know, fight the traffic or whatever, right? Yeah, and I don't have to convince my boss to buy a bunch of pizzas. Yeah, exactly, exactly. All right, I got a couple of extras as well.

I ended up doing a fast API webcast with the folks over at JetBrains, so people can check that out. They want to go learn more about that. And then also, let me close my notes here, but I'll get them back. Brian, you go next. I got to pull this up somewhere else. I just went. So let's-

Sorry. Anthony, what are you up to these days? What extra you got to share with folks? Yeah. So I'm going to be starting at Microsoft in February, which is big news for me. Yeah, congratulations. That's awesome.

Yeah, yeah, I'm really excited about it. So yeah, I'm going to be the Python developer advocate working with Nina Zakarenko. So Nina's the current Python developer advocate, and I'm going to be, yeah, helping out. So yeah, really looking forward to it and working with a bunch of smart people over at microsoft so yeah all things python um do you know any particular projects you're working on so i'm not going to be i'm not going to be i'm not on the engineering team um and i'm definitely not working on pigeon this has got nothing to do

with pigeon um contrary to the rumor, I guess. You were hired to work on Pigeon. That's what I heard. Come on. Yeah, definitely not. So yeah, this is kind of a side thing. But yeah, it's kind of definitely the Azure platform and using Python and Python working with Azure and on Azure. VS Code and the Python extension

and just the tooling in and around that, as well as the Python ecosystem with Microsoft products in general or Microsoft technologies and Python in general. So all things Python. And yeah, really looking forward to starting on the 1st of Feb. And yeah, I have a long list of things

that I want to get stuck into. Yeah, I'm guessing that's remote, not just now, but permanently, right? Yes. Yeah, that's super exciting. But that does mean, Brian, that he'll be somewhat in our neighborhood more often. I suspect he'll come up to Seattle and Redmond sometimes

when that happens again, when the world is let loose. Yeah, I expect so. We're not even again, when the world is let loose. Yeah. I expect so when we're not even allowed to leave the country at the moment. Yeah. Well, we'll see you in the Northwest once in a while. Yeah.

Yeah. When, when maybe 2022. Yeah. What? 2030. We'll definitely see you 2030. No problem.

Also from, from the YouTube stream, Piling says, congrats Anthony on the new job. piling yeah yeah so we should have like a like a web page with like a graphic to show like how many like well-known python people there are and the like maybe a death star microsoft sucking all the python people into mic. Yeah. They're pretty evenly distributed. Yeah. No, it's all good.

All right. Well, I feel like it might be time to wrap this up with a joke, you think? Sure. Yeah. So this one came to us over Twitter from Lars. And there was apparently a question. Someone named Kate Maddox asks,

what's the best programming language for coding your own therapist? You know, like we're all stuck at home. People are depressed, have social anxiety or just anxiety about the world in general. And Wes comes along with an appropriate answer. What do you guys think here? Python, of course, is the language. So you can call it therapi, E-H-E-R-A dot P-Y.

Yeah. I think it's pretty much on par for our standard jokes though. So you can call it TheraPy, E-H-E-R-A dot P-Y. Yeah. I think it's pretty much on par for our standard jokes, though. Yeah. Might be better than average, actually. Yeah. Yeah, it could be better than average. You need a little drum kit in the background, Brian, that you can. Yeah.

Some cymbals. We'll set you up with that. It's a business expense. I'll totally get a drum kit. Yeah. Fantastic. All right. Well,

Brian, thanks as always. And Anthony, thanks for joining us. Thank you. Yeah, it was a pleasure. Yeah,

you bet. And everyone listening, see you next time. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're

always on the lookout for sharing something cool. On behalf of myself and Brian Auchin, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.