Python Bytes - #245 Fire up your Python time machine (and test some code)

Episode Date: August 4, 2021

Topics covered in this episode: State of the community (via Jet Brains) Cornell - record & replay mock server pyinstrument Python 3.10 is now in Release Candidate phase. RC1 just released. Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/245

Hey there, thanks for listening. Before we jump into this episode, I just want to remind you that this episode is brought to you by us over at TalkPython Training and Brian through his PyTest book. So if you want to get hands-on and learn something with Python, be sure to consider our courses over at TalkPython Training. Visit them via pythonbytes.fm slash courses. And if you're looking to do testing and get better with PyTest, check out Brian's book at pythonbytes.fm slash PyTest. Enjoy the episode. Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 245, so it's not the first time. Recorded August 4th, 2021. I'm Brian Ocken.

I'm Michael Kennedy. I'm Wompe. So Wompe, thanks so much for coming on the show. Can you introduce yourself a little bit before we get into it? Thank you very much for having me. So my name is Juanpe. I'm from Spain and I did my PhD in particle physics. I've been working at CERN for four years.

Then two years after my PhD finished, I decided to step away from academia and start working in industry. And right now I'm working at FinancialForce, which is a company that develops products for Salesforce. So I'm not developing products, I'm in the analytics team. So my job is to analyze internal data as well as product usage from a customer to allow the board to take data-driven decisions on how the company should go forward. Nice. Yeah. Super interesting. Give us your thoughts real quick on one hand working for a place like CERN and then the other working on a place that like provides like enhancements to Salesforce. Like those sounds so different. Are they really that different or are they

similar or what's the story? Part, I mean, of course they're different, but there is a big part, which is very much the same, at least in the team that I'm working on. Because at CERN, basically what you do is you don't know the answer to anything and you need to first know what you need to ask yourself. And this is very similar to what happens today in my current job. is the marketing come and say, we have this whatever campaign and we want to know if we're targeting right. And I need to know what do I need to do to answer that question, but neither marketing knows. So it's like, let's figure things out. But yeah, I mean, it's a pretty drastic change, but I don't know. I got the feeling that I needed to switch. I like coding a lot. And I felt at some point that I was enjoying more the coding part of being a physicist than the physics part.

So I said, I mean, you basically described why I'm not in math anymore. Yeah. I was working on projects and I was having so much fun and writing code on these silicon graphics, like huge computers and stuff. And then there'd be parts where I'd be like, ah, this part's not so fun. This part's amazing. And I realized the programming parts were amazing. And when it had to get down to solving the math bits, I'm like, ah, darn, I gotta put

a video away, go work on the math again. I mean, I remember the last year and a half I was working on a project that was literally designing a system that worked within GitLab CI to automate paper review publishing. So you don't need to have a lot of people reading the paper and say, oh, this rule is not matched or you need to fix this image. So I built an entire pipeline in Python to check all of this that works based on pull requests and groups and so on in GitLab CI. And I thought, I've been a year and a half not doing almost any

physics. So my physics work, it was related to review paper because I was the chair of an editorial board. So I had an analysis. It was pretty cool, but I wasn't doing it. I received version, reviewed them, made comments, fixed this, fixed that, then go back to write a pipeline to make the pipeline.

So it was... Yeah, exactly. Yeah, that sounds really cool. Yeah, go ahead. Will you kick us off today? Will. You want to hear about the state of the developer world? How's that sound? I like state.

Yeah, yeah. So here's an interesting survey results, I guess, put together by JetBrains, the state of the developer ecosystem 2021. And I thought this would be fun to talk about because we do cover like the PSF state of Python survey and things like that. But I thought it'd be fun to just have a quick look at the broader landscape, what people are doing and where Python fits into that.

And, you know, JetBrains has done a really good job with a PSF survey. So I thought, you know, this, this will be similar. So let's check that out. So let me give you some stats and some rundown on this. So basically, the idea is it presents the results of the fifth annual developer ecosystem survey conducted by JetBrains, and it went out to 31,000 or had input from 31,000, 32,000 people. All right, so there's a couple of interesting things they're doing in the presentation here. But let but say in that world, JavaScript is still the most popular language. Not if you ask Stack Overflow, but of those 32,000 people or whatever, Python's more popular than Java overall. However, Java is used more as the main language. So there's more people using Python for extra things or for other things and so on, which I think that jives pretty well with my understanding of Python is that it's this really amazing way to like

bring in a little interactivity, bring in a little bit of analysis, a little Jupyter notebook or something, even if it's not your, your main focus, right? You might be an economist, you are not even a programmer and, but you're still a Python person in a sense, whereas you probably wouldn't be a Java person as an economist most of the time. Yeah. I use Python for testing. Yeah, for sure. So let's see. The top five languages that developers are planning to adopt are Go, Kotlin, TypeScript, Python, and Rust. And the fastest growing languages are Python, TypeScript, Kotlin, SQL, and Go. So for example, JavaScript was the most popular language. Java was the most popular main language,

but they're neither the fastest growing languages. So that's pretty interesting. Of this group, 71% people work on some sort of web backend, APIs, Flask apps, or Go apps, or whatever. So they have a lot of interesting stuff here in terms of analysis so they have these blocks that show how popular a language is it's been used or how likely people are to adopt it and pick it up so there's a bunch of grids if you go to the report and you can check them out and basically the the orange is the current state of the world and there's a darker, almost black,

that is like the derivative. Like how quickly is that world changing for the upswing, right? So for example, JavaScript has more orange squares, but it doesn't have as quick of a growth or a planned growth, I guess. So Python has one of the larger ones of those. So does TypeScript as well.

And those are interesting to look into. You can compare those against different things. You can also see the popularity of the language over time. Python's been going up and up and up, although this year is kind of plateaued in this report. So that's maybe something worth noting. There's obvious things going down like Objective-C. You'd be insane to work on Objective-C right now when Swift has replaced it, although that's going down as well. Let's see, there's a few more things. They have these really interesting graphs

that are both like grids, but also heat mapped. So you can, it'll let you answer questions like, okay, if I am currently a Swift developer, what is the likelihood that I'm going to adopt Python? 6%. But if I'm a Kotlin developer, that's 8% likelihood that I'm going to adopt. Oh no, I'm going the wrong way. If I'm a Kotlin developer, that's 8% likelihood that I'm going to adopt. Oh, no, I'm going the wrong way. If I'm a Kotlin developer, I'm 10% likely to adopt, to move

to Python and so on. So there's a lot of sort of like flow from one language to another. I haven't seen any analysis like this anywhere else. Have you? No, that's pretty interesting. My head's looking at correlation or something. What's the first row? I'm curious. I'm not planning on changing. So they are the most likely to change yeah they're just stuck yeah all right let's see uh also interesting operating systems people use for development 61 windows 47 linux 44 mac os which that's pretty high for mac os given its general uh popularity amongst like the computing world i think yeah i think linux is pretty high too uh this doesn't surprise me but

yeah exactly and they're one percent other uh who knows what that is also questions about people using the windows subsystem for linux and stuff like that um there's also if you're interested a similar heat map for like what type of software do you develop? So if you're trying to understand where, like if I develop this kind of software, what is the distribution for programming languages there, right? Like it's interesting to say Python is popular or JavaScript is popular, but if I'm an embedded system developer, is JavaScript still popular? I don't know. Probably not. Maybe, but maybe not. Right. Maybe C is like really popular. So there's a really cool thing called what types of software do you develop? There's

like a grid plus heat map plus intersection of language and type. So if I develop security software, there's a 9% chance that I would be doing Python versus a 6% chance of Java. On the other hand, if I do blockchain, how does that break down and so on? Let's see where is Python kind of notable. On utility little scripts, it's quite popular there. Database backends, pretty popular in that area. Let's see, another one that maybe is standout would be programming tools.

Actually, that's pretty interesting and so on. Yeah, what do you guys think of this? I think it's weird that there's 39% of the C++ developers are developing websites. What the heck? Yeah. Yeah, what are they doing back there?

Maybe, yeah, maybe the backend. Yeah, should we put in the middle? I don't know. But it's weird, yeah. Yeah, yeah, that is quite interesting. And then you get the standard business intelligence. It makes sense.

Yeah, the business intelligence one, that one Python is definitely crushing it there, right? It's like 30% versus 10, 15, 20% for the others. Yeah, I guess one more, there's all these different things you all can dive into. I guess one more area that might be worth interesting is they broke down like the type of coding and software activities you do based on gender.

So for example, if you're male like how likely are you to do straight programming versus testing versus user experience type stuff or versus female and um let's see so there were some takeaways says women are more likely than men to be involved in data analysis machine learning uh ui design and research but less likely to be in directly doing infrastructure development or DevOps. But I mean, I kind of had that sense as well, but just... I mean, my personal experience

is completely the opposite. So most of the DevOps people I work with are women. But I think it kind of makes sense. I mean, in the industry for what I'm seeing. But for me, it's completely the opposite interesting yeah so i'll leave this out here for people to go dive into and explore more i feel like i've gone probably over enough details there to give you all a sense but there are some interesting things to to be learned i think yeah definitely pretty cool yeah and and matt out there

in the live stream uh points out that that might be more than 100 i'm not sure which part you're talking about i do believe a lot of these had a multiple uh you could check multiple things like which languages am i willing to adopt well i might be adopting both sql and python in the next year something like that yeah i think a lot of people are perpetually going to start learning rust or go um but never for starting. That's true. It's always 12 months out. All right. All right. Cornell. So this was suggested by Yale Mintz.

And Michael, you know where Cornell comes from, apparently? I'm thinking Soundgarden. Some good 90s grunge. I mean. Okay. Maybe.

So Cornell is a record and replay mock server. And we're going to link to the tool, but also there's an introduction blog post about it. And it supposedly makes it really simple to record and replay features to perform end-to-end testing in an isolated test environment. So kind of the gist around it, there's a cool tool called VCRPy, which saves cassette files for, you send a request and you get replies back, and then you can save those request reply sessions and stuff. And this is bundling VCRPy with Flask to make a little server.

And it's actually really kind of a cool idea. The idea, one of the things around it is that you can do, you're not just mocking one service, you can just mock in any external service that you're dealing with. It'll, you know, do replays on that. And one of the benefits over rolling your own mocks or rolling your own test server or test service is um that you can

that you don't really have to think about designing the whole thing it just kind of replays everything yeah that is cool pretty looks pretty fun i haven't played with it yet but definitely want to hey speaking of play with it click on uh documentation i think it is on that page right there on the bottom left. Okay. Documentation. And then click on the documentation of that page. There you go. And so you have this kind of like a series of animated GIFs of scene in action. And I think that that's kind of cool, right? Like you can, it'll go along and say, oh yeah, here you're recording a bunch of API calls

and then the workflow of like how you create it. I just want to give a shout out to the animated GIFs for like, is this interesting to me? Let me just watch the GIFs instead of actually take the time to try to adopt it. It's simple, but it seems really effective. Juanpe, what do you think? It's a good idea. No, it's a really good idea. I mean, there are many projects that you think this might be cool to work with.

And then you start reading walls of text and halfway through, I don't know if it's interesting, but I mean, it's been half an hour. So having a bunch of GIFs, I got you. if it's interesting, but I mean, it's been half an hour. So having a bunch of give is, I got you, yeah. Yeah, for sure, for sure. Yeah, also, I just want a quick shout out to the live stream. German points out from his experience, the data analysis have more men than women, but it could be biased due to the tech sector

having more men in general. I do think that that's true. I think what they said is, if you're a woman, what are you more likely to be doing? If you're a man, what are you more likely to be doing? If you're a man, what are you more likely to be doing? And it's like, of that population, what areas do you work in?

Not that user experience has more men or women. I don't think it addresses that question. I think my thoughts here, there's a lot of women who end up in programming, not down the traditional computer science path. You know, they go into biology and then they're like, oh, I've actually learned a little Python and now I really like it and I work here and it's amazing. But, you know, they kind of get pulled in tangentially where there's a lot of guys that

like sign up for computer science and they just go through that path. And some of the areas that were called out are more likely to take the straight computer science path people rather than the, I got interested and I came in through like psychology or something else, um, where there would be more women. So I don't know, I would love to have more women in there, but I think that this is my, in broad, broadly speaking, but I think this is my, my thoughts about why maybe those different areas seem to attract people not so directly down the computer science path. Yeah. yeah. All right. Juanpe, you're up. Talk to us about the next thing you got here.

Sure. So I want to talk about Factory Boy. I think it's a very well-known library to basically mock objects when you're running tests. And both this and the next tool I'm going to talk about came because we were working on a system that replicates an entire Salesforce org. So we have an infrastructure we've built that takes everything

you have, every object you have in Salesforce and copy it to a database. This is a way we have to have daily snapshot of the data that we can do a time series and analysis and all the models that we have on it instead of being furious, let's say. When you modify, it's lost. So for this, we obviously need to communicate a lot with the API in Salesforce. And when you get API responses, you need to not only treat the JSON plainly, let's say, just the plain JSON object,

but you would like also to have some kind of object representation. And for this, I think it's not news for anyone. The PyDantic right now is taking the floor. And the biggest issue came when we needed to start writing tests for it because we get the JSON file, we stick it in the PyDantic object,

it validates everything, everything's beautiful and works fine. But then we have a bunch of fields on the object that cannot be nulled, for instance, or they are not optional. So they need to come in the API and we need to validate for those because if the API does not return any of those, it should break and tell us, look, this is wrong, it's not what you expected. So when we wanted to write tests for those and we wanted to create objects for those in each test, we

noticed that out of hundreds of fields, we might need to fill, I don't know, probably 80, 90 of them because they were mandatory. And it started to be very tedious. And I remember I opened an issue in the Pydantic and say, hey, have you thought about probably allowing creating an object with random fields that validate properly like this feels an integer between 10 and 20 so i just don't want to feel it i don't want to feel any of those because i don't care for this test is there a way that i can say okay just feel whatever it validates and they say no it's out of the scope of pedantic which also makes sense i just wanted to ask in case and they said that probably in factory boy they might be interested in this so i went to factory boy and i read the documentation it was pretty cool because

it allows you to create uh unifying an inside class so it's a meta class not a meta class in the terms of uh python meta classes but it's a class called meta within the the factory that you want it's weird because everything you yeah, this is the meta class. Why do meta class? No, it's a class meta. So you inherit from factory, then you define a class called meta. Meta would you define what is your model? So this is the object I want to mock with this factory.

And then you can define many fields with their default values. The cool thing about this is that it implements also Faker. So you can say, if I have a username, I don't want to fill it, just give me a username and Faker will give you. Yeah, Faker is really cool for generating stuff like that. Yeah. And the amount of plugins you find for Faker is outstanding.

So you can fake almost anything you think of. So the cool thing about this is that it's not only you plug in the class that you have and it will fill it, but you also can work with ORMs. Like you can use SQL OCRM or Django ORM and it will generate an object for this ORM based on whatever you set these are the default values. So I thought it would be great if I could do this also for Pydantic. So I could just say, okay, these are the mandatory field that put something fake, you can think about it, and then we're ready to go.

But reading the documentation, it didn't appear anywhere, and I thought, hmm, maybe I cannot use it for this case. So I went ahead and opened an issue and say, are you thinking about putting this also to work with Pydantic? I mean, it's now booming and everyone is using it, and if you are reading JSON from an API, it's very likely that you have hundreds of fields you don't care about.

You might want to fill it with whatever. And I remember the author said, I didn't know this. I didn't know PyTantics. Cool, you mentioned it. But internally, what FactoryGo is doing is creating a dictionary with the parameters you want to fill in and just unpacking it

in the construction of the class. Have you tried it? And I was like, no, I have not tried it. And when I tried, it worked. So of the class. Have you tried it? And I was like, no, I have not tried it. And when I tried it, it worked. So it works perfectly. I mean, maybe there are some quirks of PyDantic that it cannot cover. But if you're using PyDantic to store your data from API calls and so on, from JSON and validates and so on, FactoryVue is pretty cool.

I mean, the amount of things you can do with this, you can create many factories for the same class. You can create fixtures like, I don't know, if you want can do with this, you can create many factory for the same class. You can create fixtures like, I don't know, if you want to mock an user, you can have an admin or a buyer or whatever. And then you can just define different factories and it will give you the usage you've defined. And it's also pretty cool because the faker is randomized beneath it. So if there are parts of your object that your code does not care about, it's also a good test to have those parts being random. Because if it really doesn't care, you don't care what those fields are. And at some point, your tests fail, it happens once, it means that you actually can fix something.

Yeah, absolutely. I did see that if you need repeatable tests, but you want Faker to generate random stuff, there's a way to seed Faker. Exactly. Generate the random values, but do it in a consistent way. And one way you might want that is if you have an edge case or some value that breaks the test, and then you want to put a break point and press debug

and go through it again. But how are you going to get it to hit that case again in a predictable way, right? So if you trigger, if you tell it it to say always do the same thing but randomly yeah you know you'll make it so you can go back and look at it a second time and figure out what's up yeah you can fix that sometimes it's also good to have them fixed even if you don't care i mean you need to have a date time and for some reason you need to have the date time being whatever and whatever but you can validate for it so you can, or either set it on ensure that it's fixed.

Yeah. There are many use cases that you can exploit that thing. And it's actually really cool. Yeah. I usually, I almost always seed faker because I just, I don't, I don't, I'm not using it because I want the randomness. So I'm using it because I don't want to come up with the data. Yeah, exactly.

So make it so that it does the same thing every time. Just gives you the random data that you want. That's right. Agreed. Very, very cool. All right. The next one up actually is pretty sort of related to that. It's called PyInstrument.

Have either of you heard of PyInstrument? Not until now. When I read the notes and it sounds pretty cool. Yeah. Brian? No, I haven't. Yeah. So it's a call stack profiler for Python, which is pretty cool yeah right yeah i haven't yeah so it's a uh call stack profiler

for python which is pretty cool right it's just going to tell you where your code is slow but it's you know it's it looks really clean and when you look at the output it can actually give you the results in the terminal so if you want to see you know like you run this thing instead of saying Python, my example, my Python dot py file, you would just say py instrument that same file and it'll run it. But then at the end, it's going to generate a whole bunch of things about how long it took and whatnot. And then you actually get like colored output in the terminal showing which lines of code are spending how much time in different places. And it seems like it's a real good way to just sort of quickly dive in on where you're spending your time.

Yeah, I'm definitely going to try this. It's cool. Yeah. One thing I like about it is the simplicity of like pip install py instrument, py instrument, your file. That'll give you the answer, right? That's for me, sold it. I mean, every time you want to do some profiling, you spend some time tweaking things so you get what you want. The fact that this is just running with PyInstrument, whatever script you want,

I mean, I'm going to try for sure. Yeah, yeah, for sure. And when you do profiling, you end up in this sort of quantum mechanics world of if you observe a thing, you've changed it. Yeah. And so there might be code that is like, this half is 50-50 and this half is 50, 50, and this half

is 50, 50 at a time, but one is calling an external system once. And the other is a tight loop. And if you profile that with instrumentation, it's going to wreck it. It's going to make the loop look way slower because now you've added a bunch of overhead each step, whereas there's very little overhead to this external service sort of thing. And this one uses sampling and the sampling doesn't really have that effect. It just every so often, every millisecond or something, it says, what are you doing now?

What are you doing now? Who called you? What are you doing now, right? And so it's more of a polling sort of thing rather than slowing down line by line code. So that's probably worth doing as well. Yeah, that's pretty cool. Yeah, it looks like you can specifically jump in

and just do a section of your code that you care about also. Exactly. If you want to say, you know, so one of the things that I hate about profiling is it'll say 87% of your time was in the startup code and the imports. You're like, yeah, okay, that's not relevant to me.

What I want to know is the part that I's not relevant to me what i want to know is the part that i'm actually trying to test here what how long did i spend there and please don't pollute that with other junk about like starting up python or loading modules or whatever right and so you can there's an api and you can say from pyramid import profiler and then you can do a context block in there and run it. And just that code will tell you like how long it takes. Does anything else jump out there at you, Brian? And like with this example I got on the screen here, that would be hard.

It's an async example for one. Yeah, as an async and a wait. And so they recently released PyInstrument 4, which will actually give you the information about the async code as well, right? So it'll, let's see what it says. Has async support, PyInstrument now detects when an async task hits an await and tracks the time spent outside of the async context under the await. Whereas before it would basically just profile the async IO event loop or something

silly like that. Right. So if you're trying to profile async and await and async IO, this might be your best option because it specifically supports that. That's good. So what happened before, if you use a different profile? It would say, yeah, it says you only see the time spent in the run loop. And it'll basically tell you, like, here you see, like, run once, the select, and then the Q control built in. It's just like there's this async IO event loop that's cranking around waiting for the signal for the IO to be done. And it just says, well, you're waiting on this. You're in the loop. You know what I mean?

Yeah, yeah, yeah. Yeah. So, yeah. Very cool. So now you get a little bit better. Like it says, okay, you're waiting on this line of your code for a second or whatever it is. Yeah, there's also, I'll shout out a few more things here. Is it in the stock?

Yeah. So they also, there's also a bunch of simple authentication they did previously about network calls and stuff. And there's an interactive Vue.js app that you can get with FlameGraphs. So instead of looking at it in the terminal, you can look at it in your web browser and explore into those. Yeah, there's a lot of neat little things here pulled out of the show notes. But it seems like a really nice way to do some profiling. And you just high instrument your code and you have a look. Yeah, I personally kind of like the default output.

I know that a lot of people like flame graphs. They don't really do much for me. They look like I don't see the data. But it's cool that it has both. Yeah. A couple of things from the live chat. Maddie says, Pi Instrument is a statistical or sampling profiler, which is better prepared for profiling.

I think it depends. I mean, the instrumentation ones do give you more precise information, but it's also skewed with the overhead of that information. So it depends, but this is the least influential one for sure. And then Avaro says, how would you use PyInstrument with an entry point? That's a good question. Not knowing the answer off the top of my head, maybe make another Python file that just imports your library and calls the entry point and then profile that. But there know there's a real quick uh cheat you know just make it call it and then um uh pi instrument that file but there may be some way to say like dash m and give it a module and a thing to do so yeah that's it brian that's it for pi instrument cool um well i just

wanted to remind everybody that uh python 310 release candidate one came out yesterday so pablo announced it uh just just on the third i think i think it was yesterday um the fourth today yeah anyway uh so 310 is out um if you've got um if you've got the well 310 rc1 is out the timelines that we're looking at then, we're getting excited. It's coming up. So the September 6th is the plan for RC2. And then October 4th is the plan for the official release. So we're just really right around the corner.

It's nice. And this is definitely a time, I know we've brought this up before, but if you maintain any third-party Python packages, you probably should have already been testing it against 3.10. But if you haven't, definitely do it now to make sure that people that use your stuff doesn't break when they need to.

And then in the show notes, we put just a reminder of some of the new changes in 3.10. We've definitely talked about some of these before, structural pattern, structure pattern matching is the switch statement kind of thing. And then, yeah, lots of these other things we've covered. I'm kind of actually, I like the union type, union types. So you can like, because there's a lot of stuff that I write that the default is none, but the normal type is something else. So you can really easily say the type is none or int or something like that.

And that's a lot cleaner than before. I've already started using 310 to test everything that I support. So I hope everybody else has as well. Yeah, cool. I like the optional length checking in zip, right? Zip taking two collections and you want to pair up the items. Like if those things don't match that should be a problem also like the or for the types uh information and i think dick and some of those types are now don't require a from typing imports oh right yeah um yeah i don't think i don't see it called out here but either one of the problem

was um maybe that's explicit type aliases i'm not entirely sure but if you want to say this type is a dictionary of strings and um integers you would have to say from typing import capital d dict and then dict square bracket uh string comma int whereas now you can just use the lowercase d i c t and you don't have to have that import and you can do that sort of thing too so that i'm looking forward to that yeah a lot a lot of the, with this, a lot of the common type hints, you won't have to do the import anymore. And that's great.

So I think that's really all I was using the import for was things like dict and set, things like that. Yeah, exactly. Didn't that, I mean, I seem to remember that 3.10 was the one that was including these built-in types without having to import from typing. Didn't that update might break some of the libraries that use typing? Like Pydantic and FastAPI.

The thing that that was, was to use it and basically use it as a string and not actually evaluate the type. Okay. use it as a string and not actually evaluate the type i think that like so if you had your own type your own pydantic type that was a customer i think you could put customer but it wouldn't be actually evaluated until a type checker hit it or something like that like a forward typing yeah yeah exactly so this this ability to specify the type on like lowercase d dict is related but it's not the same and i'm pretty sure that that that fear around Pydantic is not in 3.10. Yeah, it either got postponed or rolled back or modified. Yeah, yeah.

I just wanted to talk about the one that says, what was the number? Six to six, do you have it? Yeah, the precise line numbers for debugging and other tools. I think it's very underrated. It's going to be one of those things that when people get used to it, it's like, I don't know how you live without this. Oh, yeah.

There's not a good example shown right off the bat, but it's pretty cool. Yeah, absolutely. Very cool. And we also have better stack trace error messages, right? Yeah. Yeah, those are coming. A lot of good things to look forward to.

All right, Juanpe, you got the last item. Great. I think it's time for it. You want to take us out, right? Sure. Yeah. So let's talk about Time Machine. I said we were building this tool that copies an entire cell for sort.

One of the things that we need to work is to timestamp almost every action we do this means that in many places all over the code we have daytime UTC now method call and when we are testing day we need to be able to mock it

and if you've tried to patch daytime UTC now with the usual patch method it you know, it works. You can do it. But you need to do it with a patch and then you pass the string, but the module where this UTC now call is. And then you're good to go. But when you have this in many files in the same test, you need to patch every one of those because otherwise it wouldn't work. So I tried to use patch object and patch daytime and say, okay, I patch the it's in our method this object and it will of course complain and

say you cannot patch a built-in uh built-in type like daytime that time so i was looking for how we could patch this and i found um friskan which is a very well-known library thing to patch this kind of things but suddenly i noticed that once I started using Frisk and all of my tests took much longer to complete. It's not like a deal breaker. So it went for probably five minutes to seven or seven and a half. But I was very surprised because our pipeline or deployment pipeline already take a long time.

So every time I can reduce a minute, it's good for me. And when I saw it going up two minutes, I was surprised, why is this happening? And then I learned that what Friskan is doing is actually scanning all your dependencies and make a patch for every call you make to the methods of data. And then trying to see if there was something else, I found out Time Machine. Time Machine is a very cool, not so well-known, I think, library that allows you to do basically the same that Freezgen allows you to do. So you can just patch almost any method call in daytime or time with a simple decorator in your test. It also supports PyTest fixtures that you can use.

The good thing about this is that it does not scan for imports of date and daytime. And what it does is actually change the underlying C-level calls that you make to get the time. So every time you say, I want to patch any call to be on january 1st of 2019 for instance it will just call it normally but the c the underlying c calls that will be made will return this time instead of the other one so you don't need to scan everything to patch it another thing that says that i thought it was pretty cool is this you can let the time tick after you patched it so you say this is for February 1st of 2018. And once you enter the mock,

either with a decorator or with a context manager, you can also use like standard patch call. Then time start passing, starting on that time that you mocked it for. So you can do perf counters and all this thing normally. But if you need to stay in a given day for a test, you can do perf counters and all this thing normally, but if you need to stay in a given day for a test, you can do it. So I thought it was pretty cool. It solved my two extra minutes running because we have many places and many files in the project where we use C2C now.

And it was pretty well. So this must have had incremental, I mean, it has a little bit of time that it has to do its work, but it's fast enough that you're not noticing it then? No, I'm not noticing anything. I mean, it runs more or less the same. Okay. Wow. I mean, I imagine there should be some delay, but it's not as noticeable as what happened with FreeScan. Yeah.

Because it took some time. Yeah, I'm really glad you brought this up. This is cool. Yeah, we have a bunch of tests. Yeah, exactly. I was going to say, Brian, this is kind of in your world right like dealing with time as a dependency definitely and there's there's there's sometimes it's really you want it fixed because you really want fixed answers because like your time stamp stamps and

stuff are in your data you're gonna have to i mean it's good to compare against known oracles but there's all the also times where you you uh and this is where freeze gun isn't so bad is but maybe this would be really useful too is is if you want to test certain things there's weird quirky dates you want to make sure that your software deals with certain times uh fine does it work fine when it's running over like uh overnight on december 31st to january 1st things like that when the year changes and things like that. Yeah, exactly. Yeah. Yeah. Yeah. You always want to test your boundary conditions, right? And crossing over time or weird cases like March 29th and stuff like that. You're like, let me just try

that and see if this is going to survive. Yeah. Yeah. But then, I mean, to be fair, I think most of the time things like this are used are, like was brought up, is that the time shows up in the data. So in order to compare the, you know, or the log or something, in order to compare those apples to apples, it's nice to have the same dates there. I can't tell you how many times I've had to compare two log files and strip out the times because those are the only, like those are the every line is different

because the the timestamp is different so yeah yeah very cool nice find so that's all for time machine yeah yeah super um well that's our that's our six items uh everybody uh um have you got anything extra michael well i have the old thing that is new again. Let's see. I have some bandits. So the drama around supply chain vulnerabilities and open source repositories goes on. So this one, I think actually the other article I'm going to talk about comes to us from Joe Ridley. Thank you, Joe, for sending that in. But basically there's some more malicious things in PyPI again, and people just remind everyone to be careful and be white list stuff.

Yeah. This one, I don't know what this one was. If it was typosquatting this time around, or it was just something else that got put up there. Yeah, there's one headline is credit card stealing malware found in official Python repository. And the other one is the same one about Ars Technical article. It says software downloaded 30,000 times from PyPI ransacks developer machines, developers' machines. Expect to see more of these Frankenstein-type things

because it's basically a systemic threat. Like, how does it get dealt with? I'm not sure if they list out. Yeah, so they did interesting stuff as well. Like, they did simple obfuscation of the code that was being run so you couldn't look at it and you know say look for a credit card or look for a bitcoin wallet and then go do your evil deeds in python source code so they would do things like um base 64 and code the python code and then just in memory decode it then

run it so they were trying to get around things like that. So anyway, people can check that out. It's not ideal, but just a reminder to beware. Yuck. Yuck, yuck, yuck. This is why we can't have nice things. Come on, people. This is why we can't have nice things.

Well, I got a couple of things I wanted to bring up, just things I've been up to. I just released episode 162 of Testing Code Code and I kind of run through all the different flavors of test-driven development that I know of. So there are quite a few versions. So check it out if you're interested in test-driven development. And then I'm just working on wrapping up the talks and continuous integration chapter

for the second edition of the PyTest. It'll be coming out hopefully within a week. Very cool. Good to see you making progress there. Do you have anything extra, Juanpe? Nope, not for my time. I mean, I'm very happy to be here. Let's go to a joke.

Yeah, it's good to have you here. All right, let's go to a joke. So this one's a visual. If you're listening, you're going to have to go to the, scroll down to your podcast show notes at the bottom, just click on the joke link. One of the things you guys, I like about Python is there's a lot of stability in the code

that we write. You know, if I wrote something on Flask five years ago, chances are it'll still run, right? If I write my Python code now, it's probably still going to run. Yeah, there's new things. There's new shiny visualization frameworks and stuff, but generally it's pretty stable. You know what is the opposite of that? JavaScript. So here's a little animation and it says JavaScript developer bouncing from framework to framework. And it's this incredible, like almost people are awesome type of thing where somebody set up, you know, 50 workout balls

on a running track, the whole straight of a quarter mile running track. And somebody jumps on it and just like glides from one to the next. What do y'all think? The fact that he's able to do this is surprising. It's really impressive that he pulls it off. And it's on one of these like sandy, gritty running tracks. It's going to hurt like crazy if he misses it. So maybe there's the motivation you know yeah actually so i remember the gem brain report you said before i was thinking i

didn't say anything but i didn't want to mean but you're saying how likely are you to change languages and it was like what javascript they're going to change a lot then i thought oh their language is not frameworks exactly if it's how likely are you to change your framework, well, that's like nearly 100%. Yeah, that's true. I mean, people stick around. Like you got Django developers have been doing it for years.

Yeah, 10 years and they're more excited today than ever about it, right? They're not like, we're ditching this. Yeah. All right. Well, does that count as a joke? Yeah. Oh, I laughed. All right, perfect. Well, that's, does that count? Does that count as a joke? Yeah. Oh, I laughed.

All right. Perfect. Well, that's what I brought for you all. Well, thanks everyone for showing up and had a fun day today. Hope everybody else did. Thanks a lot for having me here. Thanks, Brian. And thanks for being with us, Juanpe.

Bye. Bye-bye. Thanks for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. Get the full show notes over at PythonBytes.fm. Bye. and click live stream to get notified of when our next episode goes live. That's usually happening at noon Pacific on Wednesdays over at YouTube. On behalf of myself and Brian Ocken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.