Python Bytes - #201 Understand git by rebuilding it in Python

Episode Date: October 2, 2020

Topics covered in this episode: Under the hood of calling C/C++ from Python * ugit: DIY Git in Python* Things I Learned to Become a Senior Software Engineer Profiling Django Views Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/201

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 201, recorded September 24th. I'm Michael Kennedy, and Brian is not here this week. Instead, my friend and special guest, Cecil Phillip. Welcome, Cecil. Hey, hey everybody. I've been a long fan of the show. And so, you know, Brian, I'm sorry that you're not here, but I promise I'm going to do a good job and do you justice. So I'm going to do my best Brian impression today on the podcast. Beautiful, beautiful. So Cecil, you've been on the show before. You're a developer advocate at Microsoft. Is that right? Yep. Yep. Still developer advocate.

And as folks could imagine, we're all at home and not traveling as much as we used to anymore, which is usually something that's attached to the job. But now we're just trying to find different ways to kind of reach out to the community and work with folks and make sure that they have what they need to, you know, just be successful and productive. Yeah, absolutely. And you're doing a lot of live streaming stuff these days rather than appearing in conferences. Yeah, we're doing a lot of live streaming, a lot of live events, and even, you know, live conferences, too. I think a lot of conferences now, and you've seen this a lot as well with PyCon

and tons of the other online conferences, is just because we can't do it physically together like we used to, you know, now a lot of folks are just doing it online. So, you know, I'm sure if you'd look online, you see tons of, you know, register for free. Here's my conference on Tuesday and Wednesday and Thursday. And, you know, everyone's just trying to find a way to connect.

Yeah, it's got some interesting implications for accessibility, right? Like I could necessarily go to EuroPython, but I could attend the virtual one just as easily as long as I'm willing to get up a bit early. That's kind of cool, actually. Yeah, that's one of the things I like about it the most is that, kind of like what you said,

like before, I didn't want to go buy a plane ticket to go to Australia or somewhere that's maybe halfway around the world to me. But now I can have maybe not the same experience, but I could still engage with that community of folks virtually by just signing into their YouTube or their Twitch stream or whatever the case is. And I can get the content the same way. And it's great. Yeah, absolutely. So let me kick this off with our first item where we go from Python to speak another language, a little different cultural here.

So we're going to talk about talking from Python to C++. Maybe not what you thought when I said that, but really C++ is one of these or C really at the more low level is one of the reasons I think Python is doing as well as it is these days, right? Like a lot of what you do in Python might be a little bit slow, but you know what? You implement part of that package in C and then all of a sudden,

it's a lot faster than things you might compare it to. And that's a lot of the magic. A lot of the data science works that way and whatnot, right? Right. Yeah. So there's this article that I'd like to point folks to that basically walks you through what it's like to call C and C++ code from Python.

So there's a lot of things to cover that's pretty interesting. It talks about, first, what do you get when you compile C? Obviously, you get something that's executable. What it is is actually different depending on the machine you're on, right? So if you're on Linux, you'll get an executable and linked format, which is ELF. On Windows, you get get an executable and linked format, which had ELF. On Windows, you get a portable executable. On Mac OS, you'll get a mock object, a mock.o, and so on.

So it compiles to whatever machine instructions you need. And then depending on what platform you're on, again, you can put that in some central location so that compiled C library can be used by Python. So on Linux, it'd be user slash lib. On Windows, what do you think? Windows slash system, system 32, something like that. System 32 or something crazy like that. Yeah. Yeah, exactly. Something there. But once you put it there, then your program can find it and you can just import it. So there's different ways you can do this right like you can use the c types library which is simple sort of sometimes simple and you can just go and import c types and then just say

here's a cdll and point literally at the file the output from compiling some c code and import it and call it that's pretty simple coming from microsoft world made this DLL import type of equivalent. That's all I'm not entirely sure, but there's some interesting things that you run into. Like, so you get this library back, but it doesn't necessarily know what the return types are, right? Like, Oh, you get a function, you call it returns a float. Maybe it doesn't know what to do with that. So you've got to like, as you import it, tell it, Oh, you return a float here and here are your functions that you can call and so on. So there's a lot of interesting things you can do.

If you remember your C, you have to set the calling convention, which can be really tricky, like extern C. Oh my gosh. Oh, I remember that. Is it standard call? Is it a C decal? I mean, there's so many variations. Do you remember this?

Did you ever do some C and just go, why won't this library work with that other one? I know. Now, I'm looking through this document as you're talking about it, and I'm like, oh my gosh, this is bringing back so many memories. Yeah, I mean, you remember how hard these things were to get them right. And it reminds you how nice it is to work with more modern languages like Python. So there's actually an interesting library called libffi.

This is a C library. So if you're going to write the C code, you can use libffi. And when you compile it, as you compile it for different platforms and stuff, it'll automatically figure out the calling convention for the functions at runtime. Oh, nice. So that's pretty sweet. So you can, I mean, this didn't exist to my knowledge when I was trying to do these things. That's interesting. Another thing you can do is you can add an entry point function to your code. And then Python will

ask, basically can call that and say, you know, what functions do you have? And what are their signatures? So there's a way to have your library kind of introduce itself over. It's kind of like providing metadata, I'm guessing. Yeah, exactly. Here's the structure of my stuff and I'm going to give it to you. And now you can kind of understand it. Exactly. Because you don't have a header file or anything like that, that defines it.

It's just, here's the executable compiled functions. Go, right? And so it gives you that metadata. Exactly. Of course, the goal is to have C++ code that you can call from Python. functions go right and so it gives you that metadata exactly of course you find you the goal is to have c++ code that you can call from python and there's a python specific a python friendly way to do this as well if you're willing to work with a python type so like pi object pointers and so on and the way you would do that is you create this entry point, but it's the function name is pi init underscore module name. So if it was requested, it'd be pi init underscore request.

Exactly. And so that returns all this metadata that says, here's all the C stuff and the signatures and whatnot. So after going through all this stuff, how you write the C code and you do it, it says, you know what? Maybe you just want to skip that yeah and can we please what can we please yeah so what you could do is you could write it in cython and which compiles to c which compiles to machine instructions or there's also boost.python i wasn't aware of boost.python that's pretty cool and pybind11 which allows you to bind i believe uh to c++11 which's pretty cool. And pybind11, which allows you to bind, I believe, to C++11, which is pretty cool as well.

Really nice integration stuff there. And some examples of libraries that people might know of that are written in Cython, but basically natively compiled and do awesome stuff is AIO HTTP. Oh, I didn't know that. That's cool. Yeah, use the Cython for the HTTP parsing, so that string stuff is super fast.. Oh, I didn't know that. That's cool. Yeah. Yeah. Use the Cython

for the HTTP parsing. So that string stuff is super fast. If you do async, you can, one of the things that's interesting in Python about Python's async as a weight and a weight is you can change the implementation of the underlying of the event loop. Right. And so one way you can do that is you can replace it with UV loop, which is based on libUV, which I always thought of as a C library, but it's actually fully written in Cython. Oh, I didn't know that either. Really? Yeah, that's pretty cool.

Then finally, HTTP tools bindings to NodeJS HTTP parser is fully written in Cython, along with Sanic, UVCore, and a bunch of other stuff that use this library. So if your goal is to try to get a little bit lower level, maybe work with some C, make your code go faster, here's a really cool write-up of how to do all those things. That's really cool.

You know, I'm a big fan of language interoperability in general. Yeah. And, you know, one thing I could tell you, it feels like every language knows how to work with C, at least. Like C is like the common denominator that every language knows how to like talk back to. And I really wish there was a way for us to, I mean, I love C. I used to love C. But I really wish there

was a way for us to like, have like a very standard way to like interoperate with each other a little bit better. And I remember, I think the last time that i was on the podcast i talked about wazi so i think it was a web assembly yes it's like a web assembly module that you could use and now all of your code compiles down to web assembly and it exposes different functions and whatnot but now that that'll be like a portable runtime that essentially again i can import it in python and i can can import it in Java or JavaScript or.NET or whatever the case is, because they all have that, you know, that knowledge of how to work with that WebAssembly binary, right, that portable WebAssembly thing. Yeah, I should look that up. I don't know where they are with that. But I know that was the thing that folks were getting really excited about,

not just from the fact of, hey, I can run my Python code or my language X code in the browser, but also, too, I can consume it from other languages. Right. If somehow you can package your code in whatever language you use in a WebAssembly format, all of a sudden it becomes interoperable with other WebAssembly libraries. Anything else that can be packaged into WebAssembly, basically, right? Right. And what's good about that is it's relatively a standards compliant way, because WebAssembly is a standards based thing. It's not owned by a company or one particular runtime. All of us run and play in the web and use browsers and use browser technologies for

one thing or the other. So it's definitely something that everyone can kind of get involved in one way. Yeah, for sure. I'm super excited about that as well. All right, moving on to the next one, your topic here. You've got something very small that's actually quite cool. Yeah. And so just kind of talking about interoperability a little bit. One of the things that we all use a lot as developers is Git, right? I use Git a lot, and particularly in the GitHub space. Actually, I don't think I know a developer today that does not have a GitHub account, but you know, maybe that happens. Do you remember when people used to debate about what kind of version control they use like oh we use perforce oh we use

subversion oh we're still on cvs we use a win c i use tortoise cvs and i got a right click like that used to be a debate that debate is gone it seems like i remember man i remember there was a debate back when git first started too there was a debate between git and mercurial because those were the two distributed version control systems and people were like oh i'm going to use this one i'm going to use that one and trying to figure out like which one that was going to be the thing to use but i think it's pretty safe to say that in github of one yeah pretty much yeah you know github too yeah github's awesome yeah i think they're pretty much just one so now this article

why i think this is really interesting is you know as a developer since we again get is pretty much like the standard of source control for all of our projects it's important for us to understand how git works or maybe you might be curious about what the underlying internals of that looks like and so what this is is a really cool. It's an interactive tutorial that you could do kind of in your browser as well, where you could play around and implement Git yourself, your own version of Git in Python. And I'm like, oh, that sounds cool. I definitely want to do that. Because one, I think, hey, you're just giving me another project that I could get better

at writing Python. But two, I can also really try to understand what Git does. I know a lot of folks, you may know a few Git commands, but oh man, I can also really try and understand what Git does. I know a lot of folks, you may know a few Git commands, but oh man, like I really don't understand what rebase does. And I don't know, what's the difference between Git branch and Git switch and, you know, like some of those types of things. And so I'm hoping like we could go through this tutorial and by implementing it, we could really understand like how the underlying data structures are different and how that get database

is different that's so interesting and thinking about understanding get by re-implementing it from scratch is pretty interesting yeah and i think it's worth pointing out just the format of this article tutorial thing it's it's quite unique so if you open it up on the left hand, the browser split 50-50, the left hand column is the description and message. And on the right, it's basically got a diff of the code that you're writing at each step. So it doesn't look like you can execute or anything, but it's a little bit like REPLIT or something like that.

Yeah, I'm looking at this and I think, I was trying to see if you could click on this thing, if it'll take you to the source. But I think this is, in addition to just being a really cool topic, I think it's an interesting- Oh, it does. As you click around it,

it'll reload the side to like, take you to different parts in the code. Yeah. Oh, does it? Okay, yeah. Yeah, so there it does. There you go. There it does it.

But I think this is a really interesting experience just for learning in general, right? It's very interactive. So I could read the text, the verbiage on the left-hand side, and I could look at the code and I can see how it changes on the right side. So it's not just like, hey, I'm just reading this super complex book about like Git internals. Like I can actually see the code, which, you know, for folks that like to learn by seeing, it's a really good experience to be able to understand like what exactly is going on.

I might have to go through this. This looks really interesting, and I might learn a couple of things by doing it for sure. Yeah, yeah. This definitely looks really cool. But no, speaking in terms about learning, you had a topic you want to talk about, right? That has to do with learning, right? I would say you and I, probably these days we qualify as senior software engineers.

I don't know. I don't really want that title on me anymore because, I don't know, it sounds like really want that title on me anymore because I don't know. It sounds like I go to an office. That's my title. But nonetheless, we've been around software for a while, right? And there's this cool article by Neil Kakar

who wrote it. It is an in-depth write-up of things I learned to become a senior software engineer. I love these things where people sort of look back on what they've done in their career, how it's, you know, what went right, what went wrong, and just trying to help other people shorten that cycle to get farther, faster, quicker, that kind of thing.

So there's a lot of lessons he talked about. And I just wanted to cover some of them. I said it's really long, and so I'm not sure I want to go into all of it there. So the first one was he advanced his career by growing his skills and experience using different ladders of abstraction, is how I put it. So the first year or so, he had basically learned the ins and outs of software. He knew his language, his compiler was working, and I believe he was doing Python,

so maybe not his compiler. But all the stuff that he's doing Python, so maybe not his compiler. But all the stuff that he's doing to work there is pretty much figure out the software side of things. But his software development lifecycle was like a small cog within a bigger cog and so on. So he said, look, there's really the lifecycle of the product there's a life cycle of the infrastructure and the business and like working your way and sort of out of that loop to understand the bigger pieces really uh helped him do more than just like write code but like really be a bigger part of the team i guess this is really interesting and i'm kind of scrolling to the article again as you're talking about it and it brings up to mind like a discussion that i get a lot from some of the students that we talk to, you know, with the different engagements we do. And that's always their question. It's either one, how do I become a good software developer? Or how do I level up in

my career? Like, what are the things that I need to do to step forward in my career? And sometimes I feel there's certain things that for sure you could definitely do, but then I think that's only like 50% of it. And I think there's another 50% that's could be a little subjective based on the area that you're in and the region that you're in. And like, what are the things that you really want to do? Like, what are the things that you really care about? Like I could tell you when I first started, honestly, I was just, I was just trying to get a job, to be honest with you. I wanted to leave school. I wanted to work because being an international student, then I had to worry about visas and green cards and all that type of stuff.

And I'm like, the first person that hires me, I'm gone. That's it. But obviously, that's not something you base longevity on when you think about a career. You want to think about, okay, now I'm working. And, you know, I maybe I've spent a year or a year or two inside of the environment. And, you know, you know, you know how to code, you know, how your deployments go, you understand your ticketing system and all the scrum meetings that you have to sit in on. But now you got to understand, well, well, how do I get better? And one of the things I always tell

folks is, well, it's kind of like being a specialist, right? Like what are the things that you want to get better at? Because there's so many different things to look at. What are some of the key things that really interest you? So do I want to be really good at doing security? Do I want to be really good at doing performance? Do I really want to get good at maybe microservices and distributed systems is interesting, right? Right. You get pigeonholed and people will just leave you there if you don't try do i really want to get good at maybe microservices and distributed systems is interesting thing right right you get pigeonholed and people will just leave you there if you don't try to stretch right but you have to want that though right like you have to want to get better and

then after you have that one you got to figure out where exactly do you want to go and then for me then the next step after that is really trying to understand well what are the fundamental pieces that i need to understand like what are the fundamental pieces that I need to understand? Like, what are the fundamental algorithms and data structures I need to understand? And I'm not talking about like link lists and bubble sorts and that type of stuff, but like within that space, right? Like, so in, when we talk about security, like what are the things I need to know about cryptography, right? When I talk about like distributed systems, what do I need to know about networking and how networks work? You know what I mean? Like,

how do I get the knowledge to understand that so now I could continue to move forward and really become like very knowledgeable and successful in that particular aspect of software development? Yeah, I think that's a really big part of it. And that's definitely some of the stuff they were touching on. I think one of the big levers you have to pull is to try to get yourself to do stuff that is uncomfortable, right? Like I don't really know about continuous deployment, but we don't have a great answer for that. So I'm going to force myself to learn that and I'll volunteer to be the person to make that happen, even though I'm not totally there yet, right? But that'll get you there if you take those steps. Another one that Neil pointed out was learning

what people around him are doing. So not just what are the developers doing, but what are the project managers, the salespeople, the analysts, and so on. For you working in a company like Microsoft, there's layers and layers and layers of that, right? Yeah, for sure. And there's good and bad things to that. One of the good things I like about that is that there's always people that you could ask a question to. And sometimes you might be in an environment that that's not really the case. Sometimes you're in a company and everyone is at the same level that you are. You can't talk to other people, at least, or maybe you don't have access to talk to other people. But when you're

in an environment that there's a lot of folks that you can ask questions to, one of the best things that you could do for your career is make sure that you spend time with them, however that works, to really understand how does their world kind of move, particularly if it's something that you're interested in. So again, like, hey, I don't know a lot about security and I have a security feature I need to work on and it's something I really want to dive deep into. Let me go and find the PM that does security for this thing, or let me go and find the person that works under Docker integration for this thing. Or let me go and find the person that works under Docker integration for this thing, or the Python integration for this thing, and really have like a deep conversation with them. And the good part about it, if that

person is willing to sit down with you and give you the time that is like gold, like it is the best thing ever to be able to have someone pass on knowledge to you. I say that too, because we move forward. And as we become quote unquote senior engineers, we have to remember too that the folks that are coming that are not so senior, they're going to want to do the same thing for us, right? And we have to have that mindset of being willing to share that information,

not try and keep it inside and take it with us when we leave the company kind of thing. Yeah, well, the other flip side is, the best way to learn something is to teach it often. And that's your opportunity. Yeah. Yeah, exactly. Exactly. Another interesting lesson was strategies for making the day-to-day more efficient, which I suppose the larger the company, the more important this can be. And Neil said some good habits. Notice was never leave a meeting without making a decision or having a next action to do. Meetings drive me crazy. And it's so often you just have a meeting because we always meet on Mondays for half an hour,

even if we don't have to. Like, no, please, not again. Decide who's getting a thing done because if it's not on somebody's responsibility to get it done, it's probably not getting done. And then document design decisions made during a project. So that was pretty cool. Those are good ones.

Yeah. Acquiring new cool. Those are good ones. Yeah. Acquiring new tools for thought and mental models. So one example in the software space would be like, I was recently struggling with the domain of a lot of complex business logic and all these weird edge cases and stuff. And so found the domain driven design book by Eric Evans and that whole space and said oh

there's a whole world over here around like modeling objects and relationships that i didn't understand or didn't know about and and diving into that stuff's uh pretty interesting here's my favorite and this has driven me crazy about so many different things protect your slack so not your slack the not competitor not your channel but slack as in i've got a little bit of time in my day-to-day schedule that is not clearly assigned right not like i'm spending the next two hours on this feature because i agreed to it in the sprint and then i'm done and i have to move on to the very next thing, right? If it's so tightly packed that you don't have the freedom to explore, like that didn't quite work the way I expected it. Let me dig into that. Or what if we re-implemented it this way? Let me just try that

idea or I don't know anything about domain-driven design. So let me learn. Let me spend the next hour reading about that and see if that'll help us later right if all those opportunities are taken away in your times like squished just back to back to back you end up just doing 20 years of the same thing instead of actually growing and learning new stuff i'm a big fan of companies that allow their employees developers or not time to sit down and experiment and try out new things and and whatever you know prior to coming to microsoft i was in a company that they used to have like every time there's a release right after the release they had this thing oh man what was it called i don't remember what it was called but there was it was like in events it was like a whole company event

where for like the next week all they could do was just work on side projects. Like it was a whole week, right? Oh, wow. Obviously the service folks had to be on service, but for everyone else, like the PMs, the engineers, the developers, you know, they spent this entire week just hacking on stuff. And then the next week, you know, there was a day where they gave presentations and everyone talked about what they did and it was a whole showcase, but it was only internal. It was just for the folks inside of the company. And not only did you have time to do that and learn and try new things out,

but there were prizes, right? Like they were, whether it was stock or money or like just, there was stuff that you got. Which was really cool. So now it shows like how much the company like really believed in you. And so again, I bring it up because I think it's an important one.

Not only do you give your employees, if anyone is an employer that's listening, that you give your employers, employees, the ability and space to learn, but also incentivize them to do it, right? Like really encourage them and let them know that you're supporting it. And regardless of whether that means it's something that's going to go in your product or not, like the fact is like they're becoming better, right? And so if you, all of your employees are becoming better, like that means that they're going to be able

to do better work for you at the end of the day. And so I think that's always a really cool thing to do. Yeah, absolutely. I think this concept of Slack is super important and so much people

looking from the outside, they try to squash it and compress people's time down and I think it's, in the end, it's not so good. So there's a ton of other stuff.

I want to just throw out really quick one last thing on the sake of time so we don't spend too much time on this the last was this idea of building up superpowers so are you getting into the source code of some project some package and the documentation isn't enough all right you have a quest that's to go read the open source code and spend some time like learning from the code not from the docs. You want to quickly build a mental model for code you're looking at? Reading open source code again.

Embracing fear. Something you're not confident or feeling comfortable with. Do it as a side project. That's your quest. Confidence to express ignorance or not knowing. Fine, right? We all got to ask questions we don't know

and get over it. And practice with that. There's just a lot of good takeaways from here and like we really just scratched the surface on it sure and kind of going back to that point about reading the open source code i think that's one of the reasons too why github is so popular because it made searching source code so easy and i'm not gonna lie there's a lot of times throughout my day that I'm thinking, hey, the documentation is not the best for whatever it is that I'm looking at. I'm going to search for like the class or the method. I literally just look for the class name and the method and filter by language just to see, well, I just want to see how are other people using it.

And, you know, what are the types of issues that are created around it? And hey, should I wrap this in a try catch? Or what kind of parameters do you pass in? How do you deal with these things? And being able to kind of look and see how other folks do it gives you some ideas now. Okay, well, this is how this thing was meant to use. Now, the flip side of that is that you might run into a bad example. But again, that's a learning experience, right? You'll run it and you'll figure out and then we're working out.

Yeah, exactly. That's great. So speaking of learning, you guys have a pretty cool project, a data source, I guess, to learn from or context to learn from? Oh, yeah. Yeah. So what's happened over the past, oh man, over months and months and months. So Microsoft actually started a project where they're working along with NASA to create

these learning modules on Microsoft Learn, where you can kind of go in and learn how to do like machine learning with Python. But you could do it within the context of exploring space, which is really cool. So you could do different things like classifying space rocks and predicting weather, it'll help you like predict, you know, rocket launch delays based on the weather, like tons of great stuff. And so there's a, I'm trying to think about

how many are there. There's a one, two, three. Yeah, there's at least six learning modules. Again, all based on Python and AI with machine learning and NASA, where you can kind of kind of go in and learn all these different things.

So definitely recommend that folks go in and check it out. It's 100% free. So you could, you know, just click on the link and kind of go through them yourselves you don't have to sign up for anything which i love like i love free learning resources right but um yeah i thought this was really cool and so if anyone's a space buff at all like this would be a great adventure for you to kind of go through and kind of play around with some some data from nasa yeah i think that's super cool space yeah and so much of being successful

especially when you're learning early is about having context that inspires you yeah because we all get stuck or things get frustrating you're like why won't it why is this function not doing what it's supposed to do but if you're like and if i get it to work i get to understand a space rock or something like that all of a sudden it's like yeah that's not so frustrating right right and i think nasa also has for folks that want to get a little it's like, yeah, that's not so frustrating. Right, right. And I think NASA also has, for folks that want to get a little bit more into NASA, NASA also has like, I think it's called NASA Socials, which is like a national social club

where you could, again, work along with other folks in a community. Back before social distancing, you could actually go to a social event and you just learn different things about the work that they're doing with NASA. So that's also really cool too. I think you could do those social events online.

So if you're interested, probably like, hey, maybe you learned a little bit in doing some of these courses and you want to engage with other like-minded folks that are into space exploration, but they don't necessarily work at NASA, that might be another thing to definitely check out. Yeah, absolutely.

The next one I want to cover has to do with performance. And I don't know if you've ever had this experience, but I have definitely had it where the code is going slower than I expect. And I'm like, oh, it's got to be this part of the code that is so slow. And then I profile it or I try to fix it. And it's absolutely not, it has nothing to do with it. That's like 5%. It's somewhere else. It's completely somewhere else. Have you had this experience? Yeah. And you know what happens? It's always a database have you heard that no indexes yeah people always say that there's like oh you have a you have a secure you have a performance problem it's always the database i know it's not always the database but i totally get what you're saying like there's definitely

cases where because you who else would know your code better than you would and so if you're thinking oh man if this is where the problem is this has to be where it's yeah well we just have such a often we have a bad conceptualization of what is fast and what is slow yeah it's i don't know computers are so fast it's a little bit crazy yeah but profiling is usually the first step to figure out what that answer is and then if you're going to change it you can see do they get better did it get worse where did it get better and worse and so on right so if you're doing to change it, you can see, did it get better? Did it get worse? Where did it get better and worse and so on? Right. So if you're doing Django, I want to talk about profiling requests there because there's a couple of interesting options. So there's one library that will let you plug in basically C profile, which is the low level C profiler from

Python that chips with it right into your request. So if you plug it in, you can just do a question or an ampersand prof or question mark prof at the end of the URL and you'll get a profiling output. Oh, nice. It's pretty cool. So there's a thing called Django-cprofile-middleware. You just install that. And then if it's installed, you can put basically prof

in the query string and you'll get that, which is great. And that gives you an output there or you can put ampersand download and it'll download you can visualize it graphically with snakeviz so that's even better so you get like this download profiling report for your request all that's pretty good but what would be a lot better is to be able to like visually dive into the request and do things like understand you talked about the database being slow like understanding like the time you're spending in the database versus other places so there's this thing called django silk okay and you also add that as a middleware but then it keeps track of all the requests on your site

hopefully you're doing this in development not production it keeps track of all of those and then it'll show you like this is the general time for this URL. Click on it. It'll show you things like these are the database queries that were run. Actually, as SQL, even though using ORM, it'll show you the SQL that was run. And it'll show you how much time the database is spending, how much time other parts of your code are spending. So, yeah, a really cool way to explore the performance from like all the tiers of your bingo app. Yeah, that's really cool. And I always find database interceptors to be a really powerful

concept, like across all languages, right? Like being able to see like, like, let's say I'm using SQL alchemy, or I'm using, you know, Mongo engine or something like that, right? Like, I'd really want to know what's the query that I'm sending over. Like, what does it think I'm asking for and versus what I wrote in code, right? And what am I actually getting back and being able to see, oh man, like this thing is doing like four or five joins. Like I don't need to do that. Like I could try and optimize this query a little bit more to one, make it more performant from a database perspective. And then two, obviously passing on that that performance over to my application or i thought i did one database request why are there 51 right right

exactly oh that lazy property that that we forgot about every single time through that loop we're hitting the database again like such a bad idea maybe we should do a subjoin like eager eager load sort of thing yeah for sure yeah. So I think this is really nice. Yeah. Yeah, it is really nice. And this is one of the scenarios too, where I think it's interesting to get like a log collector. So, you know, there's tons of services that do this,

but like something like Datadog or Rollbar or something like that, that not only capture your logs, but allow you to kind of go in and search through them, search and filter your logs. So you could say, okay, well, hey, I'm having performance problems, but it only happens at two o'clock in the afternoon, for some reason, or, you know, my memory always spikes within these hours of the day,

I'm being able to have a history of logs and be able to query them, and then get that rich information. And I don't know whether you plot it or turn into a report or something like that. Being able to do that, I think, is hugely important. Yeah, absolutely. I totally agree. And being able to visualize it, right? So often having the picture is what you need. Sometimes, though, it's good to just get a text.

Yeah, that's true. Sometimes it's good to just get a message or get like an alert. Hey, Michael, the website is going down. You need to get here ASAP. Exactly. And so that kind of leads into the next one. That was a good segue. And so that kind of leads into the next one. That was a good segue.

I like that. Thank you. So that can lead into the next topic I did want to talk about. At the time that we're recording this, this week is actually Microsoft Ignite, which is one of the conferences that we do. And it's all online. And we announce products and tons of stuff like that. Well, last time that we got to actually see each other in person and hang out a little bit was at Microsoft Ignite in Florida.

And I was blown away at how ginormous that conference is. Like you were on TalkPython then and Python. Right, right. That's 30,000 people. So I guess it's 30,000 video streams now. Yeah, I remember that last time we were all together. It was in Orlando and Dan Bader was there. I remember. Yeah, that's right. Yeah, episodes remember that last time we were all together, it was in Orlando and Dan Bader was there. I remember.

Yeah, that's right. Yeah. Episodes with Dan. I don't even know what the numbers are this year. That would be interesting to find out. I should look that up. Yeah, for sure. See like what those numbers kind of seem like.

So it's been a year and you guys have a new announcement. Yes, it's been a year. And so there's a new service. And so this is Azure communication services, which is cool. One, because, you know, we have a new service. But two, like I had no idea this was coming out. I kind of found out like everyone else did on social media. So this is Azure Communication Services.

And essentially, it's a way for you to add, well, communication services to your application. So it does a lot of cool things like it does voice, does video, it does text messaging. And you can obviously receive and send phone calls and things like that, which is really cool. But what's even better about this, more specifically for the folks of this show, is that there's a Python SDK on day one. I guess we could call it day zero since it's not officially out yet. But on today, in preview, while the service is in preview, there's a Python SDK, there's a JavaScript SDK, and there's also a.NET SDK. And I looked at the code for the Python SDK, and I almost want to say there's nothing there. I mean, obviously there's something there, right? But most of the code is create a directory, create a virtual environment, install pip, sms client, that send message. I'm

like, wait, that's it? Yeah, it's like literally the meaningful bit is like five lines of code. Right, exactly. And so it looks like a really easy thing to kind of get started with. And so I haven't played with it yet. Again, I just literally discovered this yesterday, kind of like everyone else did. So I'm really interested in plugging it into some of my demos and seeing what the experience is really like. Yeah, this looks really cool. If you want to add text integration, like text alerts to your app, seems like a really good choice. Right, and so then we could plug in. Imagine if we could plug that into the Django profiler,

and then as things are happening, it sends us text message alerts. And so now we can, uh-oh, there's a problem with the website now. That's right, it's running slow. The memory's too high. Go check it out. Yeah, that's awesome.

Nice one. People can check that out. I guess that's it for our items for the week quick shout out i just say i was on you mentioned dan beta i was on real pythons podcast with christopher the host there last week so i'll be sure to link to that it was really fun look back on the trends and stuff i've seen over the last five years of podcasting in the Python space. So if you want to hear about that chat, follow up on that show. And then how about you? Anything else you want to share? One of the things that our team does is they work with students from different

universities across the world. And they join a program that we have that's called the Microsoft Student Ambassador Program. Now, if you're familiar with like Microsoft MVPs or Docker captains or what is it called? GDEs, Google developer experts. It's something very similar to that, but it's just this time we're talking about students, right? And so, you know, these are students that are going out and they're creating content and doing sessions and workshops and, you know, writing blog posts and different types of things within their school and within their communities.

And so, you know, we have them join, you know, this program where we're able to support them a little more and give them Azure credits and Visual Studio things and LinkedIn learning things and some private events. Yeah, yeah. They have little private events and mentorship sessions and things like that. But, you know, it's a really great way for us to kind of engage with the students, right? And I really just understand, like, what are they doing? What do they care about?

Or what are some of the ways that we can affect their learning? And so what I wanted to do was share the link to the Microsoft Student Ambassador website. And you can actually kind of go here and you can search by region, by language, by area of focus, and you can kind of see what the students are doing, which I think is really great. And if anyone that's listening is a student as well, a university student that's interested in joining the program, you know, you go ahead and hit that big blue apply now button and, you know, go ahead and submit an application for the next, for the next cohort. I think that'd be a great opportunity for students. So people should check that out.

All right, let's finish this off with a joke. And we're better together than XKCD. I love XKCD. I know they got some good stuff. So this one is entitled Dependency. And I think it just captures the whole bring together different languages and technologies and especially a bunch of

open source libraries. Do you want to describe this picture for the listeners? You can always grab graphical stuff for our jokes. This reminds me of a game of Jenga. Like a super Jenga this is like recursive jenga yes that's what this looks like and so there's some big blocks and there's some little blocks and it looks like as the closer you get to the top they start to become a lot

more specific with like the type of blocks that are up there and then at the bottom it's much bigger at the bottom and you know that looks that looks like that could be there. And then at the bottom, it's much bigger at the bottom. And, you know, that looks, that looks like that could be trouble. Oh, and then on the right side, there's one little part that's holding everything up and it's really, really small and fragile. At the top, it says all modern digital infrastructure. And at the bottom does the little tiny stick is a project. Some random person in Nebraska has been thanklessly maintaining since 2003. That's amazing. I love this. In the hover over messages, someday image magic will finally break for good.

And we'll all have a long period of scrambling as we try to reassemble civilization from the rubble. To me, I think of this as open ssl that little stick that's open ssl down there there's like one person who maintained it and everything seemed to be built upon it and when there was that bug that needed to fix them it was a huge problem i find anytime i try to home so i use a mac and every time I homebrew install OpenSSL and it updates the version that's on my system, like everything just stops working. Like I

can't install any pip packages. Like nothing works. I'm like, what happened to my virtual environment? Like what's going on here? Why can't I install anything? Then I'm like, oh, OpenSSL. Yeah, exactly. That's the stick that can bring it all down. Well, thank you for being on the show, Cecil.

It was great to catch up with you. And you had a lot of good stuff to share. Hey, thank you for having me on. And Brian, I know you're not here, but I hope I did you service. And I hope you come back soon. And also, too, I didn't get to say this in the beginning. Congratulations on 200 episodes. Well, over 200 at this point. Remember, I listened to episode 199 and i'm like wow has it been like that many episodes already like it's crazy yeah that's

coming up on like four years almost it's insane wow that's awesome well congratulations to you both man that's a great achievement for for podcasting yeah thanks a bunch and uh take care take care thank you for listening to python bites follow the show on twitter via at python bites Take care. Take care.