Python Bytes - #102 Structure of a Flask Project

Episode Date: October 31, 2018

Topics covered in this episode: QuantEcon Structure of a Flask Project Overusing lambda expressions in Python Asyncio in Python 3.7 * Giving thanks with* **pip thank** Getting Started With Testing in Python Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/102

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 102, recorded October 30th, 2018. I'm Michael Kennedy. And I'm Brian Ocken. Hey Brian, how you doing? I'm doing great. Are you frightened to be recording this on the day before Halloween? Or is it all okay?

Yeah, it's okay. It might release on Halloween, but I think it's still going to be okay. Yeah, that'd be neat. Actually, it would be cool. You know what is neat is that DigitalOcean is sponsoring the show like they are all of our shows for the rest of the year. So check them out at pythonbytes.fm slash DigitalOcean. You'll get $100 credit if you're a new user there. We'll come back and tell you more about that later, some cool stuff they got going on.

Brian, what's the first cool thing that you found? Quanticon, or Quanticon, however you say that. Didn't we cover like economics? Somebody getting a Nobel Prize in economics. Was that last episode? That was last episode, yeah. The guy who won the Nobel Prize, one of the guys who won the Nobel Prize in economics is all about Jupiter. And is very much an advocate for it.

Yeah, and we did a shout out then saying, hey, if anybody knows other economic stuff, we'd like to hear about it. And somebody did. Somebody let us know. I really should have wrote their name down, but thank you. Gave us a pointer to this site called Quanticon, and it's quanticon.org. It says open source code for economic modeling. Quanticon is a NEM-focused, fiscally-sponsored project dedicated to developing documentation for modern open-source computational tools

for economics, econometrics, econometrics? I don't know. And decision-making. Anyway, it's an educational resource for people wanting to do economics and stuff like that with Python and other programs. I guess it does Julia also.

But it's got a bunch of lectures and a bunch of workshops and seminars that are taped and upcoming ones, and then some Jupyter notebooks and some cheat sheets. This is really cool what you found here. If I cared about economics, this would be a massively cool resource. Yeah, you could just sit here all weekend and learn all sorts of stuff on this. It's pretty great.

Yeah, it says fiscally responsible. And looking at the bottom, it's even got Sloan Foundation support. So that's pretty cool. There's a whole bunch of stuff here. Yeah, it's great. It's all part of the NumFocus thing, which is really cool. Like NumFocus is sort of the umbrella that sponsors things like NumPy and stuff. And this is one of the areas of open source where, like, legitimate money is going to these projects.

Remember we covered a while back, like, multi-million, like $3 million going to the SciPy, NumPy world? That's really, really great. It'd be cool to see that in some other areas of Python as well, but it's great to see it here. Yeah, it's neat. And, yeah, maybe I'll check it out also. Yeah, it says there's a, it's basically a high-performance library. It comes in the form of Jupyter Notebooks if you want it, and it has a whole series of lectures.

And what's cool is you can pick the Python version, the Julia version, and you just get started watching the lectures. It's pretty great. And it has an area for if you want to contribute to it where they'd like some help and yeah it's good yeah so that's a good find i like it i'm sure people will enjoy having it so i think we've talked about how you structure packages in python a lot it's kind of one of our themes right yeah so this next one that i want to cover is about structuring Flask projects

and also treating them as packages as well, which is kind of interesting. So I think Flask is one of those things where it falls over itself to show you how simple it is, and then it doesn't have much opinionated ways of sort of telling you, now this and that right so you're kind of left to your own devices to figure out how to structure your flask program so there's a cool article called structure a flask project and it says you know look it seems really easy to get started but when you're new like as your app grows you're going to find you start running the challenges with circular references and other things like that. So he actually, the person who wrote this, has a couple of different structures. And you can take the ones that feel best to you and run with it.

So one of it is to have, like, your project and then have a models folder subpackage. They call them routes. I think it should probably be called views or controllers or something. But routes, templates, services, things like that. So like if you're going to have some code that sends mail for you, put that in a separate thing in a services folder. Don't cram that all into just one giant file, which kind of flask encourages. So I really, really love it. And I think it's nice. What do you think? I do. I like it. And actually, when you structure things in packages, then you have the dot notation in your code.

It separates the different parts of your system a little bit better. Yeah, absolutely. The one thing I would add here is some kind of concept of a view model, which is a class whose job it is to exchange the data between the HTML template and the view or controller method and just do all that validation instead of, you know, trying to cram that into your view, right? Usually the validation is more code than the operation you're trying to do with the data anyway. So yeah, I'll throw that one in there. And those are easier to test, right? You test your validation separate. Yeah. So I threw this little question for you. I see you already caught it quickly. What's missing from this structure? Tests. Yeah, there's no tests in this layout. I mean, there is a dot, dot, dot

somewhere. So maybe in the dot, dot, dot there's some tests, but probably not. Also, this is sort of a zoom in on the project. And in a lot of projects that I've seen that are Flask applications, there's a test directory like at the top level. So it might be parallel to this top level project directory. Right, right, right. Cool. Yeah, there's another structure that they propose as an alternative called the app-based structure. And it takes all the models, the view methods,

the templates and everything and puts it under the different categories. So maybe you've got a blog sub part of your app, you've got authentication, you've got a store, and it sort of replicates this structure underneath all of those. I don't know how I feel about that. I kind of like the first one better personally. But it's more Django-like.

There's a couple of options here, which I thought was a nice tradeoff to show how different people were doing it. And then also, I would like to just give a shout out to Flask Blueprints, which is part of Flask. There's nothing like extra. It's just built in. But it's kind of challenging the way Flask works to easily and clearly share or spread your view code across different files so it's not into one giant file.

And Flask Blueprints is really a great way to break one big file with all your view methods up into a whole bunch of little ones that are grouped by what they do. They're easier to test because they're more focused things like that and is this the blueprint something you could use with either any of these project structures yeah i think so okay it just lets you take one big file instead of trying to figure out how to you know in flask you create the app dynamically at runtime, and you say app.route. That makes it tricky to share that thing without being circular or things like that.

So this lets you do the reverse. You create a blueprint for each of your separate files, and you say blueprint.route, and then you just register the blueprints with the app and the main startup, and you're all good. Oh, nice. It's a really nice thing. I haven't seen it used a lot, but it's great.

I learned it from the court guys. All right. So anyway, I think that puts a nice structure on the Flask world and people should check that out if they're doing Flask. What you got next for us? I've got an article from Trey Hunter that's overusing Lambda expressions in Python. And I actually kind of love lambda expressions. However, I've so where I often use them is in like ID functions for test cases. So if I've got a test parameterization, you can rewrite the ID function to take a all the parameter data and spit out an easier to read, easier to read string. And this this is something that where something where lambda expressions are often used. However, I do have some of them that are kind of getting big and they're hard to read.

So this is a good time to read Trey's article. So it starts out talking about some of the differences between lambda expressions and just defined functions. And some of the things I guess I didn't realize, I mean, it's obvious if you think about it, but I didn't really think about it, is that one of the things is there's no doc string. You can't have a doc string for a Lambda expression. And the representation also is a little weird. So if you're stepping around or if you assign a Lambda expression to a variable, if it gets passed in somewhere, the value of it is going to be something ugly that you can't, if you're debugging, it's going to look bad.

And you might have trouble figuring out where that came from. Whereas if you just define a function, it's easier to figure out. So basically, this is an argument that it doesn't really save you a lot in a lot of places and perhaps even just use normal functions. Then the other thing is he talks about some misuses. And some of them are, I've seen this before, is having a lambda expression calling a single function with a single argument. Well, that function that you're calling, you could just pass that instead of the lambda expression that you're passing. Right. Like so, for example, if you have a sort function and you'd

like to take the number and sort by the absolute value of it. You could have a lambda that takes the n and it says returns a b a b s of n or you could just pass a b s. Right? And this is basically the same thing. You're just putting a layer of function calls in between them, make them slower.

And then actually, I didn't know about this module before, so I'm glad I read this. Defining little custom lambda expressions for operators that happen to exist in the operator module. There's an operator module that comes with Python, and it has a whole bunch of things that you might have wanted to create a Lambda expression for. Yeah, this is interesting. I didn't know about this either, but you can say like operator.add, operator.truth, and so on. Yeah, these are supposed to be high-performance, little efficient functions. Quite cool. The last thing is I've seen this before also is I used to be using Lambda expressions for map and filter,

but just useless comprehensions instead nor other kinds of comprehensions but yeah that's a good one certainly the comprehensions set comprehension dictionary comprehensions all that kind of stuff is better than a lambda function generally right yeah it's more readable i think that might even be more efficient in terms of performance i'm not not 100% sure, but I think so. But anyway, these are all good things. Yeah, and I think there's a lot of good guidance here. For example, it says you can't have a doc string. Well, if your Lambda expression requires a doc string,

you're probably doing it wrong, right? It's probably too complicated. The value of the Lambda is I'm going to take such ridiculously simple code instead of moving it way far somewhere else into a function that then i have to document so i know what the heck it does i could just put it right here right like you know abs of a number type of thing i do think i personally i personally love lambda expressions i feel like trey's a little harsh on them i find that i more often see code that should be using the Lambda than that is over using Lambdas. So grain of salt there, but I think there's a lot of good advice too.

Yeah. I guess there's sometimes where you see a little tiny function that is only used once and it's small and maybe, maybe it could have just been a Lambda. Right. It's probably badly named and not documented either. So yeah. Doc string so yeah doc string what doc string exactly exactly so anyway yeah people check it out it's uh it's a good article it's kind of a long one it really is uh in depth we've got a couple of uh good solid long articles i cover this week now before we get on to another one that i think is really interesting because it introduces a lot of structure around features introduced two versions ago, two dot versions. So we'll get to that. But before we do, I just

want to tell everyone about DigitalOcean. If you're not familiar with them, check them out at pythonbystuffm.com. They've got all sorts of cool stuff going on. One of the things that I want to highlight this time that's cool, I've talked about a few times before, is the ability to bring your own custom images. So you can go to DigitalOcean and create a droplet. That's what they call their virtual machines. And say I want Ubuntu 18 or CoreOS or whatever, and you get those stock images. They even have little pre-built apps like give me a MongoDB server or give me a Ghost server, things like that.

But if you want exactly the machine that you'd like to run there, and you want to create that locally, you can upload your own images. All right. And that's super easy. Once you've registered them and upload them, you can just click start my own image. And off it goes. It's really cool. So check them out. They're doing us well. And I'm sure they'll do you well as well. So if you wrote some code, Brian, and it used the word async, all lowercase, and that was working just fine in Python 3.4, and then Python 3.5 comes along and it introduces two new keywords, async and await. Do you think there's going to be some problem with your 3.5 code?

Yeah, probably. Well, there should be, right? Like there's a new keyword and you can't use the word class. You can't use other core keywords, right? You can't use def as like a variable name. But it turns out for whatever reason, I don't know really why they chose this,

but async and await were valid like parameter names and stuff in Python 3.5 and 3.6, but they and stuff in Python 3.5 and 3.6, but they're not in Python 3.7. Okay. So if you've got some sort of package and it uses the word async for a variable or a parameter or something like that, you're all of a sudden going to have your package or the one you're using no longer running.

That's big, right? Yeah. That's not good. So there's actually, I saw this article. This is from Chris Medina called AsyncIO in Python 3.7. And I thought, oh, okay, well, it's going to talk about this keyword thing. Maybe a little bit something else. Oh, no.

This is like a series. There are so many changes to AsyncIO and async and await in Python 3.7. Like pages and pages of changes. So there's really quite a bit of new stuff. And if you cared all about this, you should go check it out. And you might not care at all about it, but you still maybe should check it out, at least for the fact that parameters called async and parameters called await no longer work. And if you have a library like that, it won't run on 3.7.

Yeah, it's definitely concerning. Yeah. So I kind of feel like this maybe should have been the way it was done when 3.5. As soon as those keywords appear in the language, they should be officially keywords. But anyway, here we are three years later. Now async and await are enforced as keywords. They're reserved words

okay so i definitely want to we definitely want to heavily test your applications as you shift to three seven yeah maybe using something like docs or something like that right test across the different versions yeah oh yeah that's a great idea so there's a couple other things i'll go through really quick uh here like i said check check out the article. There's tons there. One of the challenges with asyncIO is it simulates parallelism by slicing up all these different methods and running the other ones while one of them is waiting. But what's interesting is you can no longer have thread local variables because even though it feels like you're doing concurrency,

it's actually all the same thread. So there's no mechanism to distinguish across threads right with that that's sort of how flask works for example they've got the request as a a thread local variable but it doesn't work in async io so there's this new concept called context variables that are like thread local variables but for co-routines so they can have different values on the same thread, which is kind of crazy. There's a bunch of stuff around how that works. There's a new asyncio.run function.

You just have to get a hold of loop and call that. Simpler task management, simpler event loop management. You can call asyncio.getrunningloop from within the library to see if you're already being run asynchronously or if you have to kick that stuff off. A cool decorator to turn async functions into async context managers. Remember, context managers are things you can put in a with statement, right? Like with open file name as fin, right? There's a new part of the language that'll let you say async with so it will like

allow concurrency and you sort of do waiting type of work on those context manager enters now there's a really cool decorator you just add to your function and it automatically implements this asyncable async context manager, which is pretty awesome because that sounds complicated, doesn't it? Yeah. And then there's a bunch of performance improvements. AsyncIO event loop is 15 times faster. That's good. Gather is 15%.

Sleep is twice as fast for not really sleeping. Future is faster. All sorts of good stuff. There's tons and tons here, so go check out Chris's article. I'm just blown away at how much new stuff Async.io got in 3.7 actually when you see it all put together

like that. Yeah, wow, that's great. I like seeing it all together like that. Yeah, it's like, wow, there's still a lot of stuff happening here and that's good. I feel like we should say, you know, thanks to all the people that work on it. That'd be awesome. Yeah, I think we should. Would like with an email or how do you propose this?

Well, there's a proposal in PIP. So this is a proposal from Brian Skin. Brian Skin, yep. Yep. And he sent us a heads up on this that he submitted just this morning. And it's a proposal to add a sub command thank to pip so that so you got pip install yeah pip uninstall these and so another one would be like pip thank requests or pip thanks equal alchemy something like that yeah

um there's this other proposal is to add that and then also to change install. So the install, if you install a bunch of stuff at the end after everything's installed, if there's the right metadata in the modules that you've installed, add a little information that says, hey, if you'd like to thank the contributors, run pip thank on pip thank requests or pip thank something. And the idea is to look in the metadata for the project for URLs, specific like thanks tag in the metadata or donate possibly.

It's still kind of in a little bit in flux as to what would be in there. But there's already some information. Apparently it's all the info's there. A lot of it is. And there's a way to possibly just print that out and say, hey, if you want to say thanks, this is how you can do it. I think it's a simple thing. It's not going to be very intrusive.

And I think it's cool. I think it's cool. There's a lot of conversation happening on GitHub around this. I like it. Well done, Brian, for coming up with this idea and actually putting it out there as a proposal. Yeah. And I don't know if it'll happen, but why not? There's so much negativity in the world. A little bit of positivity is a good thing. Yeah, I agree. Yeah, quite cool. So, I feel like I kind of snuck this last item from you because this one belongs in

your realm, but I grabbed it. So So this one is an article, a booklet. Should I call it a booklet? I don't know. I'll tell you why in a second. On real Python called Getting Started with Testing in Python by Anthony Shaw, friend of the show. So this is not just a little quick tutorial on getting started with testing. This is like a little bit of a mini book type of thing.

I threw it into Instapaper and Pocket, and they both said it's a 33-minute read. So that's a serious little write-up. Yeah. Yeah, it's an interesting read. I heard about this before it went through, and it's sort of one of the things that highlights is some of the process around real Python articles.

They treat it like a magazine, and there's a review process and stuff. So that's pretty cool. Yeah. Dan Bader and crew over there definitely creating some serious articles and things over, over at real Python. And so this is Anthony Shaw's. I'm just going to touch on some of the topics covered in there because like I

said, 33 minutes, it's like not, I can't really go into too much detail, but it covers things like automated versus manual testing. It's really for people getting started, I think, but it's pretty interesting for everyone. Unit test versus integration tests. It does a comparison of unit tests, NOS and NOS2, why there's actually two NOSes, and PyTest and which one you should use. Do you have an opinion on that, Brian? You care about which one you use? Yeah, use PyTest.

It also talks about things like writing your first test, where to write it, how to structure tests, how to write assertions, the dangers of side effects, testing in PyCharm and VS Code. PyCharm is pretty straightforward. VS Code has that command palette thing, which is cool, but you've got to figure out how to do it. So that's kind of nice. It also talks about testing web frameworks like Django and Flask, advanced testing scenarios, and even testing for security flaws in your app. So there's a bunch of cool stuff here. Like the testing security flaws, like that's new to me. So yeah, I don't think it's really that much of a beginner article. It just starts at the beginning. Well, okay. Yeah. But it's a lot of these, I like that he touches on a lot of this stuff, but a lot of these things are touches on them.

There's not, like testing some of the web applications, it's kind of a pointer to, yes, you can do it, and go read about it in other places. But I do like it. And like you said, the security flaws, that needs highlighted more, and I think that's a great thing to add to people's tool belt. Yeah, just to know that, hey, this is the way you can test for it, right? There's even tools and whatnot.

So are you telling me that this could be a book? Yeah, I think it could, but I think it would be a lot bigger book. Yeah, somebody should write it. I'm just kidding. People should check out your book if they want more than 33 minutes worth. I'd also like to add, so it does talk about PyCharm and VS Code and it's a little bit, I don't see a lot of opinion,

but the testing story in PyCharm right now is a lot stronger and I'm not just saying that because I like PyCharm. It's just true. The VS Code team has something on there. I think they're going to address some of the testing shortfalls

in the upcoming years and I hope they're going to address some of the testing shortfalls in the upcoming years, and I hope they do. But for right now, if you want to test within an IDE, do it in PyCharm. Yeah, it's definitely nice in PyCharm. Awesome. All right, so those are our main items.

I have a few other things I want to throw in here because I don't feel like they really warrant a whole thing to be covered, but they're fun. Anything you want to cover first, Brian? I will just say that speaking of testing, I'm ramping up testing code episodes and doing them actually more frequent than one a week. Not up to two a week, but I'm going to get a lot of episodes out.

And there's a couple of recent ones on flaky tests. Yeah, speaking of Anthony Shaw. Yeah, Anthony Shaw's there talking to me with flaky tests. And then I talked with the Automation Panda about feature tests. Yeah, that's really cool. Great to hear you doing more. I've seen those coming out. All right, I have a few that I would like to throw out here,

a few quick, simple ones. First is just this fun project from Vicky Boykus. See where it says hack your name in our notes, Brian? Click that. So the idea here is that you've got a startup. You're trying to come up with a name. And it seems like there's all these silly names that are often talked about on Hacker News, which is from y combinator so why not just take the words from hacker news break them into syllables and generate new words which can be the the name of your new cool startup and so there's you go to this url hack your name you are name.com and it has just one big button

and it says pivot me bro so i get guised back let see. If you click it a few times, you'll get some pretty good ones. I got your chair. Yeah. I had shy Fox the other day. I'm pretty happy with shy Fox. So why is this interesting? I mean, it's funny obviously, but it also is a open source project that is written in Python that does all the download from hacker news and does all the work in words and stuff. So it's kind of cool. You can check it out. I'll link to the GitHub repo as well. Oh, so it's a good example project.

Yeah, exactly. It's a pretty simple one, but it's pretty fun. So if you are doing a startup, like seriously, click that thing like 20 times. You'll probably find your name right there. All right. The other one, just some quick news. Python 3.7.1 and 3.6.7. So modern Python, last two versions,

got a new release, and there's actually a decent amount of little bug fixes and stuff, at least in 3.7.1. So check that out. Links there for that. Not a lot to say more about that one. And then the last one is, we were talking with Tom Baker on Twitter

about a project that's a command line interface app that he's converted over to Python. And he used Qlik. And he wanted to propose a new acronym. We've got TDD, right, test driven development, we've got BDD for behavior driven development, he wanted to propose CDD for Qlik driven development. So use the Python Q click package to mock up your suite of commands. You put little print functions in there to show what's supposed to be happening. And then you just start filling out the placeholders,

take away the print functions and make it start working. And so there you have CDD. Sure. Yeah. Why not? Right? Yeah. We always have a shortage of acronyms in the programming world,

so why not? Yeah. I think CDD may have been taken already, but I don't remember what it was for. Yeah, well, I'm not sure how well it's going to actually take off, but there it is. CLDD, I don't know. I don't know.

I do love CLIC. CLIC is cool. Yeah, it's pretty cool. All right. Well, that's all of our news for this week. Brian, thanks for being here and sharing with everyone. Yeah, thank you. Yep. Bye.

Bye. Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at PythonBytes.fm. If you have a news item you want featured, just visit PythonBytes.fm and send it our way. We're always on the lookout for sharing something cool.

On behalf of myself and Brian Auchin, this is Michael Kennedy. at pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Ocken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.