Python Bytes - #56 The pendulum of time swings beautifully in PyPI

Episode Date: December 14, 2017

Topics covered in this episode: Pendulum for datetimes Flask asynchronous background tasks with Celery and Redis Building a Simple Web App With Bottle, SQLAlchemy, and the Twitter API Python extension for VSCode updated, now brought to you by Microsoft A Comprehensive Guide To Web Design Requestium Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/56

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 56, recorded December 12th, 2017. I'm Michael Kennedy. And I'm Brian Ocken. And we got a bunch of awesome stuff, like always, lined up for you. Before we get to it, Brian, let's just say thanks to Rollbar. Thanks, Rollbar. Yeah, thank you, Rollbar, for supporting the show. Tell you more about what they got going on later. I want to hear the TikTok of the pendulum, if you will.

I was reading an article called Dealing with Date Times Like a Pro in Python. Which can be way harder than it should be, right? Yes, and I think that there's things like Arrow and Maya that try to get so that, you know, I really have to deal with date times directly. But this is actually a nice article on dealing with date times

if you want to use the direct library. However, at the end of it, they mentioned Pendulum, which I hadn't run across before. So checking it out, there's a link here for the Pendulum Python datetimes made easy. And the website is beautiful. But also the interface to it, it looks really nice. I mean, you can do things like just import Pendulum and then Pendulum now in Europe, Paris, or in America, or something. And then you can do things like add dates together and do, you know, subtract dates. And the normal things you have to do with date times, but just the model looks nice. I like it.

Yeah, it looks really, really nice. And the website is quite stunning. the normal things you have to do with date times but the just the model looks nice i like it yeah it looks really really nice and the website is quite stunning and you know to me like on one hand like okay whatever it's documentation on the other it really shows the people working on this actually care about this project and it's not just some random thing thrown on github so yeah it's really cool so you say like pendulum.now and you give it a time zone say like europe slash paris and then you can subtract weeks from it. Or you can, one thing I like to do is put times relative to humans, like a week ago, five seconds ago in 10 minutes. And so it just has a diff for humans, you know,

that's really nice. I'm going to include in the show notes, links to Arrow and Maya. And I think it's, it's a matter of developer taste really and what looks more natural to you. So check them all out and see what's nicer for you. Yeah, it's really nice. The other one to throw in here while we're thinking about all this is parsing datetimes, which I think there's 700 different formats for datetimes and text, which is insane. But I really like to use Python dateutil. So pip install python-dateutil, but just import date util. And it compars like lots and lots of stuff. So you can just go here, load that into time, you know, and that's one more thing.

But this is really, really slick. It has more sort of time delta things like how many weeks, how many days, how many hours, as opposed to just like the seconds, total seconds, the built-in one. Really nice. I like it. Good find. So the next thing that I want to talk about, we're going to talk about some web stuff, aren't we? So the next one I want to do is, it's sort of a web thing, but it doesn't really have

to be. It's any kind of service or long running thing. So one of the things that can be a challenge with web applications is long, long running tasks or tasks that are slow, but you don't really need to like wait on their outcome. So I'll give you an example. On my training website, training.talkbython.fm, on that site, there's a capability to email everyone in a class. And it turns out, if you in the same request, actually email thousands of people, one at a time, like through Amazon SES or something, that request times out. And then you've emailed some people and you don't know how far you got. You've got to go figure out how to rewrite that some other way. Right? That's a problem. Yeah. So there's certain things like email, like logging, like other sort of kick this job off and forget

about it, that would be much better served in some sort of asynchronous way. One of the best things you can do to add scalability to your app is to push those kinds of things onto some background sort of get to it when you can style. And probably the most formal way is to use some kind of queuing. So this, the thing I want to talk about now is this article called flask asynchronous background tasks with celery and redis. Redis, although it has nothing really to do with Flask, although it just uses Flask as an example. It could be Django, it could be Pyramid. Like really it's the Celery bit that's interesting.

So you take Celery as the worker, right? It's going to run back there. It's going to look at the Redis message bus. And what you do is you just kick jobs off to it and it'll just start pulling them off and working on them. So what I could have done is said, oh, I want to email, you know, 3000 people. So go to the database, get the 3000 people and just put 3000 email this person jobs

onto a Celery queue and been done straight away. And it would just, you know, do it over the next minute or two. Actually, that's pretty cool. Yeah. I've got my, the wheels are turning in my head thinking things that I want to use Celery for. I know. I feel the same way. It's really nice write up of of how you walk through these things. So it talks about using the Celery

client to connect Flask to Celery, the Celery worker process, the message broker that works with the Redis backend. And the other thing that's interesting about this is so often these Python examples, these especially Python web examples are done on Mac OS or Linux, and this is all on Windows. Oh, wow. And I don't think Redis is even supported on Windows or Celery, one of them. I think it's Redis. There's like an unofficial build or something.

So I think it's kind of extra interesting to show this other angle and show people how to do it where it might not be so obvious. Also, something like taking on Celery, it's kind of nice to look at an example project to see how it's been used, because just jumping right into the entry of the documentation can be a little scary sometimes. Here's a nice little simple example. Take it and do what you will with it. Yeah, well, we have another web example that I liked, and this is from Bob Belderboss of PyBytes. He wrote a guest article on RealPython called Building a Simple Web App

with Bottle, SQL Alchemy, and the Twitter API. And the reason why this caught my attention is because I was curious about having some way to interact with Twitter and Python. So I think I'm going to walk through this a little bit. And from start to finish, it's a project that pulls, it isn't pushing things into Twitter, it's reading Twitter and then displaying them on a little bottle website that they pushed up to Heroku. And it's kind of cool that they document everything from start to finish for this. Yeah, it has the deployment bit as well, like pushing it to Heroku and making it work there, which is also kind of cool because your web apps are fun, but they belong on the web, not your laptop. It's on my celery queue of things to learn because I'm busy learning Flask at the time, but maybe I'll check this out next.

Yeah, that sounds really interesting. Definitely. Nice job, Bob. Now, before we get on to talking about something that's actually not on the web, but has to do with Python, let me just tell you a quick story of yesterday. Yesterday, I decided, let me see how far I can push the performance of MicroWhiskey and Nginx on the TalkPython sites. Luckily for you, Brian, I didn't do this on PythonBytes.fm. But I'm like, all right, I think I can push the parallelism out better.

I could crank up the number of threads. I can crack open the process affinity, processor affinity. So, like, get the maximum parallelization out of MicroWhiskey, which is one of the better sort of application tier worker process web servers for Python. Okay? So, that seemed like a good idea, right? Yeah. Turns out something about the threading was bad and I don't know what it is. I don't really care to look into it, but I started getting roll bar notifications saying that the template files could not be found for the website, which has been basically largely unchanged for months.

And then it couldn't convert this template to a string and all sorts of random errors. And I'm like, what is going on? And there's some kind of corruption timing problem in this whole series. And it happened like one out of a hundred thousand times. I mean, really, really rare, but it started coming in like a couple of times yesterday. So I'm like, all right, well, I guess we're going to turn that back down and turn it back

down. Errors went away. And without roll bar, I never would have known it. So if you guys want to make sure that your apps are working, go check out roll bar, just pythonbytes.fm slash roll bar and you sign up and you too may start getting weird messages that you need to go look into it'd be awesome yeah nice yeah so that was yesterday but today i want to move on and talk about something that actually covered way back on talk python let's say about a 40

episodes ago so not quite a year 40 weeks ago And that was the Visual Studio Code extension written by a guy named Don Jayamana. Jayamana, sorry about that. I'm sure I'm messing up your name. And he wrote a really cool Python extension to basically add Python capabilities to VS Code. And it turned out to be the first or second most popular thing in Visual Studio Code. It had like millions of downloads. That's pretty cool, right? That's very cool, yeah. Yeah, and so one of the things I looked at, you know, when we spoke, it's pretty interesting.

He took, I'd say maybe 10 different packages, Jedi and other, you know, Flake 8 and those types of things. And he basically wove those together into an extension for VS Code. He didn't really build a huge amount, but he definitely put it together in a really, really usable way. So that was pretty cool. So today, this week, the news is that he is now officially hired by Microsoft because of his open source project. That's nice. Really cool. Yeah. So here's a guy who started an open source project and got this pretty influential job just

because he had a really successful plugin for this for this open source thing so pretty pretty cool you know it doesn't really mean many changes just there's a few more people working on the Python extension for VS code which if you're into VS code it's a lot like Atom or sublime it's quite quite nice if that's your style editor it does a lot of cool stuff and so basically there'll be more people working on it, fewer bugs, faster turnaround. The other big news is they're also hiring for Visual Studio Code Python developers.

So if you're looking for a job and that sounds pretty awesome, there's some really cool guys over there. There's Brett Cannon, there's Steve Dower, some of the big hitters on the Python core dev team in addition to Don. So that might be worth looking into if that interests you.

Nice. So I feel like you're kind of doing this web crash course, getting into a lot of different things on the web, right? I'm trying to embrace the web a little bit more. Nice. So what'd you find? Yeah. So one of the things with, we talk about a lot of mechanics of how to get websites up with Flask and Bottle and Django and all that. But sometimes the design of your website, as it turns out, at the end leaves a little bit lacking. It looks like a developer designed it. Yeah, it definitely looks like a developer designed it, which is usually better than when it looks like a database engineer designed it. But still, there's an article called a comprehensive, it's on Smashing Magazine,

called A Comprehensive Guide to Web Design. It's a long article, but it's all one page. And it's a nice, I think a good overview of basically the basics of make sure you have this stuff down and to make sure your website doesn't suck too bad. That's really cool. I feel like design is one of those things that feels like a barrier to developers. You're like, I'm not a designer. I can't do stuff that's pretty. I can't build these types of things. But if you put in just a little effort, you can turn that barrier into

a superpower because most people don't have it. And it's actually not that hard if you just sit down and focus and try to study it for a month or something. Especially a lot of times, there's a lot of projects that get done, like internal work projects or different tools for a small group of people that you don't have the money to hire a front-end designer. So something like this to go through and make sure that you do your breadcrumbs appropriately and you put your search box in a findable place. And some of these page structure overviews that are good to just make it more usable. And this doesn't add a lot of work to your workflow. It actually, I think it takes away some of the stress of designing a webpage if you're not a designer. Yeah, it's cool. It gives you a

framework to think through and stuff you got to do anyway. That's cool. All right. Final item. What would you name the baby of requests and Selenium coming together to create some new library? Well, I know the answer, but... Requestium. Maybe one of the candidates you throw out there, right? Yeah. So the thing I want to talk about is a thing called Requestium. And we have requests, which everybody knows, super, super popular library for just downloading html and we have selenium which fewer people know but it's basically a headless chrome browser there's other options

as well but let's say chrome for now and you control it from python and so use them in different situations requests is super lightweight takes takes no very, very few dependencies. I mean, it doesn't depend on say Chrome, it just depends on a few packages. And you can download the HTML and do what you will with it. But if you happen to run into, let's say, an AngularJS, or VueJS, or any other JS site that has like a front end JavaScript framework that basically requires the JavaScript to execute to get the content, you end up with just a bunch of curly braces that have no meaning when you hit it with requests, right? Oh, I didn't know that. Yeah, you just get the HTML that's basically, here's where JavaScript will fill out the holes. Good luck with that. I mean, and so that's kind

of useless. So you might use Selenium. But you know, Selenium is a way more heavyweight. So that Requestium basically merges the power of request, Selenium, and this thing called parcel, which is like Beautiful Soup or Scrapey. It understands HTML and lets you ask questions of it. So the idea is that it's written for mostly when you're doing requests, but that one time or two that you're like,

oh, you need this login sequence has to happen in a richer JavaScript way. And it lets you do crazy things like automatically switch between request sessions and Selenium sessions. And it lets you basically, I think, share cookies across them and all sorts of interesting stuff like that. So if you need like a little bit heavier weight than request, but not full on Selenium, here's a nice combo sort of marriage of the two. Oh, that's nice. Cool. Yeah.

So very good for either testing or web scraping or some other types of things. But if you need to test something that's JavaScript heavy, this is a pretty good option here. Nice. All right. Well, that's our news, Brian. At least the ones we've selected for everyone else. But how about yourself? What do you got going on?

Well, I finally got a testing code episode out not too long ago. I don't know if I brought it up already. I don't think so. Not here. It was a nice interview with Catherine Jarmal or K-Jam about testing and data science. And it's a testing code episode 33. And it's been very well received so far. So a lot of people get some value out of it.

Yeah, that's great. Catherine's done a bunch of cool work. So definitely go over there and check it out. And the last thing I wanted to mention was I had asked people to leave Amazon reviews for books that they like, including mine, hopefully. And so far, nine people have left reviews on Python testing with PyTest. So thanks a lot. Yeah, that's great. How about you? Any news? Not a whole lot of news. I'm working hard, but I'm getting things ready to talk about. Nothing to talk about yet, but someday soon I'll, I guess the big, big thing I have coming up, I just recorded some nice TalkPython episodes, one actually on optimizing web servers.

And then later this week, I'm recording one, which is a panel session on machine learning and artificial intelligence at the large Hadron Collider and in particle physics. Oh, that sounds interesting. And the last few episodes that you've put out, I mean, you always have good episodes, but I've really enjoyed the last few quite a bit. Thanks a bunch. Yeah, same for you.

All right, man. Well, thank you for sharing this stuff. And everyone, thank you for listening. Thank you. Thank you for listening to Python Bytes. Follow the show on twitter via at python bytes that's python bytes as in b-y-t-e-s and get the full show notes at pythonbytes.fm if you have a news item you want featured just visit pythonbytes.fm and send it our way we're always on the lookout for sharing something cool on behalf of myself and brian ocken this is michael kennedy thank you for listening and sharing this podcast

with your friends and colleagues