The Changelog: Software Development, Open Source - Python, Django, and Channels (Interview)

Episode Date: November 25, 2016

Django core contributor Andrew Godwin joins the show to tell us all about Python and Django. If you've ever wondered why people love Python, what Django's virtues are as a web framework, or how Django Channels measure up to Phoenix's Channels and Rails' Action Cable, this is the show for you. Also: Andrew's take on funding and sustaining open source efforts.

Bandwidth for Changelog is provided by Fastly. Learn more at Fastly.com. I'm Andrew Godwin, and you're listening to The Changelog. Welcome back, everyone. This is The Changelog, and I'm your host, Adam Stachowiak. This is episode 229. And today, Jared and I are talking about Django and Python with Andrew Godwin. Andrew is known for raising money for Django features. We talked about

sustainability, a lot of fun stuff around how he's funded open source. We talked about Python, what's new in Python, how to get into Python. We also talked about Django channels, what they are, how they work, what the status of the project is, and how they compare against Action Cable or Phoenix Channels. We have three sponsors today, Heap Analytics, GoCD, and also our friends at TopTile. First sponsor of the show is Heap Analytics. Check out heapanalytics.com slash changelog. Heap automates away the annoying parts of user analytics.

No manual event tracking, no messy tracking plans, no custom ETL pipelines, just insights for everyone on your team. And I talked with one of their customers, Alan DeSalsa, the director of product analytics for LendingClub.com. Alan shared some insights into the value they get from Heat. Take a listen. Very quickly, we realized that we were not good at anticipating every single question that would come up. We could get questions from different departments about how many people are looking at this agreement on this page or how many people clicked on this particular link in a seven-day period.

You know, 99% of the time, that link actually has no value until that discrete question comes up. And those moments are tough, right right so we really wanted something that addressed that okay so here you are you've got heap installed very low burden on your engineering team to get in place you don't have to go through and individually track events or taps or predict the future basically but you've got all this data in heap and obviously we're all data driven talk to me about how you get value from all this data being in heat. Data is really hot. Everyone's talking about it. People are creating tons and tons of data. Every tool lets you export raw data and do this and do that. People have a ton of data, but that doesn't necessarily mean that you're getting anything valuable out of it or you're able to answer

questions or inform your decision making process using data and incorporate it into day-to-day things. And that's not valuable. What is valuable is the tool that might have large amounts of data in the background, makes it seamless for you to phrase a question and get an answer to that question in real time. What you really need is the tool that is able to work with you and create value out of all this data that might be there in the backend. I mean, Keep is the only tool that I've encountered that lets everyone instantly answer

business questions. Very cool. Thanks, Ellen, for speaking with me. For the listeners out there who want to check out Heap, go to heapanalytics.com slash changelog. Do not Google it. Go to that URL. It's the URL we're getting credit for, heapapAnalytics.com. Now on to the show. All right, we're back. We've got a fun show lined up. It's Python Django Day, Jared. What do you think about that? J-J-J-Django.

I just can't think. I just think about Django Unchained every single time. Every single time. I don't know if it's been helpful or hurtful to the brand of Django. What do you think? Of course, surely the Django project predates Django the movie. Yeah. But not in my mind, it doesn't.

That's for sure. We can't talk much further, though. We've got to introduce our guest, Andrew Godwin. Hello. That's right. Andrew Godwin, Thanks for joining us. No problem. It's my pleasure to be here.

Jared, you should open up with that thing you said your friend said. What was that about? So the first person that ever introduced me to Django was a fella that pronounced it Dajongo. And when he said it, it's like one of those double takes, like, what would you say? You know? And he's like, the John go.

And he was the first person that had ever, you know, said the word to me. And he's talking about the framework. And I was like, it didn't sound right, but I wasn't quite sure. And so I was like,

okay. I kind of rolled with it for a little while. So in my head, I was reading it to John go, but I had a feeling that wasn't it. Um, and kind of just waiting for somebody else to confirm or deny and then didn't take very long i have never heard anybody else in the history of my life say it like that ever again but would it be somebody else's life i guess it would have been so andrew what do you think about that how do you say it you don't say the jango do you i i definitely

say jango i i have heard the jango before um one of the jokes when i joined the community was that a good number of people said to jango um and there was actually a talk one of the first jango cons about why by a linguist about why it's actually jango not to jango but yeah it's definitely more common than than you might think the jango well see i think jerry's is different though because the jango is fine and that's that's okay well yeah but the jango well not if you're a linguist it's far-fetched i would disagree with the jango as well but what does the word even mean i've never even looked it up i i like the origins of it it comes from jango reinhardt who is a famous gypsy jazz guitarist um and oneist. One of the founders of the framework was a big fan of his,

and so that's where the name came from. It's just a name, basically. For some reason, I think about a kangaroo. Is that a different thing? Not that I'm aware of, no. It might be a dingo. Yeah, you've got the two films as well, Django and Django Unchained. Oh, Jared.

Dingo is a wild dog from Australia. I'm just going to be quiet for a while. Let you guys talk. Dingos are great. So in the interest of getting to good content, Andrew, take us back. We like to get kind of a history of our guests. And obviously we're going to dive deep into Django, not the Dj or django or whatever else you want to say it might be take us back into your history where did you get in your start in programming or even open source well so my my start in programming is a little bit unusual um i started programming on a palm pilot 3x i think it

was wow um so my mother had a palm Pilot and it had a basic sort of, had a basic interpreter on it. And so I was there on holiday just like writing basic games with a stylus on a touchscreen, which is the suboptimal way to program in any situation. And that's, I sort of got my start there,

like writing like basic, like music, keyboard stuff and basic games. And then from there, I kind of branched out into some web stuff. So I've been doing a little bit of HTML around the edges of there and like going to my local library because you know this i was like i say 14 15 at the time so we had dial-up internet but the library had

proper internet so i'd go down the road to the library i'd use their computers do some websites and that suddenly sort of morphed into well how do i add javascript to the website how do i add some php and then i started doing php development and my very first open source project um which i think is about 2005 2006 was one called byte horde which is this php interface for uploading and downloading files it was very simplistic had several rather bad security vulnerabilities in it which of course is part of the course for your first php project but it was this wonderful feel-good thing. I released it open source and there weren't many, I'd say, in retrospect, moderately well-designed ones at the time. And so I got some people using it, like an energy company in the Netherlands. At some point,

some part of NASA said they were using it, which I found kind of crazy when I was a teenager. I think it was just some tiny broom closet somewhere in NASA. But still, it was fun to hear from somebody over there with a nasa.gov email address. And that's kind of my first experience. For about two or three years, I ran the ByteHorde project, making my PHP skills better, doing some freelancing, but very much a single-man, open-source operation. Just like, it's just me, I'm just fixing these bugs. Did one rewrite to try and fix security holes.

It was a lot better after that, thankfully. And then I think it was about, let's see, it was 2006 it would have been. I'd just gone to university. During university, I was working part-time in the holidays and the vacations at an agency called Torchbox

in Oxfordshire. And it was there I met Simon Willison. Simon Willison is one of the co-creators of Django. And if you've ever met Simon, or if you've ever even heard him speak, he is one of the most enthusiastic people I've ever met. And it was about a week and a half of being exposed to his enthusiasm, I'd

switched to doing Django rather than PHP at that point. So it was a very tour-de-force entry into doing Python and Django. I'd done a bit of TurboGears and old WebPrimer at the time, but to switch to doing mostly Django, that was the big change, catalyzed by Simon in many ways. And then I was doing that, and about two years I was just doing Django projects on the side at university. And then it was 2008, I believe.

I was sitting down at the agency. We had this problem. We wanted to change our database schema. Django at the time was very bad at this. It made new tables and nothing else. And so me and a friend of mine, a co-worker there, sat down, planned out what we'd like to see in a migrations framework, borrowing something from Ruby on Rails at the time, from ActiveRecord and borrowing stuff from java's hibernate and a few other things

and i made south the jenga migrations framework and i sort of we wrote it it worked for our purposes very very bare bones stuck it up on the internet published it somewhere got a pretty reasonable uh sort of level of interest, I would say. And then I think it was a couple of weeks later, Jacob Kaplan-Moss, who is another co-creator of Django, emails me. He says, hey Andrew, we have the first DjangoCon happening, I think it was four weeks away at that point, in San Francisco. And we have a panel on database migrations. And we already have. They had Simon there talking about his one called demigrations. And Russell Keith-McGee talking about his one, which was Django Evolution.

He said, would you like to come and talk with the panel? We can fund your flight and get you over here. At the end of the time, I am a student in university. I am not well-off by any means at this point. So I go, if you're going to fly me to San Francisco, sure, I will take that offer. And so with a couple of weeks to spare, I booked a flight to San Francisco. I foolishly booked, I think it was three days in the Bay Area for an 11-hour flight from London, which you never do, is my advice. And then flew over here and did that first DjangoCon. That was the trial by fire of my

involvement with Django. And since then, South has got more and more popular and eventually ended up being sort of merged into Django 1.7. Very cool. So just for the listener's sake, we forgot to mention that Andrew is one of the Django core developers. You're also an engineer currently at Eventbrite.

That's right. Also, this show was put together in thanks to our ping repo and a whole host of people who hopped in when python was mentioned and said yes let's do this so we want to give a shout out to a few people jl duggar the gustav frank wiles and freak boy 3742 for helping us line this show up and saying if we're going to talk about django we got to be talking to andrew godwin um for many reasons but also the major reason is because of your work on Django channels which

we'll be talking about in detail upcoming but what I find interesting about your your history is that Django really brought you to Python and it wasn't like you didn't come to Python for Python you kind of came for Django and you stayed for the Python? Yeah. I had experimented with Python before. It was about that time I was reaching the end of what PHP was capable of. PHP 4 was a very limiting language. It's got a lot better since, but back then it was a very limiting language. So I've been experimenting with different sort of alternative languages. You, of course, had Java, you had Python. Ruby was a lot more nascent back then i hadn't really heard of it so i played around a little bit in python but django is really that catalyst

that got me to sort of go whole hog into python be like no like this is the language i'm going to learn become my primary language and i like many people i think i learned a lot of python through django yeah so interesting we were talking before the call adam and i and the conversation was kind of like why would you choose python or i think adam even more generally your question was like why do people choose one language over the other yeah yeah like he always wonders like what's the impetus the reason and in your case it was very much because of a web framework that you liked or wanted to use um but if somebody's getting into python today let's say they don't do web like what are some of the draws of python

the language uh why somebody might pick it up as their first language or their next language before he answers that though i want to pause for one second because something he said jerry there was he said primary language so i think that'd be a good tweak on the question i was asking which is like why would they choose a certain language as their primary language? Okay. Yeah. So it's an interesting question because like, you know, I say primary language. I am very much a person who believes you should choose more than one programming language.

Like use the right tool for the right job. I think Python works very well for the sort of work that I do because like, so there are various things. It's very hard to explain. Like if you've ever seen the Zen of Python, it's sort of like this short poem, it's very hard to explain. If you've ever seen the zen of Python, it's this short poem about what Python really means to be in. Things like explicit is better than implicit, for example. One of the differences that I personally prefer Python over Ruby, say, is that imports are very explicit, you can trace where things happen very easily. But in the majority, it's very much like Python is an easy language

to write, it's got a decent typing system that isn't too crazy. Like it can enforce some stuff, but it's a little bit flexible, but it's not as, say, lax as JavaScript is. It has a good sort of set of like data models and built-in stuff. Like the standard library is very useful. A lot of stuff comes with it. And that's true of Django as well, sort of extended out to web stuff. And it's just like, and the community is really nice.

And like this sort of combination of all that stuff, there's no one thing, right? It's this sort of bundle of like, it's nice to use, it's easy to write, it's easy to read, which is a super important thing. I used to write some Perl, but it's not easy to read at all. It's easy to maintain, you can start it pretty well. The combination of all those things, I think, really kept me in that language. I approached it with one thing, and then a whole set of things conspired. This is a nice place to be. I enjoy doing this. The cycle of writing stuff in this language works really well for me.

I spent about six months myself writing Python day by day, and I really enjoyed it. I don't have any problem with significant whitespace, even being a person who mostly writes Ruby and JavaScript. I've always, I like significant white space. It saves on parentheses and curly braces and doesn't bug me, but some people just can't get over that. One thing that I loved about it was the documentation. And that just seems like you talk about the community and kind of the overall thing is you add all these things up and you get something that's really nice and it just seems

like the python community is so documentation oriented or first or um there's this like built-in thing that makes it really easy to pick up and me i mean i didn't know anything about python coming into it but i got up ramped up very quickly because the docs both for the language in the saren library and for anything else you might grab off the shelf they're all pretty good yeah it's definitely like a community thing like like i'd say that ruby is probably better at testing this community and maybe python's a bit better documentation but like it's definitely a thing of like it is considered like a product isn't even launched until there's like a decent documentation like a lot of the solutions for doing good documentation have come out of the bike community like sphinx

and read the docs and stuff like that like they're all sort of rooted in python itself and that's probably one of the things that got me to stay too is like as a as a beginning programmer out in python going oh there's this huge like extensive standard library like full of examples as well like i i'm an example oriented learner so like that was an amazing thing for me there were good books as well um I think dive into python was the thing I learned at the time that was really good back then so like just this whole host of like endless reams of information that you could just turn to and like even now like I don't know all the standard library I just I know whereabouts in the documentation to look if I forget right that's what I learned it's like oh once I remember exactly

how say like order dicts work I can go and look in that part like the collections page it's more like having this index of where to look rather having to know everything off offhand you mentioned read the docs and just reminded me of a recent show of requests for commits episode five i'll just submit that as something that people should go listen to eric holscher came on the show and talked to nadia and michael all about read the docs and documentation and write the docs, which I had never heard of. A very cool conference for technical writers and people who care about documentation, get together, talk about the best ways of doing it, tips and tricks, just this whole community around documentation that's like really refreshing and like seems to be underground in the open source community, at least.

You know, like I, I try to stay abreast of what's going on and i was like blown away by how much eric and everybody involved in read the docs and write the docs have been up to and many of us don't seem to know about it so um curious your thoughts on read and write the docs uh andrew and everybody go check out rfc episode five it's a good one yeah like to me it's a really important thing like like i have turned into a documentation first developer like you might say like testing first but like i will generally like especially with like channels we'll discuss later i wrote down the entire sort of basis documentation api i wanted to give to people before i actually wrote the code like

the idea being that like that's kind of that's kind of my ux right that's how i like how would i as uh like being my own devil's advocate want to approach a project i'd never seen before like how would i want to be introduced to that thing what would i expect to see how how should things behave and like for me writing documentation and getting that down on paper is not both a great way of telling other people it's a great way of like getting yourself to be on the right track and understanding what you're doing and sort of cementing what you're trying to work on. It reminds me of something Jameis Buck said years ago about how he designs libraries

because he's a Ruby developer amongst other things and we're like Capistrano and NetSSH and a whole bunch of tooling and libraries in the Ruby world. And he was talking about his, the way he goes about building these projects. Like, where do you start with?

And of course we've had the movement read me for read me driven development, which, uh, is kind of along the lines of documentation driven, but it's like, if all your docs can fit in the read me, that's good. Um,

a lot of times you need to go beyond that. But he said like, he always designed the API first. Like he wouldn't write any code. Like he would think, how would I want to use this? Right. And always designed the api first like he wouldn't write any code like he would think how would i want to use this right and he designed the library api and the beauty about the ruby language is it is so flexible and the dsls are so easy to craft that you can come up with like what is a fun and beautiful to use api and then you back into the code once you have the api finished and it seems like as you it's a very similar idea that you're talking about

where it's like if I'm writing the actual documentation first for this library or this tool that I'm building, I get to think about it as the end user, the person who's coming to it and reading the docs and saying, is this going to be what I need or not? And then I can go and fill in the gaps with the code. I think it's a really powerful way of building things. Right. And like, even in Python,

like Python is less flexible than Ruby, but you still have a lot of flexibility. There's still metaprogramming. There's still like various things, tricks like that. And so generally like in my perspective, like part of it is also is the principle of least surprise. And so an API should still be like what we call Pythonic.

It's just still like not be full of like crazy hacks or like it's full of declarative classes that magically work. Magic is a bad word, but let's say part of it is an API that is both nice to use, but also is similar to other stuff and is not very surprising and is easy to debug. There's a whole spectrum of different things I would consider, but part of that

is documentation. Trying to write down how would you run this? How would you test this? How would you deploy this? And that kind of stuff. Yeah, it's like Kennetheth writes has made himself you know very famous not just in the python community but in the programming community the open source community because of how great you know the apis he's created around hdp requests and a lot of that honestly is like he does a really great job of documenting that and like selling it as a thing that you'd want to use. And so that that's an example, I guess, of a success story inside of just caring a lot about your documentation.

What exactly do you classify as documentation? Simply where you go to learn about how to use it or like getting started guides? Where does documentation begin and end? I'd say there's a whole different set of things like you've got reference documentation, which is usually things like, oh, here are the methods, here's the errors you can raise, and here's things you can call. And that's usually at the end of a project, I would write that stuff. It's like, well, this is the exact spec of

what you should expect. These are the four errors this function could raise, and this is the set of scenarios. And then there's more sort of an overarching guide, which would be almost an architectural overview or an introduction of what the thing is, what it's trying to achieve and the sort of the concepts and that's that's kind of what i put in the the getting started guide slash sort of introductory end of stuff and so i'd usually probably write a a getting started tutorial first like the goal

being that a novice programmer should be able to use the thing i'm writing or rather someone with no experience of the domain i'm writing it to be able to use the thing i'm writing or rather someone with no experience of the domain i'm writing it to be able to use it but generally i try and think like you know i want any programmer to come along and be able to use this and not only go to use it and get stuff done but make it really hard for them to shoot themselves in the foot which is a very important part of writing a library so like how can we make it nice but safe and like approaching it from that end generally. Yeah, there's definitely a couple of different aspects.

There's this reference and even release notes are important too. Like release notes is this sort of temporal documentation. Like what's changing in a project? Like what do I have to do when I upgrade? And like a lot of projects don't have good release notes. And that does sometimes irk me a little bit. I think sometimes, you know, as a maintainer, you might get so focused on the future and where you're going, you kind of forget where you came from and maybe how you're currently using whatever might be and this is totally just a abstract example but you know sometimes you just forget like how do you go from one version to the next or a bigger upgrade or you know move from one version to the next it's just a larger you

know transition than maybe before. Yeah, I mean, this is a problem you have. One of the things of being an open source maintainer that I did not really appreciate before I was one was that it is very difficult to see the wood for the trees, to use a phrase. It's very difficult to understand

the external... You're always living in the most current code with all the current bugs all the issues being filed like the perspective of somebody who's like oh there you know like on a two versions behind their site's stable they want one new feature you've just announced like that's a very different perspective and getting a good insight into what that perspective is is sometimes very difficult for an open source yeah but that's also crucial to have like and and having been on both sides of the fence

like having used open source as well as maintaining it i try and think about like what i appreciate in when i use open source tools and try and provide at least as much as i can myself well i think it's a natural place to take our first break on the other side we will dive into django what's new what's the flagship features of dango? Why do you want to use it from Andrew's perspective as well as a deep conversation on Andrew's baby, which is Django Channels. So we'll be right back and we'll talk about those things after

this break. Our friends at ThoughtWorks have an awesome open source project to share with you. GoCD is an on-premise open source continuous delivery server that lets you automate and streamline your build, test, release cycle for reliable continuous delivery. With GoCD's comprehensive pipeline modeling, you can model complex workflows for your team

with ease, and the value stream map lets you track a change from commit to deploy at a glance. The real power is in the visibility it provides over your end-to-end workflow so you can get complete control of and visibility into your deployments across multiple teams. To learn more about GoCD, visit go.cd slash changelog for a free download. It is open source. Commercial support is also available and enterprise add-ons as well, including disaster recovery. Once again, go.cd slash changelog. And now back to the show. All right, we are back with Andrew Godwin, and we're here to talk about Django. So Andrew,

let's kick it off like this. Many of our listeners have used Django. Many have not. We have lots of web developers. We have a whole group of listeners out there, and they're coming from different perspectives. So let's just start off kind of with the high level of what Django is and does in 2016, and then we'll kind of drill down from there. So what are the flagship or the major features of Django as a web framework?

I know one of them, which I've always been jealous of, is the built-in admin, but surely there's lots of other things it does for you. So give us the rundown on what Django looks like today. Sure. So yeah, as you said, the main one, sort of our main star attraction that everyone gets on into the project is the admin.

And that's sort of the thing that when it was launched was the big thing. So basically the idea is Django has this fully featured ORM that sort of you declare models. It has migrations on those models. You can declare custom field types. It has a very extensible query framework, all this kind of stuff. But you declare your models and then it just makes an admin for you.

And that's one of the big sort of get developing fast features. Like, oh, we can just take the admin and then we can just immediately put things into the database and play around with it. And some sites even run with the admin in production, although usually when we're trying to sell a little bit different, but there's many other, many other features as well. So one of my favorite ones is the GIS framework, which is a little less known maybe, but Django has a very extensive, very powerful GIS framework for doing sheer spatial data or queries, that kind of stuff. It's built into the RM if you want it.

So you can say like, oh, find me all posts that were made inside this polygon. It's all in there and integrated. That's super nice. It has other things too. It has a forms framework for doing display stuff. It's a templating language with accessibility as well. It has a views and URL

routing framework. All the pieces you expect from a web framework really. You can make URLs, you can route them, you have views. It has what we call generic views, which sort of takes some of the replication

out of doing something. Like, oh, if you want to just have a view, which is like, you know, just accepts a form and saves it, there's built-in code for that already.

And all this kind of stuff. I'm trying to think of other stuff it has. Great documentation. Obviously, yes, of course, great documentation. I mean, I just kind of assume that, right? That's kind of just part of like, oh yeah, of course there's great documentation.

It's part of my, yes, but of course, Django has this, not only is there a Django tutorial, there is full like prose, like article style documentation on every major feature we have and reference documentation. And there's excellent other, so like the Django girls tutorial is an an excellent second tutorial apart from the Django one

as well. So there's a really good culture of documentation around that. And also there's a lot of security considerations. Django has a lot of CSRF protection built in, has places to put cookie protection and HSTS and all this stuff. We have things called middlewares. There's a security middleware where you can say, is my site secure? And it has run some checks for you.

There's a command where you can check for common errors. And so all that stuff is bundled up into one set of stuff. But one of the things I think is most important about Django, and the thing I try and stress, especially to people who are maybe new to Django and don't use it a lot, is that one of the key features is that not only is it got all these components, but you can individually remove each component. They're all optional. So if you don't want the admin, it's a couple of lines to turn it off. If you don't want to use templates, you can swap in another one straight away. It's super easy. And so part of what Django's appeal to me has been over the last eight, nine years I've been using using it is the ability to as i grow projects i can replace the generic parts of django with bits that custom or slightly

customize them and like i don't have to just throw the whole thing away it's very loosely coupled and so i can say oh i just i want to use ginger rather than django templates here i just swap do you do a little bit of work swap it out and it'll work it all works pretty well so that's what i personally appreciate a lot about django it's like when you get to that point, it steps away and like sort of falls away and then you should put your own thing in place. Let's hop back to the GIS feature because that really caught my ear.

Is that tied to us? Like, is that using post-GIS or is it using a specific backend? How does that whole thing fit together? How does it work? Also, much like Django has pluggable database backends, the GIS framework does as well. So it supports Postgres,

it supports Oracle, it supports MySQL. There may be more supports as well, but those are the three I know about. But it has support for multiple backends. And it has Oracle and PostGIS, especially,

they are fully featured geospatial databases and as is the new MySQL as well. And it just supports all of those functionality out of the box, pretty much. It's just like, you'd install a few libraries that are a little bit arcane right once you've

got that done it's pretty much blow and play yeah i've used post gis a handful of times and via rails and i think directly a couple times and it's it's always a little bit of uh you know wiggle this and you know pull that lever over there to get everything all working together but you can do amazing things once you have it up and running so that's really cool that it provides kind of a adapter pattern to those type of queries and and lookups inside of the framework that's pretty sweet yeah and even better like you know keeping with a theme it has admin widgets so like if you make a point field or a polygon field and load the admin up you get a map with a polygon on it you can drag around and edit so like it's all just all that it can just like start using it straight away like oh

i want to see my what's my shape of london look like and the admin will just show you on a map which is super nice now correct me if i'm wrong but i believe django came out of a newspaper correct in lawrence as far as where it was born and so it's kind of very much in that community of like online publishers and newspapers these types of companies. I believe Eventbrite's using it. It makes sense since you work there. But who else is using Django? What's the community look like and the companies and the organizations that are involved in the greater Django community?

Yeah. So as you mentioned, like Lawrence Journal World, the Lawrence newspaper was the first place Django was created and born out of and open sourced from. And it had initially a big traction in publishing newspapers, but these days it's pretty well spread. I'd say the biggest one you've probably heard of is Instagram. Instagram runs on Django as a backend. That's sort of the biggest one we have that we know about

and other ones we don't know about. Generally, knowing what things run django is is interesting um but obviously like event right runs it as well uh we like i'm very bad at knowing the list of companies who run django it's one of my um i know i know that like a decent number of government agencies like run at least some django as well uh let's see um one second here that's all right i didn't tell you to think about it i put you on the spot should i get should i prepped you but i just thought of the question because i just wanted to kind of nice to get an idea not just like hey how many big companies use this as like a street cred but just to kind of get an idea of what the community looks like in terms of

participants and support obviously it's huge and you guys have django con and you have you know it's a long-running project that thousands and thousands of people are using but um no i think that's a good enough idea don't need to bang your head too much against knowing which sites run on django yeah so let's do what ones i can now i can i've now i can now like remember um i've got some too, because I went to Wikipedia. I think maybe just like you used to be typing just then.

So among other things, I believe parts of Pinterest at least used to run on Django, which they still do. Parts of Spotify certainly used to. And also like the, a lot of Firefox and the Mozilla sites

run on Django as well. Like the whole of add-ons at Mozilla.org, I think increasingly more and more of all the Mozilla sites run on Django as well. The whole of add-ons at Mozilla.org, I think increasingly more and more of all the Mozilla sites run on Django as well. The ones on Wikipedia are Pinterest, as you mentioned, Instagram, as you mentioned, Mozilla, as you mentioned. Then also the Washington Times,

Discuss, Bitbucket, and Nextdoor. Yes, of course, Nextdoor. I think NASA does as well. Yeah, part of NASA runs on Django. Is it Nextdoor? Is it a mobile app, or is there also a website, maybe just running their API?

Maybe an API thing. It's one of the things with Django, right, is that Django is, it's not easy to tell as long as it's a Django site. Like, there's no obvious giveaways. Like, Django is this flexible backend framework. There's no, like, oh, you know, like the same WordPress you could usually tell from the URL structure or something. But Django is like, well, anything could be Django. And

increasingly, a lot of things use Django as a backend or as an API for a native app or a rich JavaScript web app or something like that. And so sometimes the site won't even look like Django at all, but behind the scenes, there's a lot of Django churning away to just serve everything. Like Disqus, for example. Disqus is mostly embedded, of course. But the whole Django thing there is the API is driving all these embedded comments and JavaScript systems they're using.

And so a lot of Django is this sort of secret, behind the scenes, getting stuff done kind of approach. And we're all quite fine with that. Like Django core developer community, kind of like we take pride in that. We are a slightly boring, predictable framework. It's kind of a point of pride for us. It's like,

you know, like, I don't know if you want something that's like stable and reliable, like Django is a very like decent choice these days. Like personally, especially I see a lot of pride in that. It kind of touches a little bit on what Jared and I talked about earlier, which wasn't really like back to that choosing the language.

We talked a bit about, you know, Django in terms of, Python in terms of, I guess, Buzz, so to speak. And it seems like you, that community seems to be more focused on stability, as you just mentioned, versus like banging out brand new things and all this new stuff, know you know similar to how javascript is just always new something's always new and there's a lot of fatigue in that arena well i mean part of it is the age of the language too right like javascript as a development community like javascript is an old language by itself and sort of the modern community around

it is still quite young and like all the tooling and all that kind of stuff is still developing python is you know several decades old at this point. When I joined Python, there was still some of that buzz and different options going around. And there still is in certain arenas too. But there's now this happy medium between, well, we still have new stuff coming out, and things still happen that are interesting. AsyncIO was one of the big things in Python 3.5, for example. But at the same time there's also this sort of stability and predictability, and also this history of Python. Python is now so old that big enterprise companies and universities will happily use it and teach

it. So we have this whole spectrum of from novice to very experienced programmers, from single developers to entire huge enterprises using using it and like that kind of diversity of use is i think actually really positive factor in what drives python forward and i think javascript especially will get there as it sort of matures and people and you know yeah it'll go through the same process and we'll get we'll get the same place this is funny that you said the word boring and i wonder if your you know counterparts that uh are core team members with you or users of django or you know participants in the python community will appreciate or agree certainly some of them will i don't speak for everyone of course but like make everybody happy um but

certainly like i would say one of the things i can say is that in in the public community the word magic is frowned upon like if something something is magic, it's considered bad. And so in my mind, the opposite of magic is like boring and predictable, or at least like, you know, very obvious. And so that's kind of what I aspire to is like,

this software is predictable and easy to deploy and easy to understand and does the thing it says. And it's not terribly exciting, but it will save you a lot of time in your daily work. Like that's what I really appreciate.

Like as I, as I developed as a programmer, especially especially i think what you makes that choice to um like we said before choosing that language is that uh if you're choosing something with more fatigue so to speak you know with more bleeding edge more modern however you want to phrase that you know you can expect some bumps but if you need something to go smooth, you need something that's going to not drive you crazy, that's going to have clean, good documentation, or even good tutorials to get started, then what you said there was a good example of the right choice.

Yeah, and of course it doesn't hold entirely true. Like Python, so the Python 2 to 3 change was a big overhaul, and that's one of the most major things in recent history of Python. It's like, this is a big change, it broke a lot of stuff. It was perhaps done the wrong way.

Some people now consider things like that. And so you're not free from that kind of stuff. And in the same way, I'd say that a brand new language, you're probably going to find more recent posts about it, more recent developments and tutorials. But as you said, they do get outloaded quicker as well as things move faster. So it's definitely a balance of those kind of things. part of what i'm trying to do with django is trying to like find where that balance is like where is the balance between keeping things

predictable and then pushing forward what what is the state of the art like how do we advance as well interesting well speaking of state of the art and advancements we do want to talk about django channels which is kind of seems to be the most exciting thing going on in the Django community right now. And something that you're heading up. We are hitting our next break. So let's tee up Channels and we'll talk all about them after this break. I talked to Daniel Reed, head of design at TopTile, about their new expansion into TopTile Designers, doing for designers what they've done for developers. We talked about why TopTile works for designers,

and this is what she had to say. As a designer, the big, or as any kind of creative person, the big overarching question is always like, how can you find inspiration? And for me personally, and for a lot of creatives that I've spoken to, it's really about traveling, exploring, and being accountable for your own career. And I think as a TopTile designer

or a remote designer in general, the ability to be able to switch up your lifestyle, change contexts, meet new people, have new ideas sort of infiltrated into your life by having that freedom and flexibility is something that's absolutely fundamental to doing great work. That's the real power of TopTile, I feel.

You're not just stuck with one product, one company, or even one agency, but you can choose to work on multiple occasionally or a range of different clients. And I think that that keeps you fresh. It gets you involved in new technologies, different people, and is really fundamental for being sort of

switched on as a designer. All right. That was Daniel Reed, head of design for TopTile. To learn more, go to TopTile.com slash designers. That's T-O-P-T-A-L.com slash designers. Tell them Adam from the changelog sent you and now back to the show. All right, Andrew, let's talk about Django Channels. Seems like these days most web frameworks are either adding channels as a feature or they're trying to. Some of them started off with channel support, but you talked about boring. And I don't think channels are boring. I think the general public is very interested in by general public. I mean,

our general public, uh, not the people on main street, but the developers were interested in channels because, uh, it's, you know, it provides a new way of interacting and some new features.

There's lots to it. And like you said, I guess during the break, it is a lower level kind of a behind the scenes thing, but lots of excitement around Djangoango channels so tell us uh give us the short history of the channels bit because i believe it started as a as a plugin or something and perhaps it's being integrated but you know what you're talking about why don't you give me the rundown and i'll quit talking of course um so yeah channels is actually stems from about three or four years ago so

i'm sure your audience is aware of this but if if they're not, WebSockets came onto the scene about five or six years ago. It's sort of like this, oh, we need bidirectional communication for the web. Like HTTP is not good enough. And so like a team of developers and the various browser manufacturers are sort of, mainly Google, in fact. So developing this protocol, like, well, what kind of protocols do we have? How's it going to work? How are we going to encode it? It has to start HTTP.

I was following at the time this development of what is WebSockets? How is it going along? Back then, I was actually using a Python framework called Eventlet, which is sort of an asynchronous programming framework for Python. They had a WebSocket library, and the spec was changing pretty much every month at that point and so i put a few patches in to try and like keep it with the current spec they like found security holes and found like replay attacks and stuff um but then about i think about three my history is about three four years ago i think it's stabilized pretty much and then i sort of got interested in like well well, as the web evolves, we see this trend towards

websites being much more of a heavy application. Single page applications is often one of the buzzwords you hear. And a website is this application, it's full of logic, it has to communicate with the backend server, and a lot of that's still being done over REST, over normal HTTP. And that's great. That works really well for a lot of stuff. I appreciate the boringness of that. It's very well understood. But for some things, it just doesn't work.

So an example to say from my real place of work at Eventbrite, there's a seat map, what you can pick seats from. And the seats gray off in real time as they're picked. If we did that using HTTP, we'd have to sit there doing long polling and bundle it up on the backend and make sure it's all done efficiently. We did used to do that as well. But the key thing is WebSockets give you this really easy, quick, efficient way of sending data backwards and forwards to and from the browser. It's not standardized.

It's a slightly crazy protocol on the wire, but pretty much it is a datagram-based protocol. You can send and receive packets, that's pretty much it. There's a lot of stuff under the scenes. And it got bigger and bigger. And I saw it as like, well, obviously as these applications get bigger and more complicated, they obviously require more and more communication with the backend, and at some point, Sockets makes sense. And then Meteor comes along, and Meteor is the backend, and at some point, sockets make sense.

Then Meteor comes along, and Meteor is the thing I see and go, ah! Everyone else is like, this is a really clever way of doing this. Meteor is this great idea of, we can use sockets to do data binding and send data both directions across a socket. Meteor was the spark for me initially. I was like, oh, we could do that in django maybe and then the problem is of course that um for listeners who aren't aware python is a single threaded language at its core like we have a threading model but the threading model is just time sharing on a single core there's no actual multi-threading in there um and on top of this like support for what's called asynchrony as in you can open lots and lots of like virtual threads and wait on sockets and do things out of order, only came in in Python 3.5.

And so Django supports Python 2.7 and up. So we have the limitation there. Well, Python has to support, Django has to support all these different versions of Python. And in addition, when I was thinking about this a couple of years ago, AsyncHair wasn't even mature yet. And so for a little while, Amarik Orgastan, one of the a couple of years ago, Asynchro wasn't even mature yet.

For a little while, Aymeric Orgastin, one of the core developers of Django, did some work with WebSockets with Django, and his work was really quite good. He had a game of life where every cell in the game of life was an open WebSocket, which was actually kind of amazing. I really enjoyed that demo. I sort of did that, and he moved on from that but it was still like a really nice proof of concept and then about a year and a half two years ago i sort of jango migrations which had previously been my baby but as you as you would um was done like merged um some of the core developers had very kindly taken up a lot of the mantle of maintaining it um i got a bit burnt out merging

in in the 1.7 release. And so I was still helping out occasionally with migrations, but I was thinking well, what else is Django missing? I spent eight years fixing this big hole in Django that I saw in 2006. What do I do now? And so I sat down and thought, well, in my opinion

and what I'm excited about is WebSockets. I'm excited about these things in the browser. I also do game development on the side. Sometimes I'm like, well, can we do games in the browser? Like, what's that look like? Like, we obviously need Sockets for that, for like proper real-time communication. And so I sat down and started like sketching out

sort of like a prototypical, like, how would this API work? Like, how would I want it to work? And then saying, could I do this with Django? Could I do this as Python even? And it's difficult. There are solutions out there already, and they always have been for a long time, that rely on a separate process running that that process is using an asynchronous framework, be it Twisted or Tornado or AsyncIO or G-Event or one of the many other ones that Python has. And that framework is capable of handling the idea of lots of simultaneous connections.

And if you have this framework running a separate process, that can handle WebSockets. And then everything's good. And it sort of communicates somehow with the main server over some kind of socket. And then you can get sort of socket support, but it's not properly there. And there are a lot of downsides to this solution. Like the first thing is that server generally only handles WebSockets. You can't just point your server at that server. You've got to have Nginx in front and then have a path-based routing.

If you see the path slash WS, go to the WebSocket server. Otherwise, go to the main server. It gets a bit tricky. And on top of that, another part of it is HTTP2. And WebSockckets are not the only thing changing in the web. HTTP2 is also coming in. And it's coming in in force. It's been a slow build, but it's getting there. And HTTP2 also has a lot of changes in terms of how things are served. It no longer has this pure request-response model that HTTP has. HTTP is

great. You get a request, you single-threadedly handle the request, you return a response, and then you just return to the top of the loop and keep going. It's easy to write. CGI sorted this many, many decades ago. It's an understood paradigm. And then HTTP2, part of it, you can have concurrent requests that you can work out with multiple server processes, but also things like you can push information from the server in real time and other things. All this brings on the same feeling and texture, like, well, this is all real-time stuff that is no longer a request-response cycle.

And at that point, I sat down and went, well, let's think bigger than WebSockets. What would it take to bring Django to be a general protocol framework? What would it take to handle HTTP 1, HTTP 2, WebSockets, like Internet of Things protocols coming up as well. Like there's several of those. They're very lightweight, of course, because the Internet of Things is very limited in bandwidth.

But like those also have API servers. And like, what do we do for those? And like trying to envision this plan of like, what would it take to build a framework or like build into Django a framework for the future of the web, the future of the framework, or build into Django a framework for the future of the web, the future of the internet. How do you write a system that handles all these different protocols and styles of communication

while still being like Django? That's the track I settled on back in my third year at channels. This doesn't sound boring at all. No. Well, it's all infrastructure, right? And I find infrastructure really interesting, but some people find it very cool. The future of the web is what you said. The future. You're building the future of the web here.

Well, I have the ability to make it sound very exciting. How about that? You're succeeding. You need a hype person behind you, like that Geico commercial. No, it's progressive. There's like a hype man behind that flow. He has them.

I've seen them out there on the web my pr team is here behind me giving me advice so you know i'll defer to them on that stuff but but yeah it's it's it's you know it's it's i like big challenges right so i'm attracted to as a developer like as i've grown as a developer i've gone more and more towards the sort of like software architecture slash networking infrastructure end of stuff and so that's always held my interest and channels is very much the idea of like well how do we do this and for the longest time i couldn't do it like i kept doing ideas none of the work or they were ugly or they performed really badly which it didn't look nice if it performs like way worse than django it's not worth doing um and then eventually the current design channels I hit on about a year

and a half ago now I would say um sat on it for a while like sat there trying to prototype it and test it and like going is this really okay does it really work and then like asked a few like close developer friends of mine saying like can you can you like double check this and make sure I'm not crazy or like overestimating this and then finally like when I was happy that it was like at least only a little less performant and had all the features and seemed to map into django did i go okay i want to talk about this thing and here's my design and when was that uh so that would have been probably it was a june 2015 so you know a little over a year ago i posted this first blog post about

this is what I think the challenge facing Django is. The blog post is called Beyond Request Response. That's my encapsulation of the whole thing. This is what I want to do. This is my rough outline. And then it starts with a proposal to the Django community of, this is the major fundamental change to the way Django works I would like to do. And it's changed the way Django runs views. That's where the discussion started. Well, Andrew has this crazy idea, let's talk about it. And then at Django Under the Hood that year in November 2015, I wasn't talking, but a

good half the talk seemed to at some point mention channels. I think at that point I had some idea that it may have been a very popular thing. This growing, already interest in it and what it mean for django is really starting to bubble up then let me push back a little bit and play devil's advocate because let me just say first of all i and i get excited about web sockets too and i and i write web apps you know for a living but i've never actually had to use them and i don't do games. So I'm, I'm not saying there aren't their use cases. Um, I just feel like,

can't we, can we get away with long polling? Like maybe 95% of the time. And then like, sure. Like Slack needs to do web sockets, but you know, the rest of us are,

we can get by with long polling. Can't we? Oh, absolutely. I still recommend it for like, some people come to me for advice saying, Oh, like what should I use? So this was like, no, this is probably a long polling thing can't we? Oh, absolutely. I still recommend it. Some people come to me for advice saying, oh, should I use sockets? No, this is probably a long polling thing.

Especially with HTTP2, so a little bit of background, HTTP2 does not have WebSocket support because they maintain that chunked responses are enough. The protocol is efficient as itself, but just sending requests and responses is as good as WebSocket is.

And that's why Channels is not just a WebSocket library. It's a general asynchronous library. Long polling still requires the ability to hold open hundreds of connections at once that are all waiting on that long poll. So the same challenges apply to long polling. They apply to having hundreds of

HTTP2 connections at once. The same kind of basic structure applies. I'm very aware that WebSockets is a very niche thing. It's not for every site. Maybe it could be if the JavaScript framework is going to get a good patent for that kind of stuff, like binding it into the models in the JavaScript side. But at the same time, it's also this extreme of a whole spectrum of, well, there are all these challenges, and just serving normal HTTP 1 one after the other is the simplest thing,

and we've moved beyond that. Long polling itself is already difficult, and I think WebSockets and HTTP 2 are just the next step of that. Yeah, so what about, so does Django typically run on port 80, or is it behind a proxy of some kind?

I'm assuming there's people doing it both ways, but what's a typical way of running a Django in production? Yeah, it depends on the way you work. Like either you can run it embedded in like Apache with ModWhiskey, for example, or you can run a separate server, like much like you would with another,

like say like a JavaScript server or Ruby server and then proxy to that through your web server. So for example, Unicorn is a very popular server that you just run it listens on a port and then you just proxy through to that stuff as well and so channels stays of the latter model um of saying like here is a separate server you run that accepts both hp1 hp2 and websocket connections and you could just proxy everything through to it and just forget about the rest of it or if you want to you can switch proxy everything through to it and just forget about the rest of it. Or if you want to, you can switch different

versions and so on and so forth. Yeah, because I would think it seems like H2 and maybe it's because I don't know the intricacies very well, even though we had Ilya come on the show and tell us all the intricacies, but that was a long time ago and I can't remember.

It seems like can't that be something that you could implement at a proxy layer and then just keep your backend dumb or simple and let your Nginx do your H2 stuff and you can talk to it with H1 proxied? Of course. Yeah, so you definitely can do that.

It depends on what features you want, right? So the headline feature you might want from HTTP2 is that you can streamline multiple requests down the same single open socket to the web server. And if you're just doing that, then yes, you can just tell NGINX to take that, prox it through as three separate requests to your backends, to three separate servers,

and it will handle all that stuff for you. However, if you want the more advanced stuff of like, well, we want to send long polling chunk responses, we want to do server push, which is a feature of HTTP2, that stuff requires native support in your framework.

Like you can't, like because they're not in the HTTP1 abstraction, you can't just sort of paper them over. You have to have like native support for that stuff to pull it off. That makes sense. And then probably like the server push

aspect of HTTP2 is why they say that you don't need web sockets anymore because you can still achieve the bi-directional communication that web sockets affords is that is that is that the case that's a common misconception at least as far as i know so server push server push um as specified at least from what i've read i might be wrong on this it's just a way of pushing resources to the browser before it asks for them. So think, for example, if you're looking at a web page, you'd server push the images, the CSS,

and a few other things. So by the time it got to the full web page, it would go, oh, all these things are already in my cache, and it would just immediately serve them. The idea of doing sort of pushing things to the browser as responses is more just like you can have an open response and send things down in a chunked fashion, which is like server-centered events, I think,

would maybe be the sort of more common way you describe that. That's also a HTTPS 1.1 thing as well. Right, exactly. So yeah, I think just difficult naming there with server push. So what are the features in H2 that obviate the need for WebSockets? So, I mean, as far as...

So I am personally on the fence here. I kind of think they would still be useful. Like, WebSockets have a lot of nice things about them. But the argument goes that because you have efficient chunk responsing, like server-sense events, that's your server-to-browser part of the direction, as you were. And then because requests are so much more efficient, like headers are compressed properly and there's all binary protocol,

you can just send normal AJAX requests to the server and that counts as your in-pound. And that's just enough. And maybe it is. And one of the things I'm looking at for channels is things like abstractions like Socket.io and stuff like

that and Sock.js of like is there a good way we can implement those same abstractions and reuse those libraries and have a layer that lets you switch between both of those implementations and maybe you'll just stick you to one for performance reasons but maybe you want all of them for compatibility. So give us

the status of channels where it stands in terms of development and involvement like is it in Django? Is it a module that you use? Give us the rundown on where it stands in terms of development and involvement like is it is it in django is it a is it a module that you that you do use give us the rundown on where it stands yeah so right now it is a separate application um i wouldn't say so we usually call things third party applications that aren't part of django but that's not true of channels because channels is part of the django project so it is a first party pluggable application so basically it's a separate library install.

You put it into your installed app setting in Django, which is where you put things, apps that are loaded in. And then it just works. Like it sort of overrides the server stuff for you. And then you can just import

and start using it. And so it's done this way for a reason. So I did propose to put it into the current release of Django, Django 1.10. But there were some very rightful questions raised

about the design and how relatively immature it was at less than a year old. And so I made the call at that point to keep it out of Django and develop it as a third-party pluggable thing, and then maybe in a future release we'd look to merge it in in much the same way that South got merged

in to become Django Migrations in Django 1.7. Hmm. Okay. One last question for you just with regard to kind of the ecosystem of channel implementations across these different frameworks so feel free to dodge this one if you aren't familiar with like how phoenix does channels or how rails's action cable works but if you are familiar with those two at least or any of the others and how they're going about uh implementing channels in their frameworks you know how is django channels different or better it seems like at least the scope is larger because it's got the h2 aspect of it but i'll let you answer it yes the major difference to a lot of the common ones is that it is it is a scope thing like part of that is making up for python's deficiencies. Some of the languages already have a good solution for handling asynchronous I.O., and certainly Python 3.5 does as well.

But the idea of how do you write... Because Django is a synchronous framework. Making Django asynchronous would be a massive undertaking. So we can't do that, at least in the short term. So the difference there is it's much more of an overarching design of how do we run a system that is asynchronous across multiple different machines. Channels is, by its very nature, network transparent. If you're familiar with Go at all, Go channels are one of the patterns Django channels is modeled after. They're like, it is a data structure you put things into,

stuff comes out of the other end and it has like thread pooling properly. And the idea was that even if you have a language that supports multiple processes, that's still not good enough for most websites. Most websites are like these whole racks of servers or, you know, loads of machines in this huge cluster. And I guess channels, the idea was to have a design that would enable you to write asynchronous systems

across a network. It's a very ambitious project in that way. At the base of it, it's a distributed communication and queuing system. The Django part of it is the nice, pretty WebSocket build on top, but underneath there is a whole underlying set of libraries

that are separate libraries if you want to use them. This is how you can do asynchronous coordination and communication between different processes. It's all based on a language called CSP, which is an old academic proof language for concurrent programming, which I got taught in university. Much to my regret then, and now much better for me now actually using it. It's all based on this like idea of like, how do we write a system that handles all of this stuff together? And like, in that sense, it's a bit more

far ranging than other solutions, but also some other solutions don't have to have like, they don't have to solve the same problems in their language, say, that some other things do. But then again, all languages ultimately, apart from say, maybe Erlang are fundamentally single machine.

Right. I think the problem still applies to other languages. And one of the things I would love to do, if it proves out and works well, is look at how we could have intercompatibility across languages with this message passing solution. Could there be a way here of a more general protocol of like making things talk to each other and then like maybe solving some of the microservices headache people have? Like, well, we can write this thing in other languages and so on and so forth. Well, it's no wonder to me that so many people are excited about your channels project, even though, you know, you keep using the word boring. I would say it seems exciting and ambitious as well.

Any final thoughts on channels before we move on to the next topic? you know you keep using the word boring i would say it seems exciting and ambitious as well any final thoughts on channels before we move on to the next topic um i think i just want to say is like if people are looking at channels like i'd rather i'd like them to look at the idea like just the thing i want to say really is channels probably isn't for everyone like one of the many things like migrations when i wrote it was was for everyone. Almost every company has a database of schemas in it. Realtime communication is not for everyone. But what I would like people to think about is not just WebSockets, but as their company grows, as they expand out, will you grow dedicated hardware or will you grow other things? Think about things that will go beyond your just

normal web request response model like just start thinking about like what would that look like you know as as eventbrite as a company we move into like actual fiscal hardware and stuff like that as well like we have this problem but even like the internet of things is coming up and all that kind of stuff like we have a growing set of problems of like things aren't just normal browsers anymore they're not not just like you know ie4 sends a request gets a response job done like and just like try and bear in mind like what's the best tool for that job well said so switching gears a little bit um you mentioned a couple of projects uh of course the channels

project the big one you also had uh what was the name of your migrations uh south yeah yeah south which got merged into django and uh you've been working on the Django core team for a while now. And so you have some experience with long-term projects and sustainably funding open source projects, which is always a hot button topic for us. Share a story with us. It looks like you've had a couple Django features which you've managed to get funded the development of. Can you tell us about them? Yeah, I've been unusually lucky. Well, lucky slash well-fortunate in that regard like so the first thing i got money for um i'm one of the

first open source funding things i actually heard of um so when south was still a third party app at the time um i wanted to merge it into django and that is a lot of work um it was actually basically a full rewrite of the code pretty much much. And so I turned to Kickstarter and went, okay, I need around 3,000 pounds, US pound, I need about 3,000 UK pound sterling to pull this off. In my spare time, I had a day a week free. My freelance rate would have ended up about that. So I did a Kickstarter. I'm like, hey, this is a single feature. There's a clear defined end goal of it's in Django. Would you like to give some money and the response of that was overwhelmingly positive i think i raised almost 18 000 pounds wow from a target of two and a half i

think it was and so like i was very fortunate that the community really rallied behind that and like that gave me the funding the time to basically merge it in considerably faster than I would otherwise be able to. And just spend some of my freelancing time directly on that stuff. And that was really nice. And so that was good for the South feature. And then for channels, there's a different approach.

So Mozilla have a project called the Mozilla Open Source Support Grant System, MOSS for short. And what they do is Mozilla Open Source Support Grant System, MOSS for short. What they do is, Mozilla are doing this wonderful thing of giving money to open source projects that they rely on, which I wish every company really did. But that's an aside. I think it was Django Under the Hood last year, we were having an informed meeting of core developers, and somebody went, oh yeah, there's this Mozilla thing that we should probably apply for. Specifically, the Mozilla thing was also for a new feature,

something that you could specify and bound and say, okay, we're going to do this. This is the end goal. Here's a rough estimate of the timeframe and the cost and then apply. And so I sent an application and we were very fortunate to get given the money, $150,000 US dollars, which is a lot of money to know what to do with in many cases. And that was the estimate of the amount of work for both channels and also some Django REST framework integration to Django as well, sort of is still ongoing.

And so that was a sort of, also a sort of single lump sum raise, but from a company rather than from the public, although Mozilla is a very public thinking company in many ways. So both of those, I've been really honoured to have the public and Mozilla both be happy and confident in my proposals and my work to pull those things off. And the channel's money is more than sufficient to pay both the time it takes to develop the main part of the project, but also we are paying individual developers for features.

We have a list of big features on the GitHub issue tracker. If you want to work on one, you can send in a small proposal. We have a small team of DSF and Django members who will vet it and make sure you have a relatively decent history and then just approve you to work on it. And then we'll pay you your going daily rate to work on that stuff. As a sustainable way of doing open source, I really like this. It's a way of saying, we can pay you to do what you are normally paid to do and write good software. And I

really, really like that as a way of doing this stuff. We've already had, I think, six or seven major features being paid for through this program. And we have a few more that I'm sort of trying to rustle up people to do. It's sometimes surprisingly hard to find people to pay money to, it turns out. But we're getting there. So that's been a really nice way of doing it. Have you formalized the process much then, since you've done it so many times? I mean, I've done it twice, so...

I thought you said five. Oh, you mean for the channel? there we have we have a formalized channel as one from the start so like we uh django as it grows the project um we like having formal ways of doing things so like there's no skirting around the problem or like sort of especially when money's involved like you know accusations of fraud or favoritism so like we have a neutral committee that i'm not on that approve that stuff and there's a process where like like i work with the person to have a proposal you put it to the committee they discuss it and then come up with the result and then the we have a separate sort of structure already for dsf the jango software foundation they handle the money and the payouts and stuff as well

so it's all very well formalized i'm very happy with like how above board and like understandable it is like that it's very hard to accuse us of favoritism in that respect it's an interesting way to to do money raises and and i guess i compare that to you know features in ruby if you if you even might know like other languages how do other people do what they do is it just free open source just uh people's hard work and there's no money? Or what's generally the way other camps do it? Well, I mean,

this is a general problem with open source. Funding is a general problem with open source across the board. Not only in Django and Python. Obviously, those features I mentioned were funded.

Two other things in the Django world have also been funded. So we had Postgres specific fields and stuff funded for Mark Tamlin. That was amazing. And we had Django REST very much stuff funded as well. Both Kickstarters in those cases. But there's also a general problem of like, there are other Django features that are still done purely by volunteers.

Our security team is still purely volunteers. Thankfully, we have a full-time paid Django fellow, and his job is to do all our bug triaging and keep things going. He should be able to just keep things flowing and do security releases and that kind of stuff.

But it's a general problem in open source. And one of the things I like to look at and consider is how do we keep open source as a whole community going? How do we fund all this software that the world really does rely on? Certainly, yeah. Nginx and other things are the core of an astonishing number of systems.

Even Linux and Nginx, they're a commercial branch to some of those people. Red Hat employs some kernel engineers. Nginx has a commercial arm. But there are a lot of other projects. Open SSL was one of the most recent ones. There was a couple of people working on that and they weren't really paid very

much at all. And that library is one of the most critical parts of security infrastructure we have in the world. In general, I'm trying to help not raise awareness so much as trying to work out what is a more sustainable solution for ongoing maintenance. What I did was raising money for a single feature that you can define and you say, this is what we're doing, it's got a bounder time,

it finishes by this date. But in general, you need money for fixing security holes and doing code audits and keeping things maintained properly and looking at issues

and all the stuff that it seems very plain on the surface and no one wants to fund it individually, but it's still a very important part of maintaining good software. And that's really why I'm interested in,

like, for Django especially, like, because that's what I'm involved in. But also, like, in the general sphere, like, how can we get a pattern of that that works well? Well, I guess before we close the show, though, maybe give some advice. You had a successful Kickstarter.

You've done this at least a couple times. You've thought about ways to formalize it. Give people that are listening to this show some advice, ways to replicate some successes you've done this at least a couple times you've thought about ways to formalize it give people that are listening to this show some advice ways to replicate some successes you've had whether it's that kickstarter or other models you're thinking of what are some some ways that other open source camps can replicate what you've done well yeah that's a good question like it's in some ways it's tricky like the the south migrations one in particular was very much a time and place thing like I had been working on that project for seven-ish years at that point, unpaid. So I see that one more as a pressure relief of people have been using it for so long that they very fortunately came out and said, oh no, we've already used this.

We know it's good. We're going to pay you money in retrospect, in a way, is what happened. But I think also part of it was being very clear about what you're trying to accomplish, I think was very important. And also giving something that people wanted. It's bad advice what I just said about maintenance and that kind of stuff. But at the end of the day, open source needs some PR, it needs some advertising and business logic. You need to understand how to make people want to give you money be it businesses who have a different set of reasons to individuals but like both of them

you have to appeal to them on their own level like find the things that businesses businesses want find the things that people care about and appeal to those like if enterprises want like support contracts or stability maybe you can frame the conversation around like well you can pay for a support contract that actually funds all the security stuff if individuals want a new feature that sounds really exciting maybe you'd frame it around that new feature but like you still need to consider that you're still appealing to people and business with their own opinions their own thoughts and that's why you've got to really like think about your writing and what you're saying and make sure and you know, people are focusing there.

Are you focusing on businesses? You focus on the general public at that point in a Kickstarter general public. So, well, not just, but it's not the kick. So the Kickstarter I did over half the money was on the high end tiers from big companies.

Right. Okay. But like it had this cross, this cross appeal, but it really depends on the software. Like if you're writing like, you're writing archiving and checking software, that's probably more a business thing. If you're writing exciting home automation software,

that might be more personal. But it really depends. It's unfortunate that both times I've done this, I've had the advantage of, and it's definitely an almost unfair unfair advantage of I have a good history of implementing things and like people go oh yes that's Andrew he's pretty trustworthy with this stuff like we know he understands

it and even with channels I came in with like you know I'd spent a year working on prototypes I had working code at that point and so I'm sure that helped swing at least a little bit like coming in with some work already done and that's not necessarily sustainable like it is a privilege to have free time to work on open source in the first place and i would love to find a way of of fixing that that dichotomy of like we shouldn't

just have people who have the privilege of like not having children or not having partners or whatever how there's time to work on open source um and if the only answer is you have to come to table with something in the first place that's still not really helping them it helps certainly but it's not solving that problem yeah and so like i i had like an unfair leg up in that in that situation well before we close out is anything that we didn't get to ask you about django python channels the community documentation first uh different things that are tried and true for the community you're a part of? Anything you want to say?

So I'm going to put in here a really good word for the Python community because I came to Python in 2006 and I am still a very proud member of the community because of the level of friendliness and help and outreach and just niceness that you get from all parts of the level of friendliness and help and outreach and just niceness that you get from all parts of the community like you know especially in the last the last few years it's made great

strides in being more opening and welcoming as well and like like going to pycon going to janga con like they're just great places to go and so like i honestly i would say like if you run an open source project i I would advise you, don't just consider the technical part of a project, but the community is a really important part of it as well. Like, have positive discussions, welcome people, especially beginners. Like, beginners are your lifeblood of an open source project. Like, a beginner today is a core developer tomorrow, right?

Like, welcome people and try and help develop those people. And like, if we all did that, I think the world would be a better place for open source. On that note, I have to mention request for commits because we've been talking about drive-by contributions, contributor onboarding, all sorts of things on that show.

It's the human side of code. It's about sustainably running open source, forming open source businesses and licensing, all those fun things around it. So, you know, to you, Andrew,

if you haven't listened to it yet, go to change.com slash RFC. You'll find a show there or the vanity url rfc.com or sorry it's dot fm rfc.s i can't even get it out you can do it rfc.fm i promise i got it right um it's been a blast producing that show we actually just did a show that's coming out soon with brendan eich and i'm not even kidding you for like an hour plus a little bit, we got like the history of the internet and how we got here in terms of browsers and motivations from companies like Google and Microsoft and others, Mozilla, of course. But he couldn't talk much about that because obvious reasons why. But a great history, not only of like the role JavaScript played in it,

but how open source has been funded or is being funded and living based on this history of browsers. So that's a great show to listen to on that note

of what you just mentioned there. But that's all we have for today's show. Anything else you want to mention? Any last minute shout outs? No. If you're interested in channels,

go to channels.readthedocs.io. And if you're interested in Django, go to djangoproject.com. There's plenty of documentation about those to look at. And we'd love to have you come and work on stuff. Cool. All right.

With that, let's call the show done and say goodbye. Thank you very much, guys. Thank you, Andrew. Bye. We'll see you next time. I'm out.