The Changelog: Software Development, Open Source - Caddy HTTP/2 Web Server (Interview)

Episode Date: October 31, 2015

Matt Holt and Sebastian Erhart joined the show to talk about Caddy the HTTP/2 web server written in Go. It's time to serve the web like it's 2015!

welcome back everyone this is the change log and i'm your host adams dicoviac this is episode 179 and on today's show jerry went solo talking to matt holt and sebastian erhart talking about caddy the h2 web server written and and go, made for everyone. And a special thanks goes out to Justin Dorfman for creating the issue, and Carlicio Campos and many others for thumbs-upping this suggested show. So thanks for your support. And if you want to suggest a show,

go to our ping repo on GitHub. That's github.com slash the changelog slash ping. You will find issues there. Submit one and at mention any developer out there on a project. And we'll do our best to dig deep and get them on the show. We have four awesome sponsors making this show possible. CodeShip, TopTow, Imagix, and also Linode. Our first sponsor is CodeShip, a hosted continuous delivery service focusing on speed, security, and customizability.

Easily set up continuous integration for your app today in just a few steps and deploy when all your tests pass. CodeShip has great support for lots of languages, test frameworks, and notification services. They even integrate with GitHub and Bitbucket, and you can deploy to cloud services or even your own servers. Get started today with their free plan, and when you upgrade to a premium plan, use the code THECHANGELOGEPODCAST and save 20% off any plan you choose for three months. Again, the code is THECHANGELOGEPODCAST. Head to codeship.com slash thechangelog to get started, and now on to codeship.com slash the change law to get started and now on to the show all right everybody we are back and we are excited today this show actually came to us by popular demand um two of our members justin dorfman and Carlissia Campos,

were advocating for a show on the Caddy web server. They opened up an issue in Ping and didn't stop there. I think they were tweeting about it. Many plus ones came in, and we finally relented and said, man, I guess we have to have a show about Caddy. So we're excited. We're here joined by Matt Holt and Sebastian Earhart. Guys, welcome to the changelog. Thank you. Glad to be here. So Matt, were you

surprised that you had such a group of people excited to hear all about your fledgling web server? Yeah, seems to excite a lot of people and i mean i'm glad i just i am surprised because it's it's a web server it didn't seem that exciting on the surface well you know us developers we get excited about all sorts of things uh and by the way i'm quite excited as well as a a bit of a web server uh nerd as well i i do geek out on these things. So I'm quite excited about Caddy. Had not actually even heard of it previous to them opening up that issue. And I think they even had a little bit of a back channel conversation in our members only Slack room about

it. And they, Justin and Carla C ganged up and decided they were going to bombard our GitHub issues. So let's learn a little bit about you guys, and then we'll get into Caddy, and we'll find out all about it and why it's cool, why so many folks are interested in it. Matt, let's start with you. Can you go ahead and give a little bit of your background and just tell us who you are? Sure. So I'm currently a student at Brigham Young University, and we will be graduating here in December, actually, with a computer science degree. I love I don't know, I love getting outside. Hiking in the mountains is great. Bicycling down mountain roads is really fun.

And programming in when I'm not doing homework. So I look forward to the day when I don't have to do homework after a day of programming. And I actually started CADE during my hardest semester at college here. It was just earlier this year, the winter semester, January through April, when I was in four really intense programming, senior level programming classes. And I just had to step away from that for a while and work on a side project so it actually brought me a lot of sanity as well so you you were slacking so to speak or you were you were distracting yourself from all your programming tasks by starting up a web server is that what you're saying uh more or less i mean i did all right in the classes well that's good to hear you know uh most people, when they want to relax,

they'll watch TV or play video games, but, uh, they're also not here on the changelog. So I guess it's paying off for you. Um, very cool. So you're, it's still in college. Do you have plans, uh, post-graduation or are you still just feeling things out? Still feeling things out. All right. Well, maybe we'll have to catch back up with you here after a while and see what happened. Yeah. Sebastian. Yeah. Nice to meet you. Welcome to the changelog. Why don't you go ahead and introduce yourself to the listeners. All right. I'm an information technology security student. I'm from Austria.

And I'm going for my bachelor's degree at the moment. I started programming when I was about 9 or 10 years old. And doing that when I've got nothing other to do. And I got into Caddy because I was looking for a cool open source project to join. And I started off with a pretty small pull request for adding something to a middleware. And then Matt asked if someone wants to do the Let's Encrypt thing for Keri, and I volunteered, and well, here I am. Just like that. Yeah, just like that. So you were just looking for some open source to contribute to.

Yeah, right. What sparked that interest, that desire to get into open source? Well, I worked as a programmer for about three years and there is nothing really I could show off to somebody who asked me, what have you done with your life?

So I figured I'd join an open source project to have something to show off. Yeah. Good reason to do open source right out there in the open. People can see your technical chops. Matt, same question. So at BYU doing computer science things, when were you first exposed to open source software

and what got you interested and involved? I think I started doing open source in about 2011. It's about my sophomore year of college. And I don't know, I think I just started writing code and putting it up on GitHub. And then I realized that coding could be a very social thing to do. And I really liked that because I kind of grew up on a farm in Iowa, kind of away from people. And it was neat to be able to work with people remotely and doing this kind of work. And so I really enjoyed the collaboration aspect once I started putting some code out there and contributing to a few projects in little ways. It was just very satisfying.

Caddy notably is a Go project. I'm curious your interest in Go and how that got started. I picked up Go at my last job. We were looking to swap out our.NET code base with some leaner technologies, and Go was definitely a good match. And so I was assigned to write a new service in Go for our company. And so that was really fun. That's how I started it. But I just, I love the language. It's very productive. It's very simple, elegant in most ways. It's not perfect, but I found that using it in school

in my assignments, I've had kind of a competitive advantage over other students because I can crank out code, productive code, more quickly than my classmates who are using Java, for example. So your classes, they just let you pick your own language? I think when I was in school, you pretty much did what they told you to do. Yeah, in the senior level classes, a lot of it is kind of up to you. That's awesome.

Yeah. Sebastian, how about yourself? You have been doing Go for long? Well, on and off for a few years but um as the major language to write coding it's about since april this year okay um i followed the the go development since its initial release but um i just got really into it this year. Very cool. All right.

Well, let's get to the heart of the matter here, which is the Caddy web server. So from the homepage, it says that Caddy is a lightweight, general-purpose web server for Windows, Mac, Linux, BSD, and Android, which caught my eye. It is a capable alternative to other popular and easy to use web servers. Now, Matt, I went back and I saw on your Twitter, you have a pinned tweet, which was your announcement of Caddy back in April. And in that tweet, it says a capable alternative to Nginx or Apache. It seems like you've slightly altered your language there. Yeah, I figured it wasn't a good idea. I didn't actually expect it to get much attention.

But I figured once it did, I probably shouldn't call out other products by name. That's how you get attention. Yeah. So they're actually great. I'm an Nginx fan myself. But I just wanted to make it more generic. Yeah, so I'm'm an nginx fan myself um but i just wanted to make it more generic yeah so i'm also an nginx fan and i've uh i cut my teeth on apache back in the day cut more than just teeth probably um we all know that we all have been there and

very thankful for the apache project it's uh served many web pages over the years and done so admirably. But also an Nginx fan. And curious, you know, this seems like an audacious project. Even more audacious, perhaps, when you first launched it, when you're like, I'm here to take on Nginx and Apache. Now you've kind of settled down from that a little bit. And I'm just wondering, you know, what got into you when it's just like, I'm going to write a web server? I think the, you know, I make a lot of little websites, either for myself or for software projects or for other people or for school.

And I just needed a quick way to, to get a production capable web server up and running really quickly and easily. And Nginx is pretty good. It just makes it can be a little tricky to configure sometimes. I don't know if you've ever had those half day projects where you set up your web server. And that's like all you do. It can be hard to get it just right. And so I just wanted to whip up something that... I don't know if you've ever tried setting up Nginx to front a fast CGI application for a PHP site. That's the worst thing ever done.

Yeah, indeed. Nginx is much better as more of a pure reverse proxy. It seems like the fast CGI stuff. I mean, there's, there's just more hoops to jump through with that particular setup. So whenever you have a situation or I've had a situation where I have perhaps the same server that's serving a rails application and like a WordPress install or something like that for the blog section. And yeah, the WordPress side with Nginx,

it's gotten better, but historically has been a bit of a pain to get set up. Yeah, and so what I noticed is that a lot of my sites that use, for example, PHP back in the day when I was a big PHP fan, I used the PHP mostly for simple things that were kind of like minutia that I kind of wish my web server would just take care of.

But I just need a little dynamic element. But I felt like it was a pretty common use case. And then there were other complications, like, again, Googling how to set up fast CGI with Nginx and PHP. All the tutorials are different. It's a bunch of security whoopty-doos that you need to worry about. And I just didn't need to worry about. I just didn't even want to go there. So I'm like, you know, I'll just write a web server in Go.

Go seems like a great language for this. I'll just write something that's simple, and I can just type in the name of the server and the command line, hit enter, and bam, it just kind of works. And another thing that I really wanted, I wanted my configuration file, my website config, to live with my website, not with my web server. So why'd you want that? You know, because I just like to CD into the directory where the site is, and then just run the web server from there. And then it'll just pick out the config file in the present working directory. Just found

that so much more convenient than having to go over to a system folder or a central folder for the web server and then change some config files there and worrying about including and ordering as well. Yeah. Well, we may be getting ahead of ourselves a little bit, but Caddy does

support virtual hosts. So in that case, you have multiple websites under the same host. In that situation, is it run from a Etsy directory or from some sort of common configuration directory? I mean, you can. You can specify from the command line where you want to get the configuration file from. But by default, it just pulls the caddy file from the current directory. So it's just very convenient. Options, people love options. Of course, options lead to configurations

and people don't always love configurations. Let's talk about your audience. You mentioned you make a lot of small, somewhat static, somewhat dynamic sites. Who's Caddy built for? Is it built just for Matt Holt or are there other audiences in mind? It was built for me. Met my needs pretty well. And then people started opening issues. And I

figured, well, I guess I better get to work. And so I think the caddy is really well suited for people who don't want to. Well, they're suited. It's definitely suited for people who run lightweight, small websites and you can use caddy in production, not saying that it's perfect, but you can use it in production. Um, if you have kind of a complex site or a site that runs on some dynamic platform like Rails or Django or something, it might be a little trickier to configure it. But we're working on that. My main focus right now is static websites

and really bringing the power of the Go standard library out in making the web server do what you need it to do without having to make a dynamic website, if that makes sense. So if you can get away with a static website that just has a few dynamic functions that you need, maybe Caddy's a good fit for you. If you have this really dynamic and kind of heavy website going on, then stick with Nginx.

Or if you're serving 100,000 requests a second, definitely stick with something very, very high-performing. But Caddy is pretty good. It's pretty competitive for most things, I think. You said bringing out the power of the GhostHunter library. Can you give me a for instance or an example? What exactly do you mean by that? So, yeah.

So Nginx and Apache, for example, I'm used to server-side includes. So if I want to make a static site, but I don't want to have to keep repeating the footer on all of these HTML pages, then I just do a little server-side include, and the web server will kind of pull these pages in and serve them statically.

Caddy can do this too, but it uses Go's template library, which is actually really powerful. And you can do some really cool stuff, and it just parses the whole HTML file as a template. And so anything that Go's template library lets you do, you can do with your HTML file as a template. And so anything that goes template library lets you do, you can do with your HTML file without having to make a dynamic site, kind of. Okay, so it's akin to having a static site generator

kind of built into the web server. Kind of, yeah. Interesting. So it's built for Matt Holt. it's built for people who have pretty simple sites that are have a little bit of you know magic going on but not too much it also seems like it's built for cross-platform as many things in go take advantage of that universal binary or that ability to compile cross-platform yeah that's actually a good point. Have you ever tried setting up a web server,

like a production web server on Windows other than IIS? Never tried. Yeah, it can be a little weird. Because these web servers are made for Linux, basically. Unix systems. But yeah, Caddy, first-class support for Windows kind of thing. So if you need to use Windows, or... You know, Caddy's

great, too, for... We're not just targeting technical people here, but designers and writers who just use Windows. That's the environment they feel comfortable in. They can use Caddy. It's a great fit for them. They don't have to learn the technical ins and outs of a real hardcore web server.

Very cool. One thing I mentioned at the top of the show was that it promotes its Android support there. Can you speak to that, how that works, and what that's all about? Yeah, so Go compiles to Android, and actually recently it compiles in a way to iOS, which is kind of cool.

But if you download the Caddy build for ARM Linux, you can sideload it onto your Android device and actually run it there. And that's kind of fun because you can, I don't know, if you have a folder with files that you want to share with your local area network from your phone, you can do that now. It's kind of a, it's, it's kind of a cutting edge way to do it. Like this is not a refined feature or, you know, use case yet, but it can be done. Very cool. That sounds like a good place for a break. When we get back, we're going to

talk about configuration as it seems like that was an area that you focused on quite a bit is getting that user experience just right. We'll take a break here from one of our sponsors. And on the other side of the break, we will talk about caddy files. We'll be right back. Say hello to TopTile designers. Our friends at at top towel have done something really really awesome they've expanded into a new market they're talking designers top has been known as a thriving network of some of the best software developers and engineers out there many of the developers in their network know extremely talented designers, and they've always had this sort of informal relationship with designer involvement in TopTile.

They've done a little bit, but it hasn't been an exact product, so to speak, or internal model. And so they've expanded, they've evolved. Today, they are extremely excited to announce the official launch of TopTile Designers. What this means now is the same experience that you've had on both sides of the fence, whether you're someone that's looking for really awesome designers, or

you're a really awesome designer looking for really awesome opportunities, this is the place for you. Not only if you're engineers, but also if you're designers out there as well. So designers, listen up. It is time to go check out toptal.com slash designers. That's T-O-P-T-A-L dot com slash designers. And tell them the change I'll send you.

All right, we are back speaking with Matt Holt and Sebastian Erhardt about the caddy web server and HTTP2 web server. But before we get into the feature set, let's talk about configuration. That's where most people who are dealing with web servers live, sometimes die. You have a caddy file. So when I first saw this, I thought, Oh no, another file file. Uh, it seems like this, uh, thing file, file naming convention meme continues to propagate.

I'm not going to hold against you too much, but we got gem files and proc files and now caddy files. And there's all these files, files, um, just, we don't have to, we don't have to hang out here all day,

but maybe just speak on where that inspiration came from. You liked that format? Yeah, I mean, as far as the name caddy file, it just seemed to kind of flow off the tongue. But the idea for an actual configuration file is, in a way, it's actually kind of a stopgap. But a configuration file is, and I'll get to that in a second, but an actual configuration file is kind of the standard,

it's the normal way to configure a web server. You make a file, you put some directives in there that tell the web server what to do, and then when you run the web server, it just loads the file and configures itself in memory and off, you know, on its way. And that was just really easy. And I just wanted a configuration that I could persist with my website.

So I have my website folder, and then I have a caddy file in that folder. And I can just run caddy from there. And it'll read the contents of the file consistently every time on any platform and pretty much every environment, and it'll work the same way. And this config file, though, is not actually the end goal here. I don't intend for Caddy to only be configurable via a file. Do tell us more. So in the future, and there's a roadmap here,

but in the future I do intend to write an API so that you can actually dynamically set up Caddy while it's running and change its configuration while it's running without having to rely on just a file in the file system. Oh, that'll be awesome. So you're going to give us a launch date, Christmas 2015, right? I wish. No, I don't know the date yet.

I've actually started kind of stubbing out an API on a branch, and you can try it. It's a little racy, and it's a little clunky so i think i need to redesign some things but it will happen i definitely want it to i envision a nice web front end for people to just open in their browser and be able to configure their server from there with this nice gui and having maybe stats and monitoring built in one day if they want. Yeah, I wasn't expecting an actual date. This is just an old hobby of mine, which is I ask developers to commit to a ship date,

and I watch them recoil. Oh, that's nice of you. Oh, yeah, got to stay entertained somehow. Very cool. So let's talk about the caddy file itself, syntax, inspiration. Do you come up with your own thing. Is it inspired by something? It looks like it's pretty simple. You have some directives, but they're just like keywords. You type in, I like this one, browse, for instance,

and then you give a path. That's like setting up directory indexes for a specific, I'm assuming, subfolder of your system there. Where was the inspiration for your syntax of the caddy file? The inspiration was me wanting not to type very much and not having to be very clumsy with my typing. So not much punctuation. I didn't want to think about syntax, really. I just kind of wanted it to flow. So when I go to set up a web server, what's the first thing I am thinking of? Well, probably the website address, the URL, the domain name. So type that in. That's always the top of the caddy file.

Hit enter a couple times and then, okay, I want to serve clean URLs for HTML files. So ext and then dot HTML. So it'll automatically serve HTML files without needing a.html in the URL. Then let's see, I want to be able to share files in my photos directory because I have this album of pictures. So then I do browse and then slash photos. You don't have to think about curly braces and colons and quotation marks typically in the caddy file, which is really nice. Yeah, I mean, I think a recent trend with configuration files as well, we'll just type it out in JSON because we all love that format. And it seems like that is more focused on implementation

because it's really easy just to suck in a string of JSON and parse it. I'm sure Go has a built-in JSON parsing in the standard library. It does. So you had to write your own parser for this? Or how is it implemented on the Go side? Yep, it's a home-brewed parser. And I've actually gotten a little bit of flack for creating yet another configuration syntax.

This one's very specific to this use case, though. This is not a general-purpose config syntax. It meets some needs that JSON doesn't. For example example using arrays as keys ultimately you have maybe you have a set of server blocks in a caddy file

where each server block configures one or more hosts so instead of having to set up a host as a key and then have it point to some configuration because following pointers is kind of confusing and then you have to like teach people about that and i just didn't want to go there so just specify multiple hosts the key and then they all share that same configuration

is one example so yes i i know that json serialization and deserialization is very easy, but I wanted to focus on the user here. This is about the experience. I want to make it easy for people to create for the web and make the web better that way, more accessible. I'm all for that. So another aspect of this you have of the caddy file is placeholders. I think that's part of the caddy file. Can you explain what placeholders are and what they're good for? Yeah, they're just little strings enclosed in curly braces. This is one rare case where you would use a curly brace or punctuation. But it just fills in a value

specific to the request or the response usually. So let's say that you're setting up your log directive. You can say log, and then you can specify the file name, and you can specify a format, a custom format if you wish. And that format, you would use placeholders because you don't know the request time for all the requests. You can't hard code the method and the URL. So these are kind of like dynamic replaceable values that give you access to various things

like the URL of the request and the time of the request or the response body length and stuff like that. Yeah, headers, host body method, so on and so forth. Pretty much anything that you could pull out of a request, you can use it and key on that and basically modify your configuration based on that, correct? Yeah, kind of like a variable in an Nginx config,

no dollar sign, whatever, but not scriptable. I don't want to go the scriptable route. Why not? You know, as much fun as it is, it's kind of a can of worms. It's like a rabbit hole. Well, I think it's kind of a fine line line because what you have here is you have some dynamicism you have

templating for your server site includes you're like almost in a full on programming environment you're almost there and then it's like where do you draw that line so where do you draw that line that's a good question drawing the line at You're almost there. And then it's like, where do you draw that line? So where do you draw that line?

That's a good question. Drawing the line at giving you just enough power to be dangerous, but not enough to kind of hang yourself. Not enough where the learning curve is beyond what someone who doesn't know how to program can do. Not enough to the point where you have to get super frustrated at things. I feel like if you're using it and something's not obvious, there's either a bug in the documentation or

in the implementation. That needs to be fixed. If O'Reilly calls you to write a book about it, then you've probably taken it too far in this case. Come write a book about how to program the Caddy web server. At that point, you'd be like like oh man yeah i think i went too far but i got a book deal exactly blog posts are okay but yeah you shouldn't have to script caddy i think that's a little little far taking a little far

i guess uh some of the features play into the caddy file of course because you're going to be enabling or disabling features um your headline feature of course is h2 support out of the box um maybe sebastian why don't you explain uh h2 support in caddy and how it works and and all that stuff for us well uh Well, we use a library. So we didn't actually implement it ourselves, but we use GoLibrary for it. That's the beauty of open source, right? Yeah, it's open source. It's on GitHub, I think.

It's supposed to be the better version of HTTP, but not everyone agrees on that. Me neither. Oh, really? Yeah. Well, now we're getting interesting. Tell us more. I everyone agrees on that. Me neither. Oh, really? Yeah. Well, now we're getting interesting. Tell us more. I didn't know that. Uh-oh, Matt. Dissension. Do you have dissension? Do we really want to go down the rabbit hole?

I kind of do. But it sounds like you kind of don't. So let's just... You can turn off HTTP2 if you want with Caddy. No, you kind of don't. So let's just... You can turn off HTTP2 if you want with Caddy. No, I don't in principle have a problem with it being there.

It just could have turned out way better than it has. I see. So it's just the lack. It could have been better. Not that it's bad. It's not bad. It's just some features which got in and some features which got in and some features which didn't.

Well, it could have been better. Yeah. We'll see. H2 should be much easier to improve upon than H1 was. Yeah, of course. With time. Yeah, and if you are interested in H2,

the nitty-gritty details of the protocol, go back to changelog.com slash 161 with Ilya Grigoryk. We did kind of a comprehensive overview of all of its features, pipelining, multiplexing, HPAC, all the things that H2 has. And we will not talk about those today because we talked about them in detail there. And as you guys mentioned, the beauty of open source is that you're building on top of other people's work.

And in this case, you guys got to use, I think it was Brad Fitz's H2 library. And that's an awesome thing because now, so many people get the benefit of it who are using Caddy without having to know those intimate details. So nothing wrong with that. And actually, the Brad Fitz H2 library just got moved into the Go standard library on tip

and is enabled by default now. Now, obviously, caddy's not using Go tip, but it will come soon enough. It will have full HTTP2 support, whereas up to this point it's been kind of experimental but pretty soon it will be a full like production ready http2 library let's go tip is that like the master branch of the development or uh yeah basically okay okay so you have h2 and you can turn it off. It's on by default. Of course, it's only for clients, I assume, that support HTTPS or TLS. You also have some other things. IPv6, Markdown.

We can talk about that a little bit. WebSockets, Virtual Host, as we mentioned before. Server name indicators and extensions. Why don't you pick your favorite of those, Mark? Or, excuse me, Matt. And we'll... I read Mark down, and then I just pronounced your name Mark. And we'll talk about that.

What's the most exciting thing beyond H2 that's not Let's Encrypt? I'm a huge fan of SNI, server name indication, which is a TLS extension. And this is pretty standard. Like, this is not a mind blowing feature, but it's so convenient and important because now with the same socket, you can serve multiple host names that are over a secure channel.

So that's, that's a really important thing. In fact, Caddy's virtual host feature, which allows you to set up multiple sites in the same Caddy file and serve them from the same port, would not work for HTTPS sites without SNI. So that's pretty cool. So the big win there is, and correct me if I'm wrong, is that if you're hosting a series of websites, perhaps on a digital ocean,

VPS, you do not need a new static IP address for each of those hosts. Exactly. And you can secure those channels on even the same exact port just with a single IP address and port combination. You can have all these different HTTPS hosts. Yep.

Exactly. Love that. And I think that is, like you said, that's not like a unique feature of Caddy, but definitely awesome to see it there. It also doesn't have, it doesn't work on Windows XP, which is finally, I guess, starting to become not too much of an issue.

I know that was a blocker for some people. Yeah, that's pretty old now. It is, it is it is it's still out there unfortunately um i don't mind pushing the envelope a little bit i think people need to upgrade all right i absolutely agree i think there's you know your mileage may vary there are certain people who are still supporting ie8 and whatnot um and some people on xp so in those cases yes unfortunate souls So in those cases, yes, unfortunate souls. In those cases, they can't use SNI,

but I think especially if you're building a modern web server for the modern web course, you got to have that. I think one other, a couple other little technical features I want to point out too is that Caddy is a multi-core server. And so it's multi-threaded in the sense that it will spin up new Go routines or lightweight threads per request. So it's very fast and efficient that way. It has a different model than, for example, Nginx, which is multi-process.

But Caddy can utilize all the cores, just kind of like Nginx can, except that it doesn't have to rely on the operating system scheduler. Because the Go scheduler actually understands Go code and can make more intelligent scheduling decisions, which is really cool for high performance sites. And that's all on by default and just kind of works. Let's talk about extensions. Extensions seems like an interesting one. As you browse the docs and you're looking at the different directives, there's a handful of them that are marked as add-ons, such as the CMS support, Git, IP filtering, search. And as you click on those, it seems like these are using your extension feature.

Can you speak about that? Sure. Anyone can write an extension for caddy. You can choose to publish it on the website like some of these are here, or just use it internally. But basically, the idea is we don't want caddy's code base to grow too large and become unwieldy and have a lot of cruft that would kind of defeat the lightweight aspect of it, which really is a feature. It's easier to maintain. So these add-ons is how we decided to deal with this because some people, including me, like for example, I wanted a built-in site search. I didn't want to have to set up and maintain some other search infrastructure or use an external search service, which can get a little dicey. So what better way to do it than to have it written in Go, built into the web server, having complete access to the config as needed, the web server's internal configuration,

and to be able to generate an index of your site and then have it searchable. And so these are all built by, there's like a, yeah, there's the Git add-on, for example, built by Abiola Ibrahim. He built this Git add-on where you can deploy your site with Git push and your web server will automatically pull in the new changes. Super convenient things that aren't making the code base unmanageable. But you can just check them when you download Caddy and it will do a custom build for you. And you can use those. So how do those get in?

Is there like an ecosystem? Do they have to come and send you an email and say, hey, Matt, throw me up on your website? What's the situation there? It's just a pull request system. Just kind of open. It's pretty casual. You open an issue and say, I'm going to work on this.

And then you can do that. And there's some docs that kind of show you how to get started and a little template. And then it's not too bad. And then you just submit a pull request to register your add-on in the build server repository. And once we merge that in and deploy the new build server, then there you go. Then you need to submit some documentation for the website. I think the search add-on is really rad. It's definitely something that comes up all the time with static sites

is you just want to add a little bit of a search function on there, and you either have to do the Google Insight search, which is wonky, or use a third party. I think on my website I use, now I'm forgetting their names. I'm going to give them a shout out. I don't know, a third party who provides, you know, they index your site and provide a search API that you can query with JavaScript. But having that built right in, that's pretty handy.

Cool. Yeah. And it's a really good way to give some Go libraries some exposure. The search add-on built by Pedro Nasser lets you, it uses the Blevi library, which is it kind of stands out a little bit it's like well you know we support ipv6 and we support markdown it seemed like completely different things what was the logic behind supporting markdown it's kind of a first class citizen oh the fact that some people hate touching html and just love to write web pages in markdown so with this ad or with this it's actually not an add-on. It's built into the core. You can serve Markdown files and they'll render as HTML on the fly or you can pre-generate the HTML.

Was this scratching a specific itch you had or was this thinking of more general users? More general users initially, but I find myself using it more and more because you can specify an HTML template and still serve really nice, authentic HTML pages using just Markdown. And then, of course, we have this Hugo add on with caddy. So you can actually Hugo's a static site generator written in Go. It's really popular. And so that kind of does something similar. Markdown is just kind of

a more very simplified, lightweight version. Very cool. Well, I think we've hit up against our next sponsor break. So let's stop here. Take a break. On the other side, we will talk about Let's Encrypt, which is very exciting to me. And apparently to you all as a future awesome feature of Caddy. We'll talk about that on the other side of the

break imagix is a real-time image processing proxy in cdn and let me tell you this is way more than image magic running on ec2 this is way better it's everything your friend and developers have dreamt of output to png jpeg gifPEG 2000, and several other formats. And if you're like me, and you've ever argued with your boss or a teammate about serving retina images to non-retina devices, you'll appreciate their open-source, dependency-free JavaScript library that allows you to easily use the ImageX API to make your images responsive to any device. Now all this takes a platform and the ImageX platform was built on three core values flexibility and quality, performance and affordability. When it comes to

flexibility and quality ImageX has over 90 URL parameters that you can mix and match to provide an unlimited amount of transformations that you need for your images. And they take quality very seriously. And because of their commitment to quality, several top 1000 websites in the world trust them to serve their images. Now when it comes to performance, Imagix operates out of data centers filled with top of the line Mac Pros and Mac Minis, and they're set up for a completely streaming solution. This means your images never hit the disk. Images are served by the best SSD based CDN for delivery around the world anywhere extremely fast.

And while we're talking about speed, almost all the image processing happens on GPUs. This means transformations are super fast when compared to competing virtualized environments. And lastly, it's all about affordability. Everyone wants to save a buck. That's how the world works. Because ImageX processes close to a billion with a B images per day, they're able to make certain optimizations at scale and pass those savings on to you to learn more about imagix and what they're all about head to imgix.com slash changelog once again imgix.com slash changelog and tell them adam from the changelog sent you all right we are back and yes i found out who's providing my site search i think that's

the kind of thing that you just know but uh they've been just quietly uh serving me for years so i will give them a shout out swift type swift type.com they will index your static site and provide a nice easy api for you to add search to your site unless you're so lucky to be using caddy at which point you have that built right in there as an add-on so moving on let's talk about something that's coming down the pipeline currently being worked on which I'm excited about which is let's encrypt support, can you first start off by telling us what Let's Encrypt is, and then we'll go into how that works in Ducati. Yeah, sure.

So Let's Encrypt is a new thing where you can get valid SSL certificates for your servers fully automatically without really going to a CA and throwing money down its throat to get your certificate. They offer basic SSL certificates, so no extended validation validation or something. But for the normal user, it's the best thing that could happen, in my opinion.

Yeah, they're free, right? Yeah, they're free. Free SSL certs. Yep, free and valid. Free and valid. The green symbol in your browser. I'm super excited about it. I think it's been something,

is it the EFF that's doing this? I can't remember who all's involved. It's letsencrypt.org. But it seems like they have had like, it's coming, it's coming, it's coming. And we're all sitting here waiting to save some money and get all of our sites on HTTPS.

Even the ones that right now, it's like, it's my personal site. Do I really want to spend X dollars a year to encrypt it? Yeah. Is it going to come? Is it going to come or is it just going to keep coming? Yeah. So the last thing I've heard is that they want to launch

officially mid-November. Okay. So it definitely is coming. That's why we started to get pressure on for getting Let's Encrypt integrated into Caddy. We'll go

into how that works, but first let's talk about the perfect end user experience. When it's totally done and I'm a Caddy user, tell me how it will work. I just flip a switch and I'm encrypted? Tell me how it will work. You put your site into your caddy file,

hit carry up, and there you go. You are SSL encrypted without anything. Just shut up and take my money. It's free. You can't do that well figure out a way oh you got a donate button because that's going to be amazing for a lot of people um i think you know ssl but we talk about the free aspect of it and that's part of it um but just the pain and the technical uh drudgery of setting up a certificate over all these years has been too much of a high bar for many people who otherwise would be happy to just flip that

switch so this is going to be awesome um maybe talk about the where you've been so far and where you're going with it and give us some of the technical details of how this works you sure um well i started a few months ago to uh implement it in go because um we wanted it in in caddy and we just wanted it to be native in go so um we decided to write a new library in Go to handle it because the command line utility, which comes from the Let's Encrypt guys, is in Python. And we can't really interface in a good and meaningful way with Python code from Go.

So we thought it's a good idea to write our own. So have they published an API that you're coding against? Yeah, they published an RFC for their so-called ACME spec. ACME spec? Yeah, A-C-M-E. Automatic Certificate Management Environment. Oh, man. Yeah. It's the RFC.

They are now at draft number four. I think I started at the initial draft. And there were a few changes over the course of the month. First, I started integrating it or writing it quite fast. And that stopped a bit after the first few changes

because I wanted to wait a bit until it gets more stable with the API. Well, basically, the API for Let's Encrypt is a the API. Well, basically, the API for Let's Encrypt is a JSON API. It operates with JSON web keys,

JWK and JWS, JSON web signing. The technical details are that you as a user create a private key for yourself and you tie that to an account on the Let's Encrypt server. So you register with your email address, for instance, and that private key and from there on you can request certificates with your private key. Of course, that's pretty insecure.

You have to somehow prove that the domain you want this certificate for is indeed yours. That's handled by a few challenges where you have to fulfill certain tasks given by the server in order to authenticate the domain that is really yours. Is that clear what I'm talking about? Yeah, yes. So these are things

like upload a file or set a header. Yeah, the easiest challenge is the so-called simple HTTP challenge. So you tell the server, hey, I want a certificate for the domain example.com. And the server sends you back a token, which it expects you to serve at the certain URL path on that domain. And it will resolve

the IP address of your server and look if it's really there. If it's really there, then you pass this challenge. And that's it? You're done? You gotta pick one challenge? Well, it depends. As with many

things in IT, it depends. Usually, if you can fulfill a challenge without any problems, then yes, this was it. You just have to ask for the certificates and you get the certificates to download. and this challenge does not work out for you. There are other challenges to do and they always come in pairs. So the server decides what it wants to see from you. You're not the one deciding what you want to show the server. So how does the Go library and then Caddy thereafter know who I am and who my Less Encrypted account is?

Do I have to set that in the config or something like that? Well, we generate a config for the user. So we have to get somehow the email address, for example, from the user. We haven't exactly worked out how we want to tie this into the user experience. But yes, we save a configuration file along with certain other things we need to interface with Let's Encrypt

in a folder that's .caddy, I think, or Matt? Yeah, there will be a.caddy folder in the file system. In this folder, there will be a.caddy folder in the file system. In this folder, there will be other subfolders per certificate and per user and also per host. So you can peek in there and copy stuff around. But actually, it's all managed for you.

That's the goal. Yeah yeah I was working on that this morning that's going to be fun yeah are you guys moving along are you stuck tell us how it's going well I think it's going along well I'm not stuck

Sebastian understands the ins and outs of the spec, the protocol spec pretty well. And I'm just using his library now in Caddy on a private branch. Still, it's pretty early days, but I have been able to successfully generate a new certificate at startup from a local Boulder CA. Boulder is the name of the Let's Encrypt Certificate Authority server. So in a purely test environment on my local machine, I have been able to get a certificate in a private key and then serve a site using that.

So definitely making progress. The rest of this is going to have to focus now on the user experience, making sure that it's as unobtrusive as possible, and managing the certificates and keys and taking care of renewals when the time is right. Yeah, have you guys even thought about renewals? Yeah, I mean, obviously you've thought about it because you just mentioned it, but I hadn't. How's that going to work?

Every certificate which is issued by Let's Encrypt CA gets its unique URL on the server. So you can issue a simple get request, which is signed by either your private key or the private key which was used to create the site certificate. And you either get a renewed certificate back from the server, or you have to create an entirely new certificate. So that's the renewal story. And as far as the caddy integration of that, I have some ideas still kicking around the details about renewals.

It will be automatic. It will be transparent to the user. It will involve probably a graceful restart of the server when renewal happens to plug in the new certificate. But again, we expect it to be fully managed on by default. So I think all the pain, thanks to Let's Encrypt and Sebastian's work, I think all the pain of dealing with SSL certificates

will finally be gone. Yeah, I think that's the kind of feature that will set Caddy apart and hopefully other web servers and other software packages will start to integrate because it's a huge thing. And I actually, I misspoke. I said, is it an EFF thing?

And I just want to clarify, Let's Encrypt is put on by the Internet Security Research Group, which is a California public benefit corporation, and it's sponsored by the EFF and other major sponsors like Mozilla, Akamai, Cisco, and a few others automatic of wordpress fame so uh it's a community effort with a lot of companies pouring um resources into it and i think it hopefully arriving quarter four 2050 i remember when this said arriving summer 2015 so i'm sure there's a lot of uh moving parts is even just integrating for you guys there's a lot of moving parts. Even just integrating for you guys, there's a lot of moving parts,

but hopefully it all comes together. And you guys, your goal, I suppose, would be you guys going to be ready to go kind of day one, November, or what's your integration roadmap look like time-wise? Well, I can speak for the caddy integration, probably not before their launch date. Fair enough. We don't expect any major changes to their protocol

as far as the underlying library goes. But regardless of that, I want to make sure that the management of the certificates is working well. So we'll probably, shortly after their launch, we'll probably have a special download, we'll probably have like a special download that you can do of caddy or a special build that has let's encrypt. And so a few people can try it voluntarily without just kind of forcing it on everyone all of a sudden. Because

it might be kind of jarring to some because again, HTTPS by default, there's a lot of considerations. Do we redirect the plain text to the HTTPS? And just making sure everything goes smoothly, we're going to work really hard for that. So hopefully shortly after they launch. So that's definitely a headline feature that is upcoming. And, of course, H2 requiring that encrypted connection. This allows lots of folks to easily get that optimized HTTP connection instead of having to fall back to H1.

So let's take our final sponsor break. And when we come back, we will wrap up the conversation talking more about Caddy's Roadmap, future features that you're interested in building and are currently building. And then finally, how people can get started using Caddy and our closing questions. So we'll be right back after this break. We're excited about our new sponsorship with Linode. They're huge fans of the show and are excited to support what we're doing here. And they want to invite every single listener of the Change Law to try out one of the fastest, most efficient SSD cloud servers on the market.

Get a Linode cloud server up and running in seconds with your choice of Linux distro, resources, and node location. They've got eight data centers spread all across the world north america europe and asia pacific plans started just ten dollars a month with hourly billing and a monthly cap on all plans and add-on services like backups node balancers longview and even linode managed and for those who are already familiar with linode, they recently switched from Xen to KVM, and the latest Unix benchmark showed a plus 300% performance increase. We'll drop a link in the show notes for those benchmarks for you to check out. Get full-road access for more control, run VMs, run containers, or even a private Git server. Enjoy native SSD cloud storage, a 40 gigabit network, and Intel E5 processors.

Use the code changelog10 with unlimited uses. Tell your friends. It doesn't expire this year. It expires the end of next year. So use it as much as you want. Again, that code is changelog10. Head to linode.com slash changelog and tell them the changelog sent you. All right, we're back.

We've learned a lot about Caddy, what it's good for, what it might not be so good for. We've learned about Let's Encrypt and the built-in flip a switch, get your site encrypted feature that is coming to a Caddy near you. Matt, what else is coming to Caddy down the road?

Well, like I referenced earlier, I really wanted to make an API so that you can just run Caddy on a server bare bones or from scratch without a configuration and then be able to, say, log into some web-based client and manage your server remotely with a nice GUI. I think that'd be really nice, and actually give access to web servers to a lot more people who don't understand even the command line and SSH or whatever. There's a lot of potential there. And it's not a unique feature. Other web servers have similar capabilities,

but I think that it's still necessary. So that's one. Anything else? I also want to really work on... So we talked about Let's Encrypt. The API, and the API

is not a user-facing feature directly. The web-based control panel would be something else. And I don't know if that's going to be a third-party thing or if I'm going to build one as well. But we'll see. I think we'll see what the demand is and what the audience becomes here as time goes on. Very cool.

Maybe one thing we should talk about briefly before we get into the getting started because i feel like it's the kind of question that many people will say why didn't you talk about performance um i know you you mentioned it briefly earlier on but let's let's speak specifically to performance i know you have a benchmark out there. Seems like, you know, a pretty basic one. And, you know, you have a very strong and well worded disclaimer about benchmarks. But it doesn't seem like performance. It seems like there's two things. First of all, Go tends to be very performant, especially with its concurrency primitives. And then the other thing is, you didn't seem like performance was one of your goals, right? It was user experience. It was ease of use, these things, modern feature sets.

But where does it stand? And what can you say about Caddy as far as its performance abilities? So right now, Caddy performs using a bare-bones configuration compared to default configurations of a couple other popular web servers, performs competitively well. That's the word. So by that, it does perform in my testing. Again, this is just me. As far as requests per second, tends to perform better than Apache.

Again, without tuning and almost as well as Nginx but if you're serving 30,000 requests per second or 50,000 requests per second maybe performance should be your primary goal and maybe you aren't just the typical average Joe user so that is to be considered. I don't want to like put performance under the rug and totally forget about

it because if I wanted to do that, um, then this would be a much easier project, right? Because just throw any amount of mountain of code into this project and make it do all the things that you want it to do. Right. Um,

not going to go there. Definitely performance is going to be a concern, not in a bad way, but like it'll be on our minds. It's going to be, it's going to take a front seat, I think, but not over the user experience. It'd be a fine balance and it's going to, we'll have to figure it out as we go. Yeah.

And one thing I might mention is something that I read that you said, which is that if there are any people who are very interested in performance and are good with those things, A, you would appreciate if they brought benchmarks, do their own testing and publish those results. And then B, if they have specific, especially if they have GoChops and know of ways of making it faster, these are things that you're wide open to. Is that fair to say?

Yes, that's exactly right. Cool. I thought that was fair because I did read it. I didn't make it up. Glad you read it. A lot of people, I always ask for performance. They want to know requests per second. They want to see the numbers. They want to see memory allocations. And, you know, go do your own testing for your own use case. In mine, I found it to be very, like, quite sufficient. And there are several sites using Caddy in production, even under load. And it's fine. Go is a pretty competent language for that.

And that's, I mean, they're sitting on port 80 or port 443 in production mode, right? Like, there's no, it's exposed to the wide open internet and it's running in production. Yeah. Very cool. Well, let's, let's close up with getting started. So say I'm sold and I'm interested in using Caddy for my next website, serve it in production, maybe with some HTTP2 going on. How do I do that?

I'd say download it for your platform, deploy it to your machine, and run it. You'll need a caddy file. So when you download caddy, you just create a caddy file. Typically it goes, again, in the folder where your site is. Very easy to set up. You can read the docs on the web page about how to do that. But once you've got a caddy file there, just run caddy and it'll just start serving your

site. Is there any sort of daemonization or backgrounding? I mean, I'm not going there. You'll have to do that on your own. I admit, it's true. You'll have to do that on your own. I admit, it's true. You'll have to do that on your own. And also, don't run caddy as root. I mean, you can, but probably not a good idea, just in case.

So you can use the setcap utility in Linux to give caddy permission to bind to ports lower than 1024 without having to be root. So I do that in production. Works pretty well. Okay. That sounds like some opportunities to contribute at least, maybe not into caddy core, but even a tool around caddy,

some sort of wrapper that would allow it to run as a system service or in the background and manage the daemonization of it. Or, I mean, there's tools out there that do those kind of things in general with binaries, so maybe even just a tutorial on that would be good for folks. Yeah, I agree. There are a couple blog posts about it. And actually, that's a really great opportunity.

If you're a writer, a technical writer, writing about ways to use Caddy effectively in production is a really good post that you could do. Probably draw some traffic to your site. I know people are searching for it. I see it in the analytics logs.

Oh, there you go. Analytic log-driven development. That's an idea. The new test-driven development there. Yes, yes. Search your logs. Shows people want it. Yeah, that's true.

That's true. All right, well, very cool. I'm excited. Yeah, that's true. That's true. All right. Well, very cool. I'm excited. I think this is a very cool young project. What's the status? Are you at a 1.0 yet? Are you getting close?

No, we're working toward it. We want, I honestly want Let's Encrypt. I want the API to be part of it. So it's kind of an ambitious goal, but we'll get there. And Caddy is a great, can I just put a shout out to all the contributors? Please do. Yeah. Caddy is a great project. Um, as far as the community goes, it is a young project. It just launched in April this year. So it's only a few months old, but we have contributors from all over the world. Um, you know, Sebastian joining us today is in Austria. We have contributors representing every continent,

all sorts of skill sets and backgrounds. And the contributions, it would not be what it is today without them. There's no way I could do it. So huge thank you. And feel free to get involved. There are definitely many opportunities for new Go developers,

but also experienced ones to contribute in very meaningful ways without having to commit to doing too much, if that's a concern. Very good. I can echo those sentiments, just seeing the community around you that even promoted you onto the changelog. We have a lot of people on our ping repo coming out and telling us you

should have this show or that show. We appreciate all those requests and we try to, you know, fulfill all the requests that make sense for the show and timing and whatnot. But rarely do we have such support that you got so quickly with all the plus ones to, to get you on.

And I think that's a testament a little bit to the go community and to the people who are genuinely excited and involved with caddy. So that's if nothing else, a very fun thing to be involved in. Yeah, I agree. All right, well let's do our closing questions and Sebastian,

I'm going to start with you on this one. Um, who is your programming hero and why? Um, well, one. Who is your programming hero and why? Well, I guess I don't really have one single person, but I really enjoy reading code from many different open source repositories. And I just really like open source code. It's so vast learning experience. They can get so much from all the code, which is available for free out there. So everyone is my hero.

Hey, I like it. I like it. I agree with you in the terms of how much you can learn from open source. I think one area of my life where I went really from a novice developer to more of a medium range, I don't know what you curtains and read the code of the libraries that I was using. And I think GitHub really helped with that because pre-GitHub days on SourceForge, you could get open source software, you could use it, but it wasn't obvious how to read the code. And it was very kind of black box. Um, once I realized I can just go read all this and I can find out how it's working and I can see why it's erroring even though the docs say it shouldn't be. I really started to learn just by reading other people's code. So I will just echo your sentiment there.

Matt, let's move to you. Do you have a specific programming hero or is it everybody? It's going to sound like a cop out, but I have to echo his answer. Oh no. I know. It's terrible. I respect anyone who faces a challenge in coding and learns and strives to overcome it. And I do watch. I see when people have overcome challenges. And it's not just with coding, but I know there's also social stigmas in tech.

So people who just stand up to being themselves, I highly respect those kinds of people. We're getting an awful lot of meta answers here. Lately, it's not been specific people, but certain types of people. So it's a trend. It's a been specific people but like certain types of people so it's a it's a trend it's a trend i'm spotting here in changelog um very cool next question and probably the final one for today is well let's do two more so not the final one but what would you be doing if you weren't doing x where x is what you're currently doing in open source or for a living.

So if you weren't doing caddy and computer science, Matt, if you had to go completely a different direction, what would you be doing instead? That's a good question. I would love to be working with people, maybe teaching them how to code, teaching them how to think like a programmer, I think is really valuable. Or if I wasn't doing that, I would probably be improving, like focus solely on user experience. I think that's just really important to me, how we interact with computers and software. Sebastian, same question for you.

I'd probably be a lawyer by now be a what? an attorney a lawyer oh an attorney gotcha I thought you said a carny

somebody who goes in a carnival no, not really it would be kind of funny but no it would be kind of funny, but no. It would have been a valid answer. I just couldn't tell exactly what you said. So, okay, an attorney at law. Yeah.

Very cool. Okay. That's a different direction. If it were for my mom, I'll do that. I see. I see. You'd still be dealing with code, but it'd be different kinds of code. Yeah.

Anyways. All right. Now the last question, which is a bit of a call to arms. Or if you had the ear of the open source community, which you kind of do here on the show, and there's people who are excited about Caddy. They want to get involved. They want to not just use it, but they want to help. What is the best ways that we as an open source community can help you guys and the Caddy web server succeed? That's a loaded question in a good way.

Thank you. In a good way. I think that when we get Let's Encrypt, Thank you. Hopefully, I want it to change the web a little bit. I want to make sure that people are serving HTTPS. I'm a strong believer in that. And I think it's time that we secure our transmissions from surveillance and attackers. And I think there's no better way to do it to reach the average Joe user than to work on this. So I think if you feel strongly about privacy and encryption, then get on board with this. Help with the Lego library that Sebastian's worked on and help with Let's Encrypt integration and any other open issues, frankly.

Good answer. And I guess worth noting there, you mentioned Lego. In the break, we were all enjoying Sebastian's library name, which is Lego. That will be in the show notes. That's the Let's Encrypt Go library that Caddy uses or is going to use as they get it integrated. Yeah, it's still a work in progress. Awesome, awesome. Well, Matt and Sebastian, thanks so much for joining us today.

This was a lot of fun. You got me excited i like to see advances even in web servers and it seems like caddy has a lot of good ideas in it hopefully it will spawn other people to get involved and to improve on existing web servers and to build caddy into something that does you know put a dent in the interwebs. Next week, we are joined by Mitchell Hashimoto of Vagrant Fame and HashiCorp, where we'll be discussing their latest open source product, which you may have heard of, recently made a splash, Otto. So if you haven't subscribed yet, go ahead and subscribe on iTunes or your favorite podcatcher so you don't miss out on that show. We want to thank our listeners, our members who support us.

Thanks to everybody who requested this show and made sure that it became a thing. And we will see you next time.