The Changelog: Software Development, Open Source - Curl is a full-time job (and turns 23) (Interview)

Episode Date: April 12, 2021

This week we’re talking with Daniel Stenberg about 23 years of curl. Daniel shares how curl came to be, what drives and motivates him, maintaining a good cadence of an open source product, what to expect from http3, how many billions of users curl has, and Daniel also shares some funny stories like the "Spotify and Instagram hacking ring."

This week on The Change Law, we're talking with Daniel Stenberg about 23 years of Curl. Daniel shares how Curl came to be, what drives and motivates him, maintaining a good cadence of an open source product, what to expect from HB3, how many billions of users Curl actually has, whether or not it's on Mars, and Daniel also shares funny stories like the Spotify and Instagram hacking ring. Huge thanks to our partners Linode, Fastly, and LaunchDarkly. We love Linode, theyly, and LaunchDarkly. We love Linode.

They keep it fast and simple. Check them out at linode.com slash changelog. Our bandwidth is provided by Fastly. Learn more at fastly.com. And get your feature flags powered by LaunchDarkly. Get a demo at launchdarkly.com. Linode is simple, affordable, and accessible cloud computing the developers trust. Linode is our cloud of choice.

We trust them, and we think you should build anything you're working on, a fun side project, or that next big info move at work with Linode. The best part, you can get started on Linode with $100 in free credit. Get all the details at linode.com slash changelog, or text changelog to 474747 and get instant access to that $100 in free credit. Again, lino.com slash changelog. then we're back it's our good cadence of every three years which is just a random number we chose it's three people every three years sounds pretty good but it's good to have you back so the first time we had you on was like 2017 a long time ago 17 years years of Curl, and then Curl Turns 20 was our second show with you,

and now we're back for 23 years. Big moment. How you doing? Hello. It's good to be here. Time flies. 23 years. Yeah, just a number. Just a number. Well, it's good and arbitrary, but it's for a good reason. I mean, Curl is one of the most used pieces of software that I'm aware of in the world.

We've covered that. It's been debated. It's well known. We've even gotten some hate because it's so used. We might talk about that today. It's just such a used thing. You're still, from what I understand, a sole maintainer of it. There's some contributions, of course, but you're the primary person behind it. We want to dive deep into all those details. What it means to start it. What it means to lead it. You've asked your audience a ton of questions about what we should cover here. So thanks to all of your Twitter followers for kind of setting the tone for the conversation,

at least to some degree. Yeah, we've got a lot of good suggestions there. Yeah. I like to take the opportunity when I have this birthday, since I know when I first released Curl, I know when the birthday is. And I like to just go back and bring it all together and maybe think about how we all started what I've done the last few years and so on so I think it's a good opportunity to just vent through it and yeah look back a bit what's the official birthday for curl March 20 March 20

1998 wow wow that's also a bit arbitrary since that's the first release I actually called it Curl, since I actually called the tool with another name before that. And maybe I should count that as a... But I don't. So March 20, 1998, I released Curl for the first time. Gotcha. Well, the good thing is you're in charge, so you can make those choices if you'd like, right? Right.

Yes. We can give those extra two years to you. So it could actually be 25 years of something we can round it off we can round it off for if you want it's like time travel and that would actually make a good show 25 years of curl instead of 23 right and since i changed the name twice before that kind of many birthdays you know yeah why not many opportunities here and since there's it's a multiverse we can have infinite options and opportunities so anyway see but that's good i mean you know the internet was

different back in 1998 oh yes like way different right i mean the problem set the internet itself the needs for apis the needs for doing what curl does as a tool has changed drastically and so many more use cases have come into play yeah and i think back in those days of course nobody really knew where it was going or right that hdp would become this protocol that everyone would use for like apis and rest and everything everything that that was just a big unknown right so yeah it just happened yeah and of course i didn't know what where where curl would go either right right? Because in 1998, that was 2,000 lines of code, just a tiny little tool. So yeah.

What was your initial desire? What was your initial use case? Just grab a website? No, I was going to just download currency rates for my IRC bot. Okay. I wanted a little tool to just, you know, every day maybe download currency rates from a site and you hosted currency rates from a site and you hosted currency rates.

You're a Forex trader or what? No, I had a bot in an IRC channel. So you could just, you know, have a currency rate translation service in the chat. Okay. So, hey bot, what's 100 USD in Swedish kroms today? Now we ask Siri that. Hey bot, hey Siri.

Exactly. At least we try. And then she says, here's what I found on the internet for. She probably uses curl in that process, don't you think, Daniel?

Well, my exchange was roughly the same. It was rather crappy, but still, that's why I started looking into HTTP and getting HTTP. It's important to note

that big things have small beginnings. Oh, yeah. I mean, everything starts somewhere, right? So that was my sort of humble beginning. And then, of course, I released that and some users found some bugs and added some requests. Maybe we should do this. Then I found another site that hosted currency rates on Gopher.

So I added support for gopher and then it took a few months until i found a third site that hosted currency rights currency rates over ftp and then i did that that's the beginning of the story snowballed yeah so if you fast forward to now and you gave it the same answer to the same question what was your most recent use of curl that wasn't some sort of like development i'm working on curl thing do you use it as a regular user i actually use it embarrassingly little these days so yeah i still use it for for simple cases yeah more like ordinary my ordinary command line style any unique c guy you know I want to script something and I have curl there. So I rarely

use it very complicated because that would mean working with curl as a tool a lot, which I don't then since I nowadays work with curl full-time or supporting and working on it, which is one of the biggest changes right since three years ago when we did this the last time. Yeah, I was going to say you're now full-time, right? Yeah, exactly. Which, of course, has changed things for me around a bit. And you were with Mozilla previously. Tell us the story. How did you go full time? Well, I quit Mozilla in late 2018. Yes, for reasons I mostly because I was bored with that job. I didn't think it developed me. I didn't think I had enough fun. So I decided to quit that and do something else

and then i looked what i could do and i wanted to do after that and so i talked to my friends at this company called wolf ssl for for which i work now and we decided to do this together i knew them from before and then i decided that i will join wolfL, work with Carl full-time, and sell support for Carl commercially then for anyone who wants it. So basically companies who need help or whatever assistance they need regarding Carl. Nice. That's working out well for you? It does. It works really great.

It's a little bit of a dream come true since now finally what I created back in the late 90s i can now sort of you know taking it all the way as a spare time project for this long and now suddenly i can do it for a living and only that i don't have to you know save it for spare time and i can do it on spare time as well but now i can do it all the time So there's this weird phenomenon that happens to people when their hobby becomes their job. Like their passion project, their weekend warrior side gig that they love becomes their nine to five. And we see it all the time. Adam and I, in a certain way, have done that.

Like our hobby, Adam, this started as a hobby, right? Sure did, yeah. Now you're podcasting professionally. And it's like, now this is my job. And something that you just desired to work on and you wanted to do because you loved sure did yeah now you're podcasting professionally and it's like now this is my job and something that you just desired to work on and you wanted to do because you loved it all of a sudden you have to do it because that's how you make money does that change your perspective on the work daniel a little bit since of course exactly as you say now it's not only what i want to do but

sometimes really what i have to do or rather what i sign up to do and now I really have to deliver, not just because I want to do it in the afternoon. So, of course, it becomes a little bit more serious. But I think so far it hasn't taken over anything. So, yeah, that means I sometimes actually have to spend development time on Windows just because that's... There's your drawback right there. Exactly.

There are so many benefits and so much fun anyway, so i can take the few drawbacks with it i'm enjoying it fully so far so has your family and the rest of your life gained those night hours back or do you just power through because you love it so much and you're still working on it nights and weekends i still do nights and weekends but mostly nights i would say not that much weekends but mostly nights, I would say. Not that much weekends, but mostly nights. It's also because I have a very established way of working. So it's basically I'm just keeping on the same way that I did before. Now I just do curl daytime and a few hours of nighttime as well. It works really good around the clockwise too,

since then I can do stuff in the morning and late in the evening too. It works so I can fire off some pull requests and they can run ci jobs and stuff like that so yeah yeah i think i have pretty good rhythm well it's worth noting that you had sustainability down right 21 years before your full time on this thing so the last two years i'm just rough numbers 23 now the project is you've been working on it for a very long time you've kind of been the picture to me of perseverance and open source, just toiling away for years and years and years. And so this is, I mean, rhythm, this is like a lifestyle.

Oh, it certainly is. And not just the life, it's more or less my entire life, right? I'd say. Older than my kids. 23 plus years. I don't want to ask you your age, but this might be like half of your life or more. It's approaching, yeah. It's skinny. I turned 50 this year. Okay. So you're want to ask you your age, but this might be like half of your life or more. It's approaching, yeah.

It's getting... I turned 50 this year. Okay. So you're going to get there. Yes. Pretty cool stuff. Yeah. So now it's your full-time job.

If you just set that aside and you told everybody, because that's a very impressive accomplishment. To work on the same thing for that long is amazing. And I know we've actually done this before, but that was probably six years ago now, so you can share it with a whole new set of listeners. How do you stay motivated? How do you keep toiling away

at this open source thing? It's really hard to... I don't have any magic sauce or recipe. I'm just very sort of... I want to make sure that it works, and I always want happy users, and I want to have those features, and I want to make sure that curl that it works and i always want to happy users and i want to have those features and i want to make sure that curl remains a really powerful you know the swiss army knife of internet transfers really and follow along and make sure that it remains the choice

of so many users and can power all these applications and i think just keeping up with that and making users happy it's a drive enough for me. And then, of course, positive feedback that people actually appreciate it and they say they enjoy it and like it and it does fulfill the needs for them. That's enough for me to make me go and keep me going. What's a simple reason to keep going? I mean, you keep it simple at least.

Motivation by the happiness of your users, keeping it working. I think that's a pretty humble approach to remaining motivated. I mean, you didn't say make millions of dollars. No, I didn't. You know, this fantastic exit or, you know, by the way, Curl Company just got announced or something like that, where you've got like, I guess you do have professional support

and there probably is a company behind it because it makes sense. But're not saying well i'm really trying to get to that vc dollars no and that would be really weird way for me to be around about that inefficient way to get to that goal so no that's that's really no i mean of course i want money as anyone else does but that's not what drives me that's not what makes me get up in the morning and stay up late at night. Well, 23 years is a long time, and over time, you've changed your opinions about some things.

Now, it doesn't feel like it's been three years since we've talked because I'm an avid reader of your blog, and you blog pretty prolifically, so you have lots of writings out there. And you've had a bit of a change of tune lately with regarding the programming language that Curl is written in. So it's a C program. It's a library and a command line tool, right?

And it has been ever, maybe it didn't start there, but did it start in C from the very beginning? Yes. Yes. And you wrote a post a few years ago now, maybe four or five years ago, Curl is C. And then you wrote a post more recently about all of the vulnerabilities and how many of those are because of the C programming language and just the sharp edges and corners you're able to cut yourself on.

And now you're starting to talk about Rust a little bit more. So we'd love to hear your thoughts on, maybe you're changing mind around Rust and C. Yeah, I'm not sure I have changed it, but maybe refined it. I've grown into a viewpoint where I think the future could be for curl. I mean, curl is C, it was C, it's been C for 23 years, and I foresee that it's going to be C for a very long time ahead as well for several reasons. But I do think that there's certainly a way forward to introduce other languages. It doesn't have to be Rust. It could be other languages too. In the way that we in the Curve project supports

different backends, when you build Curve, you can select different ways to build it, basically selecting a lot of different knobs and switches to make your build like we can select which tls back end ssh back end name resolving back end a lot of different things and hdp3 and things like that and when we go forward i'm already and we're seeing this that's a trend from a few years now that more and more of those are going to be written in Rust. And that's a way for us in the curl project to introduce more Rust in the final binary without actually curl changing to Rust, but we're using more Rust, basically. And one of my more recent efforts sponsored by ISRG and the ones behind let's encrypt is to make sure that we can actually build curl now with a http backend written in rust called hyper which is then we

replace parts of the built-in hp powers of curl with a rust written library instead we do that also with tls with the russell's library and so on that's interesting we did have Josh O's from the Let's Encrypt project on the show a while back and he was very passionate about replacing a lot of internet infrastructure projects with memory safe languages specifically Rust and he has gone about

doing some of that work working with I think the NGINX folks on some things working with you on some things so they basically sponsored you to build this in or build this access from curl? Exactly. And what it is, is an app option to build curl with hyper instead of the built-in HTTP and HTTP2 support.

So you would use that library instead. Gotcha. I foresee a future where we can do this with more things. And in effect, that is replacing C with some other language. At the same time, I also think there's good value in remaining C too, because by having a C library, it also provides the power of curl and lib curl to a really, really large community

and operating systems and architectures and platforms out there that don't run Rust at all today. So I also think there's still, and you know, having a solid and proven and tested C library is also valuable for all those platforms. They can't switch to Rust anyway. So they would go between using my library that I claim is fairly well tested and reasonably secure compared to using something else that might not be. Well, it's worth noting that when you say it runs in a lot of places, you're not exaggerating. I mean, it runs on an unfathomable number of devices, right? Yeah, it's insane. So yeah, I used to mention the number 10 billion installations,

which is, that's more installations than humans on the planet right so but that's also because i've switched i probably did that even the last time i switched and i talk about installations these days and not number of users because most of the installations of curl they run somewhere not necessarily by a user or a human or someone knowingly using it sure what about the platforms themselves though i mean i know used to post when they'd find it in some new weird embedded place or like wasn't that the time square on the new york stock exchange or i don't know like weird places that you'd find curl being used yeah it's it's really everywhere these days. How about Mars?

I'm trying to get it confirmed that it's actually used in space, but I've failed so far. Amusingly, I've been emailed twice by NASA people, and I've asked both of them about it, but none of them would respond to that question. Come on, we need some confirmation. Probably the wrong people to ask, and they asked me about some other lame stuff anyway. But still, I'm trying. I'm trying to figure that out. I think it's safe to assume it, though. I mean, there's tech in space.

Are they using web protocols? Probably. But I guess, do you have HTTP in space? I mean, it's the most known protocol. I'm sure they've got networking there, I'm sure. There's got to be computers networked together somehow. Do they create their own protocol? Possibly.

Yeah, I would imagine that it actually exists and is used in some, I mean, at least local area networks. On ISS for sure. Yeah, exactly. Like NSS and stuff like that. They've got to communicate. Yeah. There's got to be something.

Why don't you just add a little phone home in there? Then you'll find out. Let yourself know. I love that. I love that. I don't even know add a little phone home in there? Then you'll find out. Let yourself know. I love that. I don't even know if you could have the servers that would maintain that ping. If you have 8 billion devices pinging you. Exactly.

That would be the infrastructure. That's not a problem I want to have. Since you're somebody who's in the tooling space as it relates to web servers, web, HTTP, do you share Josh's concerns about C and safety and replacing and all the things he said about essentially, I'm trying to paraphrase to some degree what he said in that episode at the end, his desires to replace as much on the edge

and in those areas with Rust or just thread-safe languages or things that didn't have the vulnerabilities that C does? I think I'm in a general agreement that it is a good thing and a good idea and a worthy mission. But I also think that it's a really long-term sort of job to go there.

And sure, I think Rust is a worthy contender to use for a lot of that but it's also a long way for us to go there to actually be that really awesome alternative and i see that already now since this work that they they're funded me to work with to introduce this rust back ending curl it really shows that rust also has a lot of things to you know fix to make sure that they can actually become that new solid pillar to lean on for stuff like that into the future but sure i i think we will go there at one point or another but i think it's it's going to be a rather slow transition So another thing that maybe has been a slow transition, maybe not so slow, is new HTTP protocols.

I know we've talked about QUIC with you last time you were on the show. Tell us about the state of the art. Where are things? I know you're right on the bleeding edge. You often are testing things in curl, right alongside the specs getting drafted and really making sure these things are, I don't know, usable in code as they get worked out. So what's the state of the art with HTTP?

Right. I like to have it be on par with the new developments so that we can use curl a little bit as a tool to try out new protocol stuff. And at the same time, that people that are developing the new servers for this protocol can also use curl to try out the servers. So HB3 is coming. I said that for a very long time.

I've done a lot of presentations about HB3. And I guess that's the slide I've changed the most times. When is it coming? Soon, I say. And that's what you're telling us right now, soon. Exactly, soon. Yeah, I've been saying it for And that's what you're telling us right now, soon. Exactly, soon. Yeah, I've been saying it for years, soon. But this time, it is actually soon.

So now, HTTP3 is a protocol on top of QUIC, right? So QUIC is a TCP and TLS replacement. So we're going to throw out TCP and TLS, and we're going to use QUIC instead. And on top of that, we do HTTP3. And both those protocols, both QUIC and HTTP3, they we do hdp3 and both those protocols both quick and hdp3 they are already pretty much finalized and done but they're not shipped as rfcs yet so that's they're in the process of actually getting out as rfcs so they're you know discussing phrasing in some descriptions and making sure that everything is done correctly so and then

there's another work going on with refreshing http in general and getting things done for generic http and i think that is going to block the http 3 spec slightly what do you mean generic http what's that mean well how to define http in a version-independent way, right? How does headers and requests and everything work? Ignore the transport over the wire. Because there's a lot of things in common, right, between HTTP 1, 2, and 3. And so that's... The work is to make sure that we have a document

that describes how HTTP works independent of which version you're using. And then there're all going... They're like a lowest common denominator thing? Yeah, pretty much like that. And then there are documents for describing each of these separate... They build on top of that or they diverge.

Exactly. They have to. Yeah, well, they sort of work together then too. Okay. What's the benefit of having that document? I think it's primarily a clarification to make sure that we're all on the same page here. To understand that

htp as a paradigm and usage it doesn't really change when we add hp2 and it is three we primarily change how it's transferred over the wire we don't really change change how we think or use htp i gotcha it's kind of for backwards compatibility as well as clarification. Yes, I would say so. Okay. I didn't know that was going on. So HTTP 3 is going to come soon, but we're still a bit off until we will see it, I mean, really deployed because there are,

this time around, HTTP 2 was a bit slow and hard to get deployed everywhere, but HTTP 3 is going to be even more complicated. Great. Well, you're not painting the best picture of it. Is it better? Is it good? It is certainly possible to get better and better as in lower latency and higher performance for at least for a good chunk of all the use cases. But I mean, it's a different infrastructure now since it's built on UDP and there's a completely different way to use TLS. So all the TLS libraries are going to have to provide new APIs.

And OpenSSL is way behind on that. And we all know that OpenSSL is the number one TLS library in the world. And as long as they don't support it, it'll be a sort of slowing down factor for deploying and adopting quick. So as a lowly application developer who just wants to get his JavaScripts onto your machine, I was very excited by the promise of HTTP2 and you don't have to bundle anymore. You don't have to put everything all in one file, blah, blah, blah. It's just send those little files. And that didn't seem like it played out in practice

to deliver as much of a win as we were hoping it would with H2. Exactly. Is HTTP going to fulfill that promise? It has other promises. What are we going to get out of this at the end of the day? Well, I think if HTTP 2 was only a minor boost for most of us, I think HTTP 3 is going to be an even more minor boost

for most of us. You're really selling it here, Daniel. Yeah, what else should we talk about? Boring. Yeah, I'm not selling it because I'm not here to sell it. He's got nothing to sell. Exactly. I'm not getting that VC dollar, so no need to sell. We're certainly going to see the big guys

use it. They're going to jump on it immediately. So, of course, we're going to see all the big ones and the CDNs are going to use it. So we're going to see it deployed big time. But I don't think when you're a small player, if you just run a few servers by yourself, you can just as well wait and see. So read between the lines.

What you're saying is the end users, the DERADs of the world, the atoms of the the world are not going to get much benefit from this change. It's really the infrastructure that's going to get the change, the big players, the edge nodes and the cloud flares. Yeah, right. Trickle down. They do it so that we as users of their services will get a better experience. So hopefully, you know, YouTube will play X percent better when we use it over Quake and HB3 and so on. So, yeah.

But possibly you need one of those infrastructures to be one of those to actually benefit or be able to provide it. How much do you pay attention then to, say, 5G and the explosion of IoT devices and the non-human devices out there? Obviously, you run Curl, so you pay attention to that stuff to some degree, considering what you build and run. But thinking about network latency, as 5G rolls out and it becomes more stable, potentially nationally or globally, we're talking about devices, network-wise, non-wired devices connected at 100 milliseconds as a dream, potentially, like when the network really speeds up. Is that what QUIC and H3 is going to deliver that kind of stuff? Like those network latencies are all meeting up in the middle, essentially, like this protocol is delivering a faster internet for that. Yeah, I think ideally they will do that.

So I think they will certainly help in that direction and also i think we're only seeing the beginning of this also so there's going to be a lot of more development with quick and stuff done with quick and over quick so i think it's going to improve further as we go forward how far is your vantage point then is your vantage point considering 23 years in is your vantage point now less one to two years down the road more like five to ten in terms of like paying attention to future tech and future direction because i had a separate conversation that sort of like gave me insight to this landscape of like 5g isn't here and winning now it's it's approaching it's coming it's rolling

out but 10 years from now it's going to be rolled, it's going to be rolled out. It's going to be much more fast. We're going to be dealing with, you know, at a global scale, all sorts of devices at a hundred milliseconds connection. Right. But for me as a, as a person here involved with curl, it doesn't really matter to me because I'm going to support my users, 3G, 4G, 5G, connected Wi-Fi. It doesn't matter because all my users, and there are plenty, they use whatever they can to do things over the networks.

So they might do more things, faster things, lower latency things in the next year, two years, three years. But they're already using curl and they're going to use more curl going forward. So I'm going to just keep support and making sure that people can do internet transfers and i'm going to pay attention to what the network development happens or you know protocols how they're changing so i don't see any particular change in in anything for me or what's going on and for me personally i don't really try to predict the future long in advance. I'm just looking at what we're doing right now and trying to see what should I work on the next few months,

really. So from Curl's present day perspective, H3 has been a pain in your butt, I'm guessing. It's a complicated beast to support. And we support it experimentally in Curl and anyone can get Curl and build with HB3 support. And there are several providers. You can go to Facebook, Instagram, Google, Cloudflare. They all support HB3, the draft versions today. So if you enable it in your browsers, you can actually use it today and try it out. So it's there and you can start fiddling with it but for me it's also i'm playing

with it just a little bit on the side and seeing how things go and since there are so many beta versions and unstable releases of everything to use to just build this and so a lot of moving that needs to get stable first. Learn more and try it out for free today at retool.com slash changelog. Again, retool.com slash changelog. So as Adam said at the top, your Twitter followers provided a whole bunch of awesome questions, things they can't wait to hear from you. So happy to make them happy. And we have this one note that says, the Spotify and Instagram hacking ring story. So I don't know what that is, but I'm excited to hear what's going on there. Tell us.

That's one of my favorite stories in Curl. So it started out with a woman who emailed me once. This was, I think, three, four years ago. She emailed me about her Instagram account being hacked. Okay. And she asked me to help her. So all out of the blue, sort of, why is this woman emailing me about the hacked instagram i had no idea you shouldn't ask me and then she sent me a screen capture of her

about window and said but look your name is here you can just ask your friends to help me out here all right you found my name instagram i had At that point, I had no idea that Instagram even used Carl. So, oh, right, that's fun. And I tried to convince her then. Exactly. There's another billion users or so, or however many they have. Oh, indeed. Yeah, a lot of users.

She didn't consider it as fun as I did. It was a good day for you. It was a bad day for her. That's fun. Thanks for emailing me. Exactly. Oh, that's really fun. But, you know, I've never talked to them. I don't know them. I don't know. I'll try to explain to her that's fun thanks for emailing me exactly so oh that's really fun but you know i've never talked to them i don't know them i don't you know i'll try to explain to her that

you know i'm they're just using a component that source code i've built it that's cool right so she really had a hard time to accept that but you know you can just ask them right to help me just you know she's like come on you know them so a little bit back and forth i said no i don't know them and you know eventually i think she sort of bought that and she went silent for i think for a week or two weeks or so and then suddenly she emailed me back again oh you've been lying to me the whole time she said because look you did hack my phone and then another screen capture spotify that was my name oh no so apparently she had sort of discovered me completely so because my name was in both instagram and spotify so obviously i had hacked her phone and just you know my secret instagram and spotify hacking ring unless i helped her

she was going to reveal this to these big companies she was going to tell instagram and spotify that you've been hacking yes and you wouldn't want that she said that is hilarious well yeah from her perspective though you know she had to have an aha moment you know when she had her second instance on spotify you know because you'd convinced her that you had nothing to do with it. And then when she saw your name the second time, I could be her in my mind and think, you know what? I got him.

He's lying to me. Here he is again. Exactly. I could sort of feel that. Gotcha. Exactly. I got you red-handed. You lied to me.

I sort of gave up at that point. No, I really couldn't. I tried. No, that's, they also tried to say no that's they also use that's hilarious she thought that was a little bit of too much of a coincidence yeah is it alphabetical i wonder why you're the one because i mean there's usually hundreds of things listed on those about pages and maybe it's just alphabetical and your and curls up there near the top i think one of the rare instances where my email address is actually in there often.

Really? Yes. At least that's an explanation I've gotten many times from people using cars have emailed me. You know, someone has a problem to figure out the language in their GPS emails. And then I ask, why are you emailing me? And then sort of, I found your email address.

And how? Well, it's in this, you screen somewhere and I scrolled and scrolled and scrolled and there it was this is a sure sign of like frustrated users who cannot get support from yeah maybe in this case she reached out to Instagram was like hey who's hacking me and she just decided to take it upon herself or the person that's driving this car that's like hey i'm having issues like with my self-driving car and it's wrecking all the time and i think that's also one of the problems i have right because the frustration level at these people at that point you know they probably trusted everything tried everything contacted everyone they can imagine and then you know banging their head against it And then they find my email address and emails me. And I say, I have no idea what you're talking about. I can understand the frustration in the other hand.

But in my end, they just appear really confused. Out of curiosity, I had to go into Instagram's section where they do disclose their open source usages. And they do list them. It seems like alphabetical. No, it's not alphabetical based on license even. The first list they have is BSD3. The second list they have list them. It seems like alphabetical. No, it's not alphabetical based on license even. The first list they have is BSD3. The second list they have is MIT.

And the third list is Apache. And I'm either have bad eyesight or what, but there's no mention of curl anymore. Is it lib curl? No, maybe they've changed that. I don't know. It says URL parser. Maybe that was a swap.

Oh. I don't see a usage of curl anymore. Maybe she contacted them and they had to take you out of there. Oh no. The hacking ring. There's that band users. They're gone. Some pointy haired boss made them remove curl from the system.

Well that one's hilarious. This other story is quite a bit more somber and tragic really. Was the fella who really attacked you via email because he had gotten hacked. I remember that one. It was called I Will Slaughter You. I mean, just terrible things being said to you because somewhere either in his software stack

or in the attacker's stack, Kerr was involved somewhere. And again, you're just writing this library, but the email came into your inbox. Do you want to share? I know we don't need to meditate on this too long, but this was the one that hit close to home for you. Yeah.

I mean, it actually hit me pretty hard exactly when it happened, at least because it's sort of, it's actually felt pretty horrible. Yeah. And then I think I got over it pretty good. And then he emailed me more and, you know know tried to post more on my blog and so on and then he appeared even more of a lunatic and i think that made me less less concerned or more less concerned actually because it then appeared more uh more just deeply confused and you know yeah rambling about whatever and then it does And then I didn't really feel as threatened as just, you know,

very focused. Yeah, exactly. So I think I got over it. It just felt a sting of nastiness there for a while. Yeah. And of course, I did a police report about it here in Sweden, but it doesn't do anything. Yeah, I mean, they could be halfway around the world.

It's hard on the internet because you can't really gauge people at all. You know, like you can't tell if this person is serious or trolling or a lunatic or seriously depressed or what it is but like here it is in your inbox and you're left to just deal with this mess right here in front of you and that can be incredibly hard to deal with just even to know how to deal with such a situation yes and and in my case also i think i replied to his first email pretty instantly and he replied to it again and pretty much said that it wasn't a mistake i wasn't just rude i'm meaning it and i think that was what made me also take it slightly more seriously than just you know someone blurting out something you

know in a hidden in a moment like that. When you're reaching out and you reach out to a lot of people, eventually you hit some really bad people too in that way. Totally. It's just a numbers game. Really. Yes. Sad numbers game. And what was the case?

Was it the case that his business had gotten hacked or something? It was really, really not easy to understand. But somehow he claimed that he had lost his business due to some hack and he lost his entire life basically his wife and his kids and his job and everything and he seemed to blame me for something but it wasn't really clear how or or what but apparently he had found my email address anyway so i guess that curl was in there somewhere yeah it was really not easy to tell yeah i couldn't understand from his because you posted some of his his correspondence on your blog and i was reading

it trying to decide like was curl in the attackers like in in the the user agent or the attackers footprint or was it in his business's cms software stack i couldn't figure out what the guy was talking about no it wasn't really possible to understand exactly you have to really make some guesses so maybe in some way it was involved i don't know yeah and of course it is involved in a lot of you know shady stuff right well it's a tool and a tool can be used for good or evil right yes so i mean 10 billion users or whatever installs or whatever that number is more than every human on earth so you're gonna hit the good and the bad right in the yeah you know among all those users that i have you know you know i said instagram and other spotify and stuff yeah i know a few really ugly malwares that

and attack softwares that are using curl as well so i know for a fact that some of them really nasty ones are using curl too does that weigh on your shoulders or are you just kind of shrug it off or what? It's sad, unfortunate, but there's really, really nothing I can do about it. No matter how much I want it to. So I just have to live with it. So, you know, if you make a hammer, some bad guys will use that hammer for something really bad. Well, in your license, you could put, you cannot use this for evil. Exactly. So even if I had done that, would they care? something really bad. Well, in your license, you could put, you cannot use this for evil.

Exactly. So even if I had done that, would they care? That's the thing is, it only keeps the honest people honest, right? Right. Yeah. The evil person does not care about your license. So no.

Yeah, so no. That does bring us to something that you can control a little bit more, but I bet does have some weight on your shoulders, which is that there are vulnerabilities over time and there are security disclosures and there are serious things that are either in Curl's code base or inside of your purview. How do you handle security exploits, vulnerabilities? And surely there's incidents that come to your desk and you have to issue a patch. What does that look like in your life? That's a good question. So of course, we have our fair share of security vulnerabilities.

In two days, we're going to do another curl release, and I'm going to announce two more vulnerabilities. And usually we do it like this. We have a bug bounty these days. So we reward security researchers or anyone actually who reports security vulnerability in curl. That is confirmed security vulnerability. And I think that's fun and good because nowadays we can use sponsor money to pay researchers off or not off, but we reward them. So we get a fair share of reports on suspected vulnerabilities, and very few of them actually are confirmed in the end. But sure, eventually they are. And then we make sure that are confirmed in the end. But sure, eventually they are.

And then we make sure that we work with the reporters. We make a fix and we announce that problem. And with a fix and everything in sort of coordinated with a release when we released a new version with that problem fixed. With this release coming Wednesday this week, we've handed out more than $5 thousand dollars now in bug bounties and we're trying to gradually increase the amounts too so that we can reward every new finding slightly more than previously that's cool is that a new thing it's a fairly new thing um because we started out we're using open collective these days to to get funds or collect funds to the project. And we have a fair amount of recurring sponsors that are funding us monthly with money.

And right now, that's the biggest way to spend money right now on the bug bounty. And we actually get more money in that we spend on the bug bounty. So right now, we are in a fortunate position like that i've learned that it's a good way to actually be able to pay these researchers because there's a lot of them are actually who are trying to do this you know for you know for a living and if you don't pay them they will go to another project that will pay them right so i think in this, we can actually get a little of their time and their attention to actually try to find problems in our product.

That's a great usage of those kinds of funds. It's also transparent too, because you got to put out there how you paid or who you paid for X. And like you said, if they're going to do that kind of stuff anyways to pen test applications or do bug bounty stuff or security research,

makes sense to use those funds from the community in a way that benefits the community. Yeah, I think it works out really good. And then, of course, when someone reports a problem, we confirm it and we can fix it. It fixes the problem for a lot of potential users or potential people that could be vulnerable for that problem. So it works out really good. And then I think over the most recent years, we've also fixed a lot of architectural things in Curl. So we've actually decreased the number of problems that we find.

The frequency has gone down. We don't find as many problems as we used to do back in the day. Speaking of a lot of people, we said earlier in the first part of the show that you've got a lot of users around the world. And one of the questions you have here is how do you interact with that many stakeholders? You mentioned before about how you keep motivated, which is a very humble portion of it. But how do you stay focused is the opposite. You know, the motivation then turned into focus.

How do you focus on the needs of that many stakeholders around the world? Luckily for me, or maybe it's both an upside and a downside. I don't have that many stakeholders as it may sound like, because most of my 10 billion installations, they're done by users I never talked to and they never contacted me and I've never interacted with them at all. They're just using my product somewhere and they never even, you know, they don't file any bugs, they don't ask for help they don't do anything so i'm not in contact with them that makes it easy because i don't have to communicate but it also of course gives me less feedback so i don't actually know of their problems or what they would want in the next release and so on so i'm trying to stay focused on i communicate with

people on the mailing list and on issues and pull requests so i have a very small stay within my little community and if people want to affect me and want to change that they come to the curl community and we talk about it and then pretty much everyone has an equal voice, an equal vote for whatever we do. Of course, if someone actually pays me, like support or whatever, help them out with something, that, of course, will have a higher priority because then I will work on whoever pays me to do something, which, of course, would also most potentially

go back into the project as well. Well, that's interesting because, I mean, you've got that many users. Say the number again was at 10 billion. Is that a confirmed number? Is that an estimated number? How do you come to that? That's a very rough estimate,

but I'm actually, I'm working on a new estimate. I think it's actually more because of the number of installations everywhere. And it's in every mobile phone. There's a number of installations in every mobile phone even. And in pretty much every server, every desktop,

every internet connected device that you're carrying around. So IoT devices are in there too, like non-human API pinging IoT devices? They're usually harder to count for me. So I usually don't count them very high, but they're certainly in there. Well, then that would be you're on your track to trillions,

because, I mean, that's the estimate. That's the estimate of what? Right, the estimate is in the billions now to close to trillions in the next few years. I think curl is often more used in slightly bigger IoT devices and not in the tiniest IoT devices, but sure. So not a doorbell or something like that. Yeah, it's impossible to really say really firm numbers.

So I'm just, you know, trying to count where I know Korg is used and then, you know, guess the rest. Might be easier to count where it's not used. Yeah, but that's also hard. Yeah, it's a smaller number. How many devices do we have on the on the planet yeah or on the non-planet the isss of the world or the exactly it's the the recent rovers name expedition not expedition curiosity no curious is that one from a few years ago what's the newest gosh i feel

terrible i have my nasa my nasa hat on here but i haven't had that confirmed either so i don't think it's on mars we're speculating for you, don't worry. You know, that's something I think is really interesting, though, the wisdom you just shared there, because while you may have seemingly infinite stakeholders to please, you've found a way to remain focused, which is staying within your lane, essentially. And that's maybe the advice you give to anybody who is in a similar shoe to you, which is stay in your lane, kind of, you know, guard your time, guard your focus. That's what you've done by not having to sort of like appease these 10 billion plus potential users. You seem to just focus on the community that needs you most and everyone else just sort of falls off your purview because it's not in your focus area.

I would say so. And also, i think that helps me also keep focus on on the actual p i mean if someone brings me an issue or brings a patch or something they are the focus right not if someone is using curl in a billion instances yeah i mean they're outside of my you know vision they're somewhere else i don't have to care about them yeah so it's better to care about them who are actually here now. And of course, make sure that we are staying on track, so that we're going in the right direction,

which of course is also really hard to say, which is the right direction. You could go that way or that way, but... It could be also a function of the tool, you know, what it does. The Swiss Army knife. If you think about a Swiss Army knife, I mean, some people just use of the tool, you know, what it does. The Swiss Army knife. If you think about a Swiss Army knife, I mean, some people just use a nail file, you know. Other people use the scissors and then somebody uses a knife and they accidentally cut themselves

and they come and tell you the knife needs to be sharpened or whatever. But, I mean, I know for me, I've never interacted with Daniel on his mailing list or his issue tracker or any aspect of his project besides i man curl is about as far as i get or i google curl how to do this thing again for the 100th time mostly i just use curl dash i because i like to see the headers like that's my biggest use case is curl dash i or just curl and then redirect the output to a file so i can inspect the file and that's pretty you know i'm just using the nail file so for a lot of people curl just works because it's very powerful but it can do very simple things and a lot of us just use it

to do i mean sometimes i'll open up dev tools and you can do the copy to curl you know and that's really cool but i see it that what that copies and i'm like holy cow there's lots of junk you can pass into curl you know but i never, ever used any of that junk. Now there's power users who do, and you're probably having them on your mailing list or in your issues, and they're maybe driving some of the project in that way. But lots of us, even if I'm not just using it on my iPhone,

completely unaware as most of your users are, I'm actually a person who types curl into my command line, I'm still not the person who's given you the feedback. And there's probably thousands thousands if not hundreds of thousands of people like me just happily using curl from their command line to download a file or check some headers and that's about it oh absolutely and of course the next level is someone asking for help but not from us anyway you know right hosting on stack overflow or asking their distro people or in a forum somewhere else. So sure, there are a lot of various degrees of users.

Most people, of course, never need any help or have any problems. They can just go on with their lives and use Curl. Yeah. Do you get involved in those forums, like the Stack Overflow or anything, or you stay purely on the code? I monitor it a bit.

I answer it sometimes. It's hard to give feedback on those sometimes because sometimes i feel that the distance between me and the users are a little bit too big it's better if someone else takes that so i mean i feel maybe i'm a bit too entrenched in the details sometimes to actually answer the user actually asked for a simple question they didn't really want to know how the engine works. Hang on, Daniel. Copy and pasting this. Let me show you the third chapter of my book.

Something like that. So sometimes we just have to, no, no, I better hold off here. That's wise. That's good. What about managing the direction of things? You sort of have a product manager role kind of thing. You got a cadence to deliver in terms of like managing the, you know,

the continued development of it. Obviously you've been doing this for a very long time. So you've either learned by the school of hard knocks, having done it yourself for so long, or you've read some books. Where do you kind of derive some of the wisdom you have or may desire to have more of as it relates to managing and directing the product itself? I think I haven't read any books on it. Well, I've read a few books on how others have done it

with open source and stuff. But I think I've primarily looked and worked with other projects for a long time since I've been into open source since way before I started Curl. So I've appreciated open source and enjoyed open source since way before I started curl. So I've been appreciated open source and enjoyed open source and worked with it and built open source code for a long time and seen how others are doing it. And,

you know, if you're joining an open source project and participate that you can see what works and what you think is good and not good. I would like this to work in my project. And I'm then trying to make sure that I'm doing it the way I would like it. I mean, if I was a participant in my project, I would want it to work like this.

And then I just try it out. And then, of course, I've done a few things that maybe wasn't that good and didn't work. And then we do something else instead. And I'm trying to listen in what people are saying, because if you're just being, you know, humble enough and just ask people, they will tell you.

Or if they don't tell you, it's probably good enough so that I don't have to ask i can just go ahead and pick whatever you want so it's actually i think it works out really good to just get if you ask people see what others are doing and then try it out and if even if it goes wrong we turn and go another direction instead so you you listen i try to which of course is um also hard if there's nobody speaking because that's also a problem we have sometimes right i have i want to make this should i do it this way or that way and then i ask on the maybe on the mailing list and there's crickets and then i guess maybe i'll take that yeah yeah

what what do you think is the most viable channel you have then in terms of inbound information to you in terms of like a response from the community saying this is the direction i'm taking it or this direction it should go i'm taking it meaning the user using it and how i use it or the usefulness of it or the downsides of it how was the most viable channel you have you think that you get that feedback loop i think i have different channels to get different kind of users so i definitely definitely if i want to do it technical things or protocol things that's that's a lib curl mailing list that's where we do all that sort of core architecture design stuff if i want to actually know how curl users you know command line users are actually

thinking then it's usually better to just ask on twitter or somewhere where people have not opted in to because even the curl users mailing somewhere where people have not opted in. Because even the curl users mailing list where people are actually using the command line tool, that's also a very self-selected bunch of people that usually had a problem when they arrived the first time. It depends. I try them all, really, and see what I can get. And I also try, since a few years back back I try to do things nowadays as experimental features so I land them in

curl and code marked as experimental so that I disable them by default to sort of try out the orders before I unmark them as experimental and ship them for real in code so just make sure that this is maybe in what people actually want and how it works that

people actually appreciate. I'm works that people actually appreciate. I'm not sure it actually makes any difference, but it makes me less reluctant to ship something in code because now it's at least not carved in stone immediately day one. Right. It could be changed. It's sort of like a beta within a product. Right. Exactly. So I had the opportunity to change the name,

change some stuff before I actually carve it in stone and say, now I'll support this forever. And there's an opt-in process for these experimental features, you said? Yeah, exactly. Then you actually have to explicitly opt-in when you build curl. So I want to have this enabled in my build. That's in curl config or it's like in the build itself? Yes, it's in the build itself. So you have to actually, like HTTP3 support is still,

you have to actually build it explicitly enabled to get it there. Is that to reduce memory footprint or bug potential? Like why couldn't I just opt in at runtime versus at build time? More of a bug since then I want to have that, I want to reserve the right to change behavior or maybe change the name of flags. I'm a little bit concerned that if I enable that by default, someone will run ahead and use that

and then will come back and be upset when I change it. So I want to make it really, when I do it experimental, I want to make really sure that everyone is aware that this is experimental, so you have to... Gotcha. You want it to be as formal as possible. Like a command line flag or even a flag you know some sort of option the library would be a little bit less formal and maybe people would do that more more likely to do that and not and rely upon it whereas if it's like

actually you have to build curl with this experimental flag not very many people are going to do that unless they actually are willing to experiment like yeah i'm experimenting here yeah makes sense but it's it's a hard sort of i have to make some sort of balance there Not very many people are going to do that unless they actually are willing to experiment. Like, yeah, I'm experimenting here. Yeah. Makes sense. But it's a hard sort of, I have to make some sort of balance there. Don't you have some sort of a survey you do as well? I'm thinking of inbound ways and some sort of a conference or at least a meetup.

That's true. Yeah, well, both actually. So I try to do an annual survey just among users, really. I try to just get as many curl users as possible to respond to an answer that gives me feedback on what people use and what they want next and what they think about things in curl so that's actually a pretty good way to get a an overview or where where people especially in what people use in terms of you know what protocols are you using with curl and stuff like that and then we also have an annual curl developers meetup which is also a way a good way to talk to people

physically not physically these days of course but at least talk to people live and see where where other curl developers want to go and what they think and that's proven to be well worth your time and effort to put that together oh Oh yeah, both of them are excellent ways to get feedback. And I think they also work really good as an inspirational source. When you read and learn where people are and where they want to go with curl,

I think that's a good inspiration and keeps me motivated. Often they help to bring out new ideas of how to do things or what to do going forward. This episode is brought to you by our friends at O'Reilly. Many of you know O'Reilly for their animal tech books and their conferences, but you may not know they have an online learning platform as well. The platform has all their books, all their videos, and all their conference talks. Plus, you can learn by doing with live online training courses and virtual conferences, certification practice exams, and interactive sandboxes and scenarios to practice

coding alongside what you're learning. They cover a ton of technology topics, machine learning, AI, programming languages, DevOps, data science, cloud, containers, security, and even soft skills like business management and presentation skills. You name it, it is all in there. If you need to keep your team or yourself up to speed on their tech skills, then check out O'Reilly's online learning platform. Learn more and keep your team skills sharp at O'Reilly.com slash changelog. Again, O'Reilly.com slash changelog again O'Reilly.com slash changelog

so we would now like to learn a few things from the curl master himself. Tips and tricks. Things you may or may not know about curl. Probably Adam and I will not know about curl. But if Daniel does know about curl, lay some on us. Help us improve our curl foo. Okay. First, of course, I'm going to keep up with what I do on a lot of forums.

I say that you should not use capital X and the keyword when you're just doing regular curl stuff, because curl does the request verb by default. So you just type curl in the URL. It will make you get, you don't have to actually tell it to do it yet. If you do curl dash D, some data and the URL, it will do a post, but that's boring. One of my favorite command line options that so many people don't know about, and it may be not always the most useful, but it's dash dash lib curl.

So you type, you know, you make up your fancy curl command line, do whatever you want. And then you add dash dash lib curl file name dot C and do the same thing again. Then it will generate a template C code for doing the same thing as a program. Then you just rebuild that linked with Libcurl

and you have your own application that does that little thing. That is cool. How many languages does that support? Can you do it in Rust? It supports C. It supports C. I knew you were going to say that,

but I had to ask anyway. It supports C. But the good thing here is that most bindings for lib curl they use more or less inspired from the api yeah from lib curl themselves so it's usually fairly easy to translate that to all other languages if you're just using a lib curl binding that's super cool so you could translate it easy to PHP or PyCurl or the other bindings. Now, why did you develop that one? What made you do that? Maybe because you were debugging somebody else's flags or something? That question is very common. I have this, I want to do this in libcurl. How do I do it? And usually a lot of users, they already know

how to do it with the command line. They just want to do that exact translation. And I figured, well, Curl knows this, so I could just do the translation. I already do it, basically. I just have to also generate the code for it in text format and just output it. It has actually been very useful many times to just show users

and help users to get started. Yeah, sounds like a time saver for support. Of course, it doesn't actually produce a completely, in all cases, a replica because there are details, but it's usually a very good template to get started with at least. Then when you have that, you can get that going

and then you start working from that. Listeners, if you want to link to this, I'm going to link to your dash dash lib curl chapter in lib curl basics in your book. And you can say that's what you do, right? Whenever you, it's TLDL, too long, didn't listen. Maybe it's just a doc. So I'll link up that in the show notes when we get there.

It's very informative. I mean, that's cool. I like that option. Hit us with another one. You got any more? One thing that I imagine that a lot of users already know about, but when you're doing a lot of testing poking you want to know about the dash w option also dash dash write out it's a way to extract extra metadata from the previous transfer so you can for example easily extract the hdp response code the size of the transfer or a lot of different timings from the previous transfer so if you

for example you want to know exactly how long time Carl spent on the name lookup phase in the transfer, you can actually output that. So they appear as little variables that you can output in a long string. So you have, I don't know, 30 different variables that you can output that gets output after the transfer is done. So maybe you threw away the output and just output a lot of data from the transfer. You can do that, you know.

If you do that on a cron job on every minute, you can see how the name result times vary over the day or whatever. A lot of fun things you can extract with that. That's pretty cool. What about the data itself? Are there any interpretation tools built in or modes where i don't think it does like json parsing or will it do anything no i've actually

tried to make that as a separation where i draw the line okay pretty much curl delivers the data or sends it data the data the other way but it doesn't actually interpret the data it doesn't handle the data you pass that on to something else. You do JQ if you have JSON. You have an HTML parser if you want to parse the HTML, but curl itself doesn't do that. Keeps it all simple, I bet.

Unix philosophy. It's simpler, and it also makes me more focused. And it's actually, I think it's a pretty good line in the sand. This side, we can do it. Do we deal with content? No. Carl has no idea about what content you're delivering.

It's just a transfer of it. Exactly. Transfer of whatever. Whatever it is. Whatever it is. It could be going good. Who knows? Now, you shared this before, but it is worth noting that you can output to a file without without a redirect right without a pipe or anything like that you can do a certain flag that will just download the file as is and

similar to the w gets default functionality you want to recover that for those who right if you add the dash capital o option it will use the file name part of the URL locally. So if you have http://example.com slash file.jpg it will use that ending file.jpg as a local file name if you use dash capital O. And it will put the data in the file.

Yeah. Way back when we first had you on the show, you blew my mind because this was the tail end of the show too. It was like a magic feature you described it as. And it was actually creating a curl RC file. And then they're putting dash dash remote dash name dash all so that it automatically passed dash O by default. Yes, exactly. You can do that then. And that option is similar, but then that option will be used for all URLs. If you specify a number of URLs, it will apply that to all of them. Which may not always be the desired case if you're a PowerCurl

user. In my case, the majority of my use of curl is that. So it'd be a nice smart default for me, but maybe not for everyone. That's cool. Right. And when I created curl, I actually wanted it to be more like the cat command in curl, you know, that you cat a file and it outputs it to the terminal. I wanted curl to be that, but for a URL instead. So you would get that. So I felt that was sort of the Unix philosophy. It should just do that. So that's why.

It totally is. Print a standard out and it integrates so much better with all the other command line tools when you do that. But then you need an optional one to save it as a file. You know, when your Coca-Cola and Pepsi comes along and they say, well, we have Pepsi Zero, you know, you have to say, well, I can do Coke Zero too. Touche, touche.

Any other whiz-bang tricks for us before we move on? Well, I could possibly mention it's a pretty specialized option. It's called dash dash next. It's a way to separate different set of options. So you can use, for example, if you want to do get and post within the same command line, you can do curl and a URL, and then you could use dash dash next, and then you could use dash D with data and a new URL,

because then you would first do a get and then you would get a post because dash dash next will sort of reset the state of the command line and start over and you can do that many times in the command line then then you can do get head post get in a long sequence you're kind of chaining commands but you're not doing separate commands right it's one curl command by keeping the same command line you can reuse connections and stuff like that. So you can make it much more performant than you could otherwise. That's why it's there.

I see. So it keeps like a TCP connection open versus if I did curl this and then curl that and then curl this, those would be new connections every time. Exactly. Then you would have to set up a new connection each new time.

I see. Which of course, when you do two commands once it doesn't matter but if you're doing in the loop perhaps a million times it will actually matter what about retrieving and maintaining session cookies and stuff like that is that all part of that as well well no because if you want to store a session if you want to store cookies you usually can do it two ways you don't you don't have to store cookies if you don't want if you for example you want to do it like that you will do a lot of requests using this one command line and just keep using cookies you can just type dash b and the file name that doesn't

exist so you just get started to use cookies and it will use cookies automatically in the session or you can use dash c which creates a cookie jar on the file as a file. So then you can store it in disk and then in repeated invokes, you can read it back from the file. There are several ways to do it. Depending on what you want to do. If you want to save the cookies for the next day, for example, you want to save it in a file and use it again next day and update that file over and over every day, perhaps. I think the common use case there is like your first request is a post to do some sort of sign in right and then you store the cookie and now you want to get a protected page as that user and so if i use dash like a login

yeah i'm logging in and now i'm getting a page that's private you know to that user that's a very common question is also because often you want to get the login page first because then you get a cookie so you store that cookie the login page first because then you get a cookie so you store that cookie in the cookie jar and then you do the post which makes a login and you get updated cookies and then you get that magic file you want to do get as a logged in so there's usually three requests so you just create the cookie file first and then you use the cookie file and create it update it so are you doing dash b on those or are you doing dash dash b is for reading you use dash dash next? Dash B is for reading.

You use dash B for reading and dash C for creating. Okay. Typically, you do both, actually. Okay. Both reading and writing, if you want to update the file as well. But if I did just dash dash next, that would not work. Dash dash next, it doesn't enable. So you have to enable the cookies. Okay.

Specifically, you would use dash B first for enable the cookie engine, and then you could use dash next and you would you would keep using the cookies and those cookies and cookie jar would carry through the continuous sequence through dash dash next and whatnot if you did it in that first part in the get part yes exactly gotcha gotcha then you can do a full login and get sequence in one command line if you if you do're crafting it good enough. Sure. Where do you send people to when they,

to discover these features? Like, for example, I will do man curl, for example, as Jerry will say before, but, you know, deriving knowledge from manuals is difficult. Let's just say it's not written to a human or from a specific use case. So I often find myself googling or something else and i discover you know features where do you point people to to discover features

like this of curl to find useful or new ways they never thought they would actually find curl useful it's of course hard because there are so many ways to use it so it's hard to cover everything my idea or my vision is rather to have that sort of description or tutorial-like things to be documented in the Everything Curl book, which is everything.curl.dev, which is meant to be more of a learning, getting into stuff like that. I mean, the monpage is good, but that's more for a reference. If you know exactly what you want to do, figure out how to do it, not getting into learning things. So that's what I want everything called book to be. Is this like a living book then, in that case, where it constantly grows and evolves, where it's never like really written? It's more like constantly in write mode?

It is constantly incomplete. Yes. Okay. It's reassuring and also disappointing. Mostly reassuring. It's fairly thorough, but since the project is moving, the book will remain incomplete, right? Yes. I mentioned that earlier, too. In libcurl basics is actually where you have that dash dash libcurl where you mentioned earlier.

We're going to link that up. So that'll at least give people the inroads to everythingcurl.dev or actually everything.curl.dev. Sorry. Cool URL. We'll link up to the main page, too, everything.curl.dev, sorry. Cool URL. But we'll link up to the main page too that where you can dig and peruse as you like. Yeah, that's also new since three years, right?

I have two new curl on domains, curl.se and curl.dev. Sure, I got late last year. Weren't you on like hacks.se or something before? Yeah, curl.hacks.se until November last year. But now you got curl.se or something before. Yeah, hacks. Yeah, curl.hacks.se until November last year. But now you got curl.se. Yes. And then I got curl.dev. Which one are you using?

I'm using curl.se actually. It's shorter. And then I figured it was more fun to use the se one than the dev one. I don't know why. Is se for Sweden? What is.se? Yes. Se is Sweden. So I got the se one. I don't know why. Is SE for Sweden? What is.SE? Yes, SE Sweden. So I got the SE one first.

Actually, I've been trying to get it for a very, very long time. It became available and someone else snatched it before I was able to buy it. It turned out that it was a friend of mine who gave it to me. Oh, that was nice of them. Oh, nice. They bought it on your behalf. Nice. Do you ever run into any sort of copyright issues in regards to curl,

considering domains and stuff like people sitting on stuff that you shouldn't? Is that ever an issue for you? You mean people using curl to download things they shouldn't? Well, do you ever have to defend, I suppose, what you have as a curl copyright? Is there ever anybody using it nefariously or incorrectly or downright illegally that you have to defend i mean considering for the most part of one person maintainership a community around it you'd mentioned the open collective fund for and how you use that for security bugs you'd mentioned how your work relationship and how that interacts

with curl and enables you to work on a full time. I'm just curious if you've got like an attorney or a legal department or a need for one. Very rarely, actually. It happens, but it's never happened big time and it's never become any serious issue or problem. So fingers crossed, it seems to be working fine. But of course, I mean, I would figure it out if I have to, but I haven't had to do it at any particular big thing. I mean, I mentioned that before

that there was once a lawsuit that involved technology that Curl uses in the US, but it never went anywhere and I don't know what happened. So it's never been, neither copyright nor patents nor anything has sort of struck us in any particular way.

I was once contacted by the company called Curl Inc. that owns the curl.com domain. It was very early on in the Curl project. They basically asked, hey, what's this Curl thing? We are Curl. What are you? And I replied basically that, yeah, we curl we're curl blah blah blah and but they never responded back again and they still exist as a curl ink so i figure they have just learned to live with us we certainly don't kind of reminds me that spider-man meme where the spider-mans are pointing at each other

and so you can have like one saying i'm curl no i'm curl no you're curling right and there was actually a point in time where it actually could occur some some you know someone would actually mistake our curl for their curl that's some sort of web programming language so i i don't know exactly what it is, but it seems to be apart enough so that it actually doesn't confuse anyone. I was curious because you mentioned domains and changes. And anytime you have that, you got separation between what I saw last and what I think is true. leverage those changes beneath you in terms of changing urls or curl.dev or curl.se and masquerade as the real curl right i know that there there have been i don't know if they're still around there are some clearly free loading sites that have registered some curl sounding

domain with almost some top level domain just you ad-filled things basically then redirects to my sites. But that seems more like a really lame attempt to make money because it doesn't really work, and I think they're mostly closed down. Well, 23 years is a big deal, and we're here to say congrats and thank you for making an awesome tool and sticking it out and sharing your knowledge back you are just such a an example of someone having a hobby that turned into the full-time

thing and it's just a big deal and we really appreciate you coming back on this show three times now and uh you know sharing your vantage point of what's changed we're talking about h3 versus h2 this time and all the changes that have happened and we just really appreciate the work you put in so hopefully that's also motivation to keep it going as well but is there anything we haven't asked you as part of this call that you're just like i just really want to share this before we go is there anything we haven't asked you at all i think we've covered a lot of, so I can't recall anything we missed. I would, real quick, just like to encourage you

to keep writing as well, because not only the work that you're doing on Curl is important, but I think that you're writing about the work you're doing on Curl and about your trials and tribulations and things you find, and even just the funny moments, is awesome, and I really appreciate reading what you write, and I know a lot of other people do too.

So keep up the good work on the blog as well. Yeah. No, thank you. Yeah, I'll try to do that. I think it's fun. And I think it's a good way to reach out with all sorts of things. When you're having a bad day,

you know, or you don't want to deal with issues or the mailing list or something else, just write, you know? You got another thing to do, you know? But seriously, Danny, thank you so much for all your efforts and to Curl. I appreciate it as a tool. I appreciate you as a human for

doing what you do in open source and being an example to follow. Thank you so much. Thank you. That's it for this episode of The Change Law. Thanks for tuning in. If you aren't subscribed yet to our weekly newsletter, you are missing out on what's moving and shaking in software and why it's important. It's 100% free.

Fight your FOMO at changelog.com slash weekly. Huge thanks to our partners, Linode Fastly and LaunchDarkly. When we need music, we summon the beat freak, Breakmaster Cylinder. Huge thanks to Breakmaster for all their awesome work. And last but not least, subscribe to our master feed at changelog.com slash master. Get all our podcasts in a single feed. That's it for this week. We'll see you next week. Thank you. you