Software at Scale - Software at Scale 42 - Daniel Stenberg, founder of curl

Episode Date: February 10, 2022

Daniel Stenberg is the founder and lead developer of curl and libcurl.Apple Podcasts | Spotify | Google PodcastsThis episode, along with others like this one, reminds me of this XKCD:We dive into all the complexity of transferring data across the internet.Highlights[00:30] - The complexity behind HTTP. What goes on behind the scenes when I make a web request?[11:30] - The organizational work behind internet-wide RFCs, like HTTP/3.[20:00] - Rust in curl. The developer experience, and the overall experience of integrating Hyper.[30:00] - Web socket support in curl[34:00] - Fostering an open-source community.[38:00] - People around the world think Daniel has hacked their system, because of the curl license often included in malicious tools.[41:00] - Does curl have a next big thing? This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.softwareatscale.dev

Welcome to Software at Scale, a podcast where we discuss the technical stories behind large software applications. I'm your host, Utsav Shah, and thank you for listening. Hey, welcome to another episode of the Software at Scale podcast. Joining me today is Daniel Stenberg, the creator and maintainer of curl, one of the most popular open source projects that deals with a bunch of different protocols of interacting on the internet like HTTP, FTP. Thank you for joining me. Hello, and thank you. Nice to be here. Yeah. So you recently mentioned in a blog post that if somebody tells you that HTTP is simple, HTTP is easy, and it's not that hard to work with it, they're lying to you.

You've been working with HTTP for the past 25 years, and you can say it's not easy. As a software engineer, I use libraries that just download stuff off the internet. I use Postman, all of these tools. It sounds like you pass in a URL and you get what you want, right? But clearly there's a lot of complexity behind it. So maybe you can just walk us through, what do you mean by that? Yeah, first I think it's, in general, at least if you go back a few years, people have always thought about it as, you know, it's simple.

And you can even do it with a telnet, right? You can telnet to the host and enter get slash blah, blah, blah, and you get a response back. And it's also easy as when it is text or ASCII, you can read it and you can sort of understand it. So yeah, I think it gives the illusion of being easy. Where possibly it also was easy back in the 90s, probably around HTTP 1.0 that was fairly easy but it wasn't very easy as soon as we

got into http 1.1 which was in 1997 the first version but but it is complicated because of several reasons the one one is that it is text so and text is not easy to parse right so and then um over time things have happened in the protocol so for example just knowing when an hdp response ends there are like three or four different ways to know when the response ends there's not just one single way and there's and then um that's just one thing and you know parsing numbers is another thing. Space pads, or you can have negative numbers. Do you have comma separators? Whatever. But those are easy things, more or less.

But I think what's more complicated is then all the different header or header names and header fields that are nowadays a huge amount of them. And a lot of them interact with each other in ways that you really can't know without reading up the docs about those particular headers and understanding the repercussions. If that header comes and that other one comes as well, which one takes precedence?

What do you do? What if you get two of the same header? And in some cases, you do one thing for another header, you would do another thing. So that adds a huge complexity. And then all of those separate headers, especially the most popular ones like cookies or a few other things,

they have sort of gotten their own lives and have developed and sort of taken off in a completely different, in a life of its own. So they have developed and sort of, I don't know what words to use, but have sort of, they have gradually changed over time and become more and more complex, have very elaborate syntaxes, and usually also in a way so that they're supposed to be then sort of backwards compatible, so that the cookies sent back 10 to 15 years ago, they still work.

But today, all the sites send cookies in a slightly different way so that you try to cram in new things in the syntax without breaking the old stuff, but still supporting the new stuff. So it really becomes a sort of a Franken header. And that's the cookie one. And that's only the cookie one. And then you have all the other things.

So yes, so there are, I haven't counted, but in HTTP, just in the regular HTTP 1.1 spec, I bet it's almost 200 headers. And there are a lot more in other specifications. And that's just, you know, the headers, HTTP and stuff. And then you add things like HTTP 2 and HTTP 3 on top of that. And then everything goes even way, way more complicated because then suddenly you no longer use single TCP connections for

transfers since HTTP2 switched so that you would have multiplexed streams over the same TCP connection. So then the headers became binaries and they're compressed. And then adding HTTP3 on that, that's not even on TCP anymore, right? So that's HTTP3 over QUIC. So that has a different compression algorithm and you have to have QUIC underneath, which isn't TCP. So it's not even in kernel. So you have to have user land libraries to support it. So yeah, so to do all that and support it in one uniform way as a one api or one command line tool to do all that in and ideally i and i try to work with that in curl both the command line tool and the library is to provide you know a uniform api so that the user mostly won't have to care about exactly what's underneath right so you would still you would use your

command line exactly as you did 15 years ago you but today you can just add a switch and then suddenly it would do it over http3 instead of http1.1 which is over the wire something completely different so yeah there's a lot of and and also course, because HTTP is such a well-used protocol, and not only then all the browsers and all the web and stuff, but also REST APIs and applications doing everything. So it's also a protocol that keeps evolving and getting polished. And people over time agree that maybe what we did back in the past we didn't quite agree now we know how we should interpret this little thing in the spec so now we agree we all

agree that we actually meant this way which we didn't back in the day but now we do so without anything actually changed we haven't changed how we look look at that specific part of the spec perhaps. So that's all of that. And then HTTP of course then isn't... HTTP 1 and HTTP 2, we do them over TLS these days always. When you go to HTTP 3 that's part of QUIC, so it's not TLS anymore. Well, sort of but anyway since and since we do http over tls as well that is also you know an entire universe of complexity and different versions and extensions and alpn and so so much so there's really a universe of stuff to know once once you start scratching on the surface and really looking into under the hood how does it actually work, all this. The story of HTTP reminds me of Windows in a sense.

Each new version adds a bunch of stuff and you can't really deprecate any of the old things. Something like that. And you need to keep the old stuff so that you can bootstrap up to the new stuff. Because nobody today only implements the new stuff right so you have to have all the old stuff as well so yes a lot of that so what happened with http 1.1 like why did it start with it sounds like that is the first release which had a bunch of complexity because i remember the first version of http get or the predecessor occurred in a sense, right? Or it's the same

tool with a different name that was implemented in like 300 lines of code, and then you contributed some fixes. What happened with 1.1? Well, what made 1.0? Well, first, actually, if we go even further back so there was there was once was an http before http 1.0 right that was the http people used with it before there was a spec and that is we call it nowadays we call it 0.9 and that was a really really really simple protocol you basically just send sent a get line and you got data back there was no headers in the response just content and you got data back. There was no headers in the response, just content. And you closed the connection when you were done. Very, very easy. No headers, nothing. So there was no extra information from the server at all. And then when HTTP 1.0 came, they added headers in the response,

basically. Actually, I wasn't around when they did this. So I didn't follow when this happened. And then also added headers in the request. So they added a lot of stuff in HTTP 1.0. But 1.0 was fairly easy in that you close the connection when the response was done. So you would only ever use the connection for one request, right? Set up the connection, do a get, close the connection again, which was easy. So then you always knew when the transfer was done

because you closed the connection. You didn't really know if it was prematurely closed. That was also sort of an unfortunate side effect. You could detect it in some cases. But anyway, but that was a really ineffective use of TCP, right? Because then you had to set it up, close it down, set up, close down. So when you started to get a lot of more objects on like your web page, it was really ineffective. So you never ever got any fast responses from servers. So when 1.1 came, the biggest change in 1.1 is that they introduced

the persistent connection. So you could maintain the connection and do more requests over the same TCP connection. And that then introduced the importance of knowing when the response ends. When is the end of the response body so that you know that you read all that is part of the response and then you could do the next request and get the next response. And so that added then the chunked encoding and stuff which is also not complicated but it wasn't there in 1.0 anyway. It also added a lot of other controls for caching and they also added support for HTTP pipelining which is a story in itself because nobody ever implemented it back in the day. And when years later, when people started trying to implement HTTP pipelining, it didn't really work because everything was already developed to a point when we hadn't

really had pipelining. So trying to get pipelining to work after the fact never really worked. So it's been one of those sad cases of protocol development that we learned that we shouldn't do that when we develop protocols we need to implement every single part of the specification because otherwise it'll never work again so when you hear about http specifications and you you hear that, you know, someone is drafting it, like some organization, who actually is working on these organizations? Who's doing the work to spec it out? Like, are you a part of it? I know that certain like big companies like Google and Facebook may be a part of it, especially with Quick. That's the story. What is that process like um so all of these protocols everything basically from iep up to hdp the transport protocols all those are usually well they don't have to be done but they are done by the ietf internet engineering task force which is an open organization. It's free for anyone to join and be part of the meetings. Everything is driven on mailing lists, basically,

and a lot of it is on GitHub these days. And there are also meetings. I think it's three times a year. They've been virtual now, of course, for the last few years, but they tend to be huge physical meetings going. So there are working groups then for different areas of interest and there's an HTTP working group that has been existing for a long time now.

So within the HTTP working group that I participate in and I'm joined in discussions and so on. So we bring in HTTP related stuff that we work on and try to write specification that we publish. And we work on specifications that a lot of people are interested in and interested in deploying and adopting in the real world. And we have the pretty fortunate position in that group that we have a lot of implementers present. So a lot of that we have a lot of implementers present.

So a lot of server authors and a lot of client authors. All the browser vendors have people there and a lot of the big, even intermediary proxy people are there too. So there are a lot of basically people with experience with HTTP for a long time and from different angles and different products from different companies. But of course, a lot of those are representatives from huge companies, right? Like Google or Apple or Facebook or Microsoft and all of those guys. Because if you want to

drive something in those organizations, you need to spend a lot of time on it, right? So you basically need someone to allow you to spend some amount of your work hours on IETF work. So that sort of favors the big guys, because if you're a smaller guy, you'll have a harder time to devote the necessary time to sort of have a real impact. And it's also sometimes fairly technical and it can be really hard to just keep up with all the lingo and the text, right? So you need to, what are they talking

about? Now I need to read up on this little background stuff because otherwise I don't understand what they're talking about. And if I don't understand, then I can't contribute right so there's that but there's there's no sort of there's no real limit for anyone sort of anyone can contribute if you just have the time and effort and skill you can be there you can speak up your mind people will listen and they will if you talk nonsense they will ignore you but you still can't do it right so it's actually a pretty good way i think it is it has its flaws i'll admit that but i think it's one of the best ways i think we can do protocols and then sort of we work on on defining things how to do stuff in protocols and you know people argue back and forth about stuff we can discuss this doesn't work this is better

no it's not blah blah blah and then sort of with consensus and and discussions we usually go forward and then end up with something that we all are reasonably happy with so you have clients all the way from things like curl to browsers like Chrome and Firefox, presumably, and then you even have some proxies, as you said, maybe like someone in Nginx or Envoy or some of those systems, all the way to do you even see like client libraries like Flask, the media Flask, Django Express, or is that too high level? Less of that, I think. But sure, they are represented too. And usually, you know, I'm not even aware of where the people come from, right? Because they're always representing themselves. They don't represent the company.

So they're all, you know, individual names. So there's John saying something and there's Peter something and then Susanna says something. I don't know. I don't necessarily know where they're from or who they represent. They have opinions and they have ideas and they have objections, whatever. But yes, all the big ones are certainly involved.

And usually they are also, especially those who drive certain efforts and initiatives, they tend to be representatives of the slightly larger companies or companies with a larger footprint. Someone who runs a lot of servers out there, for them, it might be very important, this little niche thing in a protocol, to get that better. It'll help them because they run 15 of the internet so getting stuff to so of course they have an vested interest for me maybe i have i i usually try to have the position of you know i'm i'm writing a command line tool and

library that isn't a browser that that tends to be my little niche because the browser guys they tend to look at hdp and web from a browser perspective. And a browser perspective, that's a client side, but it's certainly not as a command line or a small library as I view curl. So I think I also have sort of my own little niche that I try to raise my hand and say, hey, remember this little use case that we have that you might not and so on and generally what is the evolution of http generally had um what has been the objective of all of the evolutions like it seems to me at least from the outside things like 1.1 being able to reuse connections three with quick you want to make the internet faster in a sense more efficient send less data across the

wire is that generally true what are the other objectives generally no i think that's the primary objective since maybe since 2010 or so i think it i think it was pushed by google originally and i think was i don't remember exactly, when we all started to realize that, you know, when users using browsers, when they get the responses back faster, you know, people are appreciating that. And that has, I mean, we all knew that. But I think when at some point in time, long ago, now, we started to realize that it's actually money involved, right? So you know, that if your company is selling something, you know that your customers are more happy

if they get responses back faster. So they will be more likely to buy your stuff or watch your video or read your message. So then we started to understand that getting things faster is more sort of, there's an economy in this. And that's when we started to do everything faster and faster on the internet, on the web. And I think that's been driven a lot by browser vendors,

because they could then see if we get a response back to our users X milliseconds faster, it'll sort of be converted into happier users to this degree. So when people started to realize how do we do the web faster, you know, we have HTTP, we won't convert it into anything else, but we can change it to P. So that's one of the first, when HTTP2 came about, right, that was based on an experiment from Google called Speedy. And that was exactly like, how can we speed up HTTP,

but still sort of have HTTP around? So they worked on a lot of experiments and they thought it worked out. So they took it to the IETF to make it a standard out of their experiments. And out of that process came HTTP2. And it was faster and it did things a lot better for a lot of users. And then basically at the same time they already

started to work on their quick experiments so they had that already in progress actually when HTTP2 came about. And that was just another experiment again by Google to see what if we did this can this make HTTP even faster and address some of the issues that could still be found in HTTP2. HTTP2 fixed a few things right compared to 1.1 but there were still maybe not issues but there were still you know things that could be improved for some particular use cases and that sort of was brought into the work with QUIC. And then it should be three. So it has been that, that drive has been for maybe 10, 15 years now.

Get everything faster. Get the responses back. Shorten the run, reduce run trips, get data back to the user faster. Okay. And there's so many features to add and things to support with all of the user faster. Okay. And there's so many features to add and things to support with all of the different protocols.

And you mentioned a bunch of different headers. There's a lot of code you need to add to continuously keep up with the stream of changes. On that note, there's been a bunch of excitement around Rust and Curl. And maybe, I think generally, there is a community of developers,

Rust developers who are excited about putting Rust in everything. And I think that's a standard meme. But maybe you can walk us through how you think about Rust and Coral. Well, I'm certainly the first one to admit that, as I blogged about it a while ago then, that around half of all the security vulnerabilities we found in curl are related to memory safety, or as I call it,

C mistakes, because they weren't all memory related, but they were certainly C related mistakes. So maybe half of them. And at that time, I think we had 95 or so. Now I think we're up to 110 something. So let's say it's half of 110. That's still 55 mistakes over 20 years, right? That could have been avoided if we hadn't done it in C. So sure, it is sort of an attractive promise. If we could do it currently in a way and reduce the mistakes or the security

vulnerabilities by 50 percent that's sort of that's where we start this but and then I have my other end of it is that curl or lib curl actually since I think actually curl is primarily lib curl right that's the that's the bulk of my work and that their efforts is my my big effort is to provide the library and to make the api stable and the abi survive everything and i think that's one of my biggest selling points and and sort of for curl is that we have this api we have this api it's they're stable you know that this it'll continue working like this for a long time so if you upgrade for something you did 10 years ago, you can upgrade to it now.

It works the same way and you can rely on that. And I think that is something that is sort of sacred boundaries in the project. So therefore, rewriting everything in another language, that would be sort of, I wouldn't say impossible, but a feat that goes beyond me, right? So it's important for me to keep, to have that boundary, right?

I want that API and ABI to remain exactly as it is, but I'm perfectly open to do whatever it takes internally to improve the project if we just maintain those boundaries, sort of. If we can maintain behavior, everything else is free to up for grabs. And we already have an infrastructure in curl that is built up by a lot of different, I call them backends, but people have a hard time with that. So maybe components. So you can

replace components. You can select which TLS library do you want to build with. We can build with one out of 14 different ones. Actually, you can build with more than one. But anyway, 14 different TLS libraries. We support three different SSH libraries and a bunch of others too. So there's a sort of pick and match. What do you want when you build curl? So it wasn't really far-fetched to imagine. Sure, you can imagine build with or without Rust components as well, right? And we already support a few Rust components.

So we have, for example, we already support one TLS library written in Rust with actually called MesaLink. We support a HTTP3 and QUIC library written in Rust called Kish from Cloudflare. It's experimental, but it's in there. So when we started to talk with ISRG about what we can do in curl to replace the HTTP engine in curl with one written in Rust, I figured it was just like we do with all the other alternatives, alternatives. Pick one of those and you can build curl to use that instead. In this case then I worked on making sure that you can now when you build curl select to build with hyper the Rust written http library instead of the native http code. So you pick native or hyper for HTTP which isn't strictly true because there

is a lot of other HTTP stuff too. But a big chunk of the HTTP code can at least be replaced by the HTTP code written in Rust called hyper. So that is basically how we went into this. And then I've worked quite a lot. I've been working on it for a year now to make sure that you can actually build curl with hyper instead of the native HTTP code. And it should work exactly the same way. As a user, you shouldn't be able to tell the difference. Ideally, then you would have a safer binary in the end, right? So it would be less risk to get memory-related security problems in that part of the code, at least.

But it also, again, back to HTTP, is a complicated protocol. And talking about, you know, Curl is a very old project, and we offer a lot of features and stuff. Maybe we provided them from the beginning and it wasn't maybe that smart to do it that way from the beginning. But hey, we didn't think about that, you know, 20 years ago. Hey, fun to do it this way. And now when we insert Hyper into the mix and we say, oh, wait a minute, we have to support this little fun thing that we thought of 20 years ago.

Maybe not that clever, but hey, we still have to support it, right? Because it's in the API and we might have a user that somewhere depends on this. So we have to make sure that we can get that quirky little thing out of the hyper library too. So that is actually, we're now down into those little tiny details and differences in the weeds, sort of to make sure, oh, what about this? What about trailers in the weeds to make sure, oh, what about this? What about trailers in the HTTP chunked encoding stuff? That's sort of what I've been working on this week. Trailers in chunked encoding, how do I get that out of the hyper library?

We have, I think, we have around 900 test cases in curl that is related to HTTP in one way or another. It doesn't really say how much it tests because it could be small tests, could be big tests. But out of those 900, we're down to I think it's 35 tests now disabled for the hyper build. So that at least says something about the scale. So I'm working. My goal is to get to zero disabled tests. If you had asked me a while ago, I would have said by the end of the year, but now I think maybe January, February, in the spring. So at some point, I hope to get that down to zero.

And at that point, I hope to be able to say that this is not experimental anymore. Now it sort of should work exactly free from everyone. Build it if you want to. And then see if people actually want it and actually build it and use it for real in the wild. Because it's still marked as experimental now in the code. So you have to build it yourself

because nobody will do it as long as i call it experimental so it hasn't actually been used that much yet in the wild so but it sounds like it's living up to its promise in the sense of it seems to be able to do everything that girl needs right how is the developer experience and just otherwise the the whole experience been in the last year like working on well a hyper is uh it's a very i would say it's a competent library it it knows everything that we needed well i've been working with them mostly because I've come to them with questions like, hey, now I want this little thing. I can't figure out how to do it. Usually, they can provide it.

And usually, it has only been a matter of nobody of their existing users have ever wanted this little quirky thing that I want. So we've been working on making sure that they actually provide an API for me to extract or set that little behavior or tweak. And they're very keen on getting that input. And they're usually very fast on adopting to my desires or telling me that I'm wrong. This is the way to do it instead. And I just didn't understand it. There's a little bit of an uphill challenge sometimes that that's a Rust library, right? So the first time I approached them,

they didn't even have a C API for the library. They pretty much made a C API when I asked them about it. So there's a little bit of immaturity in that we don't have the documentation and the experience with using that C API yet. So there's a little bit of that banging my head on stuff without really understanding that maybe I didn't understand it completely. Because the API to Hyper is completely different than Libcurl or actually even how most C APIs

tend to work. So it's a little bit of a different mindset. So it takes a little getting used to. I mean, I wouldn't say that it's wrong or bad or anything, but it So it's a little bit of a different mindset. So it takes a little getting used to. I mean, I wouldn't say that it's wrong or bad or anything, but it has taken me a little bit of a time to get into. But apart from that, I don't have to care about or think much about it being written in Rust or not, because they provide a C API anyway. So I actually just interface an API, as it would have been with any other library. So it's just an API I need to work with, and make sure I understand it and use it correctly. And then, you know,

send in data, get data back. So yep. Yeah. And another exciting thing that you're working on, like implementation doesn't seem to have started yet, you're working on is web sockets so web sockets don't clearly fit into the curl model of like uploads and downloads but can you tell us you know what has been the user excitement behind like you know working on this project and how's it going on that so far right so so web sockets since web sockets is more or less like TCP for JavaScript or something. So it's more of a bidirectional connection to send anything over HTTP. And it's been one of those. I have this annual survey of users and I ask what kind of stuff,

well, when I ask users what they use and what they're happy with and what they're unhappy with and so on. And one of those repeated questions is, which protocols would you like to see us support in curl? Which of course is easy for anyone to just, you know, I want everything or without that actually meaning much. But WebSockets has always been, well, for many years, been one of those top voted protocols that people say they would like in curl. So I've always, I've had that in the back of my mind for a long time. And now this year, I started to actually work on documenting, well, assuming we would do this,

how would it actually work? Because as you said, it doesn't really fit the upload-download libcurl paradigm. So we would have to do it sort of differently. But so, yeah, assuming we think it's a good idea, how would it work? So I worked and discussed it back and forth on the main area. So I have now a design document that says basically this API probably works to do WebSockets. And I know a lot of WebSockets users are also using

Libcurl for stuff. So they use Libcurl. A lot of them do some semi WebSockets over Libcurl with code outside of it. So there's a lot of users who actually see a benefit here because then they could unify and you'd use this single API instead of many. And I've also come to understand that a lot of the WebSockets use cases that they also more or less just download stuff over WebSockets instead of over something else. But anyway, so that's where we are.

And I've been struggling to get to a point when I start to implement this. I'm actually having a conversation right now with a company about possibly sponsor this work for me so that they would pay to see it happen. And it might happen soon. So we'll see. I've been sort of holding off that work because it's quite a big chunk of work. And I would rather have someone to either help me or sponsor me to make sure that it actually happens.

It's also a sign of real interest, right? Is everyone just saying they want it or do they really want it? And if someone really wants it, maybe they could be prepared to pay for it as well, right? So maybe if nobody pays for it, it shouldn't be there.

So I've been holding it off. So we'll see where we go with that. Yeah, I mean, speaking of sponsorships and contributors like the number of contributors has been going up significantly right uh over the last few years and there seem to be some inflection points when that has happened so how do you get or how do you foster a community such that people want to contribute to a library like core right there's so much that people could do and there's so many other open source libraries that people could spend their time on how do you drive interest in

in your work that's a really good question right because that's a that's sort of the constant problem that the world is drowning in open source projects and everyone has one of their own there's so many interesting things to do where Where do you, where do you, why would people spend time in your project and not in someone else's project? So yeah, I, I, so I can't really answer that, but I try to instead make our project, make curl as good as possible in, you know,

catering for newcomers and making sure that everything is documented, welcoming, no friction, lowering the bars for entrance, and being a friendly project without any bureaucracy or weird stuff to overcome when you join the project and want to do something. So that's what I try to do in this project, to make sure that it should be really easy and friendly for someone.

Even if you just find a typo in the documentation, it should be easy for you to just get the code, submit a pull request, and we should be able to just accept that and say thanks and get your name into the credits when you've done that awesome work. Even if it's a type of documentation or you know fixing a really hairy bug somewhere so and i think that is what has paid off and then

i think we actually we're a bit of a in a good position because the more users we get the more widespread curl gets people get more of a you know boost and a feather in their hat when they think about getting their changes into curl. So they sort of think of it as a cool thing to do, which is good for me, right? I don't really mind whatever the motivation is to help us. As long as they help us, it's good for us, and it's good for everyone. But I think that also drives the sort of, if they get code into curl, they know that their code is then potentially in 10 billion devices in a few years. And, you know, not the least since curl was used in the helicopter landing mission on Mars. That's also another thing, because now you get that

little badge on GitHub that says, I'm part of this project. So that's also another driving sort of motivation for people. Wait a minute, if I contribute to this project, I might end up getting this kind of cool badge in the future. So getting popular and getting used widely is of course helping us in this case.

And then of course, we have to take care of them when they feel like they can and want to contribute. So we need to make care of them when they feel like they can and want to contribute so we need to make sure that we can do that i think you do a really good job by like celebrating every new contributor by posting about it on like social media and things it's really wholesome to see all of that yeah i try to do like that so because you know thanking contributors for their efforts that's basically the the way that we pay them, right?

I mean, they do work for It's Not To Pay. I shouldn't even say that. But they do something for free and they give it to us out of the goodness of their hearts, often from their employer too. But anyway, they give it to us anyway. So saying thanks and celebrating them

and say, hey, thanks to this person joining the project, contributing. I think that's the least we can do and i think it's uh we should all be i mean everyone actually that uses curl or somehow stands on using curl somewhere is benefiting from this right so it's we're all benefiting so i think we should all be happy and glad that people are willing to help out and on the flip side, this might not be a super comfortable topic, but sometimes people think that you've hacked their systems

because they see your name in the licenses of whatever tools that hackers might have used on their systems or just in general, they try digging up things, they see your name because it's connected to curl and then they email you negative things.

What's just your opinion or your take on that? Well, my primary take is that usually I get just weird and fun emails about it. So usually it's just a positive side effect. I get emails from people who are confused about their gps in their car or you know they couldn't finish a level in some game they found my name in so usually that's sort of you know i'm just getting happy when i see the email usually also a little bit confused because it takes a while until i understand what they're talking about so so that is actually more of a positive thing than a negative thing.

But yes, since Curl is such a versatile tool that you use for whatever you do, internet transfers, and even bad people and bad software are doing internet transfers. So sometimes my tools are used in, or my code, my name ends up in areas where I really don't want to be sort of associated with, but I can't do much about it. So yes, sometimes then people sort of draw the wrong conclusion. Well, often they draw the wrong conclusion and associate me with whatever badness that happened to them. And yeah, that is unfortunate. I don't think, there's not a lot I can do about it other than just explain to them when it happens.

That's also very, very hard. Even in the benign cases when they have a car, it's really hard for me to explain that the name they found in their product, I have no idea even that they used it. Or, no, I don't know the manufacturer of your device, whatever brand it is, and so on. So, yeah. So, I got this death threat like a year ago from someone who found my name in some it was really confused i don't quite understand the sort of everything but he blamed me for having lost his job and family and money

and companies and all sorts of stuff but of course it then begs the question if i had done that why would i leave my name everywhere but usually that kind of logic doesn't really work because when they're at that point sort of then they've tried a lot of other things they're probably desperate probably angry sad or upset or something else so they're just you know grabbing anything and i happen to be have my name there then so I got to be in the well in their target that user actually then he actually contacted me again like a half year later and apologized one thing for sure is that you've had an extremely unique life experience by being a for such a project used by so many people like some of the things that you have

to deal with probably very few people in the world have to think about it other than like actual like celebrities and stuff so there's that going yeah that's true that's true yeah in some ways i have a pretty unique situation or in many ways actually just as a starting off like a wrap-up what is the next thing big thing for curl red i know you were thinking about curl 8.0 like is the http3 working completely part of curl 8.0 like what's what's the next big thing what are you excited about well generally i don't look for the next big thing I never work on the next big thing from because I most of the stuff I work with are so gradually implemented so they never turn out the big thing like it should be three we added support for it a long time ago and it's sort of we're gradually

continually polishing it and still experimental there. So that's one of the areas that I suddenly see that we are going to improve and make better going forward. And hopefully at one point, remove the experimental tag and make sure that we can ship it sort of by default for everyone

and everyone can use HTTP3 with curl and everything will be great. But, and then we will do, there's a lot of other stuff like that. Maybe we will do WebSockets soon and I hope to get that hyper integration done properly. And there's more stuff like that, right? We're working quite a lot on, for example, the Rustles backend, which is also another Rust library for TLS, for example, so that we should be able to get more TLS done in that Rust library.

And so there's a lot of different small components that are, well, small areas of the code that we're trying to improve. And I think that's generally the way I work with curl in general. Curl does a lot of things and I try to gradually polish all of those different areas, improve it here and there, better documentation, better FTP, better HTTP, better whatever and then just go with whatever the flavor of the day is. If I have users, if they pop in tomorrow and report a bug of something or bring in a huge pull request and show an interest to go that way. So then maybe I can focus on that tomorrow. Maybe someone in the IETF will submit a draft tomorrow that suggests a new something. Oh, then I have to consider that maybe for the

next release, maybe for the next, next, or so on. Like just the other day there's um um there's the we brought up the discussion in the htp working group for a new um method and hp method right which is a sort of as a get get with a body called query and that's just as an example whoa the new htp method maybe what do i need to do about that in curl maybe nothing maybe a lot i don. I don't know. So I don't really plan long ahead. And I think that's actually luxury. I don't have to. So I go pretty much where the world takes me. And of course, I do curl full-time these days.

So I do curl support for my day job, basically. So I also need to take care of my paying customers and pay focus on their problems and their needs that also then sort of controls me a little bit but it sounds like all the tiny little things basically add up to the next big thing exactly and i think that looking back through the years that's basically the last 20 years right so a lot of tiny things all the time. And then suddenly over the years, suddenly all of those tiny things have built up to, oh, it turned out to be a big thing.

And then we just move on and then with more tiny little things. And yeah, I think that's a pretty good way to describe how we do things. Yeah. But thank you for being a guest on the show. This has been an amazing conversation. And I hope all of the future releases end up as bug-free as possible. Yes.

I actually often get that response that people have the weird view that we don't have many bugs in curl. And maybe we don't for the majority of use cases, but with all these protocols and the evolving internet and everything, there are so many niche cases, there's so many special scenarios or, oh, wait a minute, you had that particular server and that particular network and you use these 22 different switches in this combination and then suddenly something weird happened so yeah it's a constant battle and this fairly high pace of development of course also is a

adding a lot of things while also keeping it all the old stuff working the same way yep it's certainly a fun thing we're keeping busy while also keeping all the old stuff working the same way. Yep. It's certainly a fun thing. We are keeping busy. Thank you.