The Changelog: Software Development, Open Source - HATEOAS corpus (Friends)

Episode Date: December 8, 2023

Jerod is back with another "It Depends" episode! This time he's joined by Kris Brandow from Go Time and they're talking all things API design. What makes a good API? Is GraphQL a solid choice? Why do ...we do REST wrong? And WTF does HATEOAS mean, anyway?

Transcript
Discussion (0)
Starting point is 00:00:00 Welcome to Changelog and Friends, a weekly talk show about Hadeos haters. Thank you to our longtime partners for helping us bring you awesome developer pods each and every week. Fassy.com, Fly.io, and Typesense.org. Okay, let's talk. So I am here with my friend Chris Brando. Hey, welcome back, dude. Hey, glad to be back. People enjoyed our last episode together.
Starting point is 00:00:47 Yeah, I saw that. Do I sound surprised? You're like, wait, were they not supposed to? Or what were you expecting? People generally like what I have to say, Jared. Come on now. I also, I was one of those people who enjoyed it so of course happy to have you back i think you enjoyed a recent episode i did with justin searles at least the premise of it being this whole it depends that resonated with listeners as well maybe you know maybe not the particular episode
Starting point is 00:01:19 sorry justin no it did but the idea that i had for this it depends podcast and how we're just kind of doing it now uh loosely underneath the umbrella of Change Logging Friends, I think people like that whole idea of two people with experience, different experiences, hopefully, diving into the nitty gritty decision making processes that we go through, how, question each other, etc. And you like that idea. Yeah. Yeah. I thought that like, as soon as I heard it, I was like, Oh no, this is, this is a great idea. And this is definitely a show I want to be a part of, or like a mini series or whatever it is that I want to be a part of. Cause I do say it depends a lot. You do. In fact, we put together that little montage of everybody saying it depends throughout, uh, most recent 50 episodes or so. And I think you got on there twice. You were the one was it depends what TikTok you're on,
Starting point is 00:02:10 which I'm not even sure what that conversation was about. I guess it depends on which TikTok you're on. I do have a soundbite of you. Maybe I'll put it in right here. This episode is wild. Where you say this episode's wild. And I think it might have been I remember clipping that and saying
Starting point is 00:02:27 that's a good soundboard, Chris saying that. But I think that might have been the episode when you're just talking about, it depends on what TikTok you're on. Yeah, I'm trying to remember what the context that was. I remember saying it. I vaguely remember the episode, but I don't remember the context. But I also remember just like
Starting point is 00:02:43 I was walking and I was listening and then you had that little clip't remember the context. But I just remember I was walking and I was listening. And then you had that little clip of all the sounds. And I was like, oh, this is the school. I was like, oh, I wonder if I'm going to be in this, right? And then I was waiting and waiting. And then I heard that one of like, oh, it depends on what TikTok you're on. And I'm like, oh, really? Is that my It Depends clip?
Starting point is 00:02:59 It's like, that's the one. I'm like, ah. I was so disappointed. And then I was walking. And I think I might have paused right before because I was like almost to my destination. But I like messaged you and I was like, ah, I'm like so sad that this is like the one I had.
Starting point is 00:03:12 And then I got to the end and it was just like the me going like, it depends. And I was like, oh, no, that's perfect. Yes. Which is why I use it as the closer because just the amount of remorse and angst that you expressed there. In fact, one of those sounds was Matt Reier saying, I think it depends should have its own little theme song. I didn't have break master put together something special yet. I'm just kind of riffing, but I did put together a brief jingle and you may find your voice on it here it is it depends okay there you go so i think that's a good representation of this whole concept yeah i think i definitely think so
Starting point is 00:03:58 so justin and i were talking about dependencies and certainly a big topic one that we will take up again probably in different ways with different people over time because it's one of those and I were talking about dependencies and certainly a big topic, one that we will take up again, probably in different ways with different people over time, because it's one of those decisions that we all have to make on a recurring basis. And so it just requires a lot of thought and a lot of discussion. And I think a lot of data points is useful today. You and I are going to do it. It depends on APIs, which again, even that, as I say the term, is like a multitude of conversations could come out of such a thing. We're going to talk about designing APIs, building them.
Starting point is 00:04:33 Also, we'll reference consuming them, of course, and all the little things. And we're going to start that conversation by it-depensing the jargon, because not all APIs, because even API is such a generic thing, an application programming interface, that can manifest itself in so many different ways. And so maybe we start with a definition or start that by breaking down kind of generically, at least, the major types of APIs that we come across. Yeah, so like when I was thinking about this, I was like, okay, what kind of APIs exist out there? I think the one that kind of shoots
Starting point is 00:05:08 to the top of everybody's mind when they hear API are web APIs or microservice APIs. Things that are typically built with HTTP and used to communicate between separate computing entities. There's also language
Starting point is 00:05:23 APIs, which we all use, standard language, standard libraries, and all of the dependencies that you import that have their own APIs that you're using directly in your source code.
Starting point is 00:05:37 But also, there's two other types of APIs that I think are also quite interesting, and I also think they maybe can teach us a bit about how we should think about designing APIs. I think those are like the operating system level APIs, or like the system call library level stuff, like the Linux system call center library. I don't even know what to really call it.
Starting point is 00:05:57 But yeah, like that layer of like really low level, really invoking from C or assembly or something like that. And then kind of even lower level are these things called ABIs, which stands for Application Binary Interface, which is like APIs, but for your ELF or Mach0 or MachO, MachO, PE, like the form that Windows uses, like how you can make sure that if you have a DLL that's compiled and an application or a compiled two different versions of a compiler, that they'll be able to work together.
Starting point is 00:06:29 So it's kind of lower level, like machine code interfacing. Weird stuff like calling conventions in there as well, which I have a thing that we can get to later on about maybe how we can design APIs better there. The big two that people think about are definitely language APIs and then the web APIs. Well, let's start with the language APIs or the programming level APIs. I think the web APIs is the one that most of us think of most of the time and probably consume as working developers.
Starting point is 00:06:59 I think we probably are more consuming those than writing those. Surely we're writing them as well. But what we're all writing is like the, in the small APIs that we ourselves are consuming in our application code or in our library code or that we're writing for others to consume in their application code.
Starting point is 00:07:18 Literally all developers are writing those APIs. We don't really think about them. We think of them as like functions or methods or pick your particular term, depending on the context of the executable. We design those APIs all day, every day, don't we? We just don't think about it very much.
Starting point is 00:07:34 I'm sure you think that that should be designed, right? Not just kind of grown. Yeah, absolutely. I think this is a big it-depends-arity as well of what stage are you in? If you're just prototyping, like, you know, you don't need to do that much design or that much documentation of the APA you're building. But definitely, if you're, even if you're building, you know, a whole cohesive thing, that's just like a thing that's going to run like a CLI that's not going to be used as a library. So just, you know, thinking through how you lay out your functions and your methods and your types and all of that, how those things interact is very important for like kind of the long-term maintainability of that API. And I think for most of us, when we're sitting down and
Starting point is 00:08:16 writing all those things, we're just like, I just need a function that does this, or like this function is too big. So I'm going to start breaking it out into other things and kind of making more surface area of that API. Right. In a way, I'm not sure people breaking it out into other things and kind of making more surface area of that API. In a way, I'm not sure people would really do if they were thinking of this more from the library sense. I think when you jump to the library design, you're like, oh, I got to really think about all of the functions I have and what they're doing. Yeah, you think about it more because you think somebody else is going to be using this and it needs to be consumable and repeatable or has to have certain characteristics that make it good for them but when we're writing our own functions internally that maybe nobody's ever going to see or maybe you're going to use it once a lot of times i'll abstract a function simply to put a name on the logic you know and it's like does that that name needs to be relatively
Starting point is 00:09:02 good enough to be descriptive so that I know what that bit of code is doing. But I'm not really thinking about it as an API because I'm just extracting a function so that I can name some bit of code. But when you're naming functions and designing functions, whether you're going to be using them, your team, your immediate team, or once you get to the library level, do you have heuristics or thoughts on like what makes in the small like a single function api good or bad better or worse yeah but i think definitely the number one indicators is how many like how many parameters do you have really that's like kind of the first thing i always look at is like does this thing have like two or three parameters which i'm like okay cool or does
Starting point is 00:09:45 it i think one of the code bases working on we had this function that had like 16 parameters and we're like okay well this is this is a sign that this thing's kind of broken and also like the number of parameters compared to the lines of code of the actual function itself like the one that had 16 parameters was also like three lines of code because it was calling another thing that had 15 parameters and that was a sign of like oh this is a spot in like our local api that we need to fix because this is just not okay so i think like definitely looking at the parameters and the inputs to the functions i'm creating is one of the biggest signs that i tend to look at. Yeah, inputs and outputs. I mean, I guess the three things that I consider is like the name,
Starting point is 00:10:28 which if you're having a really hard time naming something, I think that's a point to slow down and maybe think you're trying to encapsulate too much sometimes. And maybe you're actually, it's two things or more, and that's why it's hard to name. But clarity and naming is something that I take very seriously, even with my own code, which there's lots of with my own code that I don't take seriously at all. But naming with clarity is one of them, probably because code comments
Starting point is 00:10:55 is not something I take seriously and so I need good names. But naming inputs and outputs, that's pretty much what you're going to see from the outside of a function. That's what you're going to care about is like, okay, what's coming in, what's coming out. And I agree with you that like too many parameters
Starting point is 00:11:10 is a indicator of, again, poor design or trying to do too much or sometimes just scope creep, right? Because a lot of those functions start with not very many parameters and then you're like, I'm just going to slap another one on at the end here. And that's fine. And it probably was was and then the next time you come back you're like i'm just gonna slap another one on at the end here and at a certain point you get
Starting point is 00:11:34 to 16 right and then and then people have meetings about your function that you wrote you know and that's a problem then there's the cheat code right which is where you pass in a hash or a map or a struct, right? And you're like, see, it takes one. It only takes one input. It only takes one. Yeah, that's one of the things that when Go introduced the context object, there was a lot of push in the community to be like, don't put random junk in here.
Starting point is 00:12:03 That's supposed to be a parameter. Like, don't do random junk in here. That's supposed to be a parameter. Like don't do parameter hiding in this thing. Cause that'll kind of, that'll allow you to put too much past too much stuff into a function and not really rethink if this function needs to be broken up or if you need to start restructuring your code. Cause that's one of the things I really love about go is that because things are pushed to be so simple in a lot of ways,
Starting point is 00:12:24 it becomes much easier to see when there's a problem somewhere. Whereas if you have that, you know, kind of bag of stuff you can pass into something, it can be a little bit more difficult or challenging to figure out, like, what stuff is in here? What am I actually getting? What am I actually using? Exactly. I find a lot of complicated, what's the opposite of confident?
Starting point is 00:12:44 I mean, non-confident, let's just go with that. Non-confident code is like dealing with what bag of stuff is in this particular, like what are the contents of this bag of stuff? And I know that my TypeScript friends are yelling right now, that's what TypeScript is for. But I'm sure you can pass some sort of an object in there and not define the contents of the object if you don't want to and end up with a bag of tricks. But are the any, any type, that's the one, right? In Go, it's pretty persnickety about that,
Starting point is 00:13:17 but I'm sure you could just pass a struct in, like the context object, right? Yeah, you can pass like, that does happen quite a bit with APIs where you have like a struct, especially when it's like a constructor where you're like, oh, there's a lot of different things
Starting point is 00:13:30 that might be in here. You have a struct or like a special object type that builds up options and you pass that in. There's definitely cleaner ways to pass a large amount of information in. But it's also sort of obvious when you're kind of looking at it.
Starting point is 00:13:44 And sometimes, you know, at the end of the day day you do just need to pass a lot of parameters into something and i think when you do like kind of bundle it into a struct into another type that's i think that's pretty okay as long as you've like thought about it as long as you can like name that type properly and it's not just kind of like input to function name or whatever. Right, or data. That's what I go with when I can't think of a better name. It's like, here comes the data, you know? I mean, context is kind of just as bad. I mean, context is a pretty meaningless thing once you start to think about it.
Starting point is 00:14:16 Like everything is contextual, right? It's all context. I mean, we've done a pretty good job of making sure that the context object just contains oddly like timeouts and then like some really like at the end of the contextual objects for like okay well like this provides you information about the environment around you so that you can you know more properly you know do
Starting point is 00:14:36 something within the function i rarely put things into the context object it's mostly just i use it for timeouts and cancellation which is the i think main reason why you're supposed to use that object but i know in like other other languages like i remember when i was still writing php back in the day there would be a lot of like uh i don't remember what it's called but you know you kind of inject everything down so you'd have this nice big bag of stuff you just pull whatever objects you need off of that and then yeah use them and uh just remember like how how much of a mess that could be and how confusing it could be to like trace where all of that came from well so in phoenix land in elixir we have the connection
Starting point is 00:15:18 that's like your web request your singular request that's flowing through all code. And we use words like pipeline. So like basically, and it's immutable. So that helps out. You know, you can't just change the connection itself, but you can return a new connection, of course, with different properties. And doggone it, sometimes, here's an it depends. Sometimes when I'm just wanting to pass something else to myself later down the pipeline, you know, like that whole deal, like, yeah, I'll just throw it on the connection and, you know,
Starting point is 00:15:50 I'll check if it's there later. And if it is, then I'll just do something with it. And it's like, I know that there's a better way and yet so darn convenient and, uh, like used sparingly, it's very productive to just, you know, add a little bit of context in your land. Yeah, I think that it's definitely a it depends sort of thing there too where I think the goal is to always try and make your parameters and your function signatures really clean and pristine and nice.
Starting point is 00:16:20 But sometimes you just can't get there yet. And sometimes you do just need to make it messy and then you'll figure out how to clean it up later after you've kind of gone through the process and better understand the structure of things or sometimes you know stuff is just messy like everything doesn't have to be clean and pristine you know i gotta get a balance the amount of time you spend thinking and designing something versus actually implementing and getting stuff done yeah sometimes when i find myself very stuck especially with like a name or how do i frame this situation that's when i just realize that i'm lacking some sort of information that i'll probably have later which is like just more knowledge about what i'm trying to do and how it's going to work and so i will slap the worst
Starting point is 00:17:01 situation on and like a bad name right that's when i'll use code comments right and i'll be like this name is terrible i couldn't think of a better one you know please do please think of something better and then i'll come back to be like oh that name is terrible i now it makes sense that it's this name or it's this parameter and so yeah sometimes you do just have to move forward but i guess then it becomes you know are you actually going to be diligent when you return to that? Are you also in a hurry at that point? And just like, I can't rename this now.
Starting point is 00:17:29 I'm too busy. And then it just stays a mess. You know, at some point you do. When you have the information, go back and change that. So that's thinking, obviously, in the very small. What about the library level API design? I mean, some things that I really appreciate in library APIs is guessability. I don't know what the word is.
Starting point is 00:17:49 Where it's named and it's used in the way that I would expect it to be. I think that's like consistency. Yeah, consistency plays into that. Principle of least astonishment plays into that. I think uniform access principle plays into that. I think uniform access principle plays into that sometimes. It's really just like
Starting point is 00:18:09 if it works the way that I thought it should, then I feel like this is a good library. Maybe that's just Jared bias. You probably use some software where you're like, you can darn near just know the way it's going to work. I feel like jQuery was like this very much. Where you could be like, I bet there's a function called
Starting point is 00:18:25 dot upcase, and then there is one. That's not jQuery, it doesn't have that, but for instance, I bet I can just upcase with dot upcase, and it's like, oh, cool, I totally can. And I feel like there's something good about that, or is it maybe, is that just very subjective? I feel like this is, one of the things I really liked
Starting point is 00:18:41 about Drupal when I was writing that in my early career, is that there was this hook system, which for a lot of reasons, it wasn't a great system. But it made it very predictable to know, oh, if I want to inject this piece of functionality, I know the words I should write and the order I should write them to actually affect that piece of functionality in the way that I want. It's like, oh, I want to modify this thing before it gets processed by this module. Oh, I know I can write this function with this particular name, and it'll get processed before this. And that predictability, that ease, that consistency made it kind of really elegant to work with that system. There are all sorts of problems with that Puck system, right? If you made a typo, it just wouldn't run your code. You have no idea why.
Starting point is 00:19:26 And you're like, well, what's wrong? It's like, oh, there's a typo in here. So there were many reasons why that system was not great, but I did like the elegance of that. And I feel like that's also one of the things that pulled me into Go when I was making that transition.
Starting point is 00:19:39 I was making a decision basically between Go and Node.js and I went with Go. And part of it was because when i was writing code there was just this ease of predictability of like what should this thing be called and then i want to call and you just type it out and it'd be oh it's that thing it's like oh no yeah i need a buffer of bytes you're like oh bytes dot buffer it's like oh that's a thing or like oh i want a reader on this like slice of bytes oh bytes dot reader because there's
Starting point is 00:20:03 all these like nice little things that made us like oh okay this just makes sense it just flows together well yeah and when i've subsequently tried to design apis i try to do that as well as just making everything consistent and predictable as much as i can and that's one of the reasons why i really did not enjoy php as a language when i used it was because and this is ancient history also i know that php is way better now but the language design was so inconsistent that like the functions the way that you work with arrays versus strings versus these things like i was constantly referencing the docs because i couldn't possibly remember the way in which you would do the same thing in this particular area of PHP that you would in the other particular area. Whereas Ruby, which is what I went from, I went from Perl,
Starting point is 00:20:51 so generally speaking, I went from Perl to PHP to Ruby to Elixir as primary languages in my life. And I've done a lot of other ones as well on the side. But when I found Ruby, the consistency of like the string objects you know functions and then like the way their arrays work uh and the way that hashes work and the way that is like so consistent throughout that i just felt like that was so much more enjoyable to use because i wasn't stopping and saying wait is it stir underscore lend this time or is there no underscore for instance yeah and so yeah that
Starting point is 00:21:25 consistency and which turns into i guess guessability is important i think it was there's an amazing rant called php a fractal of bad design i'm pretty sure i've read that written by i can't remember who wrote it but just a complete takedown this is again it's probably 15 years ago now of php just as the language design it has a lot of good properties as an overall language and ecosystem, but we're talking about specifically API. It's so inconsistent that I found it frustrating. At first, I didn't really know because Perl is kind of like that as well. Perl has short and long versions of everything because of the whole golfing angle to their community which i think is really cool but cranks out some of the most unreadable code in history because of all of the special characters and short versions of everything but i didn't realize there was like better ground out there
Starting point is 00:22:15 when i first started using php inside of wordpress until i came across ruby and i was like oh this is what a good language would look like. Just as API design, specifically. Ruby has its own warts as well, of course. Right, yeah. And I find Go to be very easy to read and straightforward and small in the same way where it is, I think, consistent for the most part in my limited use of Go, which I've written a few Go programs, but not many. Yeah, I mean, that's the thing of what I really like is
Starting point is 00:22:45 the very strong adherence to the idea of simplicity, and through that simplicity having clarity. That's always what I've looked for in the programming languages that I've used. I think part of the reason why I just did gravitate toward Go over, I was
Starting point is 00:23:01 writing Ruby for a little bit of time, I was writing JavaScript, and spent a lot of time writing PHP and like all those languages just, they didn't have that simplicity that I was really craving, that ability to just kind of sit down and just kind of write some code and have it be kind of like what I saw in my mind would be like what is actually in the document at the end of the day. But I realized that's also kind of like a me, like a weird brain thing for me, because I think a lot of people, like for them, like Python or Ruby or PHP,
Starting point is 00:23:31 that that's their language that they can do that in. But for me, it was definitely Go for a long time. I'm trying to like explore other things now. I'm getting like back into JavaScript and finding there's some things that I like, maybe controversially around like, not having static typing is freeing in some ways, script and finding there's some some things that i like maybe controversially around like oh like not having static typing is freeing in some ways even if it is a little annoying at times like you
Starting point is 00:23:52 know being able to kind of opt into a little bit of static typing when you want and having be more on the dynamic side right so are you working a solo though yes so right now it's just me solo so i realize that that's also just a different type of experience then. I eat, sleep and breathe dynamic languages. Always have. And I've used, you know, Go bugged me in certain ways. I think the type inference was way nicer than things that came before it. And I still get just annoyed at types. And so I'm very much not on the type side, but, and i even told you this in chat because we're talking about maybe talking about typing as it depends and i think that's a
Starting point is 00:24:30 good topic for it depends i don't like them but i i'm not a person who argues against it because i literally work by myself most of the time right on small projects most of the time, on small teams all the time. The largest team I've worked on is five people, and the code base was long-lasting. That changes things as well, but that's probably the biggest team I've been on. I don't feel like the person who should argue these things, because I live in this weird world that most developers don't. That's why I asked, are you solo?
Starting point is 00:25:04 I agree with you 100%. But I wonder if you would have that same feeling, working with a whole bunch of people, moving fast and breaking things. I think so, because I'm someone that's going to push very hard in every code base I'm in to make sure it's like well documented and well thought out. And what I've seen a lot over the course of my career is that people will not do good design of their code and use static typing to paper over that so like the static typing that they have allows them to be a little bit worse so they are a little bit worse with the actual documenting of their code and the actual thinking of how their code works whereas if you're in a
Starting point is 00:25:44 dynamically typed language and you try and do that same stuff, you just won't be able to function, right? Where there's some code bases I've walked into and it's like, I can barely understand this because there's types, but that's because you also haven't documented anything and everything is named terribly, mostly because you can. Like it's a crutch. Yeah, yeah. And the code bases I've walked into,
Starting point is 00:26:06 one of the things I try to do is like, okay, I'm going to try and document this. I'm going to sit down and figure this out and then write things about it so that we can all understand the page that we're on. Also, the benefits that people get from static typing isn't really the static types. It's the compiler being able to tell them,
Starting point is 00:26:21 hey, you've probably done something wrong here, which doesn't actually require declarative types. It just requires like, you know, I think there's a language called rock that you don't declare any of the types, it does type inference for everything. And like, that's the type of thing I think people actually want at the end of the day, which you can get from, you know, dynamically typed languages, because any of them today, like dynamically typed languages do have typing, they're not assembly, which has no typing. The typing just tends to happen at runtime, but that doesn't necessarily mean you can't do it statically at static analysis and compile time. Well, here we are talking about types.
Starting point is 00:26:57 Let's reroute. It's a different episode. It's an entirely different conversation and one that I'm sure our good friend Nick Nisi will want to be a part of as the TypeScript fanboy of our changelog community. What's up, friends? This episode is brought to you by our friends at Neon. Serverless Postgres is exciting, and we're excited. And I'm here with Nikita Shamganov, co-founder and CEO of Neon. So, Nikita, one thing I'm a firm believer in is when you make a product, give them what they want. And one thing I know is developers want Postgres, they want it managed, and they want
Starting point is 00:27:51 it serverless. So you're on the front lines. Tell me what you're hearing from developers. What are you hearing from developers about Postgres managed and being serverless? So what we hear from developers is the first part resonates. Absolutely. They want Postgres, they want it managed. The serverless bit is 100% resonating with what people want. They sometimes are skeptical, like, is my workload going to run well on your serverless offering? Are you going to charge me 10 times as much for serverless that I'm getting for provision? Those are like the skepticism that we're seeing, and then people are trying and they're seeing that the bill arriving at the end of the month and like, well, this is strictly better. The other thing that is resonating incredibly well is participating in the
Starting point is 00:28:35 software development lifecycle. What that means is you use databases in two modes. One mode is you're running your app and the other mode is you're building your app. And then you go and switch between the two all the time, because you're deploying all the time. And there is a specific part when you're just building out your application from zero to one, and then you push the application into production, And then they keep iterating on the application. What databases on Amazon, such as RDS and Aurora and other hyperscalers are pretty good at is running the app. They've been at it for a while. They learned how to be reliable over time. And they run massive fleets right now, like Aurora and RDS run massive fleets of databases.
Starting point is 00:29:25 So they're pretty good at it. Now, they're not serverless, at least they're not serverless by default. Aurora has a serverless offering. It doesn't scale to zero, Neon does, but that's really the difference. But they have no say in the software development lifecycle. So when you think about what a modern deploy to production looks like, it's typically some sort of tie-in into GitHub. You're creating a branch, and then you're developing your feature, and then you're sending a PR. Then that goes through a pipeline, and then you run GitHub Actions, or you're running GitLab for CICD. Eventually, this whole thing drops into a deploy into production. So databases are terrible at this today. And Nian is charging full speed into participating in the software development
Starting point is 00:30:13 lifecycle world. What that looks like is Nian supports branches. So that's the enabling feature. Git supports branches, Nian supports branches. Internally, because we built Neon, we built our own proprietary. And what I mean by proprietary is built in-house. The technology is actually open source, but it's all on GitHub under Neon Database repo. And it's quite popular. There are like over 10,000 stars on it and stuff like that. This is the enabling technology. It supports branches. The moment it supports branches, it's trivial to take your production environment and clone it. And now you have a developer environment. And because it's serverless, you're not cloning something that costs you a lot of money. And imagining for a second that every developer cloned something that costs you a lot of money in a large team, that is unthinkable, right? Because you will have 100 copies of a very expensive production database. But because it is copy and write and compute is scalable. So now
Starting point is 00:31:21 100 copies that you're not using, you're only using them for development, they actually don't cost you that much. And so now you can arrive into the world where your database participates in the software development lifecycle. And every developer can have a copy of your production environment for their testing for their feature development, we're getting a lot of feature requests, by the way, there, people want to merge this data, or at least schema back into production. People want to mask PII data. People want to reset branches to a particular point in time of the parent branch or the production branch or the current point in time, like against the head of that branch.
Starting point is 00:31:58 And we're super excited about this. We're super excited. We're super optimistic. All our top customers use branches every day. I think it's what makes Neon modern. It turns a database into a URL and it turns that URL to a similar URL to that of GitHub. You can send this URL to a friend, you can branch it, you can create a preview environment, you can have dev test staging, and you're living this iterative mode of building applications. Okay. Go to neon.tech to learn more and get started. Get on-demand scalability, bottomless storage, and data branching. One more time, that's neon.tech.
Starting point is 00:32:43 Back to APIs. Let's get off languages because we could probably live here the whole day. It's definitely, to me, a fascinating conversation. But I think most of the time when we talk about APIs, we are thinking about that proper noun, right? Like a third-party API or a web service API or a microservice API. And so we are designing these things. We are using these things. And there's a lot of questions you have to ask yourself when you're creating one. One such question is like, well, what kind of API do I want to write? That's like, I think, where you'd start, isn't it? I don't know, where do you start, Chris? Yeah, I mean, I guess when I'm trying to write these things, I'm thinking, what is the thing that I'm trying to accomplish?
Starting point is 00:33:24 What is the shape of it? Is this a request-response sort of thing that I'm trying to make an API around? Or is this a one-way feed of information? So is it an event stream that I'm trying to get access to? And even within the request-response, is it an immediate processing? Is it a long-pull processing? Is it a send a processing? Is it a long pole processing? Is it like a send a thing? It sends me an accepted back and then I wait a while and then I go and fetch the result later. So that's usually where I start thinking of like, what's the shape of this?
Starting point is 00:33:57 And then from there, I'll go in and be like, okay, well, what technologies can I use that will be advantageous for this shape or if i'm like working with others it's like what are we already using usually is that that's where you start it's like what are the things that are already in usage will those things fit the need that i currently have and if they will then i'll use them if they sort of do okay we'll probably try and use it and if they don't, okay, well now we have to have the discussion of what's the thing that will actually fit better. So you don't just use GraphQL every time?
Starting point is 00:34:31 I never use GraphQL. I never use GraphQL. Okay, well we might plumb those depths. I also never use GraphQL, but not necessarily for any reason other than I've never had a reason to use Graph QL. I thought about it once with changelog.com because there's a tool called Post Graph QL. And this is when Graph QL first came out.
Starting point is 00:34:57 And of course, we did a show on it. And after any show about a new technology, I'm nerded out, right? And I'm like, oh, this is cool. I should probably go try this. And like 99 out of 100 times, I don't actually go try it, but I have good intentions to do so. And with GraphQL, I thought, well, you know, we have a website that renders HTML. It does not have a proper API for people to consume. Maybe we could put out an API of change.com, which has people and topics and episodes and all these like typical things
Starting point is 00:35:25 that might be interesting news items, etc. And maybe I'll do a GraphQL API for that. That'd be like a cool thing that people might want to play with. Because one of the things I think is cool about GraphQL as a front end or who's like building a thing in the browser is like you can really just play with it and find stuff and do stuff. And it's very enabling, I think, for front-end devs, which I think is a powerful property. But then I found this tool, PostGraphQL. I think it was called then. It may have been renamed PostGraphFile.
Starting point is 00:35:51 And it's going to take your Postgres database and turn it into a GraphQL API, which is kind of a cool idea, but also kind of like, are you just basically exposing your database to the world? You know, like sort of like that. And I used it one time, and this and this will again probably a decade ago so i'm not going to speak on the current product or anything like that i think it was a cool piece of software and i it sure did it just was like
Starting point is 00:36:16 yep i didn't ship this but i played with it and i'm like yep basically you could just turn your entire but i don't want to just turn my database into an API. That's not designed. No, it's not. It's just like here, SQL over the wire, roughly speaking. And I'm sure there were different things that you could do. The reason why it really fell apart for me was I wanted to have some sort of other place where I would define rules and things.
Starting point is 00:36:42 And the way that Postgres file worked back then was you literally had to define everything in your Postgres database, including access controls define rules and like things. And the way that PostGraph I'll worked back then was like, you literally had to define everything in your Postgres database, including like access controls and a lot of stuff that I just didn't have at that layer. Oh, yeah. Like row level constraints and blah, blah, blah. And I'm like, I don't go that deep on my database. Like I put that code generally in my app logic. Anyways, I stopped there. That's the closest I ever got. And I haven't really considered it since. But why don't you ever use GraphQL? So I think historically, it's been mostly because I am,
Starting point is 00:37:17 I've been in the like rest camp for a very long time. Like, I mean, this is a while ago. It was almost a decade ago. But I like, I basically accidentally bought, I think it was called like R Restful Web Services or something. It was a book when I was starting to get into HTTP APIs. And it happened to be a book about hypermedia that I didn't even realize. And I was just like, oh, this is pretty cool.
Starting point is 00:37:35 I really like this stuff. And then I sat down and I read Roy Fielding's paper, his dissertation on REST. And I was like, oh, this makes a lot of sense. And then got really into HTTP and learning a lot about how HTTP works and the mechanics of it and all of the different things you can do with it. And then I think for just a while when I looked at GraphQL, I was just like, I would be more inclined to use this if you didn't put it over HTTP. I'm like, this is just an improper usage of HTTP in a lot of ways. Or just like, I don't know if it's improper, but an unfortunate usage of HTTP in a lot of ways, or just like a, I don't know if it's improper, but an unfortunate usage of HTTP. And when I started looking at the arguments for why we should use GraphQL instead of, say, a hypermedia API,
Starting point is 00:38:19 I just found those arguments to be incorrect, or just like people didn't quite understand what hypermedia APIs were. And so they were making arguments that didn't actually align with what the reality was. And once again, like if they had just said, I just want to do it this way because this is how it makes sense to me.
Starting point is 00:38:37 But I'm like, okay, cool, fine. Maybe I'll try this thing out. But because they had framed it as this like, oh, hypermedia APIs can't do this thing. And this is why you need GraphQL. I was just like, well, no, you're wrong. And now because I'm in this camp, you're silly. So I'm not going to use your thing, which is like, you know, the tribalism thing that tends to happen. Yeah. Recently, like a few months ago, tried to sit down and, you know, read a book about GraphQL. So I was like, okay, I want to give this thing a real shot.
Starting point is 00:39:08 And I just put the book down after, like, the first chapter. Because I was reading it. I was trying to give this history of HTTP and REST APIs. It was just, like, completely wrong. And I was like, well, I guess if this is, like, you know. And this is, like, a book from O'Reilly. So I'm like, if this is the way that this community is still behaving, like, I can't do this right now. Right. So that's that's like the main
Starting point is 00:39:25 reason why i just like kind of gravitated away from graphql because it just like it didn't make sense to me as something that should exist in the way it does given what you can do with hdp i feel similarly about grpc and i'm kind like, I don't really understand why this exists, given what you can already do with HTTP. Well, GraphQL is over HTTP, right? But is gRPC, is that just a TCP connection? No, that's over HTTP 2. Oh, it is? Yeah. As well or version 2? No, it's just, I think gRPC is exclusively, okay, well, gRPC is interesting because technically you can put it over pretty much any transport but the default transport is hdb2 gotcha i think graphql was an example of technology that was built to serve the needs of facebook yeah that most people don't have, but also demoed really well
Starting point is 00:40:26 and appealed to front-end developers, of which there are many. And it does provide them some freedom and flexibility and empowerment that other technologies do not. And so that combination, I think, was what led to its rise with those two things. Yeah, I definitely agree with that. I think from the perspective of a front-end person, I think GraphQL makes a lot of sense.
Starting point is 00:40:51 Or the perspective. I think gRPC makes a lot of sense from the perspective of somebody that's just like, I just need to get this thing working. I understand why they exist. And I kind of support them in that way, too. I'm like, hey, if this gets you to build the thing you're trying to build great but I from like the definitely from the back-end person perspective or from like the design perspective I'm like if you're actually sitting down to design this thing and you really want to think through what you're doing they don't feel like the tools to do that
Starting point is 00:41:18 and I've tried with gRPC and I just you know especially with proto like the fact that so much of it's built on top of protocol buffers and And I'm like, this feels awkward, then you have to manage protocol buffers. And if you don't have a way of managing them, then it's just like, okay, well, now this is just a whole other mess you got to deal with. And there's some other underlying things that I, you know, when I sat down to try and do it, when I was at a company that was actively using gRPC, and like kind of my job to evaluate these things and evolve them forward. I was just like, I'm having to modify gRPC so heavily
Starting point is 00:41:49 just to kind of get the things that I want. And I just wound up falling back to like, let's just use regular HTTP here. Let's just build on top of these building blocks that we already have access to. So generally speaking, you think hypermedia API is the way to go? Yeah, I think building an API on top of HTTP using the principles of hypermedia and the
Starting point is 00:42:13 hypermedia constraint, or as some people kind of incorrectly say, the HADIS, or the hypermedia as the engine of application state. I was just Googling that to bring it back up again because I couldn't remember how you say it and what it meant. Yeah, I got part of that wrong. But yeah, it's basically like, you know. That was pretty good. Hypermedia as the engine of application state.
Starting point is 00:42:35 Yeah, you got it right. Oh, wow, I got it. You drilled it. Hey, Dias. Yeah, and you know, I think Rohrfield brought it up once where it's just like, it's just called the hypermedia constraint. Like, that's what it is. That's the proper way to refer to this. You don't need to do this other thing. Right. I think people kind of maybe they too directly interpreted what that would mean and they didn't broaden it out enough.
Starting point is 00:42:57 So I think that's a failing of the hypermedia community and world, which also kind of like I started to move away from them as well, because I was like, a little too much zealotry of like no everything must be hypermedia and i'm like yeah okay so for us plebs like myself because i just was googling the term and maybe a few of our listeners self identify with me as pleb what is hideous what does it mean as the application state or whatever you said exactly like hypermedia as the engine of application state. What does that constraint mean? And then what were they trying to broaden it to from there? So at a base level, hypermedia is really, I think it's kind of for like hypermedia controls,
Starting point is 00:43:36 which means you have this base layer of media, which is some type of document, some type of information. And then you add controls to that, which usually come in the form of links so when you think of html the hypermedia controls in html are like the anchor tag the link tag and like form tags right and these give the application the ability to move the application to do things to kind of navigate So with forms, you can send data to the server. And the idea behind the hypermedia constraint
Starting point is 00:44:08 is that the server controls from the perspective of what the client is allowed to do via hypermedia. And then the client chooses what to do via the hypermedia that was given by the server. So the server sends you links. You use those links to do via the hypermedia that was given by the server, right? So the server sends you links. You use those links to do things and move the application state forward, right? So it's like, oh, I give you a link to create an account, or I create a form to create an account.
Starting point is 00:44:36 You fill out the form. You send that back to me. Now I have created an account, and the kind of little state engine of account creation kind of moves forward based on those hypermedia controls that you were given based on that form that you were able to fill out. Right. So the API itself, the servers, they actually publish the things that you can do via hypermedia responses.
Starting point is 00:44:59 In the responses, yes. In the response is the other things that you can do. And the idea behind this i think like the panacea or like the idea was like okay then you don't need a human at all in terms of like the client code can then just ingest that information and use it to navigate to the next page for instance or to allow you know show what to do the user what they can do and then they hit the button and it knows where to go it's like like, not exactly the way RESTful APIs are, though, because, like, we go and read the docs for the RESTful API, but, like, the idea was
Starting point is 00:45:32 ideally you wouldn't have to even do that, right, as a client developer? Yeah, so, like, the idea is, and this is where, like, I mean, I've long since given up on this, but, like, the REST APIs that we're used to thinking about are really HTTP APIs, but I realize that, like, REST as since given up on this, but like the REST APIs that we're used to thinking about are really HTTP APIs. But I realized that like REST as a term has gotten lost, which is why I think a lot of us started calling them hypermedia APIs. And REST stands for representational state transfer. I'm going back to my. Yes, representational state transfer.
Starting point is 00:45:58 Pull that one out of the old dictionary there. Which means what? Representational state transfer. I mean, even that is a conundrum of a phrase, right? Yeah, I guess the basic idea of that is that, I mean, it's kind of says what it's like, what's on the tin, right? You have a, you have a representation and you want to, you know, move it to, you want to give it to somebody. So that's to somebody else. And the state that you're transferring isn't like, I think this is in the paper, but there's this idea of like, there's the resource,
Starting point is 00:46:28 and then there's the representation of that resource. And the thing that you're moving is the representation of it. So like, you know, there's you, Jared, as a human, and then there's like your account on changelog, which represents you. But when you transfer that state to the client, you're not transferring you as a human
Starting point is 00:46:45 over it you're transferring that representation of you over it so that's like the idea of the representational change answer it's like okay i'm taking the representation and i'm moving it to somewhere else i'm taking that the state of that representation and moving it somewhere else okay moving it from the server to the client not like changing the state of the thing yes and it also has like you know rest also includes the ability for like the client to the client, not like changing the state of the thing. Yes. And it also has like, you know, rest also includes the ability for like the client to then change that state, right, to move it. Like, oh, I got the representation. Now I can
Starting point is 00:47:12 make some modifications, send it back to you, and that will change the state of that representation and potentially of the resource behind it within the server. Okay, so what you were saying before I so rudely interrupted to get that clarity is that what we call REST or RESTful, meaning it has certain properties of REST,
Starting point is 00:47:31 is the way I've just determined what RESTful means, is that they're not actually REST APIs, they're HTTP APIs. Right. Because of what do they fall short of that is not a representational state transfer? They violate a lot of the actual components that Roy Fielding defined in his dissertation where he defined rest.
Starting point is 00:47:51 The biggest one is obviously hypermedia. They are not based in hypermedia because you've got to go read a thing to understand them. And there's a few other. I think that's definitely the biggest one of why they're not. Why aren't we writing hypermedia? What was the sticking point? Is it too hard? Is it technically, is it not performant at scale?
Starting point is 00:48:13 Why don't we do that? We don't do it. I think because it wasn't the most basic thing we could do. It takes a little bit of thought and a little bit of extra writing in the beginning to actually build a hypermedia system, right? Because you have to have the server sending information to the client and you've got to structure that in some way. And then the client has to have a little bit more logic in it to be able to be like, oh, okay, like I understand what these links mean and I understand how to display
Starting point is 00:48:40 them to the user and how the user can select them. But I think that the biggest thing is that in the early days, the only hypermedia format we really had was HTML. And to some degree, you could get XML to do it. And we were missing a very big component that we sort of have now, which is the ability to define what things mean outside of the representation that you send back, right? So this is a kind of big component of hypermedia is that you have the different representations. So you might have it rendered in JSON or XML or HTML or whatever. And then you have
Starting point is 00:49:18 the actual definitions of things, which, you know, you can pull from schema.org or friend of a friend or like a whole bunch of other places that's like, okay, so when you see a field in an object with like the words name, it means this thing specifically, right? It means, okay, it's a person and it's their first name, right? Or it's a person and it's their preferred name or, or whatever, like an actual structured way of being like, these are what these words mean. And we didn't really have a good way of conveying that information. And if you don't have a good way of conveying that information, what you wind up doing is you just wind up packing that into the representation that you send. Like this is what we do in JSON APIs a lot. It's like, okay, I have a name field and it's like, okay, well, how do you know what that name field means? It's like,
Starting point is 00:50:02 well, we just write it in some docs somewhere and then the client goes and looks. It's like, okay, this name field is like the full name of the person and we just kind of know this information. Whereas with, I don't know, proper hypermedia, but the hypermedia system, you would transfer that information by saying, here's a schema that I'm using to refer
Starting point is 00:50:21 of what all of these fields mean. Use this schema when you're interpreting this. And the thing that you have to then have on both sides is that schema and keep track of that schema. But that schema is disconnected from the representation that you're sending. So you can change the representation, but still send be sending the same information. And that's what gives you some of the flexibility that you get from hypermedia APIs that you do not get from REST APIs. This is why REST APIs run into the version problem that they tend to run into, because you can't change the name of something,
Starting point is 00:50:55 or even in some cases, the location of something, because it's hard coded into the client. Hey, this string means this thing, or this location is where I find this stuff. Instead of the client, hey, this string means this thing, or this location is where I find this stuff. Instead of the client saying, oh, the server sent me this, let me interpret it. Okay, this string is mapped to this internal string I know, so I can connect those things together and process this information. so i'm here with ian withrow vp of product management at century so ian you've got a developer firstfirst application monitoring platform. It shows you what's slowed down to the line of code. That's very developer-friendly. And it's making performance monitoring actionable.
Starting point is 00:51:52 What are you all doing that's new? What's novel there? Traditionally, in errors, what's the strength of Sentry is we've taken not a stream of errors and said, hey, go look at this, like all these error codes are flowing into says, we actually look at them, we try and fingerprint them and say, hey, we've actually grouped all these things. And then we give you everything you need within Sentry to go and solve that error and close that out. And that's, I think, driven tons of value for our users. And traditionally, if you look at performance, it's not that thing. It's looking at certain golden signals, setting up lots of alerts, maintaining those alerts,
Starting point is 00:52:31 grooming those alerts, and then detecting them. And then maybe you have a war room and you try and look at traces, or maybe you realize, oh, it's this engineering team that owns it. Maybe they'll look at logs, whatever they have available. Performance is very rotated on detection and then isolating to where the problem may exist. And root causing is often an exercise left to the user. Good performance products provide a lot of context and details that an experienced engineer or DevOps professional can kind of parse
Starting point is 00:53:04 and make sense of and try and get to a hypothesis of what went wrong. But it's not like that century error experience where it's like, here's a stack trace, here's all the tags. Oh, we see it's like this particular set segment of code and Ian did the commit that changed that code. And do you want to fire a dear issue and assign it to Ian? Like it's not like that crisp kind of tight workflow that we have here this is breadcrumbs right and we said hey maybe there's no reason why we could do this for performance let's try okay so you took a swing you tried describe to me how that trial works if i'm if i go to my dashboard now and i enable apm on my application what are the steps?
Starting point is 00:53:46 Largely because we kind of encourage you to go and set up transaction information when you set up Sentry. You probably, as a user, probably don't need to do much. But if you skip that step, you do need to configure to send that data in your SDK. And what happens is we start now looking at that information. And then when we see what we call a performance issue, we fingerprint that and we put that into your issues feed, which is already where you're looking for error issues. Right. It's not a separate inbox. This is the same inbox.
Starting point is 00:54:14 The same inbox. Yeah. Now we obviously give logical filters. And if you just want to look at those, we do that. And for newer users, sometimes we detect, hey, you've probably never seen this before. We do things because we know we build for mass market that bring your attention to it. and for newer users, sometimes we detect, hey, you've probably never seen this before. We do things because we know we build for mass market that bring your attention to it, but it's the same workflow you have for errors today. So you don't have to learn something new
Starting point is 00:54:34 to take advantage of these things. So you asked for the experience. So last fall, we did the experiment, the first one, which we called M plus one, and we didn't know how it was go honestly uh but uh people liked it like we we kind of know people like it when they start tweeting and saying nice things about it and so um yeah it got traction very cool so if your team is looking for a developer first apm tool to use check out century and use the code changelog when you sign
Starting point is 00:55:02 up and you're going to get the team plan for free for three months. Make sure you tell them we sent you because they love hearing from our listeners. Check them out at Sentry.io. Again, Sentry.io. That's S-E-N-T-R-Y.I-O. so a little story uh about some more ancient history from me was that way back in the day, I did a rewrite of Grooveshark's client. So do you remember Grooveshark? I do. Before your time. Okay, cool. So Grooveshark was cool back in the day. And they had a lot of steam. This was an old music streaming app, for those who aren't aware, that was post-Napster, pre-Spotify, Apple Music, very much in like the iTunes days of like buying songs for 99 cents. And they were user-generated content. And they had a very beautiful flash, like a very well-designed flash-based player, and lots of music on there that people would upload and listen to.
Starting point is 00:56:32 And they brought me in for reasons that's a whole nother story to help them rewrite their client from the Flash version, which was their production one, to a new HTML5 player. This is when HTML5 first became a term. And so it was going to be all web technologies. And that was a fun, interesting project. And a lot of stuff didn't exist back then. So this is pre any sort of JavaScript tooling beyond basics. So I remember using a framework called JavaScript MVC, which was very loosely kind of recreating model view controller architecture that I was used to from Ruby on Rails in the web browser.
Starting point is 00:57:08 And I cobbled together a bunch of stuff and wrote a bunch of custom code as well to actually put together one of the early single page apps that was used by millions and not by hundreds. But anyways, I say all that to say this. When I started working on that, I went to their lead backend dev. They flew me down to Gainesville, Florida.
Starting point is 00:57:27 They were from Gainesville. And I got to sit in their offices for a few days and code on the early prototypes. And all I really liked was RESTful APIs and stuff. And I just thought they were the bee's knees because I love the web. I love URLs and addresses. And it makes total sense to me that you're just publishing the location of the objects. All that stuff just was sensical to me. Sensical is the opposite of nonsensical, Chris.
Starting point is 00:57:54 I know you're a Word guy. Keep up, Chris. I went to him and I'm like, hey man, I need your API docs. He's like, yeah, we don't have docs. No surprise there. And I'm like, oh cool, is it like a RESTful API? And he's like, no. He's like, it's a PHP app. And this guy coded the whole thing himself.
Starting point is 00:58:16 He was very smart, but very much like the kind of guy who didn't have docs and you had to go talk to him to find out how it worked. So not a shining example, but in some senses, very interesting. And he's like, no, there's a function and you call there's one endpoint here's the endpoint he told to me he's like then you put a question mark and then you put i can't remember the parameter was you know in the query string and that's the name of the function you want to call and then it was
Starting point is 00:58:41 all json i think and that's how it works. I'm like, this is terrible. You know what? I wrote that thing in a few weeks and it was so easy to use. He did give me a list of the different functions. He had some docs. He probably copied and pasted it out of his PHP file that actually executed
Starting point is 00:59:00 all of this stuff. He's like, here's the seven functions you can call and then just call them, and you'll see what data comes back, and you'll figure it out. And I was like, okay, I'm a big boy, I can do that. And I remember thinking, this is really terrible. And then I wrote that app,
Starting point is 00:59:16 and it was actually not so bad, just to have a single endpoint, and just, I mean, it basically was a remote procedure call. It basically was. I don't know why I told that story, just as an example of sometimes simple is just okay, even at scale, I guess.
Starting point is 00:59:32 I feel like this is also the place where the whole RESTful world went a little, it just kind of got distracted by the smaller parts of REST and not focus on the bigger parts. It doesn't do the hypermedia, but there's a lot of, because I remember sitting through and watching talks and reading about how,
Starting point is 00:59:47 oh, you got to think about the hierarchy of all your resources and how they fit together and all of these nice little URL hacking things and all the wildcard stuff, which is not the part of REST that I think matters all that much, right? The whole, oh, you have user slash ID, and then if you want to view or if you want to like edit you do like user slash id slash edit or whatever and
Starting point is 01:00:10 all of all of that structuring and i think that that once i started actually reading about hyper media and i realized that like that was actually not the thing that was supposed to matter in hyper media i think that's when i started to move away from the restful world and i think that's the thing that really captured the restful world i think it's like I started to move away from the RESTful world. And I think that's the thing that really captured the RESTful world. I think it's like a very interesting, at least from what I can remember and from what I like the little research I've done, it's like a very interesting confluence of two different things that were happening at the same time, you know, in the early 2010s of like, we were getting into the world of sharing URLs. So everybody was switching to using this whole clean URL thing. Like I remember being in Drupal and like enabling clean URLs
Starting point is 01:00:48 and having these really pretty, like, oh, this looks so nice and it's like so shareable. You copy it and you just send it to someone, which is really a way to work around a constraint. Because if you wanted to share something with someone,
Starting point is 01:01:00 all you had was what was in the URL, right? You couldn't share things like headers with them or like bodies. So you had to put everything in the url right you couldn't share things like headers with them or like bodies you so you could put everything in the url and you wanted that url to look nice but you also wanted people in the browser to be able to hack those urls right so if you had a thing that's like oh search and then it had like the the year month the day and he'd be like oh i'll just remove the day and then i can see everything from the month or i can move the month and see everything from the year.
Starting point is 01:01:26 That type of power was really nice. And so I think when we started to develop these web APIs, the people that were kind of first doing it were like, oh, I want this to be super hackable. I want to be able to go into curl and be able to just figure out what it is and be able to do this without having to document things a whole lot. That I completely get. I'm like, yeah, that's a great way to bootstrap the whole thing.
Starting point is 01:01:44 But I think we just got too caught on to that and we didn't kind of evolve past that Right. representations they get back and and kind of not have to depend on the url as like the main control of the whole thing because one of the things and i believe roy has said this quite often is like the url shouldn't matter right you should be able to encrypt all of the urls to your application and have it still work fine right that's kind of one of the things about hypermedia is that the actual addresses of things are irrelevant. And a lot of the ways, like, you know, you don't know the actual point, the memory locations of the pointers in your code. You don't think about those things. You're like, oh, I have this name that I use.
Starting point is 01:02:33 And it's like, oh, well, that name points to some memory, but I don't care what the memory is. I just have this handle that I'm going to use. And the way that REST APIs tend to work is that they are very much like, oh, I know this memory location. And if you change that memory location, my whole app is going to break, which makes everything very brittle.
Starting point is 01:02:49 And we were never really able to push ourselves out of this. Like Go in one of the recent releases actually added support in the standard library for wildcards in URLs, which once again, harkens back to that clean URL thing.
Starting point is 01:03:03 It wasn't really necessary because we have query parameters and you could always put the things that needed to be variable into the query parameters. But people are like, oh, that's ugly. It's like, the machine's looking at it. The machine doesn't care. Well, not always.
Starting point is 01:03:14 So I mean, what about humans? Like, I love passing URLs around, even to this day. Yeah, but are you passing API URLs around? Well, I would if they would allow it. I mean, sometimes I'll take an API URL and paste it into my browser and see if it renders. Yeah, not as much, obviously. So maybe when we say web APIs,
Starting point is 01:03:37 maybe we're just fashioning them too much in the image of the web. Because I think URLs and anchor tags and my ability even just to send you a google search because everything's in even though that's stinking ugly and i don't i hate amazon's urls right like but they work i can still pass it to you and you can render it oh yeah yeah is a beauty of the web and maybe we're just like we just took that and we're like it's going to make beautiful apis and maybe not yeah like again, it was like a browser constraint
Starting point is 01:04:06 that doesn't exist in API clients. And I don't think we ever shifted away from that to really rethink. I think we just didn't have a sit down and a big rethink of like, oh, this thing is different from our browser. This thing has different constraints than our browser. Maybe we should try and broaden
Starting point is 01:04:23 and use those constraints much. I also think too, like this is once again, where it's like we didn't necessarily have the right tooling and we didn't make it easy for people because i think one of the things people often think is they like oh well hypermedia sounds really cool this ability to have this flexible application that you know the client can just keep working even when you change things on the back end and it's super flexible and it'll just send you stuff and you just render the stuff you have and you don't have to make big updates. But people are like, that's going to take so much more code than what I have now. But that's
Starting point is 01:04:52 actually not true. I think Mike Amundsen, who's one of the big hypermedia people and big API people has for years been like, no, no, no. It actually takes less code on the client to implement a hypermedia API. And it's more flexible in the future. You can just add stuff to the back end and it'll just automatically appear in the front
Starting point is 01:05:09 end without you having to do deploy or anything. Because it's discoverable by the client itself. Yeah, I think people have it in their head that it's going to require more effort, more work, when really it's just different work that you're going to be doing. Okay. Are there any hypermedia APIs in the wild that you could point to? I mean, GitHub tried with their HTTP API to make it hypermedia. I think a lot of people did make efforts to, and there's a lot of remnants of them hanging around.
Starting point is 01:05:41 Like I think in GitHub's HTTP API, they still have headers you get back that tell you like, if it's a page of search results, it's like, here's the next, here's the previous, here's the current. Like, those are hypermedia controls. I think that it's kind of,
Starting point is 01:05:54 you know, for various reasons, there's not a lot of them around anymore because I think a lot of people that did have hypermedia APIs shifted them to GraphQL APIs because that was the story that was sold as like GraphQL is like the successor to hypermedia because hypermedia doesn shifted them to GraphQL APIs because that was the story that was sold as like GraphQL is like the successor to hypermedia because hypermedia doesn't really work.
Starting point is 01:06:09 So everybody should go use GraphQL and all the front end folks were like GraphQL is amazing and it's awesome. But I think part of that as well was because of the bad way we were writing hypermedia APIs, right? One of the, maybe this was the chief complaint. And one of the big reasons why facebook built graphql was that everything was kind of split up into those resources based things so it's like oh you can get like the user but then you have to like go get their friends and go get the so it's like these big chains of http requests you are making so you have to get each object then do some processing get the next object right that problem has been solved in hypermedia for a long time of being like oh no no just give me this entire representation
Starting point is 01:06:46 or give me like you know these parts of it and you know having search parameters like that stuff is built into some types like I believe it's built into collection JSON and some of the ways that you can do querying with it but at that time I think that was a very like not known thing so the big
Starting point is 01:07:03 advantage of GraphQL was I can make one http request instead of making 10 or 20 or whatever to get this exact information i want and i'm not getting all this extra information which is really just the design of the hypermedia api you had was not a fit for what you were trying to do so we needed to do some redesign and i think people are like well i just want to do something to get something done because we have a problem right it also moves the decision making process of what data is needed closer to the entity that needs it so the developer who's actually developing that client they're almost configuring the api to give them what they need and nothing else. Right. And they're the ones who have that knowledge versus saying,
Starting point is 01:07:47 hey, backend team, can you modify this endpoint to also have this data along with it? Or can you put the location of the hypermedia URL in there so that the client can discover it? I'm not sure how it would work if we're actually doing hypermedia because obviously I've never seen it done properly. And then you have that whole process, which is just inefficiencies inside of an org, right? It's like, oh, we're waiting on backend for them to do this. And in the meantime, I'm not making any progress. And so that's why I said,
Starting point is 01:08:17 I think it was like Facebook problems being solved, plus very good demos and empowerment of front-enders to just specify what they need inside the technology versus via a ticket in Jira. I think part of this too is a lack of proper design for our APIs. I've seen a lot of APIs and I think people are just like, oh, we need these objects to represent, to be put onto the front-end. And there's not a lot of thought of like,
Starting point is 01:08:44 well, what objects are we objects we need in the future or what ways can we make this more flexible? Like I think too much of the time we don't think about flexibility when we're thinking about the APIs that we're trying to design. And we just try and tack flexibility on at the end and be like, okay, well, we'll just add it here.
Starting point is 01:09:01 And that's like, you know, one of the things that GraphQL does give you a lot of, as you've mentioned, is a huge amount of flexibility for the people on the front end but it's also very challenging for the back end to implement graphql like i've looked at it before i was like this is just this is so it's kind of like you've shifted the problem like and now onto the back end folks which i think why you know there's you know that thing you found for postgres which is just like you know what we're just going to dump our database onto the network
Starting point is 01:09:26 and y'all can just program it from the front end, which is not great, especially from a security perspective. I always dislike the idea of trying to encode access controls into SQL because challenging. But I think it was a real pain point for people and it needed solving. But I think we've also kind of swung back the other way now, where it's just kind of like,
Starting point is 01:09:47 okay, well now backend people are starting to struggle and trying to figure out like, how do I implement these things? Especially when you start having huge numbers of APIs that you're then putting behind this GraphQL gateway, but your APIs are REST APIs because that's what developers like, backend developers,
Starting point is 01:10:00 and then you have this GraphQL thing. And it's like, I went through this type of thing at one of my previous jobs. And I was like, okay, this is just kind of a nightmare for everybody now. Yeah, and I guess the ambition of GraphQL to be like you said, that front layer for all kinds of data back-ends. I know that's what the Gatsby team was working on for a long time. Where it's just like, as a front-end, you just have GraphQL.
Starting point is 01:10:23 And it could be like your own application servers accessing your own database, or it could be like this other data source coming from whatever API over there that we're going to pull in and proxy and cache the crap out of, I'm sure of, and add that to our graph. And you can have just one big GraphQL API black box
Starting point is 01:10:42 that you hit. And it's kind of like when you do a function call that hits the network versus one that just does some math. You know? It's like, those functions should be different, right? Because they are fundamentally different in performance. Like, foundationally, way different. And so like, fronting a bunch of disparate API backends
Starting point is 01:11:05 with one GraphQL thing that you're all calling, to me, I always felt like, well, that's a little too ambitious, but maybe I'm just not an ambitious guy. There's simply aspects of GraphQL that I like. Like I like the idea of having this ability, especially if you have graph-like data, I think GraphQL makes a lot of sense.
Starting point is 01:11:22 It's like, okay, well, like this is literally a query language for graph style data i'm just not really convinced that most people have graph style data right i'm not convinced that like this is like the structure that you really are aiming for or that you have enough stuff the kind of graph query language is what you want this might be a place where i'm starting to become like a bit of an outlier too, because I feel similarly about SQL these days too, where I'm just like, I don't really know if SQL is the language that we should all be using for our databases, or if this is like the way we should be designing things. But I do recognize that it's like, this is the thing that's popular. This is the thing that's, you know, you can hire for this. There's other reasons why people want to use these things.
Starting point is 01:12:10 Yeah, I think a lot of new upstart database vendors who have had, again, ambitions and maybe audacity to eschew SQL and come up with something better slash different slash their own, most of those, maybe MongoDB is the exception, most of those have fallen by the wayside or have not been able to get the pickup and eventually either say, yeah, you can now speak SQL to our database
Starting point is 01:12:31 in addition to our custom language, or they just don't make it because of the mind share and the existing dominance. Maybe this is also why so many APIs look so similar to each other, is that if you're going to build a database that operates like an SQL database, you should probably just have
Starting point is 01:12:50 just thick use SQL with it. Which is the same way I feel about APIs. If you're going to build an API that's just going to return some JSON, use JSON API. Or if you just want to use protocol buffers, use gRPC. Or if you have graph-like data, just use GraphQL. I think those things are okay
Starting point is 01:13:05 to use in the small. I think where the problem comes in, and probably where the whole it depends comes in, is how do we as an industry move to build new things if everybody is always just regressing down to this mean here, going back to the homeostasis
Starting point is 01:13:21 point of, oh, well, I just want to use JSON. I just want to use an SQL-like structure for things instead of trying to maybe do something a little different. I have an answer for that that you're going to love. The answer for that is large language models. Oh, God. No, hear me out. Hear me out. So you come up with something better, foundationally better, but different.
Starting point is 01:13:46 And you also provide some sort of pre-trained or fine-tuned model that fits right in with developer tooling. I mean, no one's writing SQL anymore, Chris. You're the only one left. And even myself, who am relatively reserved when it comes to GPT tooling, I'm not writing SQL from scratch.
Starting point is 01:14:03 I'll go there first, unless it's like select star from this or that. If it's a little bit beyond basics, there's like one or two joins in there, I just don't care anymore. I don't have time. I will ask the thing and get it back. And so if it returns something that wasn't SQL but still accomplished my goal, happened to be this new thing that was better in every way, I don't care anymore.
Starting point is 01:14:24 I don't need to know. Yeah. I guess the issue at hand is that like there's a shift in the mindset of the whole stack that I think kind of needs to happen for us to build new stuff. Like I'm actually like actively going through this because I made the decision mostly when I started looking at the prices of hosting like Postgres. I was like, I don't want to pay this right now. I guess when you're a company, it's just like, okay, this is a little bit of expense.
Starting point is 01:14:49 But I'm just like, getting a high-availability Postgres server is like $50 a month, $100 a month. That's the money I'd rather not spend. So I'm going on this adventure of actually trying to, from the base level components build up a kind of new age style database and kind of build APIs just directly on top of HTTP. And it just looks fundamentally different from how so much of the stuff that we usually build looks.
Starting point is 01:15:23 And I think, I don't know if you can just kind of swap out a part of it it feels like you have to swap out the whole way you think about things you're gonna publish any of this code or yeah yeah yeah yeah i'm gonna or at least publish my thoughts behind it right like that's kind of the idea of what i'm doing now is like the whole idea of like the learning in public that i want to do is like sit down and actually go through all of this and document my journey and put it out there and then also write about, hey, this is how you can do it yourself.
Starting point is 01:15:49 The thing that I'm trying to build now is just a little based on CRDTs. Like, hey, cool, let's see if we can build some database out of these lower level components and see what we come up with. So these words you're going to write about this whole new style of doing things, can I use them to fine-tune my data model? Because if I can, then I'm set. I'm ready to adopt. I'm ready to go to the new place.
Starting point is 01:16:16 Yeah, in a way, that's part of the goal, is to give people access to information that might have been harder to find before. I think that's the big thing. when it comes to hypermedia APIs. I went through this years ago when I was first really getting into them. And I designed a system that was specifically hypermedia API based. And my coworkers at first were like, we don't know about this. Why don't we just write it regularly?
Starting point is 01:16:39 And then I'm like, no, just try this thing out. And then they tried it out and they're like, oh, wow, no, this is actually really nice to work with like i really like this like these things that we're doing are like it's so much more flexibility it's so much less code overall uh we can evolve it so much easier so i think a big portion of the problem is that like there's just not enough content out there for people to learn about how to do this stuff. You want to use GraphQL? You want to use gRPC? There are tons of resources to learn that. But if you want to learn how to build even just an API based on HTTP, it's like, I don't know, go read the HTTP specifications.
Starting point is 01:17:16 It's like that's not a good experience for people. So yeah, hypermedia APIs need some marketers. They need some people to, like Roy Fielding, he should have wrote one. Did he write one? Hypermedia API? I mean, what he defined. Was there sample code? I mean, he defined REST, right?
Starting point is 01:17:35 So he defined what web browsers use. Did he write a REST API that we could all look at? I mean, he didn't define it. He wasn't specifically talking about APIs. He was just talking about like REST is just in general the thing. Hyper-meet APIs are applying REST principles
Starting point is 01:17:49 to APIs. REST is a general thing for life or for what? What was the context here? You know, pass me a context object. Just for applications, right?
Starting point is 01:17:58 It's like, that's how the web works. The web is REST, right? Fundamentally, the entire web works on REST, right? This is the thing I'd always bring up before of just like, you know, people probably wouldn't use the web if you had to get a book in
Starting point is 01:18:08 the mail that you'd look up and see which URL you need to go to log into Facebook and which URL you need to go to to go do this thing. And, you know, oh, which URL you need to go to do a Google search. Like people probably wouldn't like that too much, especially if then Facebook updated its login page to be somewhere else. And all of a sudden you can't log in because you got to go get the new book in the mail right that would suck well the web has a different thing so the web has Google and Google I guess was the hadios right so Google is where you go to get your index of things like the number one search on Google for many years was facebook.com people type facebook.com into Google search and then they click the first link to get to Facebook.com.
Starting point is 01:18:45 And so that kind of is the phone book, isn't it? The only reason that Google works is because the web is built on hypermedia principle, right? Because it can crawl? Because you can crawl, right? That's one of the things that you can do because the controls are in there. So everywhere that you can go, all of the actions you can take are in the response you get back. So you crawl a root level domain, and then you can be like, oh, here are all the links in the page. I'm going to follow those.
Starting point is 01:19:10 And then the links in the page, I get back from there. I follow all those. And the links in the page, I get. And it's just like a continuous thing. And that's how you can crawl the whole web. You can't crawl a REST API. That's not a thing. The responses you get back, they're dead ends.
Starting point is 01:19:21 Every single one of them is a dead end. You have to know the specifics of the APIi know the specific structure all of that so like literally the web would not work as it does today wouldn't function wouldn't exist as it does today if it weren't for the hypermedia controls that are embedded into it we just don't have that and that's why apis aren't discoverable that's why we can't crawl an api to find out what are all of the endpoints what are all the things i can do with this API? So the folks who came earlier, like after Roy Fielding, but before you and me here today,
Starting point is 01:19:50 that thought, okay, this is a good idea. The Hattie Os folks, the ones who will say like RESTful APIs aren't Hattie Os, and they want to point that out. Where are their sample apps? Where are their API? Like, shouldn't there be the book and the sample app and the example? You said GitHub tried. I'm just playing a little devil's advocate here, Chris. GitHub tried,
Starting point is 01:20:11 but did they fail? There were lots of efforts to do so, but I think there were just missing pieces to translate what we had with web browsers into APIs, right? As I mentioned, right? Like in order to make a hypermedia client work, you need to have some way of defining the schema of the things that are going to come back. So you have to have a way of defining what things are because like on the web, humans do that, right? You see words and you're like,
Starting point is 01:20:40 oh, that word means a thing. Oh, sign in, log in, whatever. These are all things that mean like, oh, I can put some information in and get access to my account. We all know what that means, but computers don't understand anything. So we have to tell them, hey, these are-
Starting point is 01:20:53 Have you heard of LLMs, Chris? Have you heard of LLMs? Listen, Jared. I'm just trolling at this point. Keep going. So like with a client, we have to give it some reference point. So it says like, oh, okay.
Starting point is 01:21:05 So even if the text says sign in or log in or whatever, that really means authenticate. So we have to give it something that like basically an opaque string that means like, hey, this is the authenticate thing. Go look for the thing that's referenced in this response to see what the authenticate thing is and then present that to the user on screen. And I still think today we don't have really good tools for actually defining that information to give to people that are building clients so they can build their clients against that. Because that's still the information that you need to have at the end of the day.
Starting point is 01:21:36 You need to have the schema of something to understand what it is you can do. The thing that really hypermedia allows you to do is change the representations that you're sending back or move the representations to other places so that you can evolve one side without having to evolve the other. And I think that's the thing that I think got lost in that entire stretch that I think people got enamored with so many of the other things that they forgot that it's about like, we have to be able able to define things we have to give people something to define with and then they can do all this nice stuff with different representations and being able to evolve things and move things and all of that right like oh i want a new version of my api well if you know
Starting point is 01:22:18 when the client makes the request it sends a hey this is the schema that i can speak then the server can be like oh actually if you want to speak this schema go make a request, it sends a, hey, this is the schema that I can speak, then the server can be like, oh, actually, if you want to speak this schema, go make a request to this other URL instead of the one that you sent to. And it's like, okay, I'll just follow that redirect. Okay, now it's this another service. Like, okay, now I can talk this older protocol to you or whatever. Or it's like, oh, there's actually more newer information. But since you only speak this version of the schema, I'm only going to send you this other information back. Which, once again, is like a lot of it comes down to flexibility. Like it gives you the ability to change things independent of the other thing that's operating. And I think the biggest reason why this didn't catch on is that in the beginning, people never think that they're going to have to change their APIs.
Starting point is 01:23:03 People think that they're not going to have to evolve it. They think, oh no, this will be fine. Or I'll just add a version in the URL and that'll be great. But they don't think about, well, how do you get all of the old clients off? There's not a lot of forethought that gets put into that, mostly because a lot of the time we're just trying to get something built, trying to get something out the door. And there's not enough of a push to be like, no, we shouldn't be doing that.
Starting point is 01:23:28 We should really find a way to make sure that we do think about evolvability and that we as an industry say, no, it's important to think about evolvability. They're just trying to get something out the door. So here comes the world's first. It depends. Hypothetical. This is a test for you, Chris. I'm going to give you a hypothetical, and you're going to have to apply your knowledge to give an answer.
Starting point is 01:23:51 So you are a senior engineer, tech lead, I don't know what you want to call yourself, at a startup that has raised Series C. Do they do Series C? Yeah. Yeah, they do. So you've had some success, at least. You have some users.
Starting point is 01:24:06 You have production things. And you are tasked with, we have a website. It's a web-based company. And they want a public web API. Okay. And they want you to lead up the charge today. And you just said you just want to get stuff out the door. And this is the situation.
Starting point is 01:24:23 You do want to get stuff out the door. You don't want to go back out the door. And this is the situation. You do want to get stuff out the door. You don't want to go back and reinvent everything from the ground up. What do you do? Where do you go? What do you build? You can ask more questions and I'll give you more fake information. Yeah. Are we as a company going to build API clients for people? Or are we just going to be like, here's the API and go use it to your heart's extent?
Starting point is 01:24:49 It will be a public API consumable by anybody who has whatever permissions we decide. So it's not just yourselves consuming it, no. Right. Okay. You already have a mobile app that also consumes a private API. Okay. That threw a wrench. Or helped. I say that helped.
Starting point is 01:25:08 I think if it's an API that anybody out there is going to consume, I'd probably just, pragmatically speaking, build something that uses open API and that kind of RESTful structure where you can generate some clients for some people and focus on documenting that very well and laying everything out there. If it's an API that's going to be specifically targeted at people building front ends, then I think it would be useful to consider something like GraphQL because that community
Starting point is 01:25:39 is so familiar with that type of thing. And if our data model lends itself to something that's going to be GraphQL friendly, like if it's a read heavy API, I think it's okay. If it's going to be a write heavy API, I don't think GraphQL makes as much sense. I think writing with GraphQL is kind of wonky. But I think the kind of broad thing is, as much as I do not like opening API on Swagger,
Starting point is 01:26:00 I think that pragmatically, you know, you're trying to get something out the door. I think that's the place that you start to get something out to really see if people will use this thing. And if they do start using it and it becomes good, I think at that point, right after that, once people are like, okay, this is good,
Starting point is 01:26:17 we're getting user, we're getting traction. That's when you start planning your API strategy and you start saying, okay, do we want to commit to open API for the long term? Do we want to explore other options or do we want to build something that can actually handle multiple different API front ends? Because I think the answer that I would push toward is building a platform that can support pretty much any of the API technologies that are out there. Because, you know, you might start with OpenAPI and you might be like,
Starting point is 01:26:47 oh, wow, people are building a lot of JavaScript frontends with this. It'd probably be a good idea to support some GraphQL as well and being able to add that in after the fact. So, you know, build a quick thing to see if it gets traction. And then after that, start building out a better strategy to make sure that you stay on board with things and make sure that as you're developing the backend as well, you're developing with the idea that it won't just be OpenAPI,
Starting point is 01:27:10 it won't just be GraphQL, it just won't be this one thing. Okay. So give us the elevator pitch or the quick explainer on OpenAPI spec and Swagger, why you would pick those to get going, and then also as a follow-up why you don't like those i mean i'd pick them to get going um because i think as far as you know what i think of an api what i usually think of is someone who's writing some sort of back-end language
Starting point is 01:27:39 whether that's like ruby node python go j whatever. And those are not things that I think lend themselves to, say, a GraphQL. I probably don't want to use GraphQL from Go or Java. And I don't really like the idea of having, say, a gRPC API kind of exposed out there, because then I'm dealing with having to ship people protocol buffer files. No, thank you. And I think between, say, JSON API and OpenAPI, I would say OpenAPI gives you the most flexibility with representations. If you do want to start supporting protobus in the future or XML or something else, OpenAPI will allow you to do that.
Starting point is 01:28:21 So that makes sense. And as much as I hate YAML, it's fine to write out an open API spec, you know, and you can also like generate it from your code and all of that if you want. The reason I don't like open API is because the, I think the main thing it gives you is pretty docs. And I don't think it encourages you to write comprehensive API documentation, right? I've tried to use many open API specs for things, and the documentation is just crap. It's just so sparse.
Starting point is 01:28:50 It doesn't really give you all of the information that you need. Even the models are just like, hey, it's just like assumed you already know everything. It's like, if I already know everything, I wouldn't need to be looking at your documentation. So I think it allows people to have the illusion that they've documented their api without actually having documented their api so that's that's why i don't like it and i mean i've
Starting point is 01:29:11 been at a couple places where you wind up with like 3 000 line long yaml files to define an api and that's just not reasonable to maintain in any way shape or form that's not reasonable so i think that that's why it's like yeah i'd use it but that's also i'm like i'd use it and then i'd try to immediately figure out like how do we make sure that this isn't the thing we're stuck with forever good stuff good stuff well i am tempted to end it here even though we didn't plumb anywhere near the depths of your outline which i told you there's just too much good stuff too much good stuff to dive into here i was writing this outline and i was like oh man there's i'm like is this gonna be enough stuff and it's like
Starting point is 01:29:56 right nah there's always room for more anything from your outline or from your thoughts i know you had abis and on the brain anything that you'd want to just shove in here at the end just to get them on the air before we're done with this topic? The thing I had at the very end where I'm thinking that HTTP is more like an ABI definition than it is an API definition, which I think is a kind of novel thought that I've definitely been gravitating toward.
Starting point is 01:30:25 Okay, why do you think that? Right, so the point of an ABI is, or the way that we define APIs within, say, an operating system context or whatever, is that they really define the calling convention. So it's like, okay, you're in assembly, you want to make a call to this function. How do you do that, right?
Starting point is 01:30:43 Do you put things on the stack? Do you put things in registers? If you're putting on the stack, what order do you put them on the stack? All of that, and then, you know, you got to do the jump into the function. So it really defines the mechanics of how that function call happens. And I think HTTP gives you that same sort of base layer of like, please define your mechanics of how you would like to actually make this invocation happen. One of the things that I think about a lot is how a lot of the time people try and complete everything they want to do in a single HTTP request, right? They like make a request and they expect that request to be held until the result is available and send the result back. And that is not the only option, right?
Starting point is 01:31:25 There's an option where you can make a request and then you get immediately back or very quickly back a 202 accepted that has a location of where you can go get the result. And then you can go pull, short pull, long pull, whatever, go grab your results when it's ready. And then, you know, the system can be like, hey, it's not ready yet.
Starting point is 01:31:42 This is an estimated time we think it'll be ready. And that's a calling convention level thing because in your library that's making the API call, it doesn't have to expose that it made two HTTP requests versus making one that was just long hold. It can just hide that from you. So from the caller perspective, both look the same. But from the implementation perspective,
Starting point is 01:32:03 it can potentially give you more information. It can surface more information to ops. It allows you, especially if you do, if you do like the two or two accepted thing, it allows your servers to do some like load shedding in a way where it's like, I don't have to hold open all of these connections. I can close a bunch of them and be like, here, go get this information from this other place that might be like another pool of servers or something that's ready to handle long polls or whatever. It just gives you so much more flexibility with how you can design your architecture. But you can only do that
Starting point is 01:32:34 if you've actually thought about HTTP as this lower level calling convention-esque thing. Because if you don't, and you kind of just build your libraries and everything as if it's just everything's gonna get done on http request then you're kind of stuck with that and there's a whole bunch of pieces of functionality within http that allow you to do this like one of them that i stumbled upon is the prefer header which is a way for clients to communicate their preferences to the server of what they'd like and one of those is called wait and it's a value that
Starting point is 01:33:06 has some seconds in it and it's that value says how long the client is kind of expecting to wait for a response so you know if you want to have the ability to do both like long pulled responses and you know accept you know to or to accept responses you can enable both in the server and then when the client sends a request, it just says, maybe it says the value will wait to zero to be like, please just give me an accept back. Or it might put it at like 300 to be like, let's long pull this.
Starting point is 01:33:33 I will wait until you're ready. And when you design your APIs like this, when you kind of have this lower level substrate, you gain so much more flexibility in what you can do as far as evolvability of your system and the architecture of what you have. As I said, maybe in the beginning, you start with one API server that's just handling everything,
Starting point is 01:33:53 but then you're like, oh, I can't handle everything, but we can do some long polling result weighting. We can build some really lightweight servers that are really well optimized to do long polling and are very efficient with their resources. And then we'll have a whole bunch of those. So then when we have too much load coming into the main API server, then we start doing that load shedding, sending back 202 accepteds to push people over to the other servers. And then we have an event system that sends the responses to that
Starting point is 01:34:19 other pool of servers that then sends a response back to the client, right? It gives you more options for how you grow your APIs over time. So you're not kind of stuck with what you did in the beginning. I like that thought. I'll have to put more thought into that thought. I think my immediate thought, if I could just keep saying the word thought, is that there's a lot more to HTTP in terms of functionality and availability of things that we're probably not collectively leveraging or using in ways that would be quite beneficial if we, I guess, maybe
Starting point is 01:34:54 even just knew the protocol better. And when I say we, I'm just kind of talking about myself in the plural. And probably there's other people that are like me. Hopefully I'm not the only one who doesn't understand HTTP. Because I'm learning stuff right here, right here live, Chris. I didn't know about those things.
Starting point is 01:35:11 Yeah. I mean, some of this is stuff that I've, like, once again, I'm like building out stuff on my own now. So it's like, okay, I'm going to take a little bit of time to actually figure out how to do this right. Like the way I found this prefer header was I was actually trying to build live reload or live refresh for the website that I'm working on.
Starting point is 01:35:30 And I wanted to have it, it's all served via Go application. And I couldn't use a lot of the traditional methods like a WebSocket connection or whatever because the server would get recompiled and restarted. And I was like, okay, well, how can I make this work? And I figured out, okay, well, I can have a long-polled connection. And at first I was like, oh, we'll just send back a special header that has the version of the document.
Starting point is 01:35:52 And when I make a new version, then it'll go do that. And then I was like, oh, wait, there's e-tags, there's entity tags and HTTP that's literally built for this. And I plugged that in and it just started working. And then I was like, okay, well, how do I do this long-poll? How do I indicate to a server that I'd like to do a long pull besides just being like this thing does a long pull or do like a queer parameter long pull and that's why i went hunting and i found the prefer header i'm like oh okay so i can have the client just say if the client specifies
Starting point is 01:36:16 a value above this much then it long pulls if it doesn't then i do a short pull return the response immediately which means i get to use the same endpoint for the initial load to go get the e-tag. Because in the browser, like when you load the page for the first time, you don't have the e-tag. So you got to go get it. So you're like, OK, I send it with a weight of zero. It gives me it back immediately. And then I go turn around and do a weight of 300. So it's just long pulls.
Starting point is 01:36:39 And then when there's a change, it's just like it just responds back immediately with the, hey, there's a change, it just responds back immediately with, hey, there's a change. I think it's like a 204, no content, being like, hey, there's new content here. Or after the long poll lapses, it just gives back a not modified. And it turns out all of that stuff just works with the browser automatically, which is just beautiful. Because then I can go in and I do that special trick with CSS where you remove the DOM node from the DOM and then put it back to refresh a style sheet. And it's looking at e-tags as well. So it just handles everything
Starting point is 01:37:13 and it's just all the lines perfectly. And I was like, oh, this is really elegant. And it took me some time sitting down and looking at HTTP to be like, where is all of this stuff? But it turned out to be all there. And I was like, I now have some special bespoke thing that i'm doing that's cool so my last question for you is what is your favorite resource on http and why is it chat gpt
Starting point is 01:37:35 i know where you're learning this stuff chris i know where you're really learning it all uh yeah not not chat gpt that is that is for sure um i know it's this is a very me answer you read the spec from reading the specs yeah it's from reading the the standards i had a feeling you might say that they're very well written they're very well written standards the thing i find a lot is there's a lot of bad information out there which i assume also kind of trickles its way into chat gpt because chat gpt just pulls from written standards. The thing I find a lot is there's a lot of bad information out there, which I assume also kind of trickles its way into chat GPT because chat GPT just pulls from, you know, the web and the web just has lots of wrong and old information on it. So I think that, you know, going and just reading the specifications is a good place to start. And I also think that
Starting point is 01:38:22 one of the things I would love it if we as an industry could do is be a little bit more experimental with things and try some things out. One of the things I don't like about GraphQL is that it uses post for everything. It makes sense why it uses post for everything because we don't have
Starting point is 01:38:40 an item point retrieval method in HTTP that includes a body. right? Neither head nor get allow you to have a body. Well, actually get the body is undefined. So technically you could do it, but a whole bunch of things might break if you do. So I get starting off using post, but when this was kind of realized the IETF, like the committee within the IETF that does HTTP, it was like, Hey, we're, we have a missing method here. Um, maybe we should make one. So it started off being called search and then I changed it to being called query., it's like, hey, we have a missing method here. Maybe we should make one. So it started off being called search, and then I changed it to being called query. And it's literally just
Starting point is 01:39:08 a get with a body. And that effort appears to have kind of stalled out. Part of the reason for that is because like, you know, there isn't a lot of support behind it pushing it. But I feel like, you know, if the GraphQL folks had been like, hey, we got to use this post thing for right now because it's what we have. But what we really want is Git with a body. So let's go to the IETF and let's get Git with a body specified and put into the standard so that we can then use it for the thing that we want so we can evolve this, you know, these standards forward. And I feel like that's what we should be doing with more of these things. And I think that would make everything overall a bit better. And that's one of my gripes with a lot of these these technologies that are just kind of not using http well because it doesn't fit their need either
Starting point is 01:39:55 because they don't know how to make it fit their need or because it literally like in the case of graphql does not have a specified way to fit their need they're building the thing but they're not going back and helping to repair that and fix that. So that's the thing that I would really like people to do. And for people to just be a little bit more exploratory and be like, I don't know, make up your own headers, make up your own methods, make up your own stuff. See if it works for you internally.
Starting point is 01:40:15 Maybe even try a little bit externally and have some fallbacks, like the good old, you know, in HTML you can't use, you know, put or delete. So it's like, oh, well, we'll use post for this. We'll accept over post. But if you want to use delete, you can use delete and it's semantically better. So like finding ways to move that because that's really how we move the whole like the
Starting point is 01:40:35 platform sword. Like this is the big thing the web platform does where they like find ways to slowly add things with different types of trials and different types of shims and all of that. And I feel like we should be doing that in this API space as well, of trying to just move everything forward so that we can just have more stuff built into the platform as a whole. Very well said. Well, to our listener, if you enjoyed this conversation, which if you didn't, you must
Starting point is 01:41:03 be a glutton for punishment because you're still listening to it. It's just weird. But you were hoping for a Pound to Fine this week because you heard me say on Chainsaw News that Pound to Fine was coming up next. Well, it is coming up. It's just not coming up next. It's coming up now. It's coming up next week. So
Starting point is 01:41:18 fingers crossed, of course. Things do change, but we plan on playing that game next week and happy to provide this It Depends conversation with you, Chris, this week. I had a lot of fun, lots to chew on from here. We'll try to link up some things in the chat or we'll just put a link to ChatGBT in the show notes so you can go ask it what we all talked about.
Starting point is 01:41:42 Connect with Chris on gotime.fm. Anywhere else that you like to be contacted? Email? I'm pretty much everywhere. Scriptable is my handle. Scriptable. So I'm still on X. I guess I'm on Blue Sky now.
Starting point is 01:41:58 I am actively, as I said, working on a website, which will hopefully be rolled out in the near future. Trying to do some interesting things. I want to build an activity pub server. So lots of fun stuff. That's not there yet. So for right now, definitely still on X. Still on Blue Sky.
Starting point is 01:42:15 And obviously, go time. There you go. All right, that's all for this week. If you like these It Dependses, let us know. Because I respond positively to positive feedback and i'll do more if you don't like just be quiet just don't say anything and i won't do them anymore but uh that's all for now we'll talk to you all next time bye friends i went back and found that old blog post,
Starting point is 01:42:46 The Tech Behind the New Grooveshark, written in December of 2010. That was a trip down memory lane. Maybe you'll remember some of these open source projects from back in the day. jQuery UI, SlitGrid, Store.js, Underscore.js, Closure Compiler. Remember that one? Good times, good times. I'll add it to the show notes, just.js, closure compiler. Remember that one? Good times, good times. I'll add it to the show notes just in case you're curious. We are scheduled to record and ship Pound to Find next week.
Starting point is 01:43:16 And that's it for ChangeLog and Friends in 2023. We always take a couple of weeks off at the end of the year to relax and refresh for the next one. Which reminds me, if you're making a year-end wrap-up of your favorite things, and if the changelog happens to be one of those things, let us know about it. We love reading about
Starting point is 01:43:31 what our community members are into. You can do that in our free community Slack by signing up at changelog.com slash community, or on X, we're at changelog, or on the Fediverse, we're changelog at changelog.social. Thanks once again to our partners, On X, we're at changelog, or on the Fediverse, we're changelog at changelog.social.
Starting point is 01:43:50 Thanks once again to our partners, Fassy.com, Fly.io, and Typesense.org. And to our beat-freaking residents, Breakmaster Cylinder. Speaking of BMC, we have a couple more Changelog Beats albums in the works. Stay tuned for those. Next week on the Changelog. News on Monday. Our second and final all things open anthology on wednesday and we play that pound to find game on friday have a great weekend share change log with your friends who might dig it and let's talk again real soon

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.