The Changelog: Software Development, Open Source - HATEOAS corpus (Friends)
Episode Date: December 8, 2023Jerod is back with another "It Depends" episode! This time he's joined by Kris Brandow from Go Time and they're talking all things API design. What makes a good API? Is GraphQL a solid choice? Why do ...we do REST wrong? And WTF does HATEOAS mean, anyway?
Transcript
Discussion (0)
Welcome to Changelog and Friends, a weekly talk show about Hadeos haters.
Thank you to our longtime partners for helping us bring you awesome developer pods each and every week.
Fassy.com, Fly.io, and Typesense.org.
Okay, let's talk.
So I am here with my friend Chris Brando.
Hey, welcome back, dude.
Hey, glad to be back.
People enjoyed our last episode together.
Yeah, I saw that.
Do I sound surprised?
You're like, wait, were they not supposed to? Or what were you expecting?
People generally like what I have to say, Jared.
Come on now.
I also, I was one of those people who enjoyed it so of course happy to have you back i think you
enjoyed a recent episode i did with justin searles at least the premise of it being this whole it
depends that resonated with listeners as well maybe you know maybe not the particular episode
sorry justin no it did but the idea that i had for this it depends podcast and how we're just
kind of doing it now uh loosely underneath the umbrella of Change Logging Friends, I think people like that whole idea of two people with experience, different experiences, hopefully, diving into the nitty gritty decision making processes that we go through, how, question each other, etc.
And you like that idea. Yeah. Yeah. I thought that like, as soon
as I heard it, I was like, Oh no, this is, this is a great idea. And this is definitely a show I
want to be a part of, or like a mini series or whatever it is that I want to be a part of.
Cause I do say it depends a lot. You do. In fact, we put together that little montage of everybody
saying it depends throughout, uh, most recent 50 episodes or so. And I think you got on there twice.
You were the one was it depends what TikTok you're on,
which I'm not even sure what that conversation was about.
I guess it depends on which TikTok you're on.
I do have a soundbite of you.
Maybe I'll put it in right here.
This episode is wild.
Where you say this episode's wild.
And I think it might have been
I remember clipping that and saying
that's a good soundboard, Chris saying
that. But I think that might have been the episode when you're
just talking about, it depends on what TikTok you're on.
Yeah, I'm trying to remember what the context
that was.
I remember saying it.
I vaguely remember the episode, but I don't remember
the context. But I also remember just like
I was walking and I was listening and then you had that little clip't remember the context. But I just remember I was walking and I was listening.
And then you had that little clip of all the sounds.
And I was like, oh, this is the school.
I was like, oh, I wonder if I'm going to be in this, right?
And then I was waiting and waiting.
And then I heard that one of like, oh, it depends on what TikTok you're on.
And I'm like, oh, really?
Is that my It Depends clip?
It's like, that's the one.
I'm like, ah.
I was so disappointed.
And then I was walking.
And I think I might have paused right before because I was
like almost to my destination.
But I like messaged you and I was like, ah, I'm like so sad that this is like the one
I had.
And then I got to the end and it was just like the me going like, it depends.
And I was like, oh, no, that's perfect.
Yes.
Which is why I use it as the closer because just the amount of remorse and angst that
you expressed there. In fact, one of those sounds was Matt Reier saying, I think it depends should
have its own little theme song. I didn't have break master put together something special yet.
I'm just kind of riffing, but I did put together a brief jingle and you may find your voice on it here it is it depends okay there you go so
i think that's a good representation of this whole concept yeah i think i definitely think so
so justin and i were talking about dependencies and certainly a big topic one that we will take
up again probably in different ways with different people over time because it's one of those and I were talking about dependencies and certainly a big topic, one that we will take up
again, probably in different ways with different people over time, because it's one of those
decisions that we all have to make on a recurring basis. And so it just requires a lot of thought
and a lot of discussion. And I think a lot of data points is useful today. You and I are going to do
it. It depends on APIs, which again, even that, as I say the term,
is like a multitude of conversations could come out of such a thing.
We're going to talk about designing APIs, building them.
Also, we'll reference consuming them, of course, and all the little things.
And we're going to start that conversation by it-depensing the jargon,
because not all APIs, because even API is such a generic
thing, an application programming interface, that can manifest itself in so many different ways.
And so maybe we start with a definition or start that by breaking down kind of generically, at
least, the major types of APIs that we come across. Yeah, so like when I was thinking about this,
I was like, okay, what kind of APIs exist out there?
I think the one that kind of shoots
to the top of everybody's mind when they hear
API are web APIs
or microservice APIs.
Things that are typically built
with HTTP
and used to communicate between
separate computing entities.
There's also language
APIs, which we all use,
standard language,
standard libraries,
and all of the dependencies
that you import
that have their own APIs
that you're using
directly in your source code.
But also, there's two other types of APIs
that I think are also quite interesting,
and I also think they maybe can teach us a bit
about how we should think about designing APIs.
I think those are like the operating system level APIs,
or like the system call library level stuff,
like the Linux system call center library.
I don't even know what to really call it.
But yeah, like that layer of like really low level,
really invoking from C or assembly or something like that.
And then kind of even lower level are these things called ABIs,
which stands for Application Binary Interface, which is like APIs,
but for your ELF or Mach0 or MachO, MachO, PE,
like the form that Windows uses, like how you can make sure that if you
have a DLL that's compiled and an application or a compiled two different versions of a compiler, that they'll be able
to work together.
So it's kind of lower level, like machine code interfacing.
Weird stuff like calling conventions in there as well, which I have a thing that we can
get to later on about maybe how we can design APIs better there.
The big two that people think about are definitely language APIs and then the web APIs.
Well, let's start with the language APIs
or the programming level APIs.
I think the web APIs is the one that most of us think of
most of the time and probably consume as working developers.
I think we probably are more consuming those
than writing those.
Surely we're writing them as well.
But what we're all writing is like the,
in the small APIs that we ourselves are consuming
in our application code or in our library code
or that we're writing for others to consume
in their application code.
Literally all developers are writing those APIs.
We don't really think about them.
We think of them as like functions or methods
or pick your particular
term, depending on the context
of the executable.
We design those APIs all day, every day, don't we?
We just don't think about it very much.
I'm sure you think
that that should be designed, right?
Not just kind of grown.
Yeah, absolutely.
I think this is a big it-depends-arity
as well of what stage are you in? If you're just prototyping, like, you know, you don't need to do that much design or that much documentation of the APA you're building. But definitely, if you're, even if you're building, you know, a whole cohesive thing, that's just like a thing that's going to run like a CLI that's not going to be used as a library. So just, you know, thinking through how you lay out your functions and your methods
and your types and all of that, how those things interact is very important for like kind of the
long-term maintainability of that API. And I think for most of us, when we're sitting down and
writing all those things, we're just like, I just need a function that does this, or like this
function is too big. So I'm going to start breaking it out into other things and kind of making more
surface area of that API. Right. In a way, I'm not sure people breaking it out into other things and kind of making more surface area of that API.
In a way, I'm not sure people would really do if they were thinking of this more from the library sense.
I think when you jump to the library design, you're like, oh, I got to really think about all of the functions I have and what they're doing.
Yeah, you think about it more because you think somebody else is going to be using this and it needs to be consumable and repeatable or has to have certain characteristics that make it good for them but when we're writing our own functions internally that maybe nobody's
ever going to see or maybe you're going to use it once a lot of times i'll abstract a function
simply to put a name on the logic you know and it's like does that that name needs to be relatively
good enough to be descriptive so that I know what that bit of
code is doing. But I'm not really thinking about it as an API because I'm just extracting a function
so that I can name some bit of code. But when you're naming functions and designing functions,
whether you're going to be using them, your team, your immediate team, or once you get to the
library level, do you have heuristics or thoughts on like what makes in the small like a
single function api good or bad better or worse yeah but i think definitely the number one indicators
is how many like how many parameters do you have really that's like kind of the first thing i always
look at is like does this thing have like two or three parameters which i'm like okay cool or does
it i think one of the code bases working on we had this function that had like 16 parameters and
we're like okay well this is this is a sign that this thing's kind of broken and also like the
number of parameters compared to the lines of code of the actual function itself like the one that
had 16 parameters was also like three lines of code
because it was calling another thing that had 15 parameters and that was a sign of like oh this is
a spot in like our local api that we need to fix because this is just not okay so i think like
definitely looking at the parameters and the inputs to the functions i'm creating is one of
the biggest signs that i tend to look at. Yeah, inputs and outputs. I mean, I guess the three things that I consider is like the name,
which if you're having a really hard time naming something, I think that's a point to slow down
and maybe think you're trying to encapsulate too much sometimes. And maybe you're actually,
it's two things or more, and that's why it's hard to name. But clarity and naming is something that
I take very seriously, even with my own code,
which there's lots of with my own code
that I don't take seriously at all.
But naming with clarity is one of them,
probably because code comments
is not something I take seriously
and so I need good names.
But naming inputs and outputs,
that's pretty much what you're going to see
from the outside of a function.
That's what you're going to care about is like,
okay, what's coming in, what's coming out.
And I agree with you that like too many parameters
is a indicator of, again, poor design
or trying to do too much
or sometimes just scope creep, right?
Because a lot of those functions
start with not very many parameters
and then you're like,
I'm just going to slap another one on at the end here.
And that's fine. And it probably was was and then the next time you come back you're like i'm just gonna slap another one on at the end here and at a certain point you get
to 16 right and then and then people have meetings about your function that you wrote you know and
that's a problem then there's the cheat code right which is where you pass in a hash or a map or a struct, right?
And you're like, see, it takes one.
It only takes one input.
It only takes one.
Yeah, that's one of the things that when Go introduced the context object,
there was a lot of push in the community to be like,
don't put random junk in here.
That's supposed to be a parameter.
Like, don't do random junk in here. That's supposed to be a parameter. Like don't do parameter hiding in this thing.
Cause that'll kind of,
that'll allow you to put too much past too much stuff into a function and not
really rethink if this function needs to be broken up or if you need to start
restructuring your code.
Cause that's one of the things I really love about go is that because things
are pushed to be so simple in a lot of ways,
it becomes much easier to see when there's a problem somewhere.
Whereas if you have that, you know, kind of bag of stuff you can pass into something,
it can be a little bit more difficult or challenging to figure out, like,
what stuff is in here?
What am I actually getting?
What am I actually using?
Exactly.
I find a lot of complicated, what's the opposite of confident?
I mean, non-confident,
let's just go with that. Non-confident code is like dealing with what bag of stuff is in this
particular, like what are the contents of this bag of stuff? And I know that my TypeScript friends
are yelling right now, that's what TypeScript is for. But I'm sure you can pass some sort of
an object in there and not define the contents of the object if you don't want to
and end up with a bag of tricks.
But are the any, any type, that's the one, right?
In Go, it's pretty persnickety about that,
but I'm sure you could just pass a struct in,
like the context object, right?
Yeah, you can pass like,
that does happen quite a bit with APIs
where you have like a struct,
especially when it's like a constructor
where you're like,
oh, there's a lot of different things
that might be in here.
You have a struct or like a special object type
that builds up options
and you pass that in.
There's definitely cleaner ways
to pass a large amount of information in.
But it's also sort of obvious
when you're kind of looking at it.
And sometimes, you know, at the end of the day day you do just need to pass a lot of parameters into
something and i think when you do like kind of bundle it into a struct into another type that's
i think that's pretty okay as long as you've like thought about it as long as you can like name that
type properly and it's not just kind of like input to function name or whatever. Right, or data. That's what I go with when I can't think of a better name.
It's like, here comes the data, you know?
I mean, context is kind of just as bad.
I mean, context is a pretty meaningless thing
once you start to think about it.
Like everything is contextual, right?
It's all context.
I mean, we've done a pretty good job
of making sure that the context object
just contains oddly like timeouts
and then
like some really like at the end of the contextual objects for like okay well like this provides you
information about the environment around you so that you can you know more properly you know do
something within the function i rarely put things into the context object it's mostly just i use it
for timeouts and cancellation which is the i think
main reason why you're supposed to use that object but i know in like other other languages like i
remember when i was still writing php back in the day there would be a lot of like uh i don't
remember what it's called but you know you kind of inject everything down so you'd have this nice
big bag of stuff you just pull whatever objects you need off of that and then yeah use them and uh
just remember like how how much of a mess that could be and how confusing it could be to like
trace where all of that came from well so in phoenix land in elixir we have the connection
that's like your web request your singular request that's flowing through all code.
And we use words like pipeline.
So like basically, and it's immutable.
So that helps out.
You know, you can't just change the connection itself, but you can return a new connection,
of course, with different properties.
And doggone it, sometimes, here's an it depends. Sometimes when I'm just wanting to pass something else to myself later down the pipeline, you know,
like that whole deal, like, yeah, I'll just throw it on the connection and, you know,
I'll check if it's there later. And if it is, then I'll just do something with it. And it's like,
I know that there's a better way and yet so darn convenient and, uh, like used sparingly,
it's very productive to just, you know, add a little bit of context in your land.
Yeah, I think that it's definitely
a it depends sort of thing there too
where I think the goal is to always try
and make your parameters and your function signatures
really clean and pristine and nice.
But sometimes you just can't get there yet.
And sometimes you do just need to make it messy and then you'll figure out how to clean it up later after you've kind of gone
through the process and better understand the structure of things or sometimes you know stuff
is just messy like everything doesn't have to be clean and pristine you know i gotta get a balance
the amount of time you spend thinking and designing something versus actually implementing
and getting stuff done yeah sometimes when i find myself very stuck especially with like a name or how do i frame this situation that's when i just realize
that i'm lacking some sort of information that i'll probably have later which is like just more
knowledge about what i'm trying to do and how it's going to work and so i will slap the worst
situation on and like a bad name right that's when i'll use code comments
right and i'll be like this name is terrible i couldn't think of a better one you know please do
please think of something better and then i'll come back to be like oh that name is terrible i
now it makes sense that it's this name or it's this parameter and so yeah sometimes you do just
have to move forward but i guess then it becomes you know are you actually going to be diligent
when you return to that?
Are you also in a hurry at that point?
And just like, I can't rename this now.
I'm too busy.
And then it just stays a mess.
You know, at some point you do.
When you have the information, go back and change that.
So that's thinking, obviously, in the very small.
What about the library level API design?
I mean, some things that I really appreciate in library APIs is guessability.
I don't know what the word is.
Where it's named and it's used in the way
that I would expect it to be.
I think that's like consistency.
Yeah, consistency plays into that.
Principle of least astonishment plays into that.
I think uniform access principle plays into that. I think uniform access principle plays
into that sometimes.
It's really just like
if it works the way that I thought
it should, then I feel like this is a good library.
Maybe that's just Jared bias.
You probably
use some software where you're like, you can
darn near just know the way it's going to work.
I feel like jQuery was like this very much.
Where you could be like, I bet there's a function called
dot upcase,
and then there is one. That's not jQuery, it doesn't
have that, but for instance, I bet I can just
upcase with dot upcase, and it's like, oh, cool,
I totally can. And I feel like there's something
good about that, or is it maybe,
is that just very subjective?
I feel like this is, one of the things I really liked
about Drupal when I was
writing that in my early career, is that there was this hook system, which for a lot of reasons, it wasn't a great system.
But it made it very predictable to know, oh, if I want to inject this piece of functionality, I know the words I should write and the order I should write them to actually affect that piece of functionality in the way that I want.
It's like, oh, I want to modify this thing before it gets processed by this module. Oh, I know I can write
this function with this particular name, and it'll get processed before this. And that predictability,
that ease, that consistency made it kind of really elegant to work with that system. There
are all sorts of problems with that Puck system, right? If you made a typo, it just wouldn't run your code.
You have no idea why.
And you're like, well, what's wrong?
It's like, oh, there's a typo in here.
So there were many reasons
why that system was not great,
but I did like the elegance of that.
And I feel like that's also
one of the things that pulled me into Go
when I was making that transition.
I was making a decision basically
between Go and Node.js
and I went with Go.
And part of it was because when i
was writing code there was just this ease of predictability of like what should this thing
be called and then i want to call and you just type it out and it'd be oh it's that thing it's
like oh no yeah i need a buffer of bytes you're like oh bytes dot buffer it's like oh that's a
thing or like oh i want a reader on this like slice of bytes oh bytes dot reader because there's
all these like nice little things that made us like oh okay this just makes sense it just flows together well
yeah and when i've subsequently tried to design apis i try to do that as well as just making
everything consistent and predictable as much as i can and that's one of the reasons why i really
did not enjoy php as a language when i used it was because and this is ancient history also i know that php is
way better now but the language design was so inconsistent that like the functions the way
that you work with arrays versus strings versus these things like i was constantly referencing
the docs because i couldn't possibly remember the way in which you would do the same thing in this particular area of PHP that you would in the other particular area.
Whereas Ruby, which is what I went from, I went from Perl,
so generally speaking, I went from Perl to PHP to Ruby to Elixir
as primary languages in my life.
And I've done a lot of other ones as well on the side.
But when I found Ruby, the consistency of like the string objects you know functions
and then like the way their arrays work uh and the way that hashes work and the way that is like so
consistent throughout that i just felt like that was so much more enjoyable to use because i wasn't
stopping and saying wait is it stir underscore lend this time or is there no underscore for
instance yeah and so yeah that
consistency and which turns into i guess guessability is important i think it was there's
an amazing rant called php a fractal of bad design i'm pretty sure i've read that written by i can't
remember who wrote it but just a complete takedown this is again it's probably 15 years ago now
of php just as the language design it has a lot of good properties as an overall language and ecosystem, but we're talking about specifically API.
It's so inconsistent that I found it frustrating.
At first, I didn't really know because Perl is kind of like that as well.
Perl has short and long versions of everything because of the whole golfing angle to their community which i think is really cool but cranks out some of the most unreadable code in history because of all of the special characters
and short versions of everything but i didn't realize there was like better ground out there
when i first started using php inside of wordpress until i came across ruby and i was like oh this is
what a good language would look like. Just as API design, specifically.
Ruby has its own warts as well, of course.
Right, yeah.
And I find Go to be very easy to read and straightforward and small
in the same way where it is, I think, consistent for the most part
in my limited use of Go, which I've written a few Go programs, but not many.
Yeah, I mean, that's the thing of what I really like is
the very strong adherence
to the idea of simplicity, and
through that simplicity having clarity.
That's always what I've looked for
in the programming languages that I've used.
I think part of the reason why
I just did gravitate toward
Go over, I was
writing Ruby for a little bit of time, I was writing
JavaScript, and spent a
lot of time writing PHP and like all those languages just, they didn't have that simplicity
that I was really craving, that ability to just kind of sit down and just kind of write some code
and have it be kind of like what I saw in my mind would be like what is actually in the document at
the end of the day. But I realized that's also kind of like a me, like a weird brain thing for me,
because I think a lot of people,
like for them, like Python or Ruby or PHP,
that that's their language that they can do that in.
But for me, it was definitely Go for a long time.
I'm trying to like explore other things now.
I'm getting like back into JavaScript
and finding there's some things that I like,
maybe controversially around like,
not having static typing is freeing in some ways, script and finding there's some some things that i like maybe controversially around like oh like
not having static typing is freeing in some ways even if it is a little annoying at times like you
know being able to kind of opt into a little bit of static typing when you want and having be more
on the dynamic side right so are you working a solo though yes so right now it's just me solo
so i realize that that's also just a
different type of experience then. I eat, sleep and breathe dynamic languages. Always have. And
I've used, you know, Go bugged me in certain ways. I think the type inference was way nicer than
things that came before it. And I still get just annoyed at types. And so I'm very much not on the
type side, but, and i even told you this
in chat because we're talking about maybe talking about typing as it depends and i think that's a
good topic for it depends i don't like them but i i'm not a person who argues against it because
i literally work by myself most of the time right on small projects most of the time, on small teams all the time.
The largest team I've worked on is five people,
and the code base was long-lasting.
That changes things as well, but that's probably the biggest team I've been on.
I don't feel like the person who should argue these things,
because I live in this weird world that most developers don't.
That's why I asked, are you solo?
I agree with you 100%.
But I wonder if you would have that same feeling, working with a whole bunch of people,
moving fast and breaking things. I think so, because I'm someone that's going to push very
hard in every code base I'm in to make sure it's like well documented and well thought out.
And what I've seen a lot over the course of my career is that people will not do good
design of their code and use static typing to paper over that so like the static typing that
they have allows them to be a little bit worse so they are a little bit worse with the actual
documenting of their code and the actual thinking of how their code works whereas if you're in a
dynamically typed language and you try and do that same stuff, you just
won't be able to function, right?
Where there's some code bases I've walked into and it's like, I can barely understand
this because there's types, but that's because you also haven't documented anything and everything
is named terribly, mostly because you can.
Like it's a crutch.
Yeah, yeah.
And the code bases I've walked into,
one of the things I try to do is like,
okay, I'm going to try and document this.
I'm going to sit down and figure this out
and then write things about it
so that we can all understand the page that we're on.
Also, the benefits that people get from static typing
isn't really the static types.
It's the compiler being able to tell them,
hey, you've probably done something wrong here,
which doesn't actually require declarative types. It just requires like, you know, I think there's a language called rock
that you don't declare any of the types, it does type inference for everything. And like, that's
the type of thing I think people actually want at the end of the day, which you can get from,
you know, dynamically typed languages, because any of them today, like dynamically typed languages
do have typing, they're not assembly, which has no typing.
The typing just tends to happen at runtime, but that doesn't necessarily mean you can't do it statically at static analysis and compile time.
Well, here we are talking about types.
Let's reroute.
It's a different episode.
It's an entirely different conversation and one that I'm sure our good friend Nick Nisi will want to be a part of as the TypeScript fanboy of our changelog community. What's up, friends?
This episode is brought to you by our friends at Neon.
Serverless Postgres is exciting, and we're excited.
And I'm here with Nikita Shamganov, co-founder and CEO of Neon.
So, Nikita, one thing I'm a firm believer in is when you make a product, give them what they
want. And one thing I know is developers want Postgres, they want it managed, and they want
it serverless. So you're on the front lines. Tell me what you're hearing from developers. What are
you hearing from developers about Postgres managed and being serverless? So what we hear from
developers is the first part resonates. Absolutely. They want Postgres, they want it managed.
The serverless bit is 100% resonating with what people want. They sometimes are skeptical, like,
is my workload going to run well on your serverless offering? Are you going to charge me 10 times as
much for serverless that I'm getting for provision? Those are like the skepticism that we're seeing,
and then people are trying and they're seeing that the bill arriving at the end of the month and like, well,
this is strictly better. The other thing that is resonating incredibly well is participating in the
software development lifecycle. What that means is you use databases in two modes. One mode is
you're running your app and the other mode is you're building your app.
And then you go and switch between the two all the time, because you're deploying all the time.
And there is a specific part when you're just building out your application from zero to one,
and then you push the application into production, And then they keep iterating on the application. What databases on Amazon, such as RDS and Aurora and other hyperscalers are pretty good at is running the app.
They've been at it for a while.
They learned how to be reliable over time.
And they run massive fleets right now, like Aurora and RDS run massive fleets of databases.
So they're pretty good at it. Now, they're not serverless, at least they're not serverless by default. Aurora has a
serverless offering. It doesn't scale to zero, Neon does, but that's really the difference. But they
have no say in the software development lifecycle. So when you think about what a modern deploy to
production looks like, it's typically some sort of tie-in into GitHub.
You're creating a branch, and then you're developing your feature, and then you're sending a PR.
Then that goes through a pipeline, and then you run GitHub Actions, or you're running GitLab for CICD.
Eventually, this whole thing drops into a deploy into production. So databases are terrible
at this today. And Nian is charging full speed into participating in the software development
lifecycle world. What that looks like is Nian supports branches. So that's the enabling feature.
Git supports branches, Nian supports branches. Internally, because we built Neon, we built our own proprietary. And what I mean by proprietary is built in-house. The technology is actually open source, but it's all on GitHub under Neon Database repo. And it's quite popular. There are like over 10,000 stars on it
and stuff like that. This is the enabling technology. It supports branches. The moment
it supports branches, it's trivial to take your production environment and clone it. And now you
have a developer environment. And because it's serverless, you're not cloning something that costs you a lot
of money. And imagining for a second that every developer cloned something that costs you a lot
of money in a large team, that is unthinkable, right? Because you will have 100 copies of a very
expensive production database. But because it is copy and write and compute is scalable. So now
100 copies that you're not using, you're only using them for development, they actually don't cost you that much. And so now you can arrive into the world where
your database participates in the software development lifecycle. And every developer
can have a copy of your production environment for their testing for their feature development,
we're getting a lot of feature requests, by the way, there, people want to merge this data,
or at least schema back into production.
People want to mask PII data.
People want to reset branches to a particular point in time of the parent branch or the
production branch or the current point in time, like against the head of that branch.
And we're super excited about this.
We're super excited.
We're super optimistic.
All our top customers use branches every day.
I think it's what makes Neon modern. It turns a database into a URL and it turns that URL to a similar URL to that of GitHub. You can send this URL to a friend, you can branch it,
you can create a preview environment, you can have dev test staging, and you're living this iterative mode of building applications.
Okay. Go to neon.tech to learn more and get started. Get on-demand scalability,
bottomless storage, and data branching. One more time, that's neon.tech.
Back to APIs. Let's get off languages because we could probably live here the whole day.
It's definitely, to me, a fascinating conversation. But I think most of the time when we talk about APIs, we are thinking about that proper noun, right? Like a third-party API or a web service API or a microservice API. And so we are designing these things. We are using these things. And there's a lot of questions you have to ask yourself when you're creating one.
One such question is like,
well, what kind of API do I want to write?
That's like, I think, where you'd start, isn't it?
I don't know, where do you start, Chris?
Yeah, I mean, I guess when I'm trying to write these things,
I'm thinking, what is the thing that I'm trying to accomplish?
What is the shape of it?
Is this a request-response sort of thing that I'm trying to make an API around? Or is this a
one-way feed of information? So is it an event stream that I'm trying
to get access to? And even within the request-response,
is it an immediate processing? Is it a long-pull processing?
Is it a send a processing? Is it a long pole processing? Is it like a send a thing?
It sends me an accepted back and then I wait a while and then I go and fetch the result later.
So that's usually where I start thinking of like, what's the shape of this?
And then from there, I'll go in and be like, okay, well, what technologies can I use that will be advantageous for this shape or if i'm like
working with others it's like what are we already using usually is that that's where you start it's
like what are the things that are already in usage will those things fit the need that i currently
have and if they will then i'll use them if they sort of do okay we'll probably try and use it
and if they don't,
okay, well now we have to have the discussion
of what's the thing that will actually fit better.
So you don't just use GraphQL every time?
I never use GraphQL.
I never use GraphQL.
Okay, well we might plumb those depths.
I also never use GraphQL,
but not necessarily for any reason
other than I've never had a reason to use Graph QL.
I thought about it once with changelog.com because there's a tool called Post Graph QL.
And this is when Graph QL first came out.
And of course, we did a show on it.
And after any show about a new technology, I'm nerded out, right?
And I'm like, oh, this is cool.
I should probably go try this.
And like 99 out of 100 times, I don't actually go try it, but I have good intentions to do so.
And with GraphQL, I thought, well, you know, we have a website that renders HTML. It does not have
a proper API for people to consume. Maybe we could put out an API of change.com, which has
people and topics and episodes and all these like typical things
that might be interesting news items, etc. And maybe I'll do a GraphQL API for that. That'd be
like a cool thing that people might want to play with. Because one of the things I think is cool
about GraphQL as a front end or who's like building a thing in the browser is like you can
really just play with it and find stuff and do stuff. And it's very enabling, I think, for front-end devs,
which I think is a powerful property.
But then I found this tool, PostGraphQL.
I think it was called then.
It may have been renamed PostGraphFile.
And it's going to take your Postgres database
and turn it into a GraphQL API,
which is kind of a cool idea,
but also kind of like,
are you just basically exposing your database to the world?
You know, like sort of like that.
And I used it one time, and this and this will again probably a decade ago so i'm not going to speak on the current product or
anything like that i think it was a cool piece of software and i it sure did it just was like
yep i didn't ship this but i played with it and i'm like yep basically you could just turn your
entire but i don't want to just turn my database into an API. That's not designed.
No, it's not.
It's just like here, SQL over the wire, roughly speaking.
And I'm sure there were different things that you could do.
The reason why it really fell apart for me was
I wanted to have some sort of other place
where I would define rules and things.
And the way that Postgres file worked back then
was you literally had to define everything in your Postgres database, including access controls define rules and like things. And the way that PostGraph I'll worked back then was like,
you literally had to define everything in your Postgres database, including like access controls and a lot of stuff that I just didn't have at that layer. Oh, yeah. Like row level constraints
and blah, blah, blah. And I'm like, I don't go that deep on my database. Like I put that code
generally in my app logic. Anyways, I stopped there. That's the closest I ever got. And I
haven't really considered it since.
But why don't you ever use GraphQL?
So I think historically, it's been mostly because I am,
I've been in the like rest camp for a very long time.
Like, I mean, this is a while ago.
It was almost a decade ago.
But I like, I basically accidentally bought,
I think it was called like R Restful Web Services or something.
It was a book when I was starting to get into HTTP APIs.
And it happened to be a book about hypermedia that I didn't even realize.
And I was just like, oh, this is pretty cool.
I really like this stuff.
And then I sat down and I read Roy Fielding's paper, his dissertation on REST.
And I was like, oh, this makes a lot of sense. And then got really into HTTP and learning a lot about how HTTP works and the mechanics of it and all of the different things you can do with it.
And then I think for just a while when I looked at GraphQL, I was just like, I would be more inclined to use this if you didn't put it over HTTP. I'm like, this is just an improper usage of HTTP in a lot of ways.
Or just like, I don't know if it's improper, but an unfortunate usage of HTTP in a lot of ways, or just like a, I don't know if it's improper,
but an unfortunate usage of HTTP.
And when I started looking at the arguments
for why we should use GraphQL instead of, say, a hypermedia API,
I just found those arguments to be incorrect,
or just like people didn't quite understand
what hypermedia APIs were.
And so they were making arguments
that didn't actually align with what the reality was.
And once again, like if they had just said,
I just want to do it this way
because this is how it makes sense to me.
But I'm like, okay, cool, fine.
Maybe I'll try this thing out.
But because they had framed it as this like,
oh, hypermedia APIs can't do this thing. And this is why you
need GraphQL. I was just like, well, no, you're wrong. And now because I'm in this camp, you're
silly. So I'm not going to use your thing, which is like, you know, the tribalism thing that tends
to happen. Yeah. Recently, like a few months ago, tried to sit down and, you know, read a book about
GraphQL. So I was like, okay, I want to give this thing a real shot.
And I just put the book down after, like, the first chapter.
Because I was reading it.
I was trying to give this history of HTTP and REST APIs.
It was just, like, completely wrong.
And I was like, well, I guess if this is, like, you know.
And this is, like, a book from O'Reilly.
So I'm like, if this is the way that this community is still behaving, like, I can't do this right now.
Right. So that's that's like the main
reason why i just like kind of gravitated away from graphql because it just like it didn't make
sense to me as something that should exist in the way it does given what you can do with hdp
i feel similarly about grpc and i'm kind like, I don't really understand why this exists, given what you can
already do with HTTP. Well, GraphQL is over HTTP, right? But is gRPC, is that just a TCP connection?
No, that's over HTTP 2. Oh, it is? Yeah. As well or version 2? No, it's just, I think gRPC is
exclusively, okay, well, gRPC is interesting because technically you can put it over pretty much any transport but the default transport is hdb2 gotcha
i think graphql was an example of technology that was built to serve the needs of facebook
yeah that most people don't have, but also demoed really well
and appealed to front-end developers,
of which there are many.
And it does provide them some freedom and flexibility
and empowerment that other technologies do not.
And so that combination, I think,
was what led to its rise with those two things.
Yeah, I definitely agree with that.
I think from the perspective of a front-end person, I think GraphQL makes a lot of sense.
Or the perspective.
I think gRPC makes a lot of sense from the perspective of somebody that's just like,
I just need to get this thing working.
I understand why they exist.
And I kind of support them in that way, too.
I'm like, hey, if this gets you to build the thing you're trying to build great but I from like the definitely from the back-end person perspective
or from like the design perspective I'm like if you're actually sitting down to design this thing
and you really want to think through what you're doing they don't feel like the tools to do that
and I've tried with gRPC and I just you know especially with proto like the fact that so much
of it's built on top of protocol buffers and And I'm like, this feels awkward, then you have to manage protocol buffers.
And if you don't have a way of managing them, then it's just like, okay, well, now this is just
a whole other mess you got to deal with. And there's some other underlying things that I,
you know, when I sat down to try and do it, when I was at a company that was actively using gRPC,
and like kind of my job to evaluate these things
and evolve them forward.
I was just like, I'm having to modify gRPC so heavily
just to kind of get the things that I want.
And I just wound up falling back to like,
let's just use regular HTTP here.
Let's just build on top of these building blocks
that we already have access to.
So generally speaking, you think hypermedia API
is the way to go?
Yeah, I think building an API on top of HTTP using the principles of hypermedia and the
hypermedia constraint, or as some people kind of incorrectly say, the HADIS, or the hypermedia
as the engine of application state.
I was just Googling that to bring it back up again
because I couldn't remember how you say it and what it meant.
Yeah, I got part of that wrong.
But yeah, it's basically like, you know.
That was pretty good.
Hypermedia as the engine of application state.
Yeah, you got it right.
Oh, wow, I got it.
You drilled it.
Hey, Dias.
Yeah, and you know, I think Rohrfield brought it up once
where it's just like, it's just called the hypermedia constraint.
Like, that's what it is. That's the proper way to refer to this. You don't need to do this other thing.
Right. I think people kind of maybe they too directly interpreted what that would mean and they didn't broaden it out enough.
So I think that's a failing of the hypermedia community and world, which also kind of like I started to move away from them as well, because I was like, a little too much zealotry of like no everything must be hypermedia and i'm like yeah okay so for
us plebs like myself because i just was googling the term and maybe a few of our listeners self
identify with me as pleb what is hideous what does it mean as the application state or whatever you
said exactly like hypermedia as the engine of application state.
What does that constraint mean?
And then what were they trying to broaden it to from there?
So at a base level, hypermedia is really,
I think it's kind of for like hypermedia controls,
which means you have this base layer of media,
which is some type of document, some type of information.
And then you add controls to that,
which usually come in the
form of links so when you think of html the hypermedia controls in html are like the anchor
tag the link tag and like form tags right and these give the application the ability to move
the application to do things to kind of navigate So with forms, you can send data to the server.
And the idea behind the hypermedia constraint
is that the server controls from the perspective
of what the client is allowed to do via hypermedia.
And then the client chooses what to do
via the hypermedia that was given by the server.
So the server sends you links. You use those links to do via the hypermedia that was given by the server, right? So the server sends you links.
You use those links to do things and move the application state forward, right?
So it's like, oh, I give you a link to create an account, or I create a form to create an
account.
You fill out the form.
You send that back to me.
Now I have created an account, and the kind of little state engine of account creation
kind of moves forward based on those hypermedia
controls that you were given based on that form that you were able to fill out.
Right.
So the API itself, the servers, they actually publish the things that you can do via hypermedia
responses.
In the responses, yes.
In the response is the other things that you can do.
And the idea behind this i think
like the panacea or like the idea was like okay then you don't need a human at all in terms of
like the client code can then just ingest that information and use it to navigate to the next
page for instance or to allow you know show what to do the user what they can do and then they hit
the button and it knows where to go it's like like, not exactly the way RESTful APIs are, though,
because, like, we go and read the docs for the RESTful API, but, like, the idea was
ideally you wouldn't have to even do that, right, as a client developer?
Yeah, so, like, the idea is, and this is where, like,
I mean, I've long since given up on this, but, like, the REST APIs that we're used to thinking about
are really HTTP APIs, but I realize that, like, REST as since given up on this, but like the REST APIs that we're used to thinking about are really HTTP APIs.
But I realized that like REST as a term has gotten lost, which is why I think a lot of us started calling them hypermedia APIs.
And REST stands for representational state transfer.
I'm going back to my.
Yes, representational state transfer.
Pull that one out of the old dictionary there.
Which means what?
Representational state transfer.
I mean, even that is a conundrum of a phrase, right?
Yeah, I guess the basic idea of that is that, I mean, it's kind of says what it's like, what's on the tin, right? You have a, you have a representation and you want to, you know, move it to, you want to give it to somebody.
So that's to somebody else. And the state that you're transferring isn't like, I think this is in the paper,
but there's this idea of like,
there's the resource,
and then there's the representation of that resource.
And the thing that you're moving
is the representation of it.
So like, you know, there's you, Jared, as a human,
and then there's like your account on changelog,
which represents you.
But when you transfer that state to the client,
you're not transferring you as a human
over it you're transferring that representation of you over it so that's like the idea of the
representational change answer it's like okay i'm taking the representation and i'm moving it to
somewhere else i'm taking that the state of that representation and moving it somewhere else
okay moving it from the server to the client not like changing the state of the thing
yes and it also has like you know rest also includes the ability for like the client to the client, not like changing the state of the thing. Yes. And it also has like, you know, rest also
includes the ability for like the client to then
change that state, right, to move it.
Like, oh, I got the representation. Now I can
make some modifications, send it back to you, and that
will change the state of that representation
and potentially of the
resource behind it within
the server. Okay, so what
you were saying before I so rudely
interrupted to get that clarity is that what we call REST or RESTful,
meaning it has certain properties of REST,
is the way I've just determined what RESTful means,
is that they're not actually REST APIs,
they're HTTP APIs.
Right.
Because of what do they fall short of
that is not a representational state transfer?
They violate a lot of the actual components that Roy Fielding defined in his dissertation
where he defined rest.
The biggest one is obviously hypermedia.
They are not based in hypermedia because you've got to go read a thing to understand them.
And there's a few other.
I think that's definitely the biggest one of why they're not.
Why aren't we writing hypermedia?
What was the sticking point?
Is it too hard?
Is it technically, is it not performant at scale?
Why don't we do that?
We don't do it.
I think because it wasn't the most basic thing we could do.
It takes a little bit of thought and a little bit of extra writing in the beginning to actually
build a hypermedia system,
right? Because you have to have the server sending information to the client and you've got to
structure that in some way. And then the client has to have a little bit more logic in it to be
able to be like, oh, okay, like I understand what these links mean and I understand how to display
them to the user and how the user can select them. But I think that the biggest thing is that in the early days,
the only hypermedia format we really had was HTML.
And to some degree, you could get XML to do it.
And we were missing a very big component that we sort of have now,
which is the ability to define what things mean
outside of the representation that you send back,
right? So this is a kind of big component of hypermedia is that you have the different
representations. So you might have it rendered in JSON or XML or HTML or whatever. And then you have
the actual definitions of things, which, you know, you can pull from schema.org or friend of a friend or like a whole
bunch of other places that's like, okay, so when you see a field in an object with like the words
name, it means this thing specifically, right? It means, okay, it's a person and it's their first
name, right? Or it's a person and it's their preferred name or, or whatever, like an actual
structured way of being like, these are what these words mean. And we didn't really have a good way of conveying that information. And if you don't have a good
way of conveying that information, what you wind up doing is you just wind up packing that into
the representation that you send. Like this is what we do in JSON APIs a lot. It's like, okay,
I have a name field and it's like, okay, well, how do you know what that name field means? It's like,
well, we just write it in some docs somewhere and then the client goes and looks.
It's like, okay, this name field is like
the full name of the person
and we just kind of know this information.
Whereas with, I don't know, proper hypermedia,
but the hypermedia system,
you would transfer that information by saying,
here's a schema that I'm using to refer
of what all of these fields mean.
Use this schema when you're interpreting
this. And the thing that you have to then have on both sides is that schema and keep track of that
schema. But that schema is disconnected from the representation that you're sending. So you can
change the representation, but still send be sending the same information. And that's what
gives you some of the flexibility that you get
from hypermedia APIs that you do not get from REST APIs. This is why REST APIs run into the
version problem that they tend to run into, because you can't change the name of something,
or even in some cases, the location of something, because it's hard coded into the client. Hey,
this string means this thing, or this location is where I find this stuff. Instead of the client, hey, this string means this thing, or this location is where I find this stuff.
Instead of the client saying, oh, the server sent me this, let me interpret it. Okay,
this string is mapped to this internal string I know,
so I can connect those things together and process this information. so i'm here with ian withrow vp of product management at century
so ian you've got a developer firstfirst application monitoring platform. It shows you what's slowed down to the line of code.
That's very developer-friendly.
And it's making performance monitoring actionable.
What are you all doing that's new?
What's novel there?
Traditionally, in errors, what's the strength of Sentry is we've taken not a stream of errors
and said, hey, go look at this, like all these error codes are flowing into says, we actually look at them, we try and fingerprint them and say, hey, we've actually
grouped all these things. And then we give you everything you need within Sentry to go and solve
that error and close that out. And that's, I think, driven tons of value for our users. And
traditionally, if you look at performance, it's not that thing.
It's looking at certain golden signals, setting up lots of alerts, maintaining those alerts,
grooming those alerts, and then detecting them. And then maybe you have a war room and you try
and look at traces, or maybe you realize, oh, it's this engineering team that owns it. Maybe
they'll look at logs, whatever they have available. Performance is very rotated on detection
and then isolating to where the problem may exist.
And root causing is often an exercise left to the user.
Good performance products provide a lot of context
and details that an experienced engineer
or DevOps professional can kind of parse
and make sense of and try and get to a hypothesis
of what went wrong. But it's not like that century error experience where it's like,
here's a stack trace, here's all the tags. Oh, we see it's like this particular set segment of code
and Ian did the commit that changed that code. And do you want to fire a dear issue and assign
it to Ian? Like it's not like that crisp kind of tight workflow that we have here this is breadcrumbs right and we said hey
maybe there's no reason why we could do this for performance let's try okay so you took a swing
you tried describe to me how that trial works if i'm if i go to my dashboard now and i enable
apm on my application what are the steps?
Largely because we kind of encourage you to go and set up transaction information when you set up Sentry.
You probably, as a user, probably don't need to do much.
But if you skip that step, you do need to configure to send that data in your SDK.
And what happens is we start now looking at that information.
And then when we see what we call a performance issue,
we fingerprint that and we put that into your issues feed,
which is already where you're looking for error issues.
Right. It's not a separate inbox. This is the same inbox.
The same inbox. Yeah.
Now we obviously give logical filters.
And if you just want to look at those, we do that.
And for newer users, sometimes we detect, hey, you've probably never seen this before.
We do things because we know we build for mass market that bring your attention to it. and for newer users, sometimes we detect, hey, you've probably never seen this before.
We do things because we know we build for mass market that bring your attention to it,
but it's the same workflow you have for errors today.
So you don't have to learn something new
to take advantage of these things.
So you asked for the experience.
So last fall, we did the experiment,
the first one, which we called M plus one,
and we didn't know how it was go
honestly uh but uh people liked it like we we kind of know people like it when they start tweeting
and saying nice things about it and so um yeah it got traction very cool so if your team is looking
for a developer first apm tool to use check out century and use the code changelog when you sign
up and you're going to get the team plan for free for three months. Make sure you tell them we sent you because they love hearing from
our listeners. Check them out at Sentry.io. Again, Sentry.io. That's S-E-N-T-R-Y.I-O. so a little story uh about some more ancient history from me was that way back in the day, I did a rewrite of
Grooveshark's client. So do you remember Grooveshark? I do. Before your time. Okay,
cool. So Grooveshark was cool back in the day. And they had a lot of steam. This was an old
music streaming app, for those who aren't aware, that was post-Napster, pre-Spotify, Apple Music,
very much in like the iTunes days of like buying songs for 99 cents. And they were user-generated
content. And they had a very beautiful flash, like a very well-designed flash-based player,
and lots of music on there that people would upload and listen to.
And they brought me in for reasons that's a whole nother story to help them rewrite their client from the Flash version, which was their production one, to a new HTML5 player. This is when HTML5
first became a term. And so it was going to be all web technologies. And that was a fun, interesting project.
And a lot of stuff didn't exist back then.
So this is pre any sort of JavaScript tooling beyond basics.
So I remember using a framework called JavaScript MVC,
which was very loosely kind of recreating
model view controller architecture
that I was used to from Ruby on Rails in the web browser.
And I cobbled together a bunch of stuff
and wrote a bunch of custom code as well
to actually put together one of the early single page apps
that was used by millions and not by hundreds.
But anyways, I say all that to say this.
When I started working on that,
I went to their lead backend dev.
They flew me down to Gainesville, Florida.
They were from Gainesville.
And I got to sit in their offices for a few days and code on the early prototypes.
And all I really liked was RESTful APIs and stuff.
And I just thought they were the bee's knees because I love the web.
I love URLs and addresses.
And it makes total sense to me that you're just publishing the location
of the objects. All that stuff just was sensical to me.
Sensical is the opposite of nonsensical, Chris.
I know you're a Word guy. Keep up, Chris.
I went to him and I'm like, hey man, I need your API docs.
He's like, yeah, we don't have docs.
No surprise there.
And I'm like, oh cool, is it like a RESTful API?
And he's like, no.
He's like, it's a PHP app.
And this guy coded the whole thing himself.
He was very smart, but very much like the kind of guy
who didn't have docs and you had to go talk to him
to find out how it worked.
So not a shining example, but in some senses,
very interesting.
And he's like, no, there's a function and you call there's one endpoint here's the endpoint he told
to me he's like then you put a question mark and then you put i can't remember the parameter was
you know in the query string and that's the name of the function you want to call and then it was
all json i think and that's how it works. I'm like, this is terrible.
You know what? I wrote that thing
in a few weeks and it was
so easy to use.
He did give me a list of the different
functions. He had some docs.
He probably copied and pasted it out of his
PHP file that actually executed
all of this stuff.
He's like, here's the seven functions you can call
and then just call them,
and you'll see what data comes back,
and you'll figure it out.
And I was like, okay, I'm a big boy, I can do that.
And I remember thinking, this is really terrible.
And then I wrote that app,
and it was actually not so bad,
just to have a single endpoint,
and just, I mean, it basically was a remote procedure call.
It basically was.
I don't know why I told
that story, just as an example of
sometimes simple is just
okay, even at scale, I guess.
I feel like this is also the place where
the whole RESTful world went a little,
it just kind of got distracted by
the smaller parts of REST
and not focus on the bigger parts.
It doesn't do the hypermedia, but there's a lot of,
because I remember sitting through
and watching talks and reading about how,
oh, you got to think about the hierarchy
of all your resources and how they fit together
and all of these nice little URL hacking things
and all the wildcard stuff,
which is not the part of REST
that I think matters all that much, right?
The whole, oh, you have user slash ID,
and then if you want to view or if you want to like edit you do like user slash id slash edit or whatever and
all of all of that structuring and i think that that once i started actually reading about hyper
media and i realized that like that was actually not the thing that was supposed to matter in hyper
media i think that's when i started to move away from the restful world and i think that's
the thing that really captured the restful world i think it's like I started to move away from the RESTful world. And I think that's the thing that really captured the RESTful world.
I think it's like a very interesting, at least from what I can remember and from what I like the little research I've done, it's like a very interesting confluence of two different things that were happening at the same time, you know, in the early 2010s of like, we were getting into the world of sharing URLs.
So everybody was switching to using this whole clean URL thing.
Like I remember being in Drupal
and like enabling clean URLs
and having these really pretty,
like, oh, this looks so nice
and it's like so shareable.
You copy it and you just send it to someone,
which is really a way
to work around a constraint.
Because if you wanted to share
something with someone,
all you had was what was in the URL, right?
You couldn't share things
like headers with them or like bodies. So you had to put everything in the url right you couldn't share things like headers with them
or like bodies you so you could put everything in the url and you wanted that url to look nice but
you also wanted people in the browser to be able to hack those urls right so if you had a thing
that's like oh search and then it had like the the year month the day and he'd be like oh i'll
just remove the day and then i can see everything from the month or i can move the month and see
everything from the year.
That type of power was really nice.
And so I think when we started to develop these web APIs,
the people that were kind of first doing it were like,
oh, I want this to be super hackable.
I want to be able to go into curl and be able to just figure out what it is
and be able to do this without having to document things a whole lot.
That I completely get.
I'm like, yeah, that's a great way to bootstrap the whole thing.
But I think we just got too caught on to that and we didn't kind of evolve past that Right. representations they get back and and kind of not have to depend on the url as like the main
control of the whole thing because one of the things and i believe roy has said this quite
often is like the url shouldn't matter right you should be able to encrypt all of the urls to your
application and have it still work fine right that's kind of one of the things about hypermedia
is that the actual addresses of things are irrelevant. And a lot of the ways, like, you know, you don't know the actual point, the memory locations
of the pointers in your code.
You don't think about those things.
You're like, oh, I have this name that I use.
And it's like, oh, well, that name points to some memory, but I don't care what the
memory is.
I just have this handle that I'm going to use.
And the way that REST APIs tend to work is that they are very much like, oh, I know this
memory location.
And if you change that memory location,
my whole app is going to break,
which makes everything very brittle.
And we were never really able
to push ourselves out of this.
Like Go in one of the recent releases
actually added support
in the standard library
for wildcards in URLs,
which once again,
harkens back to that clean URL thing.
It wasn't really necessary
because we have query parameters
and you could always put the things that needed to be variable
into the query parameters.
But people are like, oh, that's ugly.
It's like, the machine's looking at it.
The machine doesn't care.
Well, not always.
So I mean, what about humans?
Like, I love passing URLs around, even to this day.
Yeah, but are you passing API URLs around?
Well, I would if they would allow it.
I mean, sometimes I'll take an API URL
and paste it into my browser and see if it renders.
Yeah, not as much, obviously.
So maybe when we say web APIs,
maybe we're just fashioning them too much
in the image of the web.
Because I think URLs and anchor tags
and my ability even just to send you a google
search because everything's in even though that's stinking ugly and i don't i hate amazon's urls
right like but they work i can still pass it to you and you can render it oh yeah yeah is a beauty
of the web and maybe we're just like we just took that and we're like it's going to make beautiful
apis and maybe not yeah like again, it was like a browser constraint
that doesn't exist in API clients.
And I don't think we ever shifted away from that
to really rethink.
I think we just didn't have a sit down
and a big rethink of like,
oh, this thing is different from our browser.
This thing has different constraints than our browser.
Maybe we should try and broaden
and use those constraints much.
I also think too, like this is once again, where it's like we didn't necessarily have the right tooling and we didn't make it easy for people because i think one of the things people often
think is they like oh well hypermedia sounds really cool this ability to have this flexible
application that you know the client can just keep working even when you change things on the
back end and it's super flexible and it'll just send you stuff and you just render the stuff you have
and you don't have to make big updates.
But people are like, that's going to take so much more code
than what I have now. But that's
actually not true. I think
Mike Amundsen, who's one of the
big hypermedia people and big API people
has for years been like, no, no, no.
It actually takes less code on the client
to implement a hypermedia API.
And it's more flexible in the future.
You can just add stuff to the back end and it'll just automatically appear in the front
end without you having to do deploy or anything.
Because it's discoverable by the client itself.
Yeah, I think people have it in their head that it's going to require more effort, more
work, when really it's just different work that you're going to be doing.
Okay.
Are there any hypermedia APIs in the wild that you could point to?
I mean, GitHub tried with their HTTP API to make it hypermedia.
I think a lot of people did make efforts to, and there's a lot of remnants of them hanging around.
Like I think in GitHub's HTTP API, they still have headers you get back
that tell you like,
if it's a page of search results,
it's like, here's the next,
here's the previous,
here's the current.
Like, those are hypermedia controls.
I think that it's kind of,
you know, for various reasons,
there's not a lot of them around anymore
because I think a lot of people
that did have hypermedia APIs
shifted them to GraphQL APIs
because that was the story that was sold
as like GraphQL is like the successor to hypermedia because hypermedia doesn shifted them to GraphQL APIs because that was the story that was sold as like
GraphQL is like the successor to hypermedia because hypermedia doesn't really work.
So everybody should go use GraphQL and all the front end folks were like GraphQL is amazing and
it's awesome. But I think part of that as well was because of the bad way we were writing hypermedia
APIs, right? One of the, maybe this was the chief complaint. And one of the big reasons why facebook built graphql was that
everything was kind of split up into those resources based things so it's like oh you can
get like the user but then you have to like go get their friends and go get the so it's like these
big chains of http requests you are making so you have to get each object then do some processing
get the next object right that problem has been solved in hypermedia for a long time of being like
oh no no just give me this entire representation
or give me like you know these parts
of it and you know having search parameters
like that stuff is built into some types like
I believe it's built into collection JSON
and some of the ways that you can do
querying with it but at that time
I think that was a very like not known thing
so the big
advantage of GraphQL was
I can make one http request instead
of making 10 or 20 or whatever to get this exact information i want and i'm not getting all this
extra information which is really just the design of the hypermedia api you had was not a fit for
what you were trying to do so we needed to do some redesign and i think people are like well i just
want to do something to get something done because we have a problem right it also moves the decision making process of what data is needed
closer to the entity that needs it so the developer who's actually developing that client
they're almost configuring the api to give them what they need and nothing else. Right. And they're the ones who have that knowledge versus saying,
hey, backend team, can you modify this endpoint
to also have this data along with it?
Or can you put the location of the hypermedia URL in there
so that the client can discover it?
I'm not sure how it would work if we're actually doing hypermedia
because obviously I've never seen it done properly. And then you have that whole process,
which is just inefficiencies inside of an org, right? It's like, oh, we're waiting on backend
for them to do this. And in the meantime, I'm not making any progress. And so that's why I said,
I think it was like Facebook problems being solved, plus very good demos and empowerment
of front-enders to just specify what they need inside the technology
versus via a ticket in Jira.
I think part of this too is a lack of proper design for our APIs.
I've seen a lot of APIs and I think people are just like,
oh, we need these objects to represent,
to be put onto the front-end.
And there's not a lot of thought of like,
well, what objects are we objects we need in the future
or what ways can we make this more flexible?
Like I think too much of the time
we don't think about flexibility
when we're thinking about the APIs
that we're trying to design.
And we just try and tack flexibility on at the end
and be like, okay, well, we'll just add it here.
And that's like, you know,
one of the things that GraphQL does give you a lot of,
as you've mentioned,
is a huge amount of flexibility for the people on the front end but it's also
very challenging for the back end to implement graphql like i've looked at it before i was like
this is just this is so it's kind of like you've shifted the problem like and now onto the back
end folks which i think why you know there's you know that thing you found for postgres which is
just like you know what we're just going to dump our database onto the network
and y'all can just program it from the front end,
which is not great, especially from a security perspective.
I always dislike the idea of trying to encode access controls into SQL
because challenging.
But I think it was a real pain point for people
and it needed solving.
But I think we've also kind of swung back
the other way now, where it's just kind of like,
okay, well now backend people are starting to struggle
and trying to figure out like,
how do I implement these things?
Especially when you start having huge numbers of APIs
that you're then putting behind this GraphQL gateway,
but your APIs are REST APIs
because that's what developers like,
backend developers,
and then you have this GraphQL thing.
And it's like, I went through this type of thing
at one of my previous jobs.
And I was like, okay, this is just kind of a nightmare for everybody now.
Yeah, and I guess the ambition of GraphQL to be like you said,
that front layer for all kinds of data back-ends.
I know that's what the Gatsby team was working on for a long time.
Where it's just like, as a front-end, you just have GraphQL.
And it could be like your own application servers
accessing your own database,
or it could be like this other data source
coming from whatever API over there
that we're going to pull in and proxy
and cache the crap out of, I'm sure of,
and add that to our graph.
And you can have just one big GraphQL API black box
that you hit.
And it's kind of like when you do a function call
that hits the network versus one that just does some math.
You know?
It's like, those functions should be different, right?
Because they are fundamentally different in performance.
Like, foundationally, way different.
And so like, fronting a bunch of disparate API backends
with one GraphQL thing that you're all calling,
to me, I always felt like,
well, that's a little too ambitious,
but maybe I'm just not an ambitious guy.
There's simply aspects of GraphQL that I like.
Like I like the idea of having this ability,
especially if you have graph-like data,
I think GraphQL makes a lot of sense.
It's like, okay, well, like this is literally
a query language for graph style data i'm just not really convinced that most people have graph style
data right i'm not convinced that like this is like the structure that you really are aiming for
or that you have enough stuff the kind of graph query language is what you want this might be a
place where i'm starting to become like a bit of an outlier too, because I feel similarly about SQL these days too, where I'm just like, I don't really know
if SQL is the language that we should all be using for our databases, or if this is like the
way we should be designing things. But I do recognize that it's like, this is the thing
that's popular. This is the thing that's, you know, you can hire for this. There's other reasons why people want to use these things.
Yeah, I think a lot of new upstart database vendors who have had, again, ambitions and maybe audacity
to eschew SQL and come up with something better
slash different slash their own,
most of those, maybe MongoDB is the exception,
most of those have fallen by the wayside
or have not been able to get the pickup
and eventually either say,
yeah, you can now speak SQL to our database
in addition to our custom language,
or they just don't make it
because of the mind share and the existing dominance.
Maybe this is also why so many APIs
look so similar to each other,
is that if you're going to build a database
that operates like an SQL database,
you should probably just have
just thick use SQL with it.
Which is the same way I feel about APIs. If you're going to build
an API that's just going to return some
JSON, use JSON API.
Or if you just want to use protocol buffers,
use gRPC. Or if you have graph-like
data, just use GraphQL.
I think those things are okay
to use in the small.
I think where the problem comes in, and probably
where the whole it depends comes in, is
how do we as an industry
move to build new things
if everybody is always just
regressing down to this mean here,
going back to the homeostasis
point of, oh, well, I just want to
use JSON. I just want to use an SQL-like structure for things instead of trying to maybe do something a little different.
I have an answer for that that you're going to love.
The answer for that is large language models.
Oh, God.
No, hear me out.
Hear me out.
So you come up with something better, foundationally better, but different.
And you also provide some sort of pre-trained
or fine-tuned model that fits right in
with developer tooling.
I mean, no one's writing SQL anymore, Chris.
You're the only one left.
And even myself, who am relatively reserved
when it comes to GPT tooling,
I'm not writing SQL from scratch.
I'll go there first, unless it's like select star from this or that.
If it's a little bit beyond basics, there's like one or two joins in there,
I just don't care anymore.
I don't have time.
I will ask the thing and get it back.
And so if it returns something that wasn't SQL but still accomplished my goal,
happened to be this new thing that was better in every way,
I don't care anymore.
I don't need to know.
Yeah.
I guess the issue at hand is that like there's a shift in the mindset of the whole stack
that I think kind of needs to happen for us to build new stuff.
Like I'm actually like actively going through this because I made the decision mostly when
I started looking at the prices of hosting like Postgres.
I was like, I don't want to pay this right now.
I guess when you're a company, it's just like, okay, this is a little bit of expense.
But I'm just like, getting a high-availability Postgres server is like $50 a month, $100 a month.
That's the money I'd rather not spend.
So I'm going on this adventure of actually trying to, from the base level components build up a kind of new age style database
and kind of build APIs
just directly on top of HTTP.
And it just looks fundamentally different
from how so much of the stuff
that we usually build looks.
And I think, I don't know
if you can just kind of
swap out a part of it it feels like you have to swap out the whole way you think about things
you're gonna publish any of this code or yeah yeah yeah yeah i'm gonna or at least publish my
thoughts behind it right like that's kind of the idea of what i'm doing now is like the whole idea
of like the learning in public that i want to do is like sit down and actually go through all of
this and document my journey and put it out there and then also write about,
hey, this is how you can do it yourself.
The thing that I'm trying to build now is just a little based on CRDTs.
Like, hey, cool, let's see if we can build some database out of these lower level components
and see what we come up with.
So these words you're going to write about this whole new style of doing things,
can I use them to fine-tune my data model?
Because if I can, then I'm set.
I'm ready to adopt.
I'm ready to go to the new place.
Yeah, in a way, that's part of the goal,
is to give people access to information
that might have been harder to find before.
I think that's the big thing. when it comes to hypermedia APIs.
I went through this years ago when I was first really getting into them.
And I designed a system that was specifically hypermedia API based.
And my coworkers at first were like, we don't know about this.
Why don't we just write it regularly?
And then I'm like, no, just try this thing out.
And then they tried it out and they're like, oh, wow, no, this is actually really nice to work with like i really like this like these things that we're doing are
like it's so much more flexibility it's so much less code overall uh we can evolve it so much
easier so i think a big portion of the problem is that like there's just not enough content out
there for people to learn about how to do this stuff. You want to use GraphQL? You want to use gRPC?
There are tons of resources to learn that.
But if you want to learn how to build even just an API based on HTTP,
it's like, I don't know, go read the HTTP specifications.
It's like that's not a good experience for people.
So yeah, hypermedia APIs need some marketers.
They need some people to, like Roy Fielding, he should have wrote one.
Did he write one?
Hypermedia API?
I mean, what he defined.
Was there sample code?
I mean, he defined REST, right?
So he defined what web browsers use.
Did he write a REST API that we could all look at?
I mean, he didn't define it.
He wasn't specifically talking about APIs.
He was just talking about like REST is just
in general the thing.
Hyper-meet APIs
are applying REST principles
to APIs.
REST is a general thing
for life
or for what?
What was the context here?
You know,
pass me a context object.
Just for applications, right?
It's like,
that's how the web works.
The web is REST, right?
Fundamentally,
the entire web works on REST, right?
This is the thing
I'd always bring up before
of just like, you know, people probably wouldn't use the web if you had to get a book in
the mail that you'd look up and see which URL you need to go to log into Facebook and which URL you
need to go to to go do this thing. And, you know, oh, which URL you need to go to do a Google search.
Like people probably wouldn't like that too much, especially if then Facebook updated its login
page to be somewhere else. And all of a sudden you can't log in because you got to go get the new book in the mail right that would suck well
the web has a different thing so the web has Google and Google I guess was the hadios right
so Google is where you go to get your index of things like the number one search on Google for
many years was facebook.com people type facebook.com into Google search and then they click
the first link to get to Facebook.com.
And so that kind of is the phone book, isn't it?
The only reason that Google works is because the web is built on hypermedia principle, right?
Because it can crawl?
Because you can crawl, right?
That's one of the things that you can do because the controls are in there.
So everywhere that you can go, all of the actions you can take are in the response you get back.
So you crawl a root level domain, and then you can be like, oh, here are all the links in the page.
I'm going to follow those.
And then the links in the page, I get back from there.
I follow all those.
And the links in the page, I get.
And it's just like a continuous thing.
And that's how you can crawl the whole web.
You can't crawl a REST API.
That's not a thing.
The responses you get back, they're dead ends.
Every single one of them is a dead end.
You have to know the specifics of the APIi know the specific structure all of that so like literally the web
would not work as it does today wouldn't function wouldn't exist as it does today if it weren't for
the hypermedia controls that are embedded into it we just don't have that and that's why apis
aren't discoverable that's why we can't crawl an api to find out what are all of the endpoints what
are all the things i can do with this API? So the folks who came earlier,
like after Roy Fielding,
but before you and me here today,
that thought, okay, this is a good idea.
The Hattie Os folks,
the ones who will say like RESTful APIs aren't Hattie Os,
and they want to point that out.
Where are their sample apps?
Where are their API?
Like, shouldn't there be the book and the sample app and the example?
You said GitHub tried. I'm just playing a little devil's advocate here, Chris. GitHub tried,
but did they fail? There were lots of efforts to do so, but I think there were just missing
pieces to translate what we had with web browsers into APIs, right? As I mentioned, right?
Like in order to make a hypermedia client work,
you need to have some way of defining the schema
of the things that are going to come back.
So you have to have a way of defining what things are
because like on the web, humans do that, right?
You see words and you're like,
oh, that word means a thing.
Oh, sign in, log in, whatever.
These are all things that mean like,
oh, I can put some information in
and get access to my account.
We all know what that means,
but computers don't understand anything.
So we have to tell them, hey, these are-
Have you heard of LLMs, Chris?
Have you heard of LLMs?
Listen, Jared.
I'm just trolling at this point.
Keep going.
So like with a client,
we have to give it some reference point.
So it says like, oh, okay.
So even if the text says sign in or log in or whatever, that really means authenticate.
So we have to give it something that like basically an opaque string that means like,
hey, this is the authenticate thing.
Go look for the thing that's referenced in this response to see what the authenticate
thing is and then present that to the user on screen.
And I still think today we don't have really good tools for actually defining that information
to give to people that are building clients so they can build their clients against that.
Because that's still the information that you need to have at the end of the day.
You need to have the schema of something to understand what it is you can do.
The thing that really hypermedia allows you to do is change the representations
that you're sending back or move the representations to other places so that you can evolve one side
without having to evolve the other. And I think that's the thing that I think got lost in that
entire stretch that I think people got enamored with so many of the other things that they forgot
that it's about like, we have to be able able to define things we have to give people something to define with and then
they can do all this nice stuff with different representations and being able to evolve things
and move things and all of that right like oh i want a new version of my api well if you know
when the client makes the request it sends a hey this is the schema that i can speak
then the server can be like oh actually if you want to speak this schema go make a request, it sends a, hey, this is the schema that I can speak, then the server can be like, oh, actually, if you want to speak this schema, go make a request to this other URL instead of
the one that you sent to. And it's like, okay, I'll just follow that redirect. Okay, now it's
this another service. Like, okay, now I can talk this older protocol to you or whatever. Or it's
like, oh, there's actually more newer information. But since you only speak this version of the
schema, I'm only going to send you this other information back. Which, once again, is like a lot of it comes down to flexibility.
Like it gives you the ability to change things independent of the other thing that's operating.
And I think the biggest reason why this didn't catch on is that in the beginning, people never think that they're going to have to change their APIs.
People think that they're not going to have to evolve it.
They think, oh no, this will be fine.
Or I'll just add a version in the URL and that'll be great.
But they don't think about, well, how do you get all of the old clients off?
There's not a lot of forethought that gets put into that,
mostly because a lot of the time we're just trying to get something built,
trying to get something out the door.
And there's not enough of a push to be like, no, we shouldn't be doing that.
We should really find a way to make sure that we do think about evolvability and that we as an industry say, no, it's important to think about evolvability.
They're just trying to get something out the door.
So here comes the world's first.
It depends.
Hypothetical.
This is a test for you, Chris.
I'm going to give you a hypothetical,
and you're going to have to apply your knowledge to give an answer.
So you are a senior engineer, tech lead,
I don't know what you want to call yourself,
at a startup that has raised Series C.
Do they do Series C?
Yeah.
Yeah, they do.
So you've had some success, at least.
You have some users.
You have production things.
And you are tasked with, we have a website.
It's a web-based company.
And they want a public web API.
Okay.
And they want you to lead up the charge today.
And you just said you just want to get stuff out the door.
And this is the situation.
You do want to get stuff out the door.
You don't want to go back out the door. And this is the situation. You do want to get stuff out the door.
You don't want to go back and reinvent everything from the ground up.
What do you do?
Where do you go? What do you build?
You can ask more questions and I'll give you more fake information. Yeah.
Are we as a company going to build API clients for people?
Or are we just going to be like, here's the API and go use it to your heart's extent?
It will be a public API consumable by anybody who has whatever permissions we decide.
So it's not just yourselves consuming it, no.
Right.
Okay.
You already have a mobile app that also consumes a private API.
Okay.
That threw a wrench.
Or helped. I say that helped.
I think if it's an API that anybody out there is going to consume,
I'd probably just, pragmatically speaking,
build something that uses open API
and that kind of RESTful structure
where you can generate some clients for some people
and focus on documenting that very well and laying everything out there.
If it's an API that's going to be specifically targeted at people building front ends,
then I think it would be useful to consider something like GraphQL because that community
is so familiar with that type of thing. And if our data model lends itself to something
that's going to be GraphQL friendly,
like if it's a read heavy API, I think it's okay.
If it's going to be a write heavy API,
I don't think GraphQL makes as much sense.
I think writing with GraphQL is kind of wonky.
But I think the kind of broad thing is,
as much as I do not like opening API on Swagger,
I think that pragmatically, you know,
you're trying to get something out the door.
I think that's the place that you start
to get something out to really see
if people will use this thing.
And if they do start using it and it becomes good,
I think at that point, right after that,
once people are like, okay, this is good,
we're getting user, we're getting traction.
That's when you start planning your API strategy
and you start saying, okay,
do we want to commit to open API
for the long term? Do we want to explore other options or do we want to build something that
can actually handle multiple different API front ends? Because I think the answer that I would push
toward is building a platform that can support pretty much any of the API technologies that are
out there. Because, you know, you might start with OpenAPI and you might be like,
oh, wow, people are building a lot of JavaScript frontends with this.
It'd probably be a good idea to support some GraphQL as well
and being able to add that in after the fact.
So, you know, build a quick thing to see if it gets traction.
And then after that, start building out a better strategy
to make sure that you stay on board with things
and make sure that as you're developing the backend as well,
you're developing with the idea that it won't just be OpenAPI,
it won't just be GraphQL, it just won't be this one thing.
Okay.
So give us the elevator pitch
or the quick explainer on OpenAPI spec and Swagger,
why you would pick those to get going,
and then also as a follow-up why you don't
like those i mean i'd pick them to get going um because i think as far as you know what i think
of an api what i usually think of is someone who's writing some sort of back-end language
whether that's like ruby node python go j whatever. And those are not things that I think lend themselves to, say, a GraphQL.
I probably don't want to use GraphQL from Go or Java.
And I don't really like the idea of having, say, a gRPC API kind of exposed out there,
because then I'm dealing with having to ship people protocol buffer files.
No, thank you. And I think between, say, JSON API and OpenAPI,
I would say OpenAPI gives you the most flexibility with representations.
If you do want to start supporting protobus in the future or XML or something else,
OpenAPI will allow you to do that.
So that makes sense.
And as much as I hate YAML, it's fine to write
out an open API spec, you know, and you can also like generate it from your code and all of that
if you want. The reason I don't like open API is because the, I think the main thing it gives you
is pretty docs. And I don't think it encourages you to write comprehensive API documentation,
right? I've tried to use many open API specs for things,
and the documentation is just crap.
It's just so sparse.
It doesn't really give you all of the information that you need.
Even the models are just like,
hey, it's just like assumed you already know everything.
It's like, if I already know everything,
I wouldn't need to be looking at your documentation.
So I think it allows people to have the illusion
that they've documented their api
without actually having documented their api so that's that's why i don't like it and i mean i've
been at a couple places where you wind up with like 3 000 line long yaml files to define an api
and that's just not reasonable to maintain in any way shape or form that's not reasonable so i think that that's
why it's like yeah i'd use it but that's also i'm like i'd use it and then i'd try to immediately
figure out like how do we make sure that this isn't the thing we're stuck with forever good
stuff good stuff well i am tempted to end it here even though we didn't plumb anywhere near the
depths of your outline which i told you
there's just too much good stuff too much good stuff to dive into here i was writing this
outline and i was like oh man there's i'm like is this gonna be enough stuff and it's like
right nah there's always room for more anything from your outline or from your thoughts i know
you had abis and on the brain anything that you'd want to just shove in here at the end
just to get them on the air before we're done with this topic?
The thing I had at the very end where I'm thinking
that HTTP is more like an ABI definition
than it is an API definition,
which I think is a kind of novel thought
that I've definitely been gravitating toward.
Okay, why do you think that?
Right, so the point of an ABI is,
or the way that we define APIs
within, say, an operating system context or whatever,
is that they really define the calling convention.
So it's like, okay, you're in assembly,
you want to make a call to this function.
How do you do that, right?
Do you put things on the stack? Do you put things in registers? If you're putting on the stack, what order do you put
them on the stack? All of that, and then, you know, you got to do the jump into the function. So it
really defines the mechanics of how that function call happens. And I think HTTP gives you that same
sort of base layer of like, please define your mechanics of how you would like to actually
make this invocation happen. One of the things that I think about a lot is how a lot of the time
people try and complete everything they want to do in a single HTTP request, right? They like make
a request and they expect that request to be held until the result is available and send the result
back. And that is not the only option, right?
There's an option where you can make a request
and then you get immediately back
or very quickly back a 202 accepted
that has a location of where you can go get the result.
And then you can go pull, short pull, long pull, whatever,
go grab your results when it's ready.
And then, you know, the system can be like,
hey, it's not ready yet.
This is an estimated time we think it'll be ready.
And that's a calling convention level thing
because in your library that's making the API call,
it doesn't have to expose that it made two HTTP requests
versus making one that was just long hold.
It can just hide that from you.
So from the caller perspective, both look the same.
But from the implementation perspective,
it can potentially give you more information. It can surface more information to ops. It allows you, especially if you do,
if you do like the two or two accepted thing, it allows your servers to do some like load shedding
in a way where it's like, I don't have to hold open all of these connections. I can close a
bunch of them and be like, here, go get this information from this other place that might be
like another pool of servers or something that's ready to handle long polls or whatever.
It just gives you so much more flexibility
with how you can design your architecture.
But you can only do that
if you've actually thought about HTTP
as this lower level calling convention-esque thing.
Because if you don't,
and you kind of just build your libraries and everything
as if it's just everything's gonna get done on http request then you're kind of stuck with that and
there's a whole bunch of pieces of functionality within http that allow you to do this like one
of them that i stumbled upon is the prefer header which is a way for clients to communicate their
preferences to the server of what they'd like and one of those is called wait and it's a value that
has some seconds in it and it's that value says how long the client is kind of expecting to wait
for a response so you know if you want to have the ability to do both like long pulled responses and
you know accept you know to or to accept responses you can enable both in the server and then when
the client sends a request, it just says,
maybe it says the value will wait to zero to be like,
please just give me an accept back.
Or it might put it at like 300 to be like,
let's long pull this.
I will wait until you're ready.
And when you design your APIs like this,
when you kind of have this lower level substrate,
you gain so much more flexibility in what you can do
as far as evolvability of your system
and the architecture of what you have.
As I said, maybe in the beginning,
you start with one API server that's just handling everything,
but then you're like, oh, I can't handle everything,
but we can do some long polling result weighting.
We can build some really lightweight servers
that are really well optimized to do long polling
and are very efficient with their
resources. And then we'll have a whole bunch of those. So then when we have too much load coming
into the main API server, then we start doing that load shedding, sending back 202 accepteds to push
people over to the other servers. And then we have an event system that sends the responses to that
other pool of servers that then sends a response back to the client, right? It gives you more options for how you grow your APIs over time. So you're not kind of stuck with what you did in the
beginning. I like that thought. I'll have to put more thought into that thought. I think my immediate
thought, if I could just keep saying the word thought, is that there's a lot more to HTTP
in terms of functionality and availability of things that
we're probably not collectively
leveraging or using
in ways that would be
quite beneficial if we, I guess, maybe
even just knew the protocol
better. And when I say we, I'm
just kind of talking about myself in the plural.
And probably there's other people
that are like me. Hopefully I'm not the only one
who doesn't understand HTTP.
Because I'm learning stuff right here, right here live, Chris.
I didn't know about those things.
Yeah.
I mean, some of this is stuff that I've, like, once again,
I'm like building out stuff on my own now.
So it's like, okay, I'm going to take a little bit of time
to actually figure out how to do this right.
Like the way I found this prefer header was I was actually
trying to build live reload or live refresh
for the website that I'm working on.
And I wanted to have it, it's all served via Go application.
And I couldn't use a lot of the traditional methods
like a WebSocket connection or whatever
because the server would get recompiled and restarted.
And I was like, okay, well, how can I make this work?
And I figured out, okay, well, I can have a long-polled connection.
And at first I was like, oh, we'll just send back a special header
that has the version of the document.
And when I make a new version, then it'll go do that.
And then I was like, oh, wait, there's e-tags, there's entity tags and HTTP
that's literally built for this.
And I plugged that in and it just started working.
And then I was like, okay, well, how do I do this long-poll?
How do I indicate to a server that I'd like to do a long pull besides just being like this thing
does a long pull or do like a queer parameter long pull and that's why i went hunting and i
found the prefer header i'm like oh okay so i can have the client just say if the client specifies
a value above this much then it long pulls if it doesn't then i do a short pull return the response
immediately which means i get to use the same endpoint for the initial load to go get the e-tag.
Because in the browser, like when you load the page for the first time, you don't have the e-tag.
So you got to go get it.
So you're like, OK, I send it with a weight of zero.
It gives me it back immediately.
And then I go turn around and do a weight of 300.
So it's just long pulls.
And then when there's a change, it's just like it just responds back immediately with the, hey, there's a change, it just responds back immediately with, hey, there's a change.
I think it's like a 204, no content, being like, hey, there's new content here.
Or after the long poll lapses, it just gives back a not modified.
And it turns out all of that stuff just works with the browser automatically, which is just beautiful.
Because then I can go in and I do that special trick with CSS where you remove the DOM node from the DOM
and then put it back to refresh a style sheet.
And it's looking at e-tags as well.
So it just handles everything
and it's just all the lines perfectly.
And I was like, oh, this is really elegant.
And it took me some time sitting down
and looking at HTTP to be like,
where is all of this stuff?
But it turned out to be all there.
And I was like, I now have some special bespoke thing that i'm doing that's cool so my last question for you is
what is your favorite resource on http and why is it chat gpt
i know where you're learning this stuff chris i know where you're really learning it all uh yeah not not chat gpt that is that is for sure
um i know it's this is a very me answer you read the spec from reading the specs yeah it's from
reading the the standards i had a feeling you might say that they're very well written they're
very well written standards the thing i find a lot is there's a lot of bad information out there
which i assume also kind of trickles its way into chat gpt because chat gpt just pulls from written standards. The thing I find a lot is there's a lot of bad information out there,
which I assume also kind of trickles its way into chat GPT because chat GPT just pulls from,
you know, the web and the web just has lots of wrong and old information on it. So I think that,
you know, going and just reading the specifications is a good place to start. And I also think that
one of the things I would love it if we as an industry could do
is be a little bit more experimental
with things and
try some things out. One of the
things I don't like about GraphQL
is that it uses post for everything.
It makes sense why it uses post
for everything because we don't have
an item point
retrieval method in HTTP
that includes a body. right? Neither head nor get
allow you to have a body. Well, actually get the body is undefined. So technically you could do it,
but a whole bunch of things might break if you do. So I get starting off using post,
but when this was kind of realized the IETF, like the committee within the IETF that does HTTP,
it was like, Hey, we're, we have a missing method here. Um, maybe we should make one.
So it started off being called search and then I changed it to being called query., it's like, hey, we have a missing method here. Maybe we should make one. So it started off being called search, and then I changed it to being called query. And it's literally just
a get with a body. And that effort appears to have kind of stalled out. Part of the reason for that
is because like, you know, there isn't a lot of support behind it pushing it. But I feel like,
you know, if the GraphQL folks had been like, hey, we got to use this post thing for right now because it's what we have. But what we really want is Git with a body.
So let's go to the IETF and let's get Git with a body specified and put into the standard so that
we can then use it for the thing that we want so we can evolve this, you know, these standards
forward. And I feel like that's what we should be doing with more of these things. And I think that
would make everything overall a bit better. And that's one of my gripes with a lot of these
these technologies that are just kind of not using http well because it doesn't fit their need either
because they don't know how to make it fit their need or because it literally like in the case of
graphql does not have a specified way to fit their need they're building the thing but they're not
going back and helping to repair that and fix that.
So that's the thing that I would really like people to do.
And for people to just be a little bit more exploratory
and be like, I don't know, make up your own headers,
make up your own methods, make up your own stuff.
See if it works for you internally.
Maybe even try a little bit externally
and have some fallbacks,
like the good old, you know,
in HTML you can't use, you know, put or delete.
So it's like, oh, well, we'll use post for this.
We'll accept over post.
But if you want to use delete, you can use delete and it's semantically better.
So like finding ways to move that because that's really how we move the whole like the
platform sword.
Like this is the big thing the web platform does where they like find ways to slowly add
things with different types of trials and different types of shims and all of that.
And I feel like we should be doing that in this API space as well, of trying to just
move everything forward so that we can just have more stuff built into the platform as
a whole.
Very well said.
Well, to our listener, if you enjoyed this conversation, which if you didn't, you must
be a glutton for punishment because you're still listening to it.
It's just weird. But you were hoping
for a Pound to Fine this week because you heard
me say on Chainsaw News that
Pound to Fine was coming up next. Well,
it is coming up. It's just not coming up next.
It's coming up now. It's coming up next
week. So
fingers crossed, of course. Things do change,
but we plan on playing that game
next week and happy to provide this It Depends conversation with you, Chris,
this week.
I had a lot of fun, lots to chew on from here.
We'll try to link up some things in the chat
or we'll just put a link to ChatGBT in the show notes
so you can go ask it what we all talked about.
Connect with Chris on gotime.fm.
Anywhere else that you like to be contacted?
Email?
I'm pretty much everywhere.
Scriptable is my handle.
Scriptable.
So I'm still on X.
I guess I'm on Blue Sky now.
I am actively, as I said, working on a website,
which will hopefully be rolled out in the near future.
Trying to do some interesting things.
I want to build an activity pub server.
So lots of fun stuff.
That's not there yet.
So for right now, definitely still on X.
Still on Blue Sky.
And obviously, go time.
There you go.
All right, that's all for this week.
If you like these It Dependses, let us know.
Because I respond positively to positive feedback and i'll do more
if you don't like just be quiet just don't say anything and i won't do them anymore
but uh that's all for now we'll talk to you all next time bye friends
i went back and found that old blog post,
The Tech Behind the New Grooveshark, written in December of 2010.
That was a trip down memory lane.
Maybe you'll remember some of these open source projects from back in the day.
jQuery UI, SlitGrid, Store.js, Underscore.js, Closure Compiler.
Remember that one? Good times, good times. I'll add it to the show notes, just.js, closure compiler. Remember that one?
Good times, good times.
I'll add it to the show notes just in case you're curious.
We are scheduled to record and ship Pound to Find next week.
And that's it for ChangeLog and Friends in 2023.
We always take a couple of weeks off at the end of the year to relax and refresh for the next one.
Which reminds me, if you're making a year-end wrap-up
of your favorite things,
and if the changelog happens to be
one of those things,
let us know about it.
We love reading about
what our community members are into.
You can do that in our free community Slack
by signing up at changelog.com
slash community,
or on X, we're at changelog,
or on the Fediverse,
we're changelog at changelog.social.
Thanks once again to our partners, On X, we're at changelog, or on the Fediverse, we're changelog at changelog.social.
Thanks once again to our partners, Fassy.com, Fly.io, and Typesense.org.
And to our beat-freaking residents, Breakmaster Cylinder.
Speaking of BMC, we have a couple more Changelog Beats albums in the works.
Stay tuned for those.
Next week on the Changelog.
News on Monday. Our second and final all things open anthology
on wednesday and we play that pound to find game on friday have a great weekend
share change log with your friends who might dig it and let's talk again real soon