a16z Podcast - Building APIs for Developers and AI Agents

Starting point is 00:00:00 So I love to say the APIs are the dendrites of the internet. So if you think about the brain being a collection of neurons that by firing together, you know, think, you can think of the internet as a collection of servers or programs running on servers that by interacting with each other, think or do. And that connection, the synapse, that's almost tautologically, almost always an API of some kind. Realize that every company is becoming an API company, and every API company needs an API platform with SDKs and docs and API versioning and just a million different things. That amazing team that I worked on at Stripe all collaborated on.

Starting point is 00:00:40 People want to be able to focus on their core capabilities, be thoughtful and exposing the interfaces to users and not have to worry about the low-level details. If you've been building with APIs, you know they're the connective tissue of modern software, powering everything from payments to cloud services. But as APIs multiply, so did the challenges. developers' experience. SDK quality, documentation, versioning, and now a new frontier. Interfaces not just for humans, but for AI agents.

Starting point is 00:01:09 In this episode, Stamos founder, Alex Ratrade, joins A16Z partner, Jennifer Lee, to share how his helmet stripe-shaped Stainless's mission to bring world-class developer platforms, SDKs, docs, and more, to every API company. They discuss why SDK is often the API for developers, how MCP Model Context Protocol, reframes APIs as interfaces for large language models and the new design challenges of treating AI agents as first class users. It's a conversation about the feature of APIs, agent experience, and what it means to build developer platforms that are robust, idiomatic, and built to scale. Let's get into it. So I started working on API tooling when I was at Stripe.

Starting point is 00:01:51 I had been working on other teams and I just saw how important the API platform was and so I moved over to that team and built out a redesign of the Stripe API docks, which was an incredible experience, and through that saw how important the SDKs were. Even internally at Stripe, people I think didn't quite see that to the developers who use the Stripe API every day, the SDK is the API. And those were kind of falling behind. So I built a code generation system there that can take the Stripe Open API spec and turn it into a great Python library and a great TypeScript library and so on. And that was really fun to build. You know, the computer scientist to me, It just had a phenomenal time building sort of the compiler aspects of that and the library design nitty gritties.

Starting point is 00:02:30 And so after I left straight, people kind of kept asking, hey, you know, how do I get good SDKs or how do I build a great developer experience for my API? And it was very painful for me, actually, because, you know, you're saying, hey, that's a great question. You really need SDKs. It's a core baseline part of the developer experience at this point. Everybody interacting with your API is going to expect a well-typed SDK in their language. Unfortunately, the bad answer to that great question is that there's no way to do it. You know, the Open API generator that's open source at the time, certainly it was pretty far from the Stripe Quality Bar.

Starting point is 00:03:02 In terms of the output, it would generate. A lot of great work has gone into that, but it's a very hard problem. The maintenance burden, the headaches involved in trying to set this up, it's just too much for almost any company other than, you know, something like a Stripe at 2,000 people, which was when I worked on that. And so it's very dissatisfying as an engineer to tell people a bad answer, like, you need this thing and also you can't have it. So eventually I started saying, hey, would you pay real money for this if I built it for you?

Starting point is 00:03:27 People both said, hey, yes, I would. And also, can you help us with everything else around our API? And realize that every company is becoming an API company. And every API company needs an API platform with SDKs and docs and API versioning and just a million different things. That amazing team that I worked on at Stripe all collaborated on. And so, you know, can we bring that Stripe developer platform in a box? and bring it to every company.

Starting point is 00:03:55 And so that's sort of the way that I came to start Stainless API as a company. That prediction you had or that statement you had for 2022 definitely turned out to be true today. Every company is an API company and we're going to dive into MCPs too. Can you explain sort of how did you approach the sort of SDK generation aspect of the product? And Stripe experience, of course, is very critical to building this product. Because till today, it's still one of the north stars of what developers look to as great, the art of experiences and great SDKs. Yeah, we had a lot of experience with this at Stripe and, you know,

Starting point is 00:04:28 saw a lot of the good and a lot of the bad and a lot of the ugly. There's so many details to get right, whether it's making a client library that's really robust. And when things get weird on the internet, your SDK does not, it's not the thing that's blowing up or causing problems. It can handle scale. It can handle great error messages and can work well with telemetry and all this stuff. In addition to being robust, an SDK also needs to be really polished.

Starting point is 00:04:51 And that means if you're building something for a Python developer, that it's idiomatic to them, it's familiar. If they hover over on a thing, there is documentation that pops up in their editor telling them about that thing, whether it's coming from the Rest API or from the core of the library. And crafting that is really quite a journey. You know, and it's something that I just personally love. Years ago, I created a small programming language, and it was all about kind of polishing those nitty-gritties and finding the best way to do every little detail. And so that's one of the things that I most enjoy at Stainless. There's an interesting thing here where, you know, we're going to be talking about MCP. And crafting, I think the Python library that we do is a great example of this because

Starting point is 00:05:28 Python was the first programming language that I learned, no offense, visual basic. It had evolved a lot since I had last used it when I was sitting down to create our Python generator. There's types in Python and there's multiple type systems. There's async. There's, you know, pattern matching. It's really incredible to see. And what's idiomatic through all of this world? And that's a moving target.

Starting point is 00:05:49 I think the industry spent years basically figuring out how to make an enigmatic Python library and figuring out what it looks like for a Python library to be intuitive and legible to a Pythonista. And that's continued to evolve. And I think that we've moved the industry forward a little bit. And I'm proud of the work that we've done there. When you come to MCP, which is this kind of new frontier where instead of exposing your API to a Python developer, writing Python software, one of the things that you can do with MCP is expose your API to an LLM agent using an MCP client.

Starting point is 00:06:23 In that world, what's intuitive to an LLM? What is usable and ergonomic and idiomatic to a large language model to an AI and interacting with a large API? And the world is starting from scratch there, almost. So there's a lot of interesting work to do and develop and figure out. So for us, it's kind of this really fun exploratory research problem. Maybe you zoom out a little bit. you called APIs being the dendrite of the internet.

Starting point is 00:06:51 Now that MCPs are everywhere, like, you know, occupying a lot of the brain bandwidth for developer tooling companies like stainless, what do you call MCPs? And how did that have impacted your business, how I think about the landscape? Yeah, so I love to say that APIs are the dendrites of the internet. So if you think about the brain being a collection of neurons that by firing together, you know, think, you can think of the internet as a collection of servers or programs running on servers. that by interacting with each other, sort of think or do. And that connection, the synapse, that's almost tautologically, almost always an API of some kind.

Starting point is 00:07:28 So it's a really critical part of the internet software ecosystem. And seeing that kind of like CPU brain, you know, extend into the GPU brain, is a really exciting moment, I think. And you need the right interface. So MCP kind of is that interface. I don't know if I have a biological metaphor off the top of my head for that. I kind of want to compare it to eyes or ears or a new sense. When a software system is communicating internally, okay, cool, but it's all in your head in some sense.

Starting point is 00:08:02 It's all in the cloud. It's not doing anything in the real world. And how does all the software that the technology industry has been building for the last 30 and beyond years, how does that kind of become real in the real world and do something meaningful? And one of the long-standing ways of doing that is sort of, I'm going to call it through the eyes, in the form of user interfaces. So you'll have someone log into a web application. You know, I used to work at Stripe, and so I use this example a lot. Someone at a small business might log into their Stripe dashboard, refund a customer, create a new payment, add a customer to the system, anything like that.

Starting point is 00:08:38 In the days before APIs, everything was this way. You know, then we saw that we needed to automate things. You know, maybe instead of eyes, I don't know, now we're going with fingers, right? And there's some part of your, you know, where there's a communication between your mind and the neurons and something tactical in the real world where all of a sudden these computers can do things in an automated fashion. I don't know if you would call MCP and AI like they're the new ears or something like that, but it's a new sense or it's a new way of interacting, getting information from the world and also putting something out. You're sort of like growing this new limb. I definitely imagine a hand raising up in the air of like, here you can use. me and like, I can do these things. So definitely, you know, having more of the touchpoints and

Starting point is 00:09:19 surface areas for AI to explore and also to leverage a lot of the existing, you know, capabilities from software. So I like that analogy a lot. And just maybe going down that train of thought, you have been developing for tools, for developers and as a developer for a long time. But now we have this like new persona that's large language models and AI. And they are part of the design aspect of how to serve large longage models as a user and building sort of tools thinking about this new persona. What are the changes and challenges in just like the thinking of designing tools when taking in this sort of new interface and new requirements that has impacted how, you know, you've been building and building the products? Yeah. So if you

Starting point is 00:10:04 think about a Python library being the interface between an API and a Python program or Python developer, I would think about MCP as being the interface between an API and an LLM. And so to us, it's all SCKs, right? Now, again, you can do other things with MCP and these other things are important. Documentation is a really big example, and I'd love to talk about that too. But in terms of sort of these like core operational, you know, let me go refund a customer and Stripe Dashboard or let me go find out when this last transaction occurred or anything like that, you know, as you say, there's a lot of difficulties and new challenges and

Starting point is 00:10:38 problems in making this work. And a Python developer might really want to make sure that async works without too much rigamarole or a Java developer might want to see that the classes that they're interacting with are typesafe, but at the same time, they may not want them to sort of blow up at runtime if something is unexpectedly null. And that's a really hard problem, right? You know, yesterday I was chatting with someone who was on the Java team back in the day and helped design some Java languages. And I was telling him about this, he was like, yeah, that's an unsolvable problem, right? And, you know, that's something that may be solved as the wrong answer, but that we've tackled at Stanliss and I'm proud of. There's really difficult challenges

Starting point is 00:11:14 for LMs. And so some of those challenges that you're asking about, a lot of it comes down to the context size. And so if right now with MCP, that the naive way, the typical way of taking an API, usually described with an open API spec, and presenting that to an NLM through MCP, is you basically take the JSON schema for every single endpoint, every single operation in your API, you turn that over into a tool in your MCP server and say, if you want to create a charge and stripe, you know, you use this create charge tool. And inside of that tool, here's every request parameter. So here's how you specify the amount of the charge. Here's how you specify the description, the currency, so on and so forth. And that's going to have the same descriptions

Starting point is 00:12:02 and documentation that you would see and Stripe API docs, it's all going to be there in the tool description. You can get accuracy there. But if you want to describe every single endpoint in the Stripe API with every single parameter fully documented, congratulations, you've just used up your entire context window.

Starting point is 00:12:17 And that basically just doesn't work. Even if you don't use up the whole thing, LLMs tend to kind of get overwhelmed. MCP clients tend to kind of get overwhelmed at seeing too many tools in use or available. And all these request parameters, like, you may not need them. You know, maybe you're only trying to,

Starting point is 00:12:34 maybe you only need three parameters from this endpoint, not all 50. Or maybe you only need one or two endpoints from this API, not all 500 or 50, or what have you. So that's a really difficult challenge to figure out how to gradually unveil what are the tools available, what are the parameters available,

Starting point is 00:12:51 what are the descriptions of those parameters, and you want to be able to do that without too much back and forth because every turn takes time and burns tokens. Yeah, exactly. Yeah. So do we have a solution for the context window challenge as well as the limited tool calls? Is there a solution yet? There's several things that we do to help with us today as Samus. And there's several things that we're doing soon. But the most simple thing that we do today is we can generate all of the tools in your API. And when someone wants to spin up this server, they can use command line flags in the CLI or query parameters in the remote URL to basically say, hey, I only want to interact with.

Starting point is 00:13:30 with these resources. So I only give me the, you know, payment intense part of the Stripe API, or only give me the customers part of the Stripe API, where I only want reads or I only want rights. And that limits things down a lot. And the end user can decide what they want to do. The API provider can decide what they want to do. And that nipping and tucking is really useful at sort of a first approximation. That could be a little bit clunky. And so maybe you don't know in advance what the limits you want to do are. And so a more dynamic approach that we also provide is the ability to say, hey, give me this in dynamic mode. And instead of, say, you know, 50 or 500 different tools, we only expose three. Right. So, you know, it scales 0 of 1,

Starting point is 00:14:17 not O of N. And you have a tool to get the list of endpoints available, the list of operations available. You have a tool to describe a single endpoint and get all those request parameters, and you have a tool to execute an endpoint. And so you've gone and made these tool definitions totally dynamic. Now, there's some tradeoffs there. It takes three turns to do one operation here instead of just one. And so at runtime, that's a little bit slower, but it saves context window. And if you end up not interacting with, you know, again, say the Stripe API, then you don't burn any of your, you know, hardly any of your context at all, which is terrific. So that's another thing that we offer today.

Starting point is 00:14:55 A third thing that we do currently, I think, is really fun and interesting. This is something that we shipped recently. It actually solves a slightly different problem, which is, okay, let's say you make your API request and you say, hey, Stripe, you know, tell me about the customer, Jennifer, right? And it'll make a retrieve request to the customer endpoint. It'll find Jennifer, or actually a list request. It'll come back with a bunch of data, right? Now, especially if it's a list request. So maybe you make a request to Stripe API.

Starting point is 00:15:21 It tries to find the right Jennifer. There's a lot of Jennifer's in there. is going to have to sort it through, and the size of the API responses here can be very large. In fact, especially for like a list request like this, they'll typically be so large that your MCP client like, you know, cursor or cloud code or anything like this, will just refuse to look at it at all. You know, you just literally can't interact with it. Even if it can, it's just too much data to sort through everything. What we've done is we've added sort of this JQ filter to all of these requests. And so JQ stands for JSON query and it's a commonly used

Starting point is 00:15:57 CLI for saying, hey, out of this large JSON object, give me only these properties and filtering down the data that you're getting. So it's sort of like the select clause and SQL for those who are familiar with that. LMs are great at JQ. So just giving them that tool has enabled them to say out of this large API request. I only need Jennifer's like name and email and description. And I don't don't need to know all these other details. And that keeps the, again, the impact on the context window light, and it keeps things really focused. That's incredible.

Starting point is 00:16:29 That sounds really useful. It reminds me another question of there's a misconception or misconceived notion that now you have MCPs, and that's how LLMs are accessing and interacting with APIs. The quality or interface of APIs or SDKs are not the important anymore. But I think that's like the polar opposite is exactly because there's more usage.

Starting point is 00:16:49 you need strongly typed, very polished SDKs as well. Do you mind elaborating on that? Yeah, absolutely. You know, so I was chatting about this morning, actually, with someone who leads developer relations at a large financial technology company. She was talking about this problem where today, if you ask a coding agent to integrate with this company's Rest API, the first thing that coding agent is going to do is install this company's SDK.

Starting point is 00:17:17 That sounds great, but what she's seen is that it's going to install a version from 10 months ago, an old version, and then it's going to hallucinate the whole interface, right? And I think a lot of people have had this experience where they say, hey, chat GPT, can you help me integrate with chat GPT? And it gets the version of the SDK a little bit wrong, right? And so something that we need to see with MCP, and this is sort of, you know, something I alluded to earlier, where the other big use case for MCP, rather than interacting with an API, is writing code and helping a coding agent basically access documentation. And so something I'm really excited for us to ship very shortly is the ability to get comprehensive reference documentation for the API of your library into the hands of an LLM. So that, you know, when people integrating with this financial technology company say, you know, hey, make me an integration, it'll know the right version and it'll have access to the full API of that version's SDK and it won't make mistakes. So that's one of the things that I'm really excited about all this kind of coming full circle. And I'll note, you know, again, there's still a reason that the, it sounds bad that the, that the coding agent kind of got the SDK usage wrong in the first place.

Starting point is 00:18:34 but it's important to note that the agent preferred to use the SDK. It didn't want to just make a fetch request given the option. And the reason is that with the SDK, you have type checking. So if you have an LLM that's kind of like trying to one shot a whole big crazy API request, that's not so easy to do with 100% accuracy. You really want those red squiggly lines saying, hey, this property has a typo. Or hey, you're assuming that this string is always, going to be string and it's never going to be null in the response. You know, a human reviewer is just

Starting point is 00:19:09 not going to catch a mistake like that for an untyped interface. They need to see their type checker kind of giving the quick thumbs up that the LLM is in the right direction so that they can basically quickly review the business logic that was produced by the coding agent. You know, I think a lot of people are seeing this that the more declarative and clean and dry the code that you're writing and the more type safe the code that you're writing is, the more you can trust LLMs, to do the coding part of your job for you and think about the higher level stuff and keep an appropriate eye on things.

Starting point is 00:19:40 Totally. So it's not just about keeping the accuracy of the LLM outputs. I mean, it certainly helps those that when you have a more constrained exploration space, but also just debugability. Like, it's so much better when you have strongly typed SDKs and very, I would call it, like you have the tools,

Starting point is 00:19:57 but also have the reference docs and menus in place with the dog's aspect of it. Like, it's going to be a more holistic experience is given to the AI agent to really perform the task. Exactly. Your coding agent is going to be able to try something out. You know, press save, see if there's a type error, and iterate by itself automatically. Rather than trying to make an API request, you know, it'll need an API key for that.

Starting point is 00:20:19 It might be a destructive API request. Having an LLM test in production is, you know, with a foreign API, with a foreign system, that is not what you want to be doing, right? And so, you know, for any company that ships an API that they want their user, to be using in production, whether it's a big business user, you know, an enterprise doing a key integration, or whether it's a developer trying to move quickly. In either case, it's really critical to make it easy

Starting point is 00:20:45 for people to move fast at that integration so that it can happen before some other project comes in and for them to do that with confidence. I love to just think about maybe the role of API developers in five to 10 years. You have been a developer, also develop really, well-known APIs, SDKs as well. If you imagine the future where the agents are going to be as common as human developers

Starting point is 00:21:12 and we're going to have this agent experience instead of just dev experience, what do you think that experience is going to be looking like down the road? Today, we're still continuing designing for both of these two personas or users. I still call it like LLM a user. What do you think it will look like in that time horizon? I mean, I think it's a really interesting and exciting question. What I was saying a minute ago, you know, where an LLM, you know, a coding agent writing your code means you want your code to be more declarative and more dry, you know, you've always wanted

Starting point is 00:21:47 that, but it becomes more important to be able to quickly and accurately review code. And so for people building APIs, you want a way to write your backend code that is going to be not like spaghetti. And that's how a lot of people unfortunately have to write the rest APIs today. And furthermore, you're going to want more clear and prescriptive standards around what API design should look like so that the LLM can follow a set of rules that the company has put forward around how they want to do their API design. And again, the LM can follow the slot into maybe the API framework that the company is using and just focus on, okay, let's declare the endpoint, let's declare the parameters, let's write the business logic, so a human

Starting point is 00:22:33 can quickly review that. I'm imagining really high-taste developers. You're kind of crafting the endpoints that are designing, both the APS themselves to, you know, the documentation with that NSD case in a very detailed-oriented way. But now we have a lot of, let's say, co-gen tools, and we have even more advanced frameworks and also much higher-level tooling that you can define the specs maybe, I wouldn't call like natural language, but just more of like a spec out way, but not having to worry too much of the like polishing on the details. And that's exactly, you know, what I feel like as soon as its

Starting point is 00:23:07 work is doing is to maybe elevate the developers to think about more of the higher level tasks because of what you want the functions to be. And we want the users to interface with instead of like caring about the errors and the details and pagination or like debugability or observability, these type of nuances. The most painful and difficult part around building APIs. There's many painful and difficult parts, unfortunately, today for people who are shipping APIs to their customers and their users and their partners, which is unfortunate. But one of the worst is sort of the design part, you know, and dealing with all the bike sheds and thinking, you know, you just want to be able to say,

Starting point is 00:23:44 we need an API that does this thing, right, that manages this kind of resource. You know, the humans should be thinking about, as you say, the high levels of what the business needs and letting the robots do all the less exciting stuff. Now, You want AI to be writing your maybe business logic. You don't really want it to be writing your platform code. Those core things that sort of operate as all the glue between everything else. For that, you want a good platform. So you want a good framework.

Starting point is 00:24:12 You want good libraries. And that's something that Stenlist is looking to bring to everything around the API so that the humans can say the high-level stuff, the LLMs can build that medial layer, and that Stainless can take care of the low-level infrastructure. The last question I want to ask is you work with some of the most cutting-edge logos and customers like OpenAI, Anthropic, Cloudflare, without disclosing any confidentials,

Starting point is 00:24:34 is there any learnings that's worth sharing as the audience on these collaborations of how they're thinking about their SDK design, their API platforms, and what they want to offer to developers? Yeah, I mean, I think there's so many lessons that every API team learns again and again. Everyone in this space sees how important it is to be thoughtful, how important it is to, you know,

Starting point is 00:24:57 consider the user. The exact way you consider that is changing a little bit, you know, just like it did when people move from writing JavaScript to writing TypeScript or when people went from having slow connections to fast connections on the internet. The fundamentals, I think, are still fairly universal. It's exciting to see people really think deeply about how to expose their capabilities, whether those capabilities are a large language model that can do all sorts of, you know, text and audio, or whether that's the ability. to spin up new servers in the cloud instantly, or whether that's the ability to move money around the world.

Starting point is 00:25:32 People want to be able to focus on their core capabilities, be thoughtful in exposing the interfaces to users, and not have to worry about the low-level details. Thank you so much, Alex. This is wonderful. Thank you, Jennifer. This has been fun. As APIs evolve into core infrastructure for both developers and AI agents,

Starting point is 00:25:51 the quality of SDKs, docs, and interfaces will only grow in importance. If you enjoy this conversation, please rate and review the podcast on your platform of choice and share it with your network. Stay tuned. We've got more great discussions on the future of software, infrastructure, and AI coming soon. As a reminder, please note that the content here is for informational purposes only. Should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security is not directed at any investors or

Starting point is 00:26:27 pretend to investors at any A16Z fund. For more details, please see A16Z.com slash disclosures.

a16z Podcast - Building APIs for Developers and AI Agents

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.