The Good Tech Companies - Here’s How We Built 1,000 API Connectors in a Week

Episode Date: March 2, 2026

This story was originally published on HackerNoon at: https://hackernoon.com/heres-how-we-built-1000-api-connectors-in-a-week. How Membrane used AI agents to ship 1,000 ...API integrations in 7 days — covering auth, actions, validation, and everything in between Check more stories related to machine-learning at: https://hackernoon.com/c/machine-learning. You can also check exclusive content about #ai-agent, #developer-tools, #api-integration, #oauth2-authentication, #good-company, #membrane, #api-connectors, #workflow-automation, and more. This story was written by: @membrane. Learn more about this writer by checking @membrane's about page, and for more stories, please visit hackernoon.com. Membrane Universe is a library of pre-built integration knowledge. It defines everything an agent or developer needs to connect to external APIs. The pipeline has two phases, each driven by its own batch script.

Transcript
Discussion (0)
Starting point is 00:00:00 This audio is presented by Hacker Noon, where anyone can learn anything about any technology. Here's how we built 1,000 API connectors in a week, by membrane. Last week, we shipped 1,000 API integrations, not over months of engineering sprints, in one week, with 10 membrane agent sessions running in parallel. Here's how we built the pipeline that made it possible. Membrane Universe. Membrane Universe is our library of pre-built integration knowledge, everything an agent or developer needs to connect to external APIs. There are many types of elements in it, but for this project we focused on two. One, connectors, which define how to connect to an external API, authentication via OAuth2, API keys, etc. Plus data collections and events. Two, action packages, which are collections of ready to
Starting point is 00:00:49 use API actions, EG, create a Slack message, list GitHub repos, that agents and workflows can call. For action packages, we don't try to be exhaustive. We generate the most common actions covering approximately 80% of typical usage. For the rest, users build ad hoc actions through self-integration. Building each integration manually takes a developer 30 to 60 minutes, research the docs, figure out off, implement the client, write tests. At that rate, 1,000 integrations would take one person roughly a year of full-time work. We've used LLMs to speed this up since the early days of GPT 3. 5, but it was always sad hoc. Membrane agent already knows how to work with our platform. We saw the opportunity to industrialize it. We built a batch pipeline to process thousands of apps
Starting point is 00:01:40 automatically. The build pipeline. The pipeline has two phases, each driven by its own batch script. Phase 1 handles authentication, the hardest part of any integration. Phase 2 layers on the actions that make each integration useful. Both follow the same pattern. Fetch eligible apps, spin-up concurrent AI agents, validate the results, publish what passes, flag what doesn't. Phase 1 authentication, build connectors. This script handles the first step. Implementing auth for each app. How it works. One, fetches all apps from our API, filters to those without a connector yet. Two, for each app, running up to 10 concurrently, it creates a connector record in membrane, creates an agent session in our engine, spawns a local membrane agent agent powered by Claude, tells the agent which connector to implement, the agent knows how to interact
Starting point is 00:02:32 with membrane from its system prompt and how to build connectors through preloaded skills, so the user message is just the app name and URL. Waits for the agent to finish, approximately two. Five minutes on average, validates the result against our schemas. This feedback loop is important for agents, as they can correct themselves when validation fails. If valid, publishes the connector and makes it public if invalid, marks the app for manual review. What membrane agent actually does inside each session? First, the agent uses web search and webfetch to find the app's API documentation. It reads through the docs,
Starting point is 00:03:09 figures out whether the API use ESSO-O-Oth 2, API keys, Basic-Oth, or something else, and configures all the relevant-oth parameters, client ID, secret fields, scopes, token URLs, the works. Then it implements an API client that properly attaches credentials to request. writes a test function to verify the connection and actually makes HTT prequests to the API to confirm its reachable and responding correctly. Finally, it uses Membrane's tools to write all the configuration back to the platform. The whole process takes about two, five minutes per app, and the agent does it completely autonomously. Do the math. Ten agents, approximately two, five minutes each, running in parallel.
Starting point is 00:03:53 That's roughly 10 connectors built and validated every couple of minutes, without a single human keystroke. And 10 is just what we settled on for now. The concurrency is configurable and could go higher. Each agent handles one connector or one action package per session. We deliberately keep it to one element per session to avoid bloating the context window. A fresh session for each app means the agent stays focused. Phase two actions, build packages. Once an app has auth configured, it's ready for the second phase, generating the actions that make the integration actually useful. This This script takes every APP that already has a connector and creates an action package for it. The pattern mirrors phase one.
Starting point is 00:04:34 The script filters to apps that have a connector with aoth but no package yet, then spawns an agent for each one. Each agent knows its connector ID and is told to implement the package. It researches the apps API, identifies the most popular and useful endpoints, and creates action definitions, complete with input schemas, API request configuration, output and optional guidelines for non-obvious behavior. After validation, checking the package actually has actions, it's published and made public.
Starting point is 00:05:04 The architecture, here's what the full system looks like when you zoom out, key technical details. At concurrency 5 to 10, we process approximately 100 apps per batch run. Here's what makes that work reliably. Session tracking every agent session is tracked in our cloud, even though the agents run locally during batch builds. The script creates sessions in our platform and after each agent finishes, and syncs all the conversation messages back. This means we can review every AI decision through our console UI, exactly as IFIT were a cloud-hosted session.
Starting point is 00:05:38 We can also continue or retry any session from the cloud if needed. Validation and error handling not every app can be automated. The script handles failure gracefully. Schema validation. After the agent finishes, we validate the result against our SD- If it doesn't pass, missing required fields, wrong structure, the app gets flagged. Dead APIs. The agent is instructed to leave auth empty and explain why if the API is unavailable. These get flagged, timeouts. If Claude gets stuck on a particularly tricky API, though it doesn't happen frequently,
Starting point is 00:06:13 the session is marked as failed and can be retried. This is where it gets interesting, failures feed back into improvement. When an agent fails on an app, we review the the session to understand why, was it a gap in the agent's skills? A weird API pattern? Bad documentation? We fix the underlying issue, rerun, and each batch gets better than the last. The agent's nulled gie this is key. The agent doesn't start from scratch for each API. Its system prompt is assembled from multiple knowledge sources. Membrane platform overview. What membrane is, how the framework works. Connector building skill, a proprietary step-by-step workflow for implementing auth. Determine auth type, red-auth-type-specific docs, configure parameters, implement
Starting point is 00:06:59 API client, implement test. Open API skill, how to find an incrementally query Open API specs without loading entire schemas into context. Detailed implementation guides for core connector functions. Our agent framework supports on-demand skill loading during a session, but for batch processing we found that pre-loading key skills directly into the system prompt works better. LLMs don't yet reliably self-load skills in 100% of cases, and at this scale you want consistency over flexibility. This means the agent has deep knowledge of our platform's patterns before I'd have and looks at the target API. The user prompt is minimal, just the app name and URL. The manual layer, not everything is fully automated, and that's by design. Some things still need a
Starting point is 00:07:46 human edge cases. Some APIs are undocumented but functional. We discovered these during review and handled them manually. Quality review. We review agent sessions through our console, especially for apps where validation failed. No real credentials. Currently, the agent doesn't authenticate with real API keys. It verifies APIs are reachable and that auth is configured correctly, but doesn't complete real OAuth flows. We're actively building browser automation for automatic test account generation to close this gap. What's next? We're publicly launching membrane universe in the coming weeks, starting with niche and obscure apps, old school APIs, poorly documented systems. The biggest gap right now is real credential testing. We're building
Starting point is 00:08:32 browser automation for automatic sign-up and OAuth flows so agents can verify integrations end-to-end. Longer term, continuous maintenance, APIs change, endpoints get deprecated. The Somme agents that built these integrations will keep them current. The bigger picture is this. A.I. Agents aren't just coding a system. that help you write functions faster. Their infrastructure builders, point them at a well-defined problem, give them the right tools and knowledge, and they candled things at a scale that simply wasn't possible before.
Starting point is 00:09:02 We pointed ours at 1,000 APIs, and they delivered. Thank you for listening to this Hackernoon story, read by artificial intelligence. Visit hackernoon.com to read, write, learn, and publish.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.