The Changelog: Software Development, Open Source - NATS and the CNCF kerfuffle (Interview)

Episode Date: May 16, 2025

Derek Collison — creator of NATS and Co-founder & CEO of Synadia — joins the show to dive into the origins, design, and evolution of NATS, a high-performance, open-source messaging system built for modern cloud-native systems and part of the CNCF. Derek shares the story behind NATS, what makes it unique, and unpacks the recent tensions between Synadia and the CNCF over the future of the project.

Well, friends, this is your favorite podcast, the changelog. Yes, we talked to the hackers, the leaders and those building the future of software. Today we're joined by Derek Colison. Derek is the co-founder and CEO of Synadia. He's also the creator of NATS. For the uninitiated, NATS is a high-performance open-source messaging system designed for cloud native distributed systems. And today Derek walks us through what it took to create NATS, where it came from, how they eventually decided to form a company around NATS called Synadia to

offer this to enterprises and more. Derek also shares the behind the scenes of the recent kerfuffle between NATats and CNCF. A massive thank you to our friends over at fly.io, our partners, our sponsors, our friends, and the home of changelog.com. You can launch your app in five minutes at fly.io. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts.

Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts.

Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts.

Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. Okay, let's nuts. in this next generation of Heroku. It's being built on open standards and cloud native. What can you share about this journey? If you look at the last half a decade or so,

like there's been a lot that's changed in the industry. A lot of the 12 factorisms that have been popularized and are well accepted even outside the Ruby community are things that are think table stakes for building modern applications, right? And so being able to take all those things from kind of 10, 14 years ago, being able to revisit and be like, okay, we helped popularize a lot of these things. We now don't need to be our own island of this stuff. And it's just better to be

part of the broader ecosystem. Like you said, since Heroku's existence, there's been people who've been trying to rebuild Heroku. I feel like there's a good Kelsey quote, when are we going to stop trying to rebuild Heroku? It's like, like people keep trying to like build their own version of Heroku like internally at their own company, let alone the public offerings out there. I mean, I feel like Heroku has been the gold standard. Yeah, I mean, I think it's the gold standard because there's a thing that like Heroku's hit this like piece of magic around developer experience, but giving you enough flexibility and power to do what you need to do. Okay, so part of Fur and this next generation of Roku is adding support for.NET.

What can you share about that? Why.NET and why now? I think if you look at.NET over the last decade, it's changed a lot. .NET is known for being this Windows-only platform. You have WinForms, use it to build Windows stuff, double-IS. And it's moved well beyond that over the last decade. You can build.NET on Linux, on Mac. There's this whole cross-platform open source ecosystem, and it's become this juggernaut of an ecosystem around it.

And we've gotten this asked to support.NET for a long time, and it isn't a new ask. And regardless of our support of it, like people have been running.NET on Heroku in production today. There's been a mono build pack since the early days when you couldn't run.NET on Linux and now with.NET Core, the fact that it's cross-platform, this.NET Core build pack that people are using to run their apps on Heroku. The kind of shift now is to take it from that to a first-class innocent.

And so what that means for Heroku is we have this languages team. We're now staffing someone to basically live, breathe, and eat being a.NET person, right? Someone from the community that we've plucked to be this person to provide that day zero support for the language and runtimes that you expect and like we have for all of our languages, right?

To answer your support and deal with all those things when you open support tickets on Heroku and kind of all the documentation that you expect for having quality language support in the platform. In addition to that, one of the things that it means to be first class is that when we are building out new features and things, it is now one of the languages as part of this ecosystem

that we're gonna test and make sure runs smoothly, right? So you can get this kind of end-to-end experience. You can go to Dev Center. There's a.NET icon to find all the.NET documentation. Take your app, create a new Heroku app, run git push Heroku main, and you're off to the races. So with the coming release of Fur and this next generation of Heroku,.NET is officially a first class language on the platform, dedicated support, dedicated documentation, all the things. If you haven't yet go to heroku.com slash change all podcasts and get excited about what's to come for Roku.com slash change log podcast Today we're joined by Derek Collison from Nats and Synadia. Derek you are on one of my favorite episodes of Go Time years ago.

Pandemic time. Challenges of distributed messaging systems. Yes, Go Time is no longer being produced, but the feed is still out there, the episodes are still out there. Please go listen to that. I learned so much from you about these things. So welcome to the changelog.

Ah, appreciate you guys having me on. Thank you. Literally years in the making. Jared, do you remember going to KubeCon a while back? I do. The circus. That's right, the circus in Chicago. Couple years back, we met the Sanedia folks and the Nats folks.

I think it was not quite early days for Sanedia, but like it was early-ish for announcements, maybe even marketing. Derek, you can kind of speak to this, but now you remember that circus and meeting those folks? I do. Yeah. I don't know if I was there.

What year was it? Because Sanayi was formed at the end of 2017, right? On the hills of Absara. But I don't remember. I don't think I was at the one in Chicago. Many years after that. Two, 2023? Okay.

So not too far back yet. Yeah. Till into 2023. Yeah, you weren't there, but we had a back yet. Yeah, till into 2023. Yeah, you weren't there, but we had a few friends there that were listeners as well, and we became to know each other. So, been nice knowing you all. I've known about NATS for a very long time. When did NATS start?

I mean, it's been around forever now, right? Yeah, NATS was kicked off as part of a system that I was building at the time at VMware called Cloud Foundry, right, which was my spin on Heroku for the enterprise back in 2010, I guess. And at the time, I had come off of time at Google, spent about eight years there, but prior to that was at a company called Tipco, which before that was called Technikron. And we have been designing lots of messaging systems that then were used to scale up distributed systems for customers and clients. So I just had a natural

way of building systems like command and control, telemetry, eventing, using these types of constructs. But Tipco was closed source, right? So we couldn't use that. So we reached out to Alexis Richardson, who was running a company that was kind of in charge of Rabbit and Q, which at the time was kind of, you know, the major open source player from a messaging construct. And, you know, Rabbit and Q is still part of, I think VMware slash Broadcom now,

part of Cloud Foundry or whatever that has turned into be. But at the time it was more of an enterprise type of a technology. What I mean by that is that if you asked it to do something it would bend over backwards trying to do it for you even at the detriment of everyone else. And we had some challenges there which led to the creation of Nats,

which was kind of like a dial tone and think of it like the electricity in your house, your condo or whatever. I can plug in a really bad blender and I might blow out something in my own place, but hopefully it doesn't take out the electric grid for the whole neighborhood. NATS was born on a very simple principle of just do a couple things very, very well. And yeah, it was the proverbial weekend project. So I left on a Friday and Monday, you know, we plugged it in. And believe it or not, applications that were written for that initial version, you know,

the protocol that speaks between the clients and servers still run today. You know, so that you can just fire up something from the Cloud Foundry days the protocol that speaks between the clients and servers still run today. So that you can just power up something from the cloud boundary days and just run it against a modern net server and it just works. So we've tried very, very hard and we're not perfect, but by any stretch, but we tried really, really hard

to make sure we were always backward compatible with the protocol. And because we knew the protocol, even though it was designed very, very quickly in only a couple of days, that was kind of the really big thing from a technology standpoint. I'm not a distributed systems person,

but NATS feels a lot to me like tail scale, but at a different layer. Is that how you would describe it to somebody who doesn't build distributed systems or have sort of the depth and knowledge you might have? Yeah, I'm a huge fan of tail scale and you know the folks that were either at Google or involved in the go programming language that are over there. But it's an overlay on top of the constructs that we still know which is IP based addressing almost everything is point-to-point. Most, I don't know about

tail scale but most cloud providers write trunk UDP broadcasts or multicast IP-based addressing, almost everything is point-to-point. Most, I don't know about tail-scope, but most cloud providers write trunk UDP broadcasts or multicasts. And so it's a point-to-point location-dependent type of a system that now is surfaced up over a very secure overlay with WireGuard. What NAS was trying to do was kind of change the notion of what we call intelligent connectivity.

And specifically when, you know, folks that are listening say, well, what does that mean? It's really fairly simple from an abstract standpoint, which is everything is location independent, right? And so one might say, oh, well, things are kind of location independent today, but I would argue that there's a lot of unnatural apps going on below the covers to get that to appear that way.

Load balancers, GSLB, Anycast, DNS tricks, right? So the biggest first one is location independence. And the second one is instead of one to one request reply is the dominant pattern, HTTP, that's all you can kind of do, SANS, SSC, and WebSockets. We are end to end and both push and pull. And the push becomes very, very interesting for certain use cases where instead of keep asking, hey, have you updated your temperature?

If you updated your temperature, just tell me whenever you update your temperature, right? And for scalability and distributed systems, just those primitives from the connective layer become pretty powerful, right, at scale. So NATS really took off, right? I mean, huge adoption. Can you tell that story? Yeah, so I mean, NATS really took off, right? I mean, huge adoption. Can you tell that story?

Yeah, so I mean, NATS was built specifically to power those subsystems in Cloud Foundry, right? And I had no desire to do anything more than that. And so it was, for at least probably the first 18 months of its lifetime, specifically geared toward just supporting what we needed in Cloud Foundry, from command and control, telemetry, eventing, things like that. When we started the company AppSara, which was trying to do a redo of Cloud Foundry on some of the things I felt I didn't spend

enough time and energy on security, governance, federation, all these different types of things, we rewrote NAT in Go. NAT was originally written in Ruby, that's what Cloud Battery was written in. Still love the language, not great to deploy things into production that are Ruby based,

although I know DHH has done an amazing job of trying to simplify that. But back in the day, it was, you know, there was a lot of moving pieces to deploy this thing. And we were starting to hit some performance issues with very, very large systems needing more performance there. So we had, at least at the time, we were looking at either Node.js, JavaScript, or Go,

which was very brand new. I think when I started playing with it, it was 0.52. And I felt that the speed advantage with the concurrency, the I-O mechanisms that they had, but mostly the fact that you could build pretty much a static binary, meaning shipping to production was just move one binary, was the direction that we took. And it was a great direction. I mean, I think Go has grown up and, you know, there's people that like it and people maybe that don't like it, which is totally fair. But we've gotten a tremendous amount of mileage out of it.

And at the time, looking back, it was the absolute right decision. Now, going forward, when we're looking at massive, massive extreme scales that some of our customers are pushing us on, we're starting to bump into some things in Go that we need to address. And whenever I'm in a language and we start to do unnatural acts to get around certain things within the language, that's when we say, all right, let's take a step back and are we at another inflection point.

And we're not there right now, but with the introduction of JetStream, which is the data persistence mechanism that Nats uses that was added on, you know, about four, four and a half, five years ago, I guess, we're starting to see a lot of friction with the way Go does disk I O and things like that. And so we're trying to figure out what makes the most sense

as customers are saying, Hey, remember when I said a million was good, then I said 10 million was good, now I need a billion, you know what I mean? And could you do that in the next two years type of stuff? And those are great problems to have because they're hard and they're fun to work on. But it challenges you to think outside the box sometimes.

What are you looking at? I don't do it anymore. I'm sorry to say that I dropped the ball, but I used to learn a new language every year, that you know, and I would just write a Nats client for it to learn it. And so I learned Rust and there's certain things in Rust that were really, really attractive.

But to be frank, I just didn't enjoy programming in it. When you were doing a low level library, if you were consuming other things and building an upper level app, there was a composite of lots of different things together. Especially the packages or crates were really well done. It felt really nice and clean, but I was at a low level library trying to do some things. We got it and it's one of our most popular clients.

I like, again, I'm older probably than the majority of the demographic that's listening to the podcast, but I like languages that I can hold most of it in my head. You know, I've never been a big IDE person. I still program in Emacs. I do use AI and I ask, you know, Claude or chat to BT questions and things like that, but I don't have and I ask, you know, Claude or chat to BT questions and things like that, but I don't have that co-pilot,

wind surfer code mentality yet, but I do like Zig quite a bit, but Zig right now, you know, Andrew was trying to take on some really gnarly problems, which I have the utmost respect for, especially the colorless async. And that kind of tied in a little bit to event loop structures.

And they kind of got pulled out. We need some sort of direction there before we finalize a Zig client. But I did write one. I have one that uses threads and does some interesting things. And then Yoram from Tiger Bureau, I've known him for a while, full disclosure, I'm a minor investor in their company, love distributed, all things distributed, I think was their conference.

And really been watching them and Bun and of course, Mitchell with Ghosty on what's possible. But each one of those event loops and IOU ring interactions is all kind of a little bit snowflake, not exactly. But I think Andrew is now thinking through a way to uplift the abstraction of IO that could tie into an event loop as something similar to allocators in the language where you have to pass it into a package or to a constructor. Say, hey, if you need to use IO, please use this essentially interface, right? And so I'm really interested to see where that lands. But I like ZIG. Believe it or not, I like Mojo, even though I'm not a Python person. I just am a huge fan of Chris

Lappner's and what he's trying to do there, I think is impressive. But ZIG's probably number one right now in my mind. But again, we need a little bit more direction, not from the server perspective, but if we wanted to do a client on, how do we interrupt between all the different event loops? Whether they reincarnate the one that they have in the language, but then Mitchell's with XEB, I think. And then of course, Bunn has one that's, I think,

a pull from Tiger Beetles and Tiger Beetle has one. So that'll be good when we get there. Well, completely different problem space, but you like Chris Latner and the Ladybird folks are probably picking up Swift from C++ to Swift. Have you considered Swift? I haven't. We built a Swift client. I wasn't involved in it. Again, I dropped the ball on my learning new language every year. I actually did program a little bit Swift. I think the interesting part about where

Zig is that we can control direct I-O, meaning we could go through a U-Ring on Linux if they're applicable or whatever. It allows us also to control more precise layouts of the data structures and of course, memory, right? So, anyone who's been programming Go for a long time and has a tremendous amount of investment in a big code base, right? You start to see things pop up. It's like, oh, avoid JSON at all costs because the built-in JSON marshallers, you know, use reflection can use lots of memory.

The network IO and the disk IO subsystems in Go are very different. The disk IO subsystems are pretty much a native thread that kicks back into your Go routine that says, okay, it completed, the write completed or whatever. I think what we need is a lot more fine control over those things. What I've been looking at is the adoption or the ability to introduce IO-Uring in a meaningful state. I think Zig's done a pretty good job there.

Maybe not within the language per se, but with folks that use it, Tiger Beetle and Bun come to mind. And so that one's kind of a big one. The data structure layout, the garbage collection and taking control of our disk IO, right? Which, you know, Jetstream, we've got folks that are coming in and says, oh, yeah, I need to do a million stored messages per second replicated three ways,

and then an async mirror over in a different region and a different clapper writer type stuff. So, we can abuse the disk, you know, the storage subsystems pretty well. And we might be getting close to everything we can accomplish with Go. I'm not saying that that's definitive, but it's starting to feel that way as, we're starting to do not unnatural acts, but we're starting to do a lot of things

inside of the server to get around how Go actually marries concurrency with this kind of. So when you said some of your customers need like a billion, is that like message passes per second? Or like, what does that mean a billion, a billion what? Some need, you know, a lot of messages per second. Nats has always been pretty performant. Last time I checked just raw speed,

a single server could do about 20 million per second. You know, messages sustained. All of our clients can't do that, right? They have more of the logic, but a raw just, you know, shove it in there. Right. But when we say billions of things, it's usually stored messages and then subject cardinality.

So in event sourcing and event streaming, right, every single message has a unique subject. And what we found was is that people are starting to treat the stream of messages. And we have materialized views like key value and object store on top of them, but they're starting to treat them like a subject-based addressing lookup thing.

Meaning that they use subjects all the time to get messages out or filter messages or do some other operation. And so you could imagine, you know, trying to figure out an index that looks like a traditional data structure that anyone who's, you know, done a comp side, you know, 200, 300 level class would understand, but overlay subject semantics

with wildcards and partial wildcards is a little bit tricky, if that makes sense. Yeah. Like a giant hash table in the sky that also has some very gnarly semantics that you can use. Yeah, and to be honest with you, it's called the S-Tree and it's an adaptive rate X-Tree with, you know, path compression and expansion, but then it has an overlay where we can say, hey, you just stored a billion subjects in here, hopefully in an efficient way, pick

out and call me back on everything that matches foo.star. You know what I mean? In other words, or a wallet card. And so we have that, but that lives in memory. And so if you have, you know, 10 million messages, but only a thousand subjects, you're fine. You'll never even see this. But we're starting to see people where they've got billions upon billions of messages and everyone has a unique subject. Next thing you know, your server's taking up 30 gig just for the indexing on the subject-based addressing layer.

And so that's one of the big challenges that the team is looking into solving for 212, the next version of the circuit. Well, it's cool that you still have your head down in the technical trenches of this project, because it seems like, from what I've seen from you all and your name, signed on things recently, it's all been higher level business,

and maybe technical legalese, not technical technicals. But it's cool that you're still close to the metal of this project and not just the business guy now. Yeah, I mean, I think I definitely, we're multiple hats obviously, and as of late I've been more concerned about upper level discussions around Cenedia, and Nats and the CNCF obviously,

which we can talk about, but also the business side of Senadia. But I identify with builders. I feel that working this hard, you wanna get to a place where you get to do the stuff you like to do, at least part of the time. And I still love doing that. Am I as good as other folks in the company?

Probably not anymore, but I still love to reason about things, think hard about things, you know, tackle really, really hard problems. That's kind of my relaxing motion in terms of work, if that makes sense. So if NATS was roughly around 2010-ish era, you really didn't plan to do anything with it

beyond its original case. At what point did you say, well, you know what, we should be a company, we should create CineD, and that was 2017, based on what you said before. How did you go from there in 2010 to there in 2017? Yeah, it's an interesting story. So one of the things that folks might not know is that when we were doing Cloud Foundry, VMware had never released anything that was open source. And my pitch to Paul Moritz at the time, who was the CEO who was running VMware, I said, I really think we'll have more bang for the buck here if this is released as open source.

And so when I did NATS, I said, I'm not going to keep this closed. So I actually just opened it as MIT, right? And then we did, I can't remember what license we put on Cloud Foundry, but it was open source, which was a really big deal. When we went to AppSara, we did the rewrite and go. And we started to contemplate whether or not this could be something bigger than just an OSS project. I mean, it was powering all of the AppSara stuff, but similar to Cloud Foundry, right?

Telemetry, command and control, and eventing. And so as we looked at AppSara, and AppSara was doing a commercial version of Cloud Foundry++ to be very, very simplistic about it. But as soon as Kubernetes came out, certain things Kubernetes did that people like, certain things that the Obsera Continuum platform did, they liked. So it wasn't a technical battle per se,

it was where's the consumer bias going? Kubernetes, because no individual stakeholder needed to make direct money off of it, drove the consumer bias to zero. Our ACV started declining. I called an emergency board meeting and said, hey, you know, I see the writing on the wall and all I can tell you for the 30 plus million dollars left in the bank over the next four years

is that we can tread water, but I don't think it'll change the outcome. And then I think pulling it into Ericsson and doing something constructive with some of the internal projects that I know they were working on made better sense. At that point in time, there were certain people that were going to leave, certain people that were going to stay with Ericsson, and then there was NATS, the team that was working on NATS in the middle. And at the time, Ericsson said, yeah, we don't necessarily have any need for that.

So it's open source, we can use it. But if you want to do something with it, we can. And a bunch of folks internal, prior to that decision-making process, had already kind of approached me saying, hey, we might be able to build a business out of this under the assumptions that distributed systems will continue to have more moving pieces,

but more importantly, that these pieces are going to start to be stretched out. So stretched out across regions, across multiple cloud providers, but more importantly out to this at the time, new thing called Edge. And the bet that we made was that Edge would dominate interaction models and would dwarf cloud within probably a decade and that cloud's not going anywhere. Don't misread that, but that cloud would become the new mainframe, but all the interactions would be at the edge and the rules are very, very different. So as many

rules changed as we went from data centers into cloud and we had the Roku guys, Adam Wright, was it the 12 things of 12, things about 12 factor apps. And then of course, cloud native architectures, of course then the CNC app, all of those things, you know, that step function from data centers to that, I believe will be even bigger stepping from cloud into edge where it's not batteries included. You can't bring all the stuff that the cloud is offering

and you have to kind of not think differently about the what, meaning you're still drawing the same triangles and squares on a whiteboard as you're architecting, but the how would be radically different. And so that's when we made the leap to create Synadium at the end of 2017 based around a foundation that modern distributed systems will look very different than they do today, and they will be pushed very hard

in terms of dynamic agility. They can change all the time. You know, again, I probably am further along career-wise than some folks on the call, but it used to be that when a system was set up like this in the 90s, early 90s, no one touched it. I mean, I could walk into any financial institution on Wall Street, I could go into their server room and point exactly where our stuff was running, and it never deviated from that, right. And that's very different than what

I think a lot of the audience probably thinks now where it's like, I don't know, it's somewhere in AWS East two, you know what I mean? But I have no clue what machine is running on. And so our thought process and our priors were that that type of step change is going to accelerate with edge and that the current tools of doing everything with like HTTP or HTTP derivatives just simply won't be able to keep up in our opinion. How has that bet played out?

Very, very well. What I mean by that is that in the 90s with Tipco, I was on the engineering side. I wasn't in an executive role or had any type of position around pricing or anything like that. But just from an engineering perspective, we cared deeply about interacting with these customers and such like that.

When we watched what was happening with Edge and how fast it was moving and how different the rules were, at Tipco we had to educate the market. And I remember how painful that was. We would, you know, a lot of the sales cycles were 80% education. We need to explain to you why we think you are going to have a problem in the near future that we think we might only be able to solve. In this world, which was good for Synadia was, is people were coming to us going, In this world, which was good for Senadia was, is people were coming to us going, hey, I tried to pull Kubernetes and JVM stuff like Kafka and put it into an Edge thing and it didn't work for me.

I need to think differently about it. Can you help us? When we started Senadia, we wanted to start a company based on if you're running a production and it's critical to what you're doing, which luckily, our stuff is critical to every customer that we have. There's no coffee breaks. If it goes down, they will light my phone up. That was the opportunity we were looking for in terms of starting a company. And we didn't have to educate the market. They were already in pain to start the company and that they would want a

commercial agreement with us to wring our neck if something went wrong, right? And I think that's a really good way to start a company. It's not the best way to scale it. So you need a backup plan as you evolve, but to start it, I think that's a really good place for folks to start, especially if it's based around OSS. Well, friends, I'm here with David Shue, CEO of Retool. David, I want to talk about awareness beyond Silicon Valley. Retool has a great presence and a great awareness inside Silicon Valley.

But what about beyond? What's really cool is I think we've done a really good and a great awareness inside Silicon Valley. But what about beyond? What's really cool is I think we've done a really good job of building awareness inside Silicon Valley. And so when you look at customers that use Retool, pretty much every big company in Silicon Valley of a thousand people today now uses Retool and builds internal apps via Retool. So that's really awesome. And I'm really proud of the progress we've made there. But I think the larger opportunity for us actually is outside of Silicon Valley, when you think about, for example,

the Kroger's of the world, the Coca-Cola's of the world, many of them are customers already today, but I think we haven't done as good of a job building awareness, if you will, around the developers and all these companies. And that, to me, is where the opportunity lies, because so much of these companies run on software. Software is so important. If you think about Coca-Cola, for example, Coca-Cola's not really gotten any cheaper than manufacturer in the last 10 or 20 years. Instead, the reason why Coca-Cola's doing well

as a company is because they are getting more productive by better software. And so every company needs to become a software company and Retro lets you do that. So Coke is a big company, but the principle rings true. Become more efficient by using better software. There you go. Well, if you're beyond Silicon Valley, raise your hand.

We want to hear from you. I want to tell David we reach people beyond Silicon Valley. That we're raising the awareness of what Retul is and what Retul does. And some big announcements coming soon here on ChangeLog, which is cool. Well, if you haven't yet, go to Retool.com, get a demo, try it out for free,

all that good stuff. Again, Retool.com. So OSS, should we open the cannon worms around OSS? Most recently there's been a, I don't know what you call it, a skirmish, a disagreement. A kerfuffle. A kerfuffle between CNCF and yourself as an ADA,

specifically around NATS and its involvement in CNCF and potential relicense. Of course, we've been here many times. In the world of open source and business and licensing, it's something that everyone's trying to figure out. And some people have or have not, and then other people think they have, and then they have not.

And then things change and people are offended. And so there's been a lot of, I don't know, maybe drama is just the best word around the business and the project. Maybe start with the CNCF and NATS's relationship to it prior to the most recent blog posts and press releases. Yeah, so what the audience might not know is that the CNCF was kicked off at a large meeting

at the Switch data center in Las Vegas, of which I attended. And I was actually part of the founding governing board. So a lot of people might think, oh, Derek's role with the Nats project, and Nats being a project in the CNCF, that's kind of the totality of it. But that's not the case, right? And even early on, we were looking at what value CNCF could bring to the ecosystem from a consumption perspective, the users to the people that are the companies that want to utilize that.

And then of course, the project themselves. And then for me, it was like the companies that are driving said projects. In the case of something like Kubernetes, where it's all of the big boys, so to speak, there's not a lot that the CNCF needs to do that, right? It's kind of taken on a life of its own and it's marching down a path. What we started to realize is that as time goes by, because this was, I think, over a decade ago, that both the CNCF and projects can evolve.

And where we were at was looking at the question, are we the best fit for CNCF? And no fault of CNCFs per se, but it just felt pretty clear to us that certain projects were at this tier and other projects were way down here, and we were in the latter category. In no way were we attempting to relicense, you know, the whole code base? I mean, one, you can't do it because once it's released, it's AP2, it's there, that will never change. But the CNCF and early discussions, because we said, Hey, would you be open to letting the project leave if the

landing spot made sense? Meaning, we could do a joint statement where it's like, no, it's landing in this new foundation made up of lots of, you know, big users and customers of, or production uses of the Nats ecosystem, commitment AP to XYZ. And during those early conversations, which started in February, we were asked, hey, are you considering a license change? And we're very well aware that we can fork

and do a Cinedia version enterprise server and XYZ. And so we said, we're considering it just for the server, that we might do like a BSL. And at the time the BSL for us, it wasn't translated well out into the media. But the reason that we were considering it was, we felt that it was the best show of commitment to both OSS and to our customers.

And I'll explain what I mean by that. The BSL has usage clauses that you have to define when you release the software. And one of them is there's a period of time that after which it converts back to whatever language you pick, which we would pick AP2. And it doesn't matter what Synadia does, the Linux Foundation does, CNCF does, that is a legal contract that once we release, let's say, a server with the BSL license,

after two years, it's converted to AP2, even if we don't actually update the copyrights or update the license or whatever. And so we felt that from an OSS ecosystem. And again, we've gotten lots of additional information, lots of additional feedback, most of it constructive, some of it not. But you know, and we've been listening, but our initial thought process as we were going down this path was, hey, this signals really strongly that we're not going to hold anything back. It will always kind of revert back to generic drug prices, right? Some of the good parts of what the patent system originally was trying to do early on. I'm not claiming that the patent system is necessarily good these days. It's a necessary evil, right? In addition, if you would imagine a two-year BSL window, and let's say Jared wanted to be a

customer of Synadia, right? And customers are worried about vendor lock-in, price gouging, all those normal things. You could say, I want to do a two year contract. So I have predictable pricing. And at the end of that, that all stuff that I'm paying for converts to AP2 and I can now use it for free. Or I could say, oh, I like the next version too. I want to use that and re-up, right?

And do that. So I'm not saying that it transpired the way we wanted it to, but I think people's thought process that we're just evil and we're just trying to price gouge and be greedy and stuff like that, I don't feel that that necessarily is correct. At least as we were discussing internally, we care deeply about our commitment to open source,

but we are starting to see a very disturbing trend where, you know, customers that, you know, everyone would recognize they're in the Fortune 50 that are using Nats to power production level services or functions or products, not only had never reached out for any type of commercial agreement with us, but actually have policies that say, if you're in the CNCF and you're incubating and graduated, we're not paying for it. Period. Really? To me, I thought that that was at least worth the discussion. It might not be a discussion

between myself or Synadia or the CNCF, but the broader ecosystem, I think, needs to look at, do we want open source projects that drive value, that can be critical, that are driven by a small group of individuals or a company that's trying to make business out of said value? And, you know, Sanadia has lots of different revenue buckets. So I don't want listeners to think that this is our only drive. But I was at least concerned enough to say, are we signaling to the ecosystem,

the broader ecosystem outside of us, that we only want projects that look like Kubernetes? We don't want any projects that are being driven by a single company. And I thought that was worth the discussion for sure. How much time before the blog posts that I think broke the news, so to speak, from the CNCF directly, how much, how much behind the scenes conversation was going on to sort of illustrate your points and your concerns? Like, was there a meeting? Okay. So this, this was a surprise.

Yeah. Was this more of a surprise? Yeah, so we were not, we had gotten to a point where we were trying to assert the fact that, you know, we wanted them to consider letting us leave and they were stuck on some of the trademark stuff, right? And so for the listeners, you know, who haven't been in this, when you're doing negotiations behind the scenes, you posture up and you do all kinds of stuff and then you figure out when you can't move the stakes anymore and then you make a decision, right? And our decision was always going to be, hey,

if they dig in, even if our legal team say that we could potentially win this, we have to consider the damage to the ecosystem and whether or not it's worth it. And we had already probably decided that if they were just going to dig their heels in and say, nope, you absolutely can't do that. If you want to go down that, it will have to be a legal recourse that we were going to say, OK, we'll figure out something else. But they, I can't speak on their behalf and I'm not trying to. And again, I'm extremely encouraged by my talks with the Linux Foundation and some of the ongoing stuff with the CNCF and that mutual statement.

Again, that's, you know, it's if everybody's had a hand in that pie, so it kind of gets diluted down. But I can tell you from my individual discussions with folks, especially at the Linux Foundation, I'm very encouraged with them saying, OK, let's see if we can do better here. You know, which is all I really kind of wanted at the time. But I did feel that we could be a good steward and if we could say, hey, here's what we're committing to, and that is commensurate with kind of what the CNCF is doing around certain projects. I thought it was worth the discussion to have.

Now, a lot of people don't agree with that and have very emotional discussion to have. Now, a lot of people don't agree with that and have very, you know, emotional responses to that. And although I don't necessarily agree with it, I understand that people will fight very hard for consumption being zero. But production is never zero, someone has to pay for it, right? I mean, you know, unless it's just something, whether it's time or money or both. And so we were concerned enough to say, hey, let's have a discussion about that. The main concern, I think, if I understood correctly, around even making this a concern for you was the fact that Sanadi was having challenges securing certain contracts because of its incubation status of NAT, which Sanadi is built on top of as the commercialized version of it. So is that the summary of like your initial concern?

No, I mean, we didn't graduate and that's a public record. And it was mostly around diversity of the contributors to the server. Right. So GitHub has all the stats so you can look at, you know, all the people that have contributed to it. And I didn't agree that that was super important for the success of a project long term. So I'm not saying the CNCF is wrong and I'm right. I'm just saying we had a differing opinion of what successful projects look like. And there's some that I've driven in my career

that now have been around for 30 plus years, two of them have been around for 20 plus years. So I kind of get what it is, but it doesn't have to only look like Kubernetes, right? In my opinion. And so that was kind of the first one where I was like, hmm, the CNCF is evolving where they bring in lots of different projects

and that's a daunting task of them having to do all that. And I, you know, hats off to them on that. But as you watched how things move through the channels and got towards graduation, there was some friction on does the project look like what the CNCF wants it to? And again, totally their right to do that. And then also I was noticing little perturbations in companies doing strange things to try to drive a business model where they were the main contributor, right?

Their company was the main contributor driving said project, you know, outside of Nats. And that's, we were watching that pretty closely. We don't get too many people worrying about the CNCF or our graduation status. Every once in a while we do, it's like, hey, why haven't you graduated? And we just say, oh, because, you know,

the CNCF's graduation criteria look like this. This is how kind of the server looks like. And it usually is that quick of a conversation, like, you know, 15, 20 seconds max with customers every once in a while. I think I've gotten two of those in the last six months, and it was just a, hey, why haven't you graduated sex stuff? So it's not that per se, it was, as the CNCF evolves,

are all projects good fit and is the only answer, you have to fork it away and then we'll kill it. And I don't feel like that should be the only answer. How about that? Right, there's so much brain equity in the Nats name. Obviously you founded it, you created it. I created the Nats name, driving home from VMware, right?

On Friday, I remember. And everyone laughs and the marketing came up with a funny term, but I remember just shaking my head, driving home going, geez, I don't wanna have to create another Net Tipco server. I was like, not another Tipco server. I just kept saying that my head driving home going, jeez, I don't want to have to create another TIPCO server. I was like, not another TIPCO server. I just kept saying that to myself driving home,

but I said, I feel like I have to. Well, at the end, not another TIPCO server is exactly how it got named. Yeah. Yeah. So when you first joined, or when you first started incubating, is that the entry level term there? With NATs, were you aware of the CNCF's desire

or were there requirements like this project will move towards a non-single corporate backer model or whatever language would fit? Like did you know coming into it that they wanted you to kind of change the format or the org structure of NATS? I don't think that was discussed at that level early on. So, you know, we were at the Switch kickoff meeting, we saw some of the early projects,

they wanted us to be one of the earliest projects. I was like hesitant, so I said, I wanna be part of this, but I wanna wait and see. And Google was doing similar things early on, they're like, we just wanna kind of see how this is gonna kind of pan out. But I think it was more from sustainability, how many people are using it, what type of impact does it have both within the CNCF, but then externally, were more of what I remember very early on of how do we classify a graduated

project. Now as the TOC, that's the Technical Oversight Committee, right, keeps changing and new people come into that, they started changing, you know, the criteria. And so instead of kind of a vote and a gut instinct, you know, on a vote type stuff, they started formalizing some of the things that they felt needed to gate between, you know, sandbox to incubating to graduate.

There might be some more levels in there, I'm not sure. And so at the time though, we weren't aware of that. And it was only kind of really solidified when we actually went through the graduation process. And that again, that's all online, the back and forth between us and the TOC on trying to graduate. So was your failure to graduate then a shock to you?

Or did you see it coming? I did not see it coming. And we, it took us a while, and again, it's all public record, it took us a while to figure out what's going on here. And it was specifically about diversity of code contributions to core pieces of the project. Now, again, CNCF, I think they've done an amazing job with all of the different pieces that are flying at them.

But if you look back historically, you could see certain projects that were like, well, how did that one graduate? You know what I mean? That one looks a lot more similar to these types of projects than others. And I think they've slowly cleaned that up and kind of solidified that process and those criteria. It's just, I didn't agree with the criteria. And again, I'm not saying I'm right and they're wrong by any stretch, but I did fundamentally disagree

that not all projects- You must think you're right then, right? Like if you disagree fundamentally, then you gotta think you're right. Well, in certain projects, right? And so, you know, could Kubernetes have been driven only by Google and gotten to where it was? No, but it has a very different role

of kind of greasing the skids between cloud provider mobility. That was the kind of some of the original that went around Diane and the folks that built it, at least from me talking with Diane back in the day of why they were doing it. But I don't think saying that that project fits means that a project like NAS doesn't fit.

Right, but they disagree with that. Correct. And they're the bosses of the CNCF. So they get final say on that. Now, when your conversations began back in February, this is when you realized you were not gonna graduate or you'd already kind of known it for a while. Like you had decided, well,

we're not gonna be a good fit anymore, right? Yeah, that was the graduation, I think it was in 2020, as well as the trademark fight. So this was more of a, hey, we just think we're not a good fit anymore and let's figure out if there's a way where both the CNCFs and Cinedia can message to the ecosystem that this is okay,

but as long as there's a landing spot that holds true to what the CNCF wants, commitment to open source, things like that, would that be possible? And obviously the answer is no, but I still think it was worth a conversation and that's where we went. So in February, you wanted to kind of slip out the back, Jack.

It wasn't going to work out. And then those conversations began. And then they asked you if you were thinking about a relicense. And you said, yes, we're thinking about BSL. And then there's probably some more conversations between there. But then they decided to publish the blog post. Or what I miss it in the interim between. So I think, you know, we were very clear our commitment to AP2 and OSS and that it would be a fork that we put BSL work into that then would again, in our

opinion, would signal a commitment to OSS because no matter what we did or said after that two years, which was the number we had picked internally, as we were going through that process, it would convert over. And so I was saying the licensing stuff is kind of separate. We're talking specifically about the open source nature of NATS and whether or not it's a good fit and if it can land somewhere else with both the blessing from the CNCF and the commitment from Synadia that allows them to feel comfortable doing that. They then started getting to the point where we were starting to figure out when the stakes would move or not move. Again, you know, just that and it's totally fine, you know, negotiating types of stuff.

And they put a hard stake in the ground. Nothing will ever leave the CNCF. We'll kill it instead. Nats cannot leave the CNCF. Nothing can leave, ever. Nothing, ever, including Nats. Correct. And that was my question too, it was like, has it ever happened before?

Which I'm appreciating. No, no, it hasn't. The details behind the scenes here. Okay, so it's like Hotel California, is that the one? You can check it out, you can check out. Right, and again, I think that, you know, that's totally fine that that's their position. I still felt it warranted a conversation of,

because that doesn't feel right to me. So if you look at like Grafana, right, had an interesting thing with Cortex and they were gonna start driving more stuff, but it wasn't necessarily compatible with the CNCF, but they were still committed to the open source nature, you know, of what they were trying to do, but it wasn't necessarily compatible with the CNCF, but they were still committed to the open source nature of what they were trying to do, but it wasn't compatible. And so the CNCF said nothing ever leaves, so you can fork it and then abandon this one. And that's what they had to do. And they, and again, I'm not saying they're right or wrong,

but when they purport that's a success story, I talked to other people and it doesn't feel as much of a success story, right? Mimir started and that's what almost everyone on the Grafana side uses. And I think Cortex is mostly supported by AWS so that they can run it, not have to pay anything to Grafana, right? Again, the consumption of zero is a sense of entitlement that I also think is kind of interesting, but production side is never zero, right? Someone's got to pay.

Right. Yeah. It's a hard position to be in whenever... And almost knowing this, I think maybe this even exposes a different scenario where maybe you are not fully aware. Like Jerry was asking, were you aware of X, Y, and Z when you first joined the CNCF? You incubated that whole process. And I think over time you get more and more privy

to certain details that are truths about what it truly means to join the CNCF. And I'm not saying anything negative or bad, but not being able to leave, like coming in the door, not being able to leave with the thing you think about over a weekend and you name is, is challenging. Because if you don't pass the bar of incubation and graduate and you're not a, I don't know what it means to not graduate.

Does that mean you're like not graduate even like what does that mean? Yeah. I don't get it all of it. Like does that mean you're not valuable? Does that, I don't understand the graduation status is either. I'm sure there are new ones too. I don't understand the graduation status as either. I'm sure there are new ones too. There is, and even after our TOC process around graduation where we did not graduate,

there was a slight change that said incubating or graduated projects are considered mature and production ready. But within the grand scheme of things, someone from the outside, a user, right, who's saying, I'm just looking for some software that might solve my problem. You know, they see graduated as a higher level than incubating. And so they're like, why haven't you graduated? Because you've been here for a while you started at incubating, you know, back in the 2018, I think is when we officially joined, if I recall correctly, I might be off on that one. And so I Yeah, I wanted a conversation. We had spent, you know, and I'm not saying that CNCF has not also worked

to promote NATS, you know, maybe not as much as Kubernetes and things like that in KubeCon where, but they did do a part there. But we lived and breathed this 7x24x365 for almost 15 years now, right? We've put, you know, everything can and not just myself, it's just, you know, all the folks that were out of Sarah. And then of course, now at at Sanadia, and not saying that what we thought was necessarily correct percent. But I find it interesting when people go, Oh, you really don't care about the project.

Or you're just trying to do a money grab and gouge people for money at this time. That's a little unfortunate. I get it. Again, I know this whole topic is very emotional and I can appreciate that. But that's about as far from the truth as you could possibly get, from what we were really trying to do. And again, if you if you looked at some of these companies, and you know,

we talked to other customers of ours, big, big customers and said, Oh, you know, these people that are kind of on the same level of you and the fortune 50, they refuse to pay. And when that kind of came up, they actually kind of changed their tone and said, ooh, so we could actually get access to stuff that we're helping fund by having a commercial relationship with Cinedia for a period of time two years before, you know, one conversation, they call them

the freeloaders. And he's like, it's not Bob and Vinny's pizza shop out of Venice, it's, you know, people on the Fortune 50 that have the means to pay and this isn't a little internal developer tool, let's say, production, you know, people on the Fortune 50 that have the means to pay, and this isn't a little internal developer tool, it's a production, you know, level critical service, they actually went, oh, yeah, we actually kind of like that idea. And so from a partner and ecosystem standpoint, those conversations, we had lots of them because we lost control of the narrative, right?

The CNCF wanted to all of a sudden move things into the court of public opinion. That's totally their right. We didn't have a heads up though, so my phone started blowing up when that happened. But we kept business as usual, we kept on commits, PRs, and we actually did a release the very next day, a full blown server release, which is a really big deal. It takes a lot of effort for us to kind of do all of those. So we've been trying to signal to the right folks

looking at the right pieces of data that, nope, there's a disagreement here. We're gonna try to work it out, but we're still committed to what we've been doing all along. You mentioned some of the emotional responses online and I read a few of them. One of which, which I'll summarize

because they use more flowery language than I'll allow myself. But maybe you have a chance to answer this particular outsider perspective which if you're just watching from the sidelines, can't remember if we hit record yet or not when we start talking about armchair quarterbacks but you're just sitting in your armchair quarterbacking.

And what you see is, Synadia brings NATS into the CNCF as an incubating project, gets all the benefits, even though it sounds like you're getting all the benefits that you maybe thought you would get, because Kubernetes maybe sucks a lot of the air out of the room, but gets all the benefits, you know, the conferences, whatever,

you know, the clout that you get of being a CNCF project and then hangs out there for a while, decides they want to relicense and take their ball and go home. And that makes people upset. What do you say to that sentiment? Why is that wrong if it is? The reaction is not wrong.

What I have an issue with is the conflation of facts within the reporting, right? And so it's easy to draw that conclusion. So I don't get upset at people being very upset at us and more so directed directly at me. The question is, is how much energy do we want to put into trying to make sure the facts are correct, right? The whole licensing and leaving the CNCF were conflated when they never should have been because they were never conflated in my initial conversations with- Unrelated conversations. They just asked me and said, hey, cause this is a problem, we're seeing this and we don't know what to do about it yet,

but we want it to know. And I'm like, yeah, cause I said, we're starting to see, you know, said three letter, you know, companies use our stuff and get a lot of value and contribute nothing back and I'll be very transparent. Yes, contributions comes in a lot of ways. But for a company like Sanadia, a commercial agreement and funds is the only one we really care about. Yes, we care about issues and PRs and all that other

stuff. But you know, just being very, very transparent, they don't keep the lights on. And again, Sanadia has plenty of money and we have lots of revenue opportunities and we're driving lots of those. But at the time I was like, this to me feels wrong. It just doesn't feel correct that the consumer bias is being driven to zero

for companies that it makes sense for them, right? Because if you sell a piece of hardware or service, it's kind of nice to have a guarantee that all the software you're going to use to augment that will always be free and keeps your margins where they are. I just think as an ecosystem, we're evolving, things are changing, the CNCF has changed a lot, that we still need to kind of think through some of these

and figure out what do we really want as a global ecosystem around this? And charity is not a business model, right? And so the whole notion of, oh, you can do donations, that's not a business model. And so what this lesson has taught me is, is that the biggest challenge, obviously, is in a company that develops OSS software that

wants to make a business off of that software, specifically as software, not necessarily as a service. Service changes the consumer bias. If it's something physical or somebody's running something for you, your natural bias is, I have to pay for that. But if I give you a piece of software that we spent over 30 some million dollars developing over the last decade or so, you start to see this notion of entitlement.

Oh, I deserve this for free, and I don't have to give anything back. We even have some people that just like bark orders at us, like you need to do this in the system. We're like, okay, you know, that's interesting. You're not a customer. We don't even know if you use it. And everyone has these problems, right? All the OSS projects have these problems. But for what we were looking at, I was like, that,

that doesn't feel right to me. And I wanted to reach out and open a dialogue and have conversations that would hopefully spur, you know spur some constructive dialogue. But just for the record, we never were abandoning open source, we were never going to blanket relicense everything. That's just simply conflation of facts in the media and then people reacting to them and then that being amplified quite a bit over the social media channels. them and then that being amplified, you know, quite a bit over the social media channels. So that's the, you know, I have no problem with people being emotional about it. But the, the way it was presented and the fact that we, we not only lost control of the narrative, but then all of these articles never reached out to us or me at any point in time and said,

Hey, what's your side of the story? You know what I mean? And I was like, that's interesting. I even have one now where someone pointed me to a YouTube video where the guy just, you know, F bombing me and trashing me, you know, left and right. And I'm like, I don't know who this person is. I've never met them. They never talked to us. You know, and I asked the whole team, I said, has anyone interacted with this person? They're like, Nope. You know, they're just, you know, a YouTuber that, you know, that's how you get the views, you know? Yeah. And I, I get that. But at the same time, you know, they're just, you know, a YouTuber that, you know, that's how you get the views, you know? Yeah, and I get that, but at the same time, you know, I kind of shake my head.

I'm like, okay. No, it's not. And again, if, as an ecosystem, we say to ourselves, we don't want open source projects that drive mission critical functionality that are driven by a single company. I don't agree with that. But if, as a global ecosystem, we say that, but I want people to think really don't agree with that, but if as a global ecosystem we say that,

but I want people to think really hard about what that really, really means. Well friends, building multi-agent software is hard. Agent to agent and agent to tool communication is still the wild wild west. So how do you achieve accuracy and consistency in non-deterministic agentic applications? That's where the agency, AGNT-CY comes in. The agency is an open source collective building the internet of Agents and what is the Internet of Agents? It's a collaboration layer where AI agents can communicate, discover each other, and work across frameworks.

For developers, this means standardized agent discovery tools, seamless protocols for inter-agent communication, and modular components to compose and scale multi-agent workflows. You can now build with other engineers who care about high quality multi-agent software. Visit agency.org and add your support. That's agntcy.org. If you had your way, like if this went without a fight and you were able to do what you were trying to do, what were you really trying to do with the project? If things went your way, this fight didn't happen, and you could leave the hotel, so to speak, what would you have done?

What was the goal? So earlier on, we were talking, I think, about, I don't know if we were being recorded or not on programming languages. And so what I was looking at originally was if we created a foundation that had a CineD representative but also had representatives of all the major users of NATS and customers of Synadia

in a 501 3C or something like that, that had a commitment to AP2, right? Open source, you know, XYZ. That's probably what we would have done. But it was simply just for the betterment of NATS. And everyone that sat on the board, similar to kind of like the TOC, although the TOC is more of, hey, we'll get involved if you need to move from sandbox, the incubator to graduation, or if something's off the rails. Otherwise we let projects manage themselves. This would be more of bringing in,

a lot of our top customers, the Walmart's of the world, things like that, to have a seat at this and be able to help direct where the project would go. And right now in the CNCF, that's just not a full-on reality. a seat at this and be able to help direct where the project would go. And right now in the CNCF, that's just not a full on reality. That's more limiting, more governed by other folks.

Yeah, couldn't you do that inside the CNCF? We could, meaning we could, I mean, right now, we stored the project. But then you graduate, wouldn't you? I mean, you did that, then they graduate you. No, the graduation criteria that we failed, and I don't know if it's still the same, I have to check with that,

was the code contributions in the server were not diverse. Okay, so it's kind of like governance, it's actually code contributions. That was what tripped us up in terms of graduate, or trying to graduate in 2020. Yeah, that seems to be off, like that doesn't make sense. I mean, it's more measurable,

but if you established even a TOC of your own, right, or TSC, like a technical steering committee, comprised of Senadia folk plus your largest customers, right? And they weren't, and they didn't write the code, like, who cares who writes the code, if they're helping vote on the direction of NATS, I would think that would be a diverse open source project. Yeah. You know, I would tend to agree, but again, I'm not trying to say, oh, the CNCS is wrong

or try to speak on their behalf. And they aren't here to represent themselves. Right, right. We also have one side of the story where we have journalists who are just podcasters, who are here to talk. But we're saying they're not here to defend their side of any of this, so. Yeah, but what happened to us

with us not controlling the narrative, I wanna be very sensitive to the fact that they do have differing opinions. And again, I'm extremely encouraged by our talks and how they're like, okay, yeah, let's see if we can do better here from a graduation flow potentially, but also, I don't know if it's this notion of,

I don't know the answer, and I don't think the CNCF knows the answer either, but like for example, modular, I don't know if you saw, they just did a license change as well. They're getting more and more permissive, right? Cause they want broader adoption. And it's AP2, which is great.

Modular, they do Mojo, that's Chris Lapner's. Okay, modular Mojo, okay. Yeah, yeah. But what interesting thing was is that they say, if you use it in production, you just have to let us use your logo in the use case. And I was like, that would be good, I would like that. Right, because we have so many companies.

I mean, if you go to nats.io and look at who uses us, those people opt in, they send us to say, hey, please put our logo on there. But there's so many more that use us and don't want that, or actually say, no, you can't use our logo whatsoever on that. So even something like that to go, oh, we can see Acme Company is actually using that,

and we know Acme Company, that's really cool. That would help a company like Snedia, right? That trust factor. And so I shot that over to the folks at the Linux Foundation and just were like, hey, would you guys ever consider something like this? I don't think they will, but I'm trying to do my best and operate in good faith

and say, okay, if we wanna make this better, I'm gonna tell you things from my perspective that could make it better for a project that looks like Nats and companies like Snaky that are driving that. And we'll see how that goes, right? I don't expect anything materially to change radically, but at least they're open to the conversations now

and are starting to be receptive to, okay, we see that there's some challenges here that we might be able to improve. So modular setting that up, effectively Apache 2 with an additional constraint on it, isn't that then technically like a proprietary license now? Like they've customized a license and now it's not exactly Apache 2 anymore, it's like Apache 2

with this other thing, which is very similar to like what MEDA did with Lama, which is like, it's pretty much open source, except for this one. Now their constraint was much more strict than we want to put your logo on our website. It was like, if you have 10 billion users, you can't use it or something like that. But.

I think, I mean, you raise a good point. My, I'm not a lawyer and don't pretend to be, but I think in that notion, the license strictly governs what you can and can't do with the code, meaning the code itself. For example, even within the CNCF, the code is AP2, but the CNCF has very other regulations around how you use the trademark and how you can use the logo and how you can do direct-to-the-products and use the naming. So I think what Modular is doing is fine.

It's outside of that pure, what can I do with the actual code? But what they're saying is, is that if you're using it in production, then you're giving us the right to use your logo. I think those are separate, but I understand it feels like it could be sticky because the difference between a trademark

versus a copyright versus a code license are all nuance in how they kind of interplay, for sure. Which reminds me of something else that I read, and this was just, again, armchair commentary. So I'm not even sure if this is true or not. But when you become a CNCF project, you typically will actually turn over IP to the CNCF, right? Like the actual intellectual property becomes part of the trademark.

Yeah. Trademark logos or whatever. Licenses, but yeah. And with NATS, did that happen? So what happened with the trademark was kind of interesting. And again, it wasn't necessarily stated. And I tried to correct some of the facts and I got pitched for it even worse.

But essentially when we did the original kickoff for the CNCF at the Switch, this is my recollection, was that you only legally transferred those upon graduation. Right. And I remember Google kind of being a little nervous that if Kubernetes started directly as a graduated project that they would have to figure out all of that stuff with all of their lawyers and XYZ. And so that was in my head. Now, you know, to the CNCFs, you know, to benefit them, I think they quickly changed that

once Kubernetes graduated. And they said, no, as soon as you're in, you essentially have to transfer, or at least once you move past sandbox. And we joined as an incubating project in 2018. So the rules had shifted from what my recollection from the original kickoff meeting was. But besides that, the thing that, to be honest with you, frustrated me, you know what I mean?

Just being transparent. But again, I like the conversations that we're having with the Linux Foundation and CNCF. But as we were trying to graduate and we had just joined, all of a sudden Major League Baseball came and said, Hey, you know, we don't like your trademark. Cause it conflicts with the nationals baseball team out of Washington. And so I was like, well, that kind of is silly. And anyone who's done patents, you know, in the, in, in, from your listeners know that that's there's a little bit of a game to that as well, you know, the whole patent and trademark and all this other stuff, but our trademarks, I was like, I can't imagine someone being able to argue

correctly to confuse a baseball team with a piece of software. Yeah, different domains. I think that's spurious. Yeah. So I approached the CNCM said, Hey, we got this opposition. We want to fight. I asked our legal team, they think it's going to cost around this much. And the CNCM at the time was not willing to take that fight on and

suggested that we just change the trademark and that if we do that and make it easier on them that they would give us a keynote slot so that we could announce the name change. But I was wed to the name. I cared deeply about the name and the logo and you know. For obvious reasons. Yeah, because remember this had been gone. it was named and we kind of started to have a logo for almost eight years before we officially joined the CNCF, right? So it had a probably longer lifespan than people realized prior to the CNCF. And so we fought it, right?

And on our own dime as a small startup that was only a year old, had just gotten seed funding and we successfully defended it. And so upon completion of the successful defense, the trademark was awarded to Sanadia with the deal. And so at the time, when I was talking to CNCF, they were like, well, we own the trademark. I go, well, technically you don't. Now this will get listeners frustrated and upset, and I get that, and I'm totally empathetic to why you would get upset.

And I'm not asking for empathy by any stretch, but I was very frustrated that something that they said they cared so deeply about, they threw away in one email and said, oh, no, you guys figure it out. And then to say, oh, but congratulations, you won. Now give it to us. That also rubbed me the wrong way. But technically what the CNCF was saying is, no, the commitment was you were going to give it to us. And so I

definitely understand their position as well. And we are committed to that. Right? So in part of our agreement, we're going to transfer everything over, the lawyers are working out the legal docs and things like that. But that was kind of my frustration, you know, to be transparent. And then they never asked for it after we successfully won that. And so, again, the way things were conflated and released out was, I think, very targeted in terms of trying this in the court of public opinion. I've got two questions, but one I'm going to pause on. When anyone, a project, maybe this one in particular, but any project, decides to take

that project, donate it to the CNCF, and do all the things you did, incubate, et cetera. Is there ever a point where you sign a specific agreement that's clear? So I feel like Jared, you've heard me say this before, clarity and expectation. Like if you have clarity on an agreement, it's a whole point of an agreement. It's less about binding you, but more like here's the line we both stand on. This is what we both agree on. Was there any clarity or is there a signatory scenario

where you're signing and it's clear and this will happen and there's a series of events? Was that, did you ever sign something? We signed something to get into KubeCon in 2018, which had a line that says, if you have a project in the CNCF, you promise, I'm making some up, but I think it's online. You promise to abide by all the rules for CNCF projects.

That link, I still can't even get into that link today. So I think there's an opportunity here to improve that process. Also, I would imagine the listeners are very well aware of those emails that say, our terms have changed or terms of service have changed. You get all these emails and most of us don't even read those. I never saw any of that. I'm not saying they weren't sent,

but I never saw any of that to say, oh, hey, what you think Derek from the kickoff meeting, we changed it and now it's this. I think there's an opportunity to make that a lot more clear, but most people who are coming from an opposition to me say that's always been the case. That should have been crystal clear to you that you donate the trademarks and XYZ.

And I understand that because they see the CNCF as it has evolved. They didn't see it when it was first getting kicked off. And so I think, you know, they're totally right in their assertion that I should have known better, but I still think there's an opportunity to make it a lot clearer. Like even now, I can't remember if it's as soon as you get part of the CNCF or once you move past Sandbox.

I think if Sandbox, you can take it back. If you don't get past Sandbox, I think, but I could be wrong there. And so even with all of this going on, it's still not necessarily clear what all of the rules of engagement are, but I just see that as an opportunity to improve for everyone, if that makes sense.

Yeah. Well, like in that case there, if the link is broken or there's some sort of like Language that's compressed. There's a whole other Document that's compressed in like four words essentially you just said and so those are usually Addendums to an agreement, you know this because you're probably signed tons of agreements in your life Those are usually addendums and that's usually compacted into a single product this the is the agreement I'm signing. And if there's terms beyond the agreement I'm signing, they're attached as an addendum to kind of,

to help everyone safeguard themselves from scenarios, potentially like this. Okay. So my second question is this, is it sounds like y'all have an agreement. The blog post is out there. It sounds like Kumbaya to some degree. Maybe not. Is any of this conversation

you're having with us right now going to get you in trouble? More so. I don't know. Because you're being very clear. Yeah, I'm trying to be very clear. For good reasons, but you're also being very clear. And it could be very clearly wrong clear.

Not wrong in terms of like truth, but more like, Hey, come on now. Don't say that in public on a podcast that gets listened to by a lot of developers in our world. Yeah, I hope not because we're entering into this in good faith. And I've tried to make sure to interject that I'm extremely encouraged by the discussions that we have committed to transfer the trademarks over. We actually did finalize early this morning, which I'm extremely encouraged by the discussions that we have committed to transfer the trademarks over. We actually did finalize early this morning, which I'm happy to talk about, you know,

our actual direction in terms of whether we're gonna do a fork or what commitment we're gonna do to the server, X, Y, Z. And so we're operating in good faith and I would imagine they are as well. And it's a trust but verify, you know what I mean, type situation. But I'm very encouraged by them being open and receptive

to some of the challenges that companies like Synadia or project like Nats might have. And hopefully there's some meaning of the minds around pass forward that might level the playing ground if that makes sense. Yeah. But I would imagine, you know, there's to be some people that are going to pitchfork again, and that's totally their right.

But that's how things went down and that's where we are right now. I still feel deeply about the Nats project and where it's going. One of the other things that listeners might not understand is there's different, in my opinion, there's different personas to projects that are kind of on maintenance mode, meaning we'll fix bugs and if there's other stuff, but the features are pretty much complete. Nats and the server are definitely not in that camp, right?

We've got over two years worth of plans and still the majority of our R&D budget goes into stuff that sits in the Nats.io ecosystem, right? The part of that open source project that sits inside the CNCF from all of the clients, obviously the server, a lot of tools, Kubernetes, sound charts, SystemD, all kinds of things like that.

We're still probably spending 70 plus percent of our R&D budget things like that. We're still probably spending 70 plus percent of our R&D budget just on that. And that hasn't changed, right? And we don't plan on slowing that down. It was, hey, is there an opportunity for us to recoup some of those costs for people that care the most about some of those crazy enterprise features,

massive scale security, whatever. And that's kind of how we started to go down the path of, hey, what if we did a commercial fork, but may get BSL. Again, it didn't land well because of the narrative, but our thought internally was that to us signaled the biggest commitment to OSS and to our customers trying to avoid vendor lock-in or price gouging.

But it was in a vacuum, and so once we got additional information, we realized that we probably would have to reevaluate that thought process. So how would the commercial fork with BSL be different than OpenCore? I'm not understanding that. We would be on, BSL also is source available.

It's not an OSI compliant license. So NAS will be the entire NAS project. Just the server. We would put all of the effort into that for, which meant that once it was released, then the two year window started. And so then that would flow into the AP2 code base that's covered by the CNCF.

That was the original thought process. Now we're not going to do that. Yeah, what's wrong with open core though? It seems like you could just accomplish the same goals without relicensing. You just have, you know, you want your billion transactions or messages per second, then we've got an add-on for you or whatever it is. Yeah, you're absolutely right and what we've heard from the ecosystem is we like that model because that's what's familiar to us. What I try to point out to people is that commercial forks, it's totally up to Cinedia

and maybe that's totally fine. And people are saying it's fine. But again, I don't think folks think all the way through some of these things sometimes. It's at our discretion what we put in there. And it's also our discretion that we don't have any commitment to put it back into the open source community whatsoever. So let's say there's something Jared that you really, really wanted and you're an avid user and stuff and Canadian sides, oh, we're only going to put it in the enterprise version. You're stuck. Now, of course you could try to do your own version

of it in the OSS side, but at the time, and again, this was in a vacuum and now we have additional information, but I was arguing hard that the BSL would signal a better commitment to saying, everything will become AP2 after two years. No matter what we put in there, it will become that way. So that was our original thinking. We knew that was going to be a fork. The question on whether or not we could use the NAT's name,

who it was, part of where we lived in the CNCF or if it came out and who technically owned that trademark. But we already knew, we knew that that was what we were, we could do. It's just if it's commercial, we have no, you know, no guarantees to either our customers from vendor lock-in and price gouging or the OSS ecosystem. And so at the time I was like, ooh, you know, I feel this is a better signaling mechanism. But we needed to control the narrative, meaning once we actually did decide, and that's the other thing that folks might not know, we hadn't finalized our decision when the can of worms got opened up. And so everyone's like, you got to tell us what you're doing. We don't even know what we're doing yet. We were just still having internal discussions on that because it was totally separate subject

from the relationship with the CNCF. Well, do you know what you're gonna do now? Are you still trying to figure it out? No, we've been meeting every day since the debacle started. And so we already had gone down a path of doing composition bias in ADIP. So what I mean by that is that a path of doing composition via Synadia IP. What I mean by that is that a lot of customers came to us and said,

hey, we want to take control over AuthNNLC. We have a pretty rich AuthNNLC system, but they go, it's not enough. We need something else. You could imagine a world where we said, we're going to do a Synadia server and we'll put all that stuff in there somehow. Instead, and this is already done, this code was done probably a couple years back, we did an auth callout mechanism, meaning the AP2 server, you can

say, hey, you're no longer in charge. Something over there is in charge, right? And then we Synadia could build lots of IP around, you know, for auth end, you know, for often, you know, XYZ, OPA integrations, things like that. And so we already were going down that path. So our decision now is, is we will not do a BSL stuff. You know, we know we have the right to reserve the right to do a commercial fork. But I think what we're going to do is we're going to post and we can do this in the show notes of everything we've committed to go into the AP two server without any delay, the BSL would have had a delay on there. And that we're going to

expand our commercial offerings through composition, meaning you're running the server, but now all of a sudden it's got security appliance, which we've been working on and some really interesting stuff. Think of it like a WAF for NATats plus a whole bunch of other stuff. And so we're going to go down that path. Well, I can tell you, there's so much nuance in this history, nuance, good intention, emotions, not a lot of bad intention necessarily, but maybe some confusion, some lack of clarity. OMG, please fix that stuff. When, when a subject matter is very emotional, right?

It puts your brain into fight or flight, which means that you have to make decisions very quickly on imperfect information. That's how we all survive, right? That's why, you know, we're all here because we ran away from the lion instead of going, Oh, look, a cat, let's put it, you know, type stuff. So I'm not being critical of folks doing that, because I understand why they're doing that. But hopefully they'll understand that they definitely had imperfect information and not even

close to a complete story. And it's not their job to understand all of that. But at some point, hopefully they go, oh, there's a lot more to this than we might have realized. And not saying we're right or wrong. And I'm not doing that at all. I'm just saying it's worth a dialogue. and there's a lot more pieces to the pie that folks might not know.

And to the credit, I think of the potential pitchfork folks or the folks in the crowd, not in the arena, is that it seems to be almost a, not literally a daily occurrence, but pretty frequently these days, something happening around a license not literally a daily occurrence, but pretty frequently these days, something happening around a license change or a rug pull. It's so much that it's become a thing we've said here, rug pull, not cool. That is our phrase. Okay. Rug pull, not cool is totally not cool. And that's an additional one, Jerry, we can put that one on a t-shirt. You know, I think they're used to, and we're used to hearing this and you think immediately negativity. You immediately think this is nefarious,

there's something going on here. And so it's almost as if there's a level of PTSD in the community. And I think as purveyors of and participants of, enthusiasts of open source software, they have a right to that in a way. But also there's so much nuance here that they have to let some of the details percolate

and it's not always clear. So. And you're right. This is why we do this podcast. This is why we have you on this show. Like, like I said, this is yours and I'm making is we're going to have you on the podcast anyways. We've been talking on LinkedIn about getting you on the pod.

And then this happened. I'm like, well, Derek, can you talk? Are you able to pod? I mean, I don't know if you can even pod. And you're like, yeah, I'm cool. And so we had you on the pod. And that's why we do this. So we give that clarity.

Yeah, and I knew the rug pool stuff. I've been listening and I know like, Euron was talking about it and others have been talking about it. And so I totally aware. And what my hope is, is that while we're focusing on, you know, license change and road pools that we uplevel to have a dialogue of, why is

this happening? What what are what are we as an ecosystem failing at? And what can we improve upon? What, you know, I said earlier on in the podcast, charity is not a business model. Well, until you're a company that is self-sustainable, you're not just running on VC dollars, that's still kind of a charity model. So in other words, right now,

our customers are funding a lot of the development, but the VCs who have funded Synadia are funding a lot of the development that everyone feels entitled to for free, right? And so my hope and, you know, my sincere hope is that we can uplevel and have a dialogue about why do we think this is happening? And are there things that we could potentially improve from an ecosystem perspective that could could do that. And again, I'm not a fan of this, but one might be, no, we don't want any projects that run critical infrastructure to be, you know, from just a bunch of folks and

or a single company. I don't think that's right. But that could be one answer. Or, hey, if you're a big, well known Fortune 100 company, and you're using something in critical production, give back. You know, it kind of like your rug pull thing. It's mine is, you know, no pay, not okay. You know, you type stuff. I like that one too.

You know, because again, production side is never zero. Someone has to put in time and money to produce this, but on the consumption side, right, we're getting to everyone expects it to be for free. And that eventually has to be for free. And that eventually has to be resolved somehow. Now, for projects like Kubernetes, this resolve because the Googles and Microsoft's and, you know, Amazon's of the world, right, are motivated to put stuff in and they have other revenue streams that can offset the cost of participating in the software. That's totally fine.

I'm not saying that that's wrong. I'm just wanting the dialogue of saying is that the only model that we really think will survive and I don't I don't want A world that looks like that myself. Yeah, well good for you for Sticking up I suppose for yourself or trying some attempt and having this clarity I know the narrative hasn't always been your favor, but I appreciate you coming on the show Even if it gets you in some trouble, I sure hope it doesn't Just coming on here and being clear. And you know, honestly, I think not that it's here at the end, but all along, anyone from the CNCF

who represents, you know, sandboxing, incubation, graduation and the clarity there, and maybe some of the things we brought up here, which was the agreement you signed and the terms that you alluded to and the lack of clarity there. I'm happy to, and we're happy to have anybody on the show. I think it's good for everyone to have some of that clarity.

So if you want to have something on the show and talk about the details of what it takes to join the CNCF, what those are, we would be more than happy to have that conversation in detail. Yeah, and I think that's great. I really do. And like I said, my hope for all of this is that

we can have an open dialogue about are there things that we can prove upon outside of CineD, outside of the CNCF, but just as a broader spectrum. I think there's some interesting questions that should be discussed on how we want to go about that, and what role the CNCF and the Linux Foundation plays, what role, you know, retainers play because one of the other things that we got a lot of pushback from is I feel deeply that the lifeblood of a project is in the maintainers. It's, yes,

people who want to concentrate on license, what foundation are you part of all that other stuff? And I think they do play a role. But at the end of the day, it's the maintainers and if also the maintainers withdraw their support from a project. I think a lot of projects that have a tremendous amount of investment and are not just a simple weekend toy, if that makes sense. You know, saying, oh, we'll just get some maintainers and they'll spend a weekend and then they'll know how to program, let's say, a NAT server, right? I think that's maybe a little short-sighted.

You know, we have someone who's awesome internally that just got promoted and is now running the server team. You know, they graciously took that from me so that I can do other things. That person came in and was under our tutelage and specifically under my tutelage and it still took them about eight months or so. So we were paying them and we were tutoring them

and most of the server's infrastructure was built by myself and someone named Yvonne and we were tutoring this person. It's still took about eight months, almost a year to become like, I can go anywhere on the server code and be effective, be productive. When you get statements that, fork it, you guys go do your own thing, and then we'll archive it or try to find new maintainers.

Again, I want folks to kind of really think through that and say, is that the best outcome for the ecosystem? I don't think it necessarily is, but someone from the CNCF might say, yes, that's absolutely the way it's supposed to work. But what people are looking at to the CNCF around stability for projects and longevity, you know, if the maintainers leave, right, there's going to be some level

of destruction for sure. Depends on the project, but there's, you know, it's not going to be zero. And in some cases, you know, in our, in our point of view, which people might disagree, Nats thrives when Synadia thrives, right, because we're driving all of that stuff. At least that's, you know, my opinion. And again, not all projects look like that. And that's totally fine as well. But that, that's another one where I was like, that doesn't feel like

it should be the only answer, right? That there could be a dialogue of how do we better support these these types of projects? What if they're not necessarily good fit as a foundation evolves? You know, what options are there for doing that? And again, I think prior to the the the dust up, but also going forward, and again, we'll release a statement next week, probably about whenever the podcast goes out, around our commitment, you know, most people might not understand, you know, of our total R&D budget, 70% is going into open source.

And it's funded by the VCs who fund Synedia, and it's funded by our customers. And the ones that can pay, that aren't paying, that are like, give me, give me, give me, you know, I'm entitled to this, you know, hey, get off your ass and change this thing for me, which we see every once in a while, like GitHub issues. That's kind of what I'm like, maybe we need to figure out a way to have them change their perspectives a little bit. I'm not saying, you know, I don't want hobbyists to be able to use our stuff for free or embed it all over the place, right? My original tagline for Nets was connect everything, meaning it had to run everywhere and was openly accessible.

But again, it's that production problem, which is non-zero. It's like, how do we solve that problem for different levels of projects and open source? For sure. And I don't have all the answers, obviously, as you can tell, but I do believe- I mean, I think the important thing is you have a lot of thoughts, you have a lot of care.

And I think that's the thing you have to really come back to is you've got a lot of care for what you're building, what you've built, the community, the spirit of open source, your customers, your employees, your investors. I mean, there's so much things I'm sure that you're holding that we're not privy to that we can definitely see that you care. But it's cool having you on this show talking about this stuff and I'm sure that you're holding that we're not privy to that we can definitely see that you care. But it's cool having you on this show, talking about this stuff, and even more cool that you're a listener and you know,

road pools are not cool. I know. That's right. And you know, no pay, not okay. That's right, no pay, not okay. You can have a t-shirt with one on the front and then that on the back. That's right.

It'll bounce. Represent both sides. That's right. Okay, get it together. Thank you so much Thanks, Derek. It's me. I appreciate it. Thank you guys and you know, congrats on all the success really really cool to see oh man Thanks, Derek So my takeaway from this conversation specifically around the kerfuffle with the CNCF and how that all played out is that there's a lot of nuance in these things. There's a lot of detail hidden behind the scenes, a lot of time spent, emotion,

investment, time, energy, and all the things to really complicate the relationships in and around open source and ultimately any commercial open source companies that spin up around open source. It's not just black and white. There's a lot of details and this conversation really helped me see Derek's side of things. Now with that being said, of course, the conversation is open. We would happily have anyone from the CNCF discuss with us the incubation process, what it takes to graduate, the clarity they offer to anyone donating

or assigning IP or copyright or a project to the CNCF because there's a lot to discover there. There's a lot of detail in there. And again, it comes back to that nuance. So if you're listening to this from the CNCF, reach out to us, editors at changelog.com. So a side note, I suppose for next week for our loyal listeners and those who listen to the end of the show, that's you listen right now. Yeah, that's you. We're out next week. We're in Seattle

at Microsoft build. I'm not sure what the publishing schedule will be. We're heading into summer. It's the last week of school and I feel like everything needs to be done right now. With that being said, if you're gonna be at Microsoft Build, make sure you say hi. Hop in Zulip, go to changelaw.com slash community, sign up. It is free to join. App mention us or whatever on the socials. DM us, email us, all the things, whatever.

Just say hello. Of course, a big thank you to our friends at Heroku for sponsoring this episode. The new generation of Heroku is so exciting. Heroku.com slash changelog podcast, our friends over at Retool, retool.com slash changelog, and of course, the Beat Freak in residence, break, master, cylinder.

Okay friends, that's it. This show's done. We'll see ya.