Screaming in the Cloud - Avery Pennarun on Tailscale's Evolution: From Mesh VPN to AI Security Gateway

Episode Date: January 8, 2026

Corey Quinn sits down with Avery Pennarun, co-founder and CEO of Tailscale, for a deep dive into how the company is reinventing networking for the modern era. From finally making VPNs behave the way they should to tackling AI security with zero-click authentication, Avery shares candid insights on building infrastructure people actually love using, and love talking about.They get into everything: surviving 100% year-over-year growth, why running on two tailnets at once is pure chaos, and how Tailscale makes “secure by default” feel effortless. Plus, they dig into why FreeBSD firewalls needed some tough love, the uncomfortable truth behind POCs, and even the surprisingly useful trick of turning your Apple TV into an exit node.About Avery: Avery Pennarun is the co-founder and CEO of Tailscale, where he’s redefining secure networking with a simple, Zero Trust approach. A veteran software engineer with experience ranging from startups to Google, he’s known for turning complex systems into approachable, user-friendly tools. His contributions to projects like wvdial, bup, and sshuttle reflect his belief that great technology should be both powerful and easy to use. With a mix of technical depth and dry humor, Avery shares insights on modern networking, internet evolution, and the realities of scaling a startup.Highlights:(0:00) Introduction to Tailscale and Security(00:52) Sponsorship and Personal Experiences(02:07) Technical Deep Dive into Tail Scale(06:10) Challenges and Future of Tail Scale(22:45) Building the Tail Net's API(23:54) Connecting Cloud Providers with Tailscale(25:22) Tailscale as a Security Solution(26:44) Innovations and Future of TailscaleSponsored by: duckbillhq.com

What's very strange about Tail Scale and very strange in the security world in general is that when you use Tail Scale to solve that problem, you accidentally make your system more secure. And also, the easiest thing for all of your engineers and people inside your company to do becomes the secure thing instead of the insecure thing. Welcome to Screaming in the Cloud. I'm Corey Quinn. It's been a while since I've had AveryPething. on the show. Thank you for joining me again. You are still the co-founder and CEO of Tailscale, which at this point is getting pretty darn close to you've heard of this company when I bring it up in almost every conversation I'm in. That is pretty exciting. I think I can't remember when I was on your show last time, but it was at least a couple of years ago and we've been growing really fast in the last couple years. This episode is sponsored in part by my day job,

Duck Bill. Do you have a horrifying AWS bill? That can be. a lot of things, predicting what it's going to be, determining what it should be, negotiating your next long-term contract with AWS, or just figuring out why it increasingly resembles a phone number, but nobody seems to quite know why that is. To learn more, visit duckbillhq.com. Remember, you can't duck the duck bill bill, which my CEO reliably informs me is absolutely not our slogan. in more and more places. I've been using you in my personal environment for many years now and the stuff that I set up once upon a time is still working. You're rolling out new stuff. That continues to be additive to this. At work, I'm paying you now, which was a big problem I had with you previously of

there's no good way for me to give you money. Could you maybe fix that? Good job. You fixed that. So things are all up into the right, which is kind of amazing. It is kind of amazing. It's amazing how long we can keep doing it. Although I've been informed that if you keep doubling revenue at 100% year over year, then in 10 years, you'll be a thousand times bigger, and that might not be realistic. But it might. At some point, you hit population limits. Last year, I gave the opening keynote at Nanog 91, and the whole theme of what I was

talking about back then was that there's been a rising tide in the level of what clouds could take over from folks who are working on-prem environments. Networking is becoming something of a lost art. When you find someone who works as a network engineer, they're usually my age and not new grads who are playing around with these things. Tailscale is in some ways an answer to some of this, where you're taking things away from the traditional network switch

and router world and into just make a big flat network and then we'll wind up handling this through policy files for access control. Even recently this year, you folks wound up redoing your policy format as far as making it a lot easier to do grants, with access grants, as well as now creating a visual builder, which I've not yet played with, because I haven't found a way to make it work in VIN yet. One of my fixations as a CEO is I insist that every change to the policy file get run by me.

Almost nothing else in the whole company runs by me, but I'm like, no, if you're changing the policy file syntax, I want to see it first. So we went through a lot of iterations of the ACL grant syntax before we finalized it, and I'm really excited about what we came up with. I realize it's a little strange to be really excited about a file syntax, but I actually am really excited. And I think a little understood feature of ACL grants is it's really extensible. Like you can grant stuff to applications that are provided by people that are not tailscale

that are running on your tail scale network. And when you connect to that application over tail scale, it has visibility into the grants that you gave it based on your groups, the tags, the blah, blah, blah, whatever is going on in your tail net routing. And it doesn't have to know about, it doesn't have to know what group you're in. It doesn't have to have its own business logic about what group you're in. It can just say, like, Tailscale says, this connection should be allowed to do this thing on this thing. And you can change all that in a central place.

And so, an easy example is like Grafana. You can say, today, everybody in the production group should have admin access to Grafana. So when they connect to Grafana, they get admin access. You don't have to set anything up in Grafana. If you change your mind later or you change who's in that group, then next time they connect your grafana, even if it's like three seconds later, their permissions are going to change. right? And that was not possible before we had this ability to just sort of like pass these things through. And so it gives you this ability to just build on top of tailscale and just stop worrying about all that stuff.

Yeah, you also have a great feature where you can effectively disallow people from modifying things in the console without going through a whole bunch of very scary warnings and mandating effectively a GitOps flow, which is fantastic, especially combined with the fact that you have test cases built into your policy files. Exactly. Well, super fun, right? Because you just said, like, you can't use our ACL editor because you like VIM. And like, that's not actually true because we did this another very nerdy thing i'm super excited about is that you can round-trip the jason of the policy file to the gui editor and back with no loss of anything and it's not just regular jason it's our special weird huge jason that has comments and extra commas which means you can actually have comments in your jason describing what goes on and then when you go to the gui and then back the comments don't get lost and so you and also gitops can like take this text store it in GitHub and then push it back when you're done. And then, of course, it's not very good to have

the GUI edit it in tailscale if you're using GitOps. But you can go to the GUI, come up with the rule that you want, or an edit that you want, and it'll tell you what text to paste back into your Git repository to get the results you want. So it's this very nice flow where like everybody who likes everything gets to have what they want. It's weird. It's just thinking back, it's been a bunch of small releases, but they add up to almost that completely different product that still does the underlying baseline thing it always did, which is flattening the network to make it work like we all used to think networks did until we knew better. It's a, it's been a very fun evolution. I think it was last year you did that partnership with

Mulvad, where I think for five bucks a month now, I can get, I get access to the Mulvad VPN stuff. It's a couple of clicks of a mouse and I'm suddenly emerging from anywhere else I want to be, which is super handy for me. And my brother who lives in Brussels, we're, but we both have EU and US citizenship. So, but there's an awful lot of government sites that, oh, you're not physically here. clearly you could never want to access these things for no apparent reason. Trivially easy. Also, my bank in Canada, whenever I go traveling anywhere that is not Canada, they're like, oh my God, nobody outside Canada could possibly need to access a Canadian bank and they kick me out.

But I, you know, I can do that. What I do is I use an exit node on my Apple TV at home, and I just bounce through my Apple TV. But I also use Mulvad for experimenting and stuff. I did that originally, and then the Raspberry Pi, I sent with my brother to his place, wound up dying, and that is not, he's a government functionary there. He's not really the technical type as far as, hey, now log into the Linux console and tell me what you see. No. That's why I went with the Apple TV, because they, you know, they're five times as expensive at least as a Raspberry Pi, but they have five times at least as much quality control as

a Raspberry Pi in the manufacturing process. And a warranty service that is comprehensible to humans. Yeah, and a GUI where you can just tell your brother like, hey, can you go to the App Store and pick tailscale, as opposed to going to the console. Yeah. I also like things that are, that have changed, some things have not changed in tailscale. They're still somewhat annoying. And I understand why. I'd love to be available to connect to two tailnets at the same time. Now, you can be logged in and toggle between them, but a device that talks between two networks is generally considered a bridge and corporate security would like a word if you start doing that. But there are ways now to share nodes between

tailnets that start making that a lot more straightforward. I would still love on some level the ability to set a custom domain for the tailnet domain that I can control the certs for. I get that that is a hard thing to do. I'm sure some big customer somewhere has it, but yeah. Yeah, it's surprising how it's, well, that particular feature, it's a little hard to do. I would say the difficulty of doing it is not actually the thing holding us up. The thing holding us up is the fishing potential when you start doing it. Because you combine that with tailscale funnel and people register some arbitrary domain that looks suspiciously like but is not quite Google.com. And next thing you know, you're hosting fishing sites for Google.com, right? If everything ends in blurbbyblurb.t.tts.tys.net,

then you don't have that problem. And it's like remarkable how much trouble that saves us. So we really want to get to the custom domain thing. We just need to like very carefully control who gets to have custom domains and minimize the abuse potential. One way is like to attach it to not easy. All the people that pay you. That was one of the things we've been thinking. It's actually, it's, I mean, it's a pretty good start. We should probably do that. Exchanging money for goods and services?

That's wild. Yeah. Yeah. It's just they do we need to live in it to only those people. It's kind of sad to have to do that. I wish we had a better idea. But like, you know, nevertheless, yeah, it's definitely on the list. Similarly with sharing, we've been in the same state with node sharing since I think like

2021. And a bunch of internal changes have been going on architecturally to finally like enable way more kinds of interesting sharing. But I really see, like, there's so much potential to newer kinds of node sharing. I don't think you ever want to be in two tailnets at the same time. I realize that everybody at first thinks you would want to do that because it would be really tempting. But it is this bridge between tail nets, and it, like, really confuses, like, as an example, I would like to be in two tail nets at the same time.

I have a personal account with my family stuff on it, and I have a work account with all my work stuff on it, and where I'm the CEO that has access to a bunch of sensitive things, right? Now, if I'm at a computer with my... And maybe your children should. should not have access to those same things. Yeah, maybe they shouldn't, right? So if I have a device that my children borrow, like an iPad or something like that,

I really should not be logged into that device using my tail scale account. But if I'm on my corporate device, I really would like to have access to my private stuff, because why not, right? But if I'm logged into both tailnets at the same time, now I'm inadvertently creating a bridge between my corporate account, so the security team should lock me out

and my personal account, right? And the security team incidentally almost locked me out a few days ago, because I wasn't on the MDM yet. So I like forcibly enrolled into the MDM, which forced me to upgrade my MacOS. And there's a bunch of new features in MacOS that I was missing. So I guess that's good.

And that yak is getting nicely shaved. Yeah, so I'm pretty far down this path. But, you know, anyway, what I think people want, and what I want to give people, is the ability to log into each device using exactly one account, and for you to be able to share many or all or a good subset of the nodes from another account

into your account. You're almost taking the GitHub identity model. Yeah, I guess so. Yeah, I have a GitHub account, but I can add to different organizations that do different things. My personal account is also what I use for work, but you can gate access to things. That part makes me nervous. Like when I log into GitHub, I have access to all my corporate stuff and my personal stuff. So if I log into my personal GitHub account, when I'm not on a work computer, I'm like putting work at risk, which is scary.

So what I think we should do is still have the two accounts. But on my personal devices, I log into my personal account that doesn't have access to my corp stuff. But I log on my work computer, I've access to all my corp stuff. and my corporate user also has outgoing access to my personal stuff. Yeah, for the last eight years, I haven't really had anything personal because my entire life has become work. Right around the time that shit posting on social media became a job. Yeah, I guess that makes sense. So yeah, I mean, yeah, I'm really talking about the experience for other people.

But yes, I mean, I have an Apple TV. Does the corporation want my Apple TV on the corporate network? Like, not really. So little things like that. And I think we can do it. We're getting very close to being able to do it. We just keep like doubling in size a lot. And so most of the engineering that we do actually ends up being just like,

hey, you now have like a tailnet with hundreds of thousands of nodes on it with like thousands of nodes churning per minute because someone is using it in a gigantic CICD cluster. Did you know that's an N-squared algorithm? Did you know that the whole system is going to crash because you did that? I'm like, oh, I didn't know that. But then we had to fix it. We learn exciting things through other people's use cases.

Exactly. So some of this stuff keeps getting delayed. But it's going to be a really good one to find a couple of things. Yeah. And you have a great list of customer references that are doing all sorts of fascinating stuff, some of whom I know reasonably well. And what I've, I also like the fact that there are options if tailskill isn't right for people, if you want one that is a lot more confusing, a lot less capable and much more expensive. I mean, AWS has launched VPC lattice. And then they've marketed it so poorly that people don't know if I'm making that up or not. Yes, I actually had not heard of them. That is maybe embarrassing. No, this is par for the course. I thought it was great when it came out. and then I forgot it existed, and then it just goes years without being mentioned by anyone until I encounter. It's like, oh, right, that exists. That's kind of neat. I should look into it. And every time I do, I come away with, or I could just use tailscale and save myself a lot of heartache,

so I do. Honestly, on some level, your next go-to-market for enterprise, you'd just be offering people a free month, the VPC lattice. We've actually had that a few times, and there's like a comparison. And we're like, can we please be like first while you're doing the comparison? and then you can, you know, install the other ones later. And they do, you know, they're done with tailscale in like 15 minutes. And then they go off and they try to install the next one. But if they try to install the next one first, they might never get to tail scale, right?

Because they don't finish. That's the dark secret of POCs. Yep. You've done a fair number of things that are, it's hard to even describe what tail scale is. You have tail drop, which is effectively an end-to-end file sharing option. It feels like you are flirting with, because, almost a service discovery tool. We have enough service Mesh in the world,

but it feels like this one makes a strong contention for being one. Well, I have, we're trying out new versions of the mission statement, because previous ones were too complicated. I will present a preliminary version that we've been trying out. It is a new layer three for every device everywhere.

It's like maybe too simple. You have to be a network person to know even what I'm talking about by layer three. I tried like new internet protocol. Sometimes people are afraid of that, it's not like IPB7, but it does the job of what layer three, the internet protocol, was supposed to do.

And let me try to explain what that means. So way back in the day, when I logged into the internet, I could connect to any device anywhere that was on the internet by using its IP address. That has not been the case for now decades, right? It's now gotten to the point where, in fact, the only things I can really connect to by IP address are like maybe my Wi-Fi router,

if I can remember what the IP address is and I'm in my house, or CloudBron. who own like most of the public IP space at this point. And that's like kind of weird. That defeats a lot of the purpose of the internet. Another thing that happened is if you have a, even if you had that connectivity, imagine you had IPV6 rolled out everywhere, which requires a bit of an big imagination.

But let us imagine IPV6 was everywhere. If I switch to a different network, like between Wi-Fi and cellular, my IP address changes. And now the connection breaks. And I actually can't find that device unless I use DNS. everyone's best friend, DNS, the thing that is not anywhere in the OSI stack, but is somehow playing some job

making some of the layers of the OSI stack work together. So now I'm like dynamic DNS, I'll just update it every time my phone jumps between Wi-Fi and cellular, like not likely, right? And so like the actual inter-networking part of the internet stack does not work anymore. It's not location independent, and it doesn't make everything in the world addressable to me.

Right? It's actually layer two. It's just a replacement for Ethernet addresses because every time my interface changes, the address is a different thing. It might as well be an Ethernet port, right? And it hasn't done this job that's like missing from the stack. And so Tailscale jumps in there

and it's a tunnel, but it's like, hey, it works the way it's supposed to work. Like, obviously the world has changed. You don't want everyone in the world to be able to access you, but everyone I want to be able to access me gets an, it can find out my name, and I get a fixed IP address,

and I'll make it work everywhere and it doesn't change when my device moves around. So Tailscale, all the stuff you can talk about, but the thing that it does is it actually produces layer three of the OSI stack for the first time in decades. That's nothing short of magical. It's weird because this gets highly technical, highly quickly and goes very deep.

But it is stupid simple to get set up. We were just traveling in France, my wife and I, and she wanted to access something that was only available from home. Great. Hand me your iPad a second. I didn't even bother to have her set up an account. I just logged it into my tailnet, so now she can get access to my shit posting nonsense if she really wants it.

And suddenly it worked when I turned it on as an exit node. I've also found, and this is what really sparked the idea of having this conversation now, is with, now I have a test Kubernetes cluster that mostly works. I have your provisioner, the operator that automatically gives access to any service I put on the thing. It's got some drama when the nodes themselves are on the telnet and that becomes, and their magic DNS becomes their resolver. It tries to pass those out to containers and that becomes a little bit of a, let's patch core DNS to make it not do that. but once I do, I can spin up arbitrary containers and not have to worry about security, which sounds like a wild thing to say,

but the only place that those things are available is on the tailnet. I'm the only person, except for my wife's iPad, on the tail net, and even then I could restrict it down further via ACL grants. Suddenly I'm doing the thing that a lot of people used to do on the open internet of,

oh, I'm not big enough to find. No one will find this weird port I've bound it to. Only there is security. It's not just pretend security. Right. And that's another thing that like, you know, if, as again, IPV sex, if it had been fully rolled out today, still wouldn't solve that problem

because it was invented 30 years ago and there's been 30 years of new problems since then, right? So it's like time for like a thing past IPV6 if we could move past it ourselves psychologically. But like there has to be identity. There has to be security. There has to be a concept of like which things are allowed to connect to which other things,

not just the dream of the late 1990s of like, you know what? If everybody could just talk to everybody, the whole world would be happier and we'd have world peace and stuff. And we sort of learned from the internet that like world peace doesn't happen when everybody can like chase you around and harass you all day, right? And so you just need that level of security, but you want the feeling that we had on the small internet before, you know, most of the really bad people showed up. I think that's the right path. It's you have, I keep forgetting this because of course, in your case, you have to deal with a, especially with a free way to get started here.

you have to deal with a tremendous amount of abuse concerns on this. But it's also not traffic necessarily passing through you. One of the smarter things you've done from pure cloud economics perspective is you're the coordination central point, but the actual heavy-duty traffic is point-to-point. Yep, exactly. So tail-scale splits in network terms, we call it the control plane and the data plane, right? The control plane decides how to distribute the keys, how do you log in, who should be allowed to talk to which other people, and then it sends those.

instructions to every device in your tailnet and then the devices themselves handle the data plane which is sending the data direct whenever possible directly point to point between between themselves so it doesn't cost us anything to transport your data and it costs us very little to be the simple coordination point between the notes and this is what makes it extremely scalable and a lot of this stuff is based on some of the original concepts of the internet right is like look it should be extremely scalable you can't have like one company that is routing all the traffic for you such as AT&T back in the day with the telephone network right you just you know it works but you shouldn't have that. You should build a system where that doesn't happen. And tail scale is very

much moving along those lines. And it is kind of magical, especially because if you get two devices sitting right next to each other on your local network, they get direct connections to each other on your local network, right? Almost any other thing will try to beam it up to the internet and back, which is pointless in situations where they're side by side. And so if you've got a data center or a VPC filled with containers and they want to talk to each other, it's really silly to send all those things to the internet back to say nothing of like the egress views you'll incur. What's weird to me is also how effective you are at routing money to other companies. Through tailscale, I use Mulvad, as we've discussed. I also pay for NextDNS because that to where I do

most of my ad blocking, which makes it super handy when I try and hit something like a link in an email that gets blocked. Great, I could special case it, but why would I do that? I'll just toggle off tailscale, hit the thing I need to and turn it back on. I do that multiple times every day. You have become something I use constantly, but also almost never think about, which is the, honestly, the Valhalla of infrastructure. Yep. Infrastructure is really tricky because we have, you know, we're trying to balance word of mouth because you want everyone to brag about how they use tail scale.

And simultaneously, the best infrastructure is the infrastructure you never think about. So it reminds me, I forget the name of this, this trendy workout campaign from like 10 years ago where, like, the joke was, like, how do you know someone's on this trendy workout campaign? is like they won't stop talking about it. So Tailscale, people, people love their infrastructure so much that they will not stop talking about it, which is a very strange situation to be in. I did not see that coming when we started the company, but it's more or less what like drives the adoption of Tailscale. This episode is sponsored in part by my day job, Duck Bill. Do you have a horrifying AWS bill? That can mean a lot of things, predicting what it's going

to be, determining what it should be, negotiating your next long-term contract, with AWS, or just figuring out why it increasingly resembles a phone number, but nobody seems to quite know why that is. To learn more, visit duckbillhq.com. Remember, you can't duck the duck bill bill, which my CEO reliably informs me is absolutely not our slogan. Yeah, every time I see weird questions come through on the AWS subreddit, which I keep a loose eye on, it's like, oh, that sounds like a tailscale usage. And sure enough, it's always the first comment someone has there. Have you considered using tailscale for this? Like a sensible person, which yeah. Exactly. Yeah. And you mentioned like these partners that we work with and

writing money to them. Like tail scale is increasingly, uh, it's a little among, you know, in the entrepreneur world, you have to be really careful with this word. But we are increasingly a platform. And what is a platform? It's like the base layer of something that people build on top of. Right. And, uh, I was talking to our investors the other day and someone said like, Look, the advice or the most important thing to know about building a platform, and the biggest mistake almost everybody makes is trying to do it, and especially doing it too soon. Like almost no company ever actually builds a platform, and if you are wrong, and you go and build one anyway, you waste a ton of time and energy. And so we've been a little bit dragged into building a platform. I've started talking about last year how maybe someday Tailscale is going to evolve into a platform.

And then this year, we made a feature that's called the Tailnets API. So a completely automated way to create a new tailnet, add devices to it, and then spin down the tailnet, share it with other people and stuff, just entirely API-based. And so now we have big cloud providers that are like, you know what, I'm going to make my inter-cloud connections just use Tailscale in the background, and our customers don't even have to know about it. And I'm going to do it all using the Tailnets API. Right?

So we're kind of like, well, this is way ahead of schedule. Now we're a platform. And I don't even... Can you cheat it under the whole? to take specific decisions on the path traffic takes to get from point A to point B? Yeah. I mean, they're basically, well, the big, the problem space that these people are mostly in is like they're, you know, lower tier cloud providers. They provide, you know, the biggest thing is usually GPUs, right, at better prices than the big cloud providers have. And then customers like, ignoring the prices, they actually have them for rent.

Yeah, or more availability, et cetera, right? Or the right ones, all kinds of things. But then the same customers want to run the rest of their stuff and a more mature cloud provider. Now you've got a connection problem between like kind of weird GPU cloud provider and the top tier provider, right? And so how do you connect between cloud providers? Well, it's actually hard.

Almost nobody makes a product for that at all. These cloud providers, they could tell you, like, go use tail scale, but then you have to go figure out a third product that kind of slows down their marketing. So they're just like, you know what? We will provide the service of connecting you to anything. Don't even worry about it. and they just like set up a tailnet and suddenly their VPC on that cloud is actually connected to the VPC on the other cloud.

And it's the right path. What I have found that is so, I guess, compelling about all of this has just been that over the years, it has solved so many weird problems. And I continue to watch the logos on your site continue to expand to going from small companies to mid-sized companies like, I don't know, Microsoft. Yeah, Microsoft recently got added to our logo list. There's a bunch of other, you know, there's subsidiaries of Microsoft. There's a bunch of other big names. Most of our biggest names are still not actually in our logo list because we didn't get logo rights for them. People often...

That is always the way that it works. It's especially true in the security world because security people are like, wait, I don't want to advertise what our infrastructure is using for security. That's just like painting a sign on our back. Do you view yourself as a security product? tradition well so i've well i'm i'm stumbling on this because the correct answer is sort of uh or yes well whose cost centers are purchased this contract coming out of sure we're a security platform i get it go go where the money is i hear you are you an analyst no unless you have analyst

budget then yes yeah so tail scale i think the best term i heard for it is a mesh vpn firewall right um and the reason for that is most people who end up adopting tail scale adopt tail scale Adopt tail scale because it solves a connectivity problem that they have right now. And it becomes the easiest way to connect things. What's very strange about tail scale and very strange in the security world in general is that when you use tail scale to solve that problem, you accidentally make your system more secure. And also the easiest thing for all of your engineers and people inside your company to do

becomes the secure thing instead of the insecure thing. And nobody really sees that coming. But then once it gets there, the security people are like, wow, how come I'm not the bad guy? I'm always the bad guy. I don't want to be the bad guy. We love Tailscale. Most of the time today,

Tailscale is not adopted through the security team because the burning problem is not like blocking people from connecting to things. The burning problem is usually connecting to things. But you get both of the same time.

And that was like from the very beginning at Tailscale. Usually you have to buy like a connectivity thing, like a router or a VPN and a firewall. And they're run by different teams and they fight with each other all day. Honestly, I found that

the most people I talk to you the most, who are the biggest champions of Thalescale, the ones that are empowered to do the thing that they want to do. It's, oh, the policy is because I said so, the end. This doesn't feel like it's something's going to be instituted top down just because it's not painful enough for the user. We have a new experimental thing that we're working on that I think is really going to appeal to security team specifically as buyers.

And I want to run by you and, like, hopefully get feedback from everybody else who's listening. You can post my email address or my blue sky or whatever you want. So people get... You say that, and yet... And you know, I get hate mail. I've received hate mail. At this point, as a C, I get hate mail from my own employees.

And so, you know, the skin gets thicker over time. But yeah, so here's the thing. AI, I think we've all heard about it. People are deploying it in their companies. And often carelessly, believe it or not, they don't always think about all the consequences before rolling out AI. And yet, many companies, and some of them we've heard about more than others, but many companies have directives from the top down

to roll out more AI. So the CISO is sitting here and is like, wow, everything you guys are doing is horrible. And this is a ticking time bomb. And I can't believe that I have to say yes to this because my job is not just to block progress in the company. My job is to ensure security as much as we can. But if they say yes, it's like there's going to be a breach.

And if they say no, they're probably going to get fired because they're blocking progress. Right. I think a solution to this is when you want to end, oh, sorry, I forgot another part of the story, which is that when you're bringing AI into the company, that's one thing. The new trend in AI is this MCP protocol, model context protocol, that you can use to connect your favorite AI agent to your favorite data source no matter what it might be, or all of your favorite data sources, right?

When you do that, all kinds of terrible and exciting things can happen. And if you Google around a bit, you can find examples of like someone hooking, hooking. Oh yeah, the attack vector now is quite literally telling the computer, trust me, bro, in those words. And it's so, so exciting the kinds of problems you can have. Like some people hooked a GitHub up to this, and like the repo that it looked at contained instructions to the LLM that then convinced it to take the rest of the data in GitHub and send it to somebody else. It's like, wow, that's a super neat attack. As a security person, I can appreciate super neat attacks. But also, like, wow, what are you going to do to defend against this kind of thing?

And I think the answer is the LLM has got to be supervised, just like any person or any weird thing that you put into your network. You've got to have auditability, control, ACLs, identity, encryption, all that stuff that you should always have that you actually don't have today when you hook an AI-ups to stuff. The way to do that is to funnel your AI traffic into a thing that has the ability to audit, log, and control, and filter, and decide what you connect to which other things, and then forward it on through. And of course, tailscale is a connectivity and security layer that makes it easy to build such a thing and deploy such a thing. But then you have a really interesting other problem. And I apologize as this is getting weirdly deep, but I hope your audience loves weirdly deep things. Once you've got a proxy that is forwarding traffic from like, it's acting on behalf of Avery, say, on its way to Salesforce, right? Avery goes into the proxy.

The proxy then wants to go to Salesforce. The Salesforce says, is like, okay, you're a proxy. you have a like service account. What did we do? Do we set up the service account to have we have global access to Salesforce and then the proxy needs to be trusted to only give Avery the stuff

Avery should have access to? Well, that sounds like a terrible idea. But it can't act as Avery by default because it's not Avery, it's running as proxy and it had an incoming connection from Avery that doesn't give it rights to Salesforce. So you have to have this little interchange.

To avoid confused deputy that way. Yeah, exactly. So you have to have this interesting interchange where Avery makes a connection to this proxy and the proxy has the right to exchange that that identity for a token that allows it to access Salesforce as Avery with a little note on it that says, by the way, it's Avery's AI, don't give it too much stuff. So it's like Avery minus minus. To do that, you can use an Oath protocol that I won't go into. But it's like, you know, originally when

the MTP standard came out 10 months ago, I think, there was like almost literally this page intentionally left blank in the security section. Since then, there has been a improvement where they said actually oath should be the way you do this and then people started implementing that and now they're at the stage where like it tries to oath to like 10 different things and each of those things leads you to a click through uh to granted permission to do some stuff so with tail scale we have this neat feature where like every connection that happens on the tail network has your identity already attached you don't have to click through anything it's just like inside your tail net everything knows who you are every request inherently becomes

authenticated exactly so the trick we did is we wrote this new tool on top of tail scale called TSIDP, the tail scale identity provider. It's open source, by the way. You can look at the GitHub repository and fork it can do whatever you want. It's only a few hundred lines. And what it does is it's a complete OOP server, but the user side is just, I already know who you are, right?

So when you try to access a service, the service redirects you to your IDP, which says, I already know who you are, and then redirects it back. No click-throughs, but it's controlled by the ACL Grant policy we talked about earlier because it's just a tool on top of tail scale. We didn't have to modify tail scale, to make any of this work.

It decides which kinds of tokens it's willing to exchange on behalf of this proxy running inside your tail net, right? But this proxy of the TSIDP server can be accessible over tail scale funnel to the outside world. So you can even use TSIDP with any service on the internet that supports custom IDP or custom OIDC. So you have this really interesting situation

where from the very beginning, Tailscale is like, I'm not going to be an IDP. We're not doing usernames and passwords. Get out of my way. That's the past. Let's live in the future. Use a real IDP. You should still do that.

But you can use that to get into tail scale. And after that, you can use TSIDP to connect to everything else, right? And this MCP thing means your AI can do the same thing, right? And all of it can be zero-click because you can set a policy, your administrator for your company, you can set a policy on TSIDP to decide which things can be zero-click. Right? And if you're worried about sort of privacy, I know that you're worried about sort of privacy, I know, a lot of people who like use Google login with Google are like ah Google's tracking me all over

the internet now because i use login with Google every time i log into a service they know every service i use now google only knows that you use tail scale right because your instance of tsitp that you ran that is open source is the one doing all the rest of your authentication and so you have access to all that private information even we don't because it's just a tool right built on top of tail scale and so the combination of all that stuff allows you to like control your AI access, but it also lets you have zero-click authentication to like everything on the internet, if you want. And it also lets you have zero-click appendication to things on your tailnet that don't understand tail-scale. All they need to understand is custom OAuth. So I think Home Assistant is a really

popular one. Grafana's another one, et cetera. So I apologize for that monologue. I'm still working on the short version. No, please. It's, it's a fascinating approach because we are definitely in a post-network world. It used to be that once upon a time you had breaches where I'm going to go and I'm going to go and take things out of your system and then send it to a different system somewhere else. Now you can do all of that just by hitting the same single endpoint that's just the AWS control plane and it's just a question of what the content of those requests are.

So you effectively have to, I don't think we call it this anymore, but you need to man in the middle, everything that is being passed through for deep packet inspection, which in turn then becomes, if you can see all the payloads, well, you now have a central point of attack for that, But people have already accepted you in a security-facing role. I think that it is a more novel approach that is likely to get further than the current security posture, which is putting the, no, seriously, bro, be secure in all caps in the system prompt. Yeah, well, exactly.

And the best thing about this MCP proxy thing, first of all, you can have it, right? We have a little default one. It's open source again. You can, like, build your own if you want. And it can run on your private tailnet. And it can access stuff that's on your private. private tail net, and it can be accessed by your favorite LLM that might or might not be running on your private tail net, and also it can access things outside your private tail net.

So there's no people coming in trying to beat on your MCP server to find the security holes, right? It's only the content that matters. And for that, you can have something filtering the content and watching what's going on to make sure the AI doesn't go wildly off track. Yeah, I think that's the, that is the right path. It's part of a defense-in-depth approach. Exactly. We're aiming for this, like, again, convenience where like the easiest way to roll out AI in your company is the tail scale way. And also, coincidentally, it's going to be way more secure. If we can get that, then I think we'll really, like, we'll be on the... The two problems I can see you're going to have. One, you use the Salesforce, Salesforce example,

but everything has to start supporting this on some level, at an application level. So they need to support OAuth. They don't need to support any of the rest of the stuff. And that's what's really neat, because everybody who makes an MCP server has to support OAF. Now it's part of the standard. And, like, where APIs were kind of hard to get access to before, The trend is that, look, everyone's going to be mad at us as a vendor if we don't support Oath for getting API keys, right? As long as you have that, all of the rest of this magic is happening behind the scenes. The Gateway has to understand all this TISIDP and everything. Everybody else just sees an OAT server.

Yeah, the other challenge that you've going to have, and this is trivial, of course, is you have to come up with a few reference implementations of this that are basically click-click on and to show folks how it works. They can modify to their own approach. But historically, my big problem with early stage products is the documentation is not there. You've got to basically read the code, come up with first principles, how you want to tell it to actually do the thing that you do. A little bit of documentation goes a long way. And not for nothing, increasingly, that documentation is being written for LLMs so that they can then explain how to do this to folks. So there's a bit of a lead time. It has to be absorbed into the models before it starts spinning out.

Yeah, the best we have right now, we have a YouTube personality that works for us. that runs the Tailscale YouTube channel, Alex, and he's got at least one video about TSIDP. That's from before we added this MCP layer, but it's actually pretty well done. There's like many, many people in their personal tail nets are already using TSIDP for their own stuff. So I think there's going to be some growth there.

But yeah, we're going to have to document it. We're going to have to do all that work. This is all pretty early stage. But we're really interested in like talking to people who think this is going to be interesting to them and like kind of working with them on making the product better. And also integrating into the open source world because tailscale personal plan is free

and it's unlimited, essentially, unlimited time, lots and lots of devices. You can do all kinds of stuff with it, and it would be nice to make people or have people who are using this in their homelab already, they can take advantage of this thing as well. Oh, yeah. I do a lot of testing in my homelab for the exact sort of thing.

I still haven't gotten quite to a level of comfort where I'm putting production nodes independently on the tailnet. I tend to use subnet routers, and that is for now. the way that I approach it, just because it feels like taking anything into a critical path, past a certain point, has risk attached to it. And that's how we built it, and that's how it works for now. If I were doing it today, I don't know that I would be as cautious, given the conversations

I've had since then with customers who are working with it in that way. Yep, there are some very big-name customers, some of which I can name and some of which I can't that are like all in on like, we're going to run Kubernetes in every single pod and every single cluster in every single store and turning them like crazy because that's what Kubernetes does. And they seem to be pretty happy. It means we have to have like pretty high uptime on our control server. Tail scale is designed so that even if the control server went down for a while, in fact, it could go down for hours. The data plane keeps on working. So there's only certain things that stop working if the control plane is like out of touch for a while. So you

have this like pretty high level of resilience that people don't expect. And it comes from us not routing your traffic for the most part. That is the bridge to cross. And you've you've hit a point now where there's enough of a community around tailscale that if someone's trying to do something that no one else has really done before, it is no longer likely that they're doing something correctly. I don't mean to be unkind. But in the early days, I was talking to your team constantly with how do I do this thing. Oh, we hadn't considered that. Now, whenever I ask any of those questions that come up, like, oh, here's a giant blog post on how to do that. Or here's the GitHub issue. We explain exactly how you're holding it wrong.

and so on and so forth, which is just, it's a maturing of the product. Yep, yeah, we've been putting a lot of work into maturing it. I think one of the hardest things as CEO is just convincing everybody to not build everything they want. And just like, let's focus on refining the core. Let's do everything we can to run this business so that the core gets better and better and better. And that's how we're going to make money, not like building tons of stuff on top, which I know is a pretty unusual, especially in the security world, not the normal way to do it. The normal way to do it is collect. I don't know, it's like collecting Pokemon cards or whatever. Well, I need a DLP and I need this and I need this and I need this and I need

this and now you can buy it from one vendor and it's going to be a collection of like sort of half-heartedly integrated tools. Right? And Tailscale is like, look, we're not that. We have this one thing. It works super well and it's going to work with all the other stuff you buy from other people. But it means we spend all our time just like, you know, writing docs like those or fixing the bugs that led to the need for docs like those. It's really neat. Any last words on what we can expect in the somewhat near future? Anything fun and exciting coming down the bike?

Which is a weird thing to say about a networking infrastructure tool, and yet. I think the two most interesting things that are going to happen. One of them is more and more stuff is going to be buildable on top of tail scale or include tail scale as an option in it. So we're starting to see more and more things like, hey, if you run my program, it's linked with the tailscale library, just paste your off key here, and that thing is just going to work. A similar one is, I think, I don't know if we've announced it or not, we're going to announce it. If not, this is the announcement. The workload identity feature that allows if you're using

tailscale with GitHub actions, for example, to just like not even use off keys, because you can set it up, it's like, oh, this is your account on GitHub. I believe GitHub, when it says it's running under this account, so now everything just has access to your tailnet automatically. That's super slick way to do it, and you don't have to manage rotating off keys and stuff like that. And I guess the third one is, for direct connectivity, you know, life is not always perfect. Sometimes firewalls are weird. So we have this new thing called... You mean there are times where they're not?

Well, some are weirder than others. So we get through almost all the weird firewalls, but there are some extremely weird firewalls out there. We have this new thing called the peer relay, also in alpha, but if you're interested, inquire within. It should be in beta sometime soon, but you can have early access if somebody asks. It allows, basically, if you remember the old days of Skype and supernodes,

It allows you to build super nodes that will route the traffic in situations where direct connections are not possible. So you can still get full speed if you put your super nodes in the right places. Including behind a firewall if you want. So then even when things can't manage to get direct connections because your internal firewalls are too weird, if they can connect to the super node behind your firewall, you can still avoid the egress traffic. So this is something that our biggest customers with, of course, the weirdest firewalls and the most firewalls are going to benefit from humongously. I would love to hear the story about which firewalls, that are doing this and how they are configured

because that is such a rare occurrence in the modern era, but... Oh, yeah, we actually sponsored a patch to FreeBSD to finally fix this problem because for a while, any FreeBSD-based firewall. Of course it's PS. Yeah, well, it was, it's intended as a security feature to be blocking this stuff.

It just turns out when you do the whole, like, decision tree, it turned out that didn't increase security at all and just made everyone's life miserable. And instead of, because it makes it so secure, that to get anything done, people start rolling out UP&P. And UP&P is never a good choice, security-wise. And yet, it's the only workaround of this problem.

So we finally convinced them of this. We sponsored FreeBST to like, can you at least make it a flag? Why does Avahi take up all my CPU core? Yeah, yeah. Yeah, so we made, now there's a flag, and I think the flag is now the default, just not be silly. But there are a few other firewall vendors that are doing the same thing,

but I'm hoping we can talk them out of it, because it's actually a relatively simple, It's called a hard gnat versus a easy gnat in tailscale terminology. And they make their hard nat hard for like, it turns out no good reason. And it's avoidable if you change your code just a little bit. But unfortunately, sometimes it's our competitors making the firewalls. So they're not always super eager to do that. Yeah.

I really want to thank you for taking the time to speak with me. If people want to find out more, where should they go? Well, there's taelscale.com. We have a blog. Sometimes I post in the blog. I also have an account on Blue Sky. I have a little used account on the system formerly known. known as Twitter. And I have my own blog on appenwar.ca.

Which has been a recurring presence on the newsletter whenever you put something interesting out there. And we'll put links to all of this in the show notes. Thank you so much for taking the time to speak with me. I appreciate it. Thank you very much. It's always a pleasure. Avery Pennerun, CEO and co-founder of Tailscale. I'm cloud economist Cory Quinn, and this is screaming in the cloud. If you've enjoyed this podcast, please, leave a five-star review on your podcast platform of choice. Whereas if you've hated this podcast, please, we have a five-star review on your podcast platform of choice, along with an angry comment that isn't going to post properly because you once again have misconfigured your crappy firewall.