The Changelog: Software Development, Open Source - Istio service mesh and microservices (Interview)

Episode Date: August 22, 2018

Adam and Jerod talk with Jason McGee, VP and CTO of IBM Cloud Platform about Istio — an open platform that provides a uniform way to connect, secure, control, and observe microservices. They cover what service mesh is, why its suddenly so interesting, who’s involved in Istio, their involvement with the CNCF, getting started, and what's next for Istio.

Bandwidth for Changelog is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at Changelog because of Rollbar. Check them out at rollbar.com and we're hosted on Linode servers. Head to linode.com slash changelog. This episode is brought to you by Indeed and I had a really interesting conversation with Darren Nix, the group manager at Indeed Assessments. And Darren is running a remote first team that operates like a startup inside Indeed. And you know Indeed, it's a huge company, lots of resources, solving big problems,

lots of big data, and Darren's team is hiring. Take a listen. Darren, tell me about the big picture problem you're solving at Indeed Assessments. What our team does is we build tools so job seekers can show off their knowledge, skills, and abilities when they're trying to get a job way better than a resume can. And that lets employers find great hires a lot quicker too and makes the process better for everybody. So you're running a remote first team looking to hire pretty

aggressively Java engineers, front end or React engineers, Ruby on Rails engineers, UX designers, business intelligence, and you operate Indeed Assessments like a startup that lives inside Indeed. Tell me more. Because we're basically a startup within Indeed, we get to hire folks all around the country, even if they're not in Austin or San Francisco or Seattle. And that means we can hire really great engineers who want to be able to work from their home city, work on really big problems, but solve those problems in a startup-y way. You know, we host our code on GitHub or Rails and Redis.

We use Postgres and React and we're push on green. So we deploy six times a day. So I've seen charts that say like, hey, we deployed 13 times this week. And I'm like, haha, we deployed like 78 times because we like to go fast. And so what we're doing here at Indeed is finding ways to be able to continue to be startup-y, but solve really big problems

and help hundreds of millions of people get jobs. So if helping out your fellow engineers get jobs sounds like an exciting problem and you like working on startup-y tools at a really big scale, send us a note, reach out. I actually interview every single person who comes to join our team. So I'll be meeting with you and I look forward to hearing from you. So if you're looking to join a remote first team working on really big problems that will literally impact hundreds of millions of people, head to indeed.jobs.changelog to learn more and take that first step. From Changelog Media, you're listening to the Changelog, a podcast featuring the hackers,

the leaders, and the innovators of software development. I'm Adam Stachowiak, Editor-in-Chief here at ChangeLog. And on today's show, Jared and I are talking with Jason McGee, VP and CTO of IBM Cloud Platform, about Istio, an open platform that provides a uniform way to connect, secure, control, and observe microservices. We cover what Service Mesh is, why it's suddenly so interesting, who's involved in Istio, their involvement with the CNCF, getting started, and what's next for Istio.

So, Jason, what is the Service Mesh? What is this, this Istio thing? Why is it important? Yeah, so kind, a strange name, service mesh. We spent a lot of time arguing about, uh, what to call it. It's kind of confusing if you don't know, right. But let me maybe, yeah, it is absolutely. And let me maybe put it in a little bit of context. Um, you know, my, my personal background, I spent, uh, the first half of my career, uh, at IBM building app servers, Java app servers, you know, 20 years ago, I guess.

And, you know, the goal of the app server was to build a platform that developers could use to build applications and to kind of solve all the common problems that you encounter when you build particularly kind of network-facing applications. What I think is happening today in the cloud is we're kind of reimagining that idea, that platform idea in the era of cloud. And there's some pieces that make up that platform. And some of them we've already agreed to, you know, containers with Docker

is kind of the foundational technology that we think new apps will get built on top of. And we've all kind of agreed on containers as the way to package and run software. The next layer that we all kind of agreed on was Kubernetes. And the problem Kubernetes was solving is, all right, I don't have one container. I have many of them. How do I deploy them and control them and scale them and keep them running and kind of solve all the lifecycle problems that exist when I'm running a bunch of containers? And we've all agreed on that as the kind of de facto technology to solve that problem.

Service mesh, I think of as the third layer of this new platform that we're building. And the problem service mesh is trying to solve is how do I actually control the interaction between different applications that I'm writing? So if containers let me write an application in real life, most interesting things are solved by having lots of pieces of applications talk to each other. And so how do I actually see and understand how those different components talk to each other and have so how do I actually see and understand how those different components talk to each other and have some control over that? At its heart, that's what Service Mesh is trying to do, is give us control over the interaction between apps. So you were one of the founders of Istio,

is that right? What's the story there? Several members, you're involved in obviously you've been a veteran at IBM. What's IBM's role? What's your role? Sure. So, you know, like many things in the industry today, there's quite a lot of collaboration and technology. I run the platform as a service team for IBM Cloud. And as part of my role, you know, we look at new technologies that, you know, we want to bring to market. And a couple of years ago, we were thinking about this space. You know, how do we help with microservices how do we help developers um you know build and manage microservice applications

we have all these different components working together and we started building a piece of technology um we actually open sourced a piece of technology called amalgamate which was kind of our viewpoint on how to do programmable routing and control between different services within a cloud architecture. And we put that out there a couple of years ago. About two years ago at KubeCon, I was talking with some of my counterparts at Google and came to realize that, you know, we were solving similar problems and we kind of viewed the microservice management space in a similar way. And, you know,

we got together and had some drinks and, and, and chatted about the space and, and decided, you know, we actually see the world the same way. Maybe we should join forces and combine some of the work that we were doing at the time google was really focused more on security and telemetry or kind of how do you get visibility of the traffic between different services and so we decided to join forces we combined our technologies together and we created the seo project and kind of worked on it, you know, not in secret, but largely just the

two of us, along with the guys at Lyft, who contributed one of the key technology components to Istio until about May of last year when we launched the project. So, you know, it was really the coming together of like minds around how to solve, you know, one of the emerging problems in the cloud native space. Maybe break down Istio for us. I know it's connect, secure, control, and observe, but break down each of those and what others are important to, say, service mesh and how that applies.

Yeah, so there's kind of three key features, if you will, or capabilities that Istio tries to provide. And remember, it's always in the context of kind of multiple services talking to each other. The first is traffic management. So providing a way to programmably control how different services talk to each other. I'll give you a really simple example. How do you roll out a new version of an application?

When you roll out a new version of an application, you might want to test that new version with 5% of your traffic to see if everything's working okay. Or you might want to have a subset of your users try the new version. Maybe everyone on the East Coast is going to try the new version while everyone else in the world stays on the original version. You need the ability of control the rollout. And so one of the goals of Istio is to give you that kind of programmability of routing so that you can decide who calls who, what versions get routed to, how the traffic gets split. And that's something that, you know, everyone has to do on some levels, but everyone has kind of solved in their own ways, right? Kubernetes doesn't solve that problem directly.

Lots of people have built custom solutions to do that. Istio tries to kind of bake that into the service mesh. So that was one key thing is kind of advanced traffic management for DevOps, for resiliency testing and things like that. The second key thing Istio was trying to do is security. And this is actually turning out to be a real interesting space with Istio. I mean, we could probably dive deeper here, but historically, the way security was often implemented was down at the network. and segment your networks to control who could talk to who, right? And which services were visible to each other. That model doesn't work very well in the era of public cloud and in the era of very distributed architectures where, you know,

maybe part of your app is running on IBM Cloud and part is running on your data center and part's running on Amazon. And, you know, you don't have one network that spans all of that. So with Istio, what we tried to do is say, can we move all that security up closer to the application? So it's not defined in terms of the network, but it's defined in terms of the application itself and automatically set up secure connections between services and allow you to specify policies about who can talk to who, but independent of the network. And that's turning out to be incredibly powerful for a lot of the early users of Istio because

it gives them a different way to secure their applications. And then the final thing that Istio was after was observability or telemetry. How do you actually see what's going on? I mean, you know, one of my kind of favorite case studies in the microservices space was some material I read around the Gilt group, Gilt, the online shopping site. And they're not a customer of mine, they're not an IBM customer, they're just, you know, people who've talked about their own journey to transform to a microservices architecture. And the kind of tidbit that always stood out to me with them is, you know,

their original architecture was a very classical kind of web and database application. You know, maybe there were 10 instances running in a cluster to handle the load a single kind of monolithic app. And, and over a period of years, they refactored that into a bunch of microservices. And when they got done, they had like, I forget the exact number, it was like 300 or 400 microservices. Each of those microservices, you have to imagine there's probably at least three instances running. So you go from running like 10 things to running 1,200 things, right? So you have this much more complex environment to manage.

How the heck do you actually see what's going on you know if there's a performance problem where does it exist where are the bottlenecks like you need a totally different way to kind of observe behavior in a much more complex network of services like that and so part of the goal of this deal was to provide ways to automatically gather all of that data about who's talking to who and how it's performing and tracing and log collection and everything in a transparent way and to make it so every developer didn't have to figure out how to solve that problem themselves. So like a single interface, essentially, it's a single, you know,

endpoint for everyone to check into essentially across, you know, multiple networks, multiple clouds, that kind of thing. Right. And one of the I think the really kind of in some sense magical things that Istio does is it manages to do all of this without really having an API that you have to write to as an engineer, as a developer. It does this through a technique of using what's called a sidecar to kind of capture all of the traffic coming in and out of every service, and then give us a control point where we can redirect that traffic or secure that traffic or collect data about that traffic without changing anything about how you write your app. You know, like Netflix OSS, Netflix open source is another kind of popular framework that people have used to build microservice apps um obviously written by the guys at netflix for their own purposes um and one of the characteristics of netflix oss was like it's written in java

originally it has api you change how you write your code to take advantage of it. With Istio, you don't have that. You can just use normal TCP connections, normal HTTP connections, and the mesh kind of captures the traffic for you and transparently gives you all these features, right? And that means it can be language neutral and it can run in any environment. And as a developer, you don't really have to do anything. In fact, you can kind of just turn stl on on your app

without changing anything and turn it off later if you don't want it which is a pretty nice characteristic yeah that's super awesome as a developer i want to do as little as possible and gain as much benefit as possible so uh that's exactly that's quite a lot of people are lazy right yeah absolutely that's like one of our core competencies so how does it plug in though like you said sidecar or side load where does it actually plug in? Like we're talking like a switching layer, like routing, how does it just capture all the traffic and redirect stuff?

Right, so the way to think about Istio is there's kind of two big pieces. There's a control plane and a data plane. And the control plane is responsible for collecting policies and giving you a way to manage the system. And the data plane actually handles all your traffic. In the data plane, there's a key component called Envoy.

Envoy is another open source project. It was originally created by the guys at Lyft. And Lyft uses it to, you know, to manage all their service interactions and their production systems. Envoy is basically a small C++ implemented layer four through seven proxy or router. Okay. It handles all the traffic. It can do, you know, load balancing.

It can do circuit breaking. You know, it knows about all the different protocols. And essentially what Istio does is run an instance of Envoy next to every single instance of every service in your applications. And that's why it's called a sidecar kind of sits right next to your app. So if you wrote an app in Node.js that, you know, right next to that app would be an instance of Envoy. And Envoy captures all the traffic. Now, when you run this on top of something like Kubernetes, we can do all that for you. We can automatically run Envoy, inject Envoy into your application without you doing anything. We can set up Kubernetes so every time you run your containers, we automatically insert this extra

container that has envoy in it and then we redirect all the networking through things like ip tables to say all inbound outbound traffic for this container goes through this envoy thing right and so it can become super transparent like it's awesome in in um in kubernetes because like literally you can just turn it on and off like a switch on individual namespaces or applications. If you run Istio somewhere else, like in a VM, you might have to do a little bit of configuration to configure essentially IP tables normally in your host operating system to redirect the traffic. But that's kind of a one-time thing you do to force the traffic to flow through envoy both directions and once that's there then envoy can kind of do all this magic and the goal

of the control plane like the istio control plane is to program all of those envoys like if you imagine the architecture i just called out if you if you mapped it to gilt's architecture which i have no data to say they're actually using istio but just to use, if you mapped it to Gilt's architecture, which I have no data to say they're actually using Istio, but just to use their numbers, if they have 1200 things, there'd be 1200 envoys running. And so then the question becomes, well, how the heck do you program and manage 1200 of these things? And that's what Istio Control Plane does, is it does all that for you. So you can define some simple policies, like when application A talks to application B use version two, that's a policy you configure. And we translate that into Envoy configuration and push it down

into all of the sidecars for you. And so you can kind of ignore them like you just manage them through SDO. We manage their configuration and you kind of transparently get all of this behavior. So it's a pretty cool approach that makes it very easy to manage and makes it very language neutral. So it doesn't matter how you built your app. You can use any programming language you want. So one question I always have with these kinds of things, it sounds like with Istio, by the time you need it,

you probably already know that you need it, like you're feeling that pain. But with microservices, more generally, the question always becomes like when to microservice or at what size or if ever. And there's a lot of opinions on that. Aaron Patterson, who's kind of a Ruby core developer, had a great tweet a while back about microservices

where he says it's how you turn a function call into a distributed systems problem. And so there's a lot of people that I think reach for them earlier than they ought to and they find that just curious your take on that because you talk about you know companies we are our audience goes from all sizes so we have like single like developer shops right all the way up to people working in the fortune 100s to the tech giants so we kind of have people in all different areas how do you know in your opinion like when to microservice and when istio is going to be something that you're going to be interested in yeah it's it's a it's actually a tough uh question uh because it depends on um technology factors and and a lot of people dynamics you know let's take the microservice part

of it first and then and then the institute partute part second. Um, so, you know, for me, microservices is fundamentally about people and teams and like how you organize, um, a team of people and how you organize an application to have kind of independent parts and, and, you know, how big each microservice is, or how many of them you have, you know, varies a lot. I don't actually think there's some magical size of, you know, code or something that makes it a microservice. I think it really comes down to microservices are useful if you have a high rate of change. Like if you are working on something that's changing a lot and you need to be able to rapidly deliver um microservices is a good approach and in fact i always think that microservices makes this explicit trade-off

like you're trading off more operational complexity in exchange for faster velocity that's a good way to look at it yeah yeah you get faster velocity because you have basically people don't have to talk to each other as much you decouple your development yeah yeah and and decoupling it really is probably less about technology more about like this group of 12 people can kind of act with some autonomy they can make their own decisions they can deploy when they need to they can make changes without having to coordinate with everybody else right um and if you're changing stuff a lot, that's super valuable. And so you tend to see microservices in environments where

there's a high rate of change. And if you look at, let's say, traditional or existing applications, a lot of people have debates about like, do I go refactor some existing app into a bunch of microservices? And I think, again, it comes down to is that app changing and what parts

of that app are changing? There might be pieces of an application that makes sense to kind of refactor into a set of independent services because it allows you to change those parts quickly. And there's other parts of that app that you haven't changed in 10 years. And like, there's no inherent advantage of moving it to a microservice. So I think you have to just get down to how do you organize and where are the places where you need change. I mean, I look at my own team. One of my other jobs on my team is I run the Kubernetes service for IBM Cloud.

So it's called IBM Cloud Kubernetes Service, and it's a managed Kube platform. And that platform itself is built with um you know more than 100 people has dozens of microservices like we built and deployed the whole thing in five months we make 100 updates a day like none of that's possible without kind of a microservice architecture underneath you to give you the ability to kind of decouple all those changes now i think if you go look at istio um the question then becomes like do i not need istio until i have a bunch of microservices yeah right you know do it until i get to some scale and i and i think i don't think that's actually true i mean i understand completely why people kind of

go there because the even the way i described histio sounds more valuable when you have lots of things um but the reality is uh observability for example or version routing are useful even when you only have a few things right being able to collect data about what's going on and being able to control updates is really valuable. And one of the things we've tried to do in the Istio project is make Istio very incremental, meaning you can adopt it in pieces and you can adopt it incrementally on portions of an application. So it's not like a big all or nothing switch where, you know, one day you just decide we're going to Istio enable the whole world and turn it all on. You know, you can, you can bring Istio in and just use it for, for monitoring,

you know, to, to see what's going on, or you can bring it in and just use it for, um, for version routing or something like that, and then kind of grow with it as you go. Um, and the sidecar thing actually helps a lot with that because you didn't have to change your app to do it. So you literally can like decide one day to turn on istio on one of your services and we'll inject the sidecars and it'll start doing new things and you didn't have to change anything about your app so it sounds like even in the small if you have microservices there's value there as long but not be aware but but know that you don't necessarily have to adopt all that Istio has to offer, I think then the deciding factor would still become the cost of operational complexity

or moving parts versus the value provided by observability or whatever specific features that you're looking for. And then is there a cheaper, operationally cheaper way of accomplishing similar means? I guess that would probably be kind of the cost benefit analysis there when you have a small amount. Yeah, I think that's a good way to look at it. And I think that's actually,

you know, one of the real challenges, if you will, in the, in the kind of container ecosystem today, like, have you guys ever seen the, the cognitive computing foundation maintains this landscape chart?

No. It's huge. It's huge. It's like this chart of basically like all of the technologies in the Cognitive space and all the vendors and startups and everybody who has solutions in each category. And it's like the most densely packed collection of icons I've ever seen. It's a good slide.

It's getting crazy out there. That's for sure. Yeah. Yeah. And I mean, getting crazy out there that's for sure yeah yeah and and that i mean on one hand that's awesome it means there's just a ton of like really amazing innovative work going on but as like the average developer or the average company like assembling all those pieces is hard and um yeah maybe too hard in some cases to warrant and so you do have to kind of make this judgment call and like what extra stuff do you want to bring into your environment and i think that holds true with this deal today like today if you want to use this deal you have to make a decision you want to use this deal and you have to install this deal into your environment uh like into kubernetes and you have to do some work to manage it and update it,

things like that. And so there's a cost to using it, and you have to kind of weigh off, does the cost warrant it for whatever I'm getting out of it right now? I think what will happen with Istio in particular is there's just a tremendous amount of kind of excitement and energy behind Istio. And I really do believe, it's hard to predict the future, but Istio has all the hallmarks of a key technology that will become a de facto part of the environment that we all work with.

And therefore, it'll start to get built in. In IBM's case, our intention is to build Istio into our Kubernetes environment. So like, it'll just be there. You know, if you use Kube to run workloads, the Istio pieces will just be there all the time. And you can just decide to turn them on or turn them off. And therefore, that kind of overhead of just choosing to use it goes down, right? And I think that's what's going to happen in the industry in general is you're starting to see people adopt platforms. Like they're either using a public cloud platform

and that public cloud combines a bunch of the capabilities in that crazy picture with all the icons or they're using a software distribution that kind of combines some of those pieces together. So you kind of defer that work to the platform provider to do the integration for you. This episode is brought to you by Linode, our cloud server of choice. It's so easy to get started.

Head to linode.com slash changelog. Pick a plan, pick a distro, and pick a location, and in minutes, deploy your Linode cloud server. They have drool-worthy hardware, native SSD cloud storage, 40 gigabit network, Intel E5 processors, simple, easy control panel, 99.9% uptime guarantee. We are never down. 24-7 customer support, 10 data centers, 3 regions, anywhere in the world that got you covered.

Head to leno.com slash changelog to get $20 in hosting credit. That's 4 months free. Once again, leno.com slash cheese log. So Jason, we are on the CNCF slash landscape GitHub repo here where some of these images that you referenced earlier are sitting. So listeners, if you'd like to look at these images, they are informative and a bit overwhelming. Definitely check out in our show notes. They're right there for you. Or if you are at your computer, just go to github.com slash CNCF slash landscape. And there you have it.

There's two. There's a trail map. And then there's the current version of the landscape, which is what you're talking about, which has all of the different icons or the avatars there. And on the, on the landscape specifically, we have service mesh and discovery as step five, excuse me, on the roadmap, which is kind of like a step-by-step thing of like how you, how you accomplish this. Or I don't know, I don't know what's at the end.

Success is at the end of the road. And step five is, you know, pick a service mesh and discovery or, or something like that. And we see Envoy, which you mentioned, we see core DNS and linker D.

Istio is not there. And if you look at the landscape, Istio is there, but there's like 16 other icons next to service management. So it seems like at least, I don't know how fresh these are, but at the time of creation of these docs, Istio was kind of one of many options. You're saying that it'll hopefully in your eyes

become kind of the de facto for this particular section of the cloud. Maybe speak to the fact that it's not in step five yet. On this, of course, the Cloud Native Computing Foundation stuff, it's just not IBM stuff, but just tell us about how it fits in there and how it's going to beat out, you know, these 16 others. Sure. So, so let me help explain the charts maybe a little bit. So the trail map and the big boxes on the landscape chart are the official CNCF projects, right? So they're the projects that are kind of managed

by the Cloud Native Computing Foundation as open source projects. The landscape has kind of everything that's going on in this space. So you see projects, but you also see technologies and companies that aren't necessarily part of CNCF. Envoy as an independent project

is officially a project under CNCF now. So the Lyft guys, Matt Klein at Lyft, donated Envoy to CNCF last year. I forget exactly when that happened, but in the last six or eight months. Istio is not there yet. So Istio is not currently an official project of CNCF. That's in progress. So that should happen over the coming months. We really wanted to get Istio to kind of a 1.0, kind of as I alluded to, is a two-year journey to, you know, bring all these technologies together.

And it's, you know, now that kind of core design is set. And if you actually looked at the evolution of Istio, there's been some pretty substantial changes in the conceptual model, in the APIs of the control plane, and kind of how it works over the last year. And so we felt, okay, let's get this thing in a good spot, and then we'll bring it to CNCF. So it'll come. CNCF is an interesting foundation because it's not trying to pick winners and losers. So you will often see what are essentially competing technologies live within the umbrella of CNCF.

So in container world, we have both container D and rocket. In service mesh space, we have linker D, and there's a related project called Conduit, which is really an alternative to Istio that exists underneath CNCF. That's cool. That's fine. This is a community of people kind of building out the best technologies to solve these problems and so Istio will slot in there

in the coming months. I thought that Conduit was folded into Linkerd, at least version 2.0. Yeah, that might be true. It's hard to keep track of all these things. Even for me, I spend every day on this and I'm like, what the heck? There's this great i'm like what the right there's this

great mitch edberg line he's one of my favorite comedians and one of the things he says he's he said i'm sick of chasing my dreams i'm just gonna ask him where they're going and catch up with them later and and i i kind of feel like i wonder like when is this landscape going to shake out a little bit and like all of the tons of innovation so it's a gift and a curse right you got all this innovation you have new projects you have excitement it's this cambrian explosion kind of thing but any space where you need a road map with like 10 steps and then there's like choose your own adventures along the way it's really kind of like i they actually have dragons on their road map or trail map which is kind of fitting. Cause it's kind of like there'd be dragons there.

I'm just curious, like when will things start to shake out and settle down a little bit to where it's like, clearly, like you said, clearly it's Docker, right? Clearly it's pretty much Kubernetes for,

for orchestration. When do you think the rest of this will be really kind of shaken out and we can just catch up with the later. Yeah, I guess if I, if I really knew the answer to that, I could make some money. And if you told me, I could make some money too.

That's right. I could create a whole business around that. Look, I mean, I think it'll settle out in layers. There's pieces we agree on and there's always kind of the hairy edge that everyone's still experimenting and trying to figure out what to do. You know, I do think the container layer is essentially settled with container D.

I think orchestration is settled with Kubernetes. You know, service mesh, you can't say is settled because we're still early in the adoption curve, but all the indicators from who's working on what projects, where different companies are invested, level of excitement, I really feel confident that Istio will take its place as that third layer. Higher up the stack of stuff that's here in CNCF, yeah, it might take a while to be honest. We're at the beginning still of what I think is a pretty formative, you know, foundational change in how software is built. Not only cloud, but containerization.

If you think about environments like Kubernetes and Istio, they are incredibly metadata rich, meaning they give you a ton of information about how software is executing and who's talking to who. And you can do some really interesting things. You can take something like Istio and watch all the traffic flowing through it and automatically detect who you're talking to or what protocols are being used and apply security in new ways. You can take all the data and apply machine learning to it and make better decisions about resource management and routing. So all these companies are exploding because we have this kind of level of access and control over the environment that just wasn't possible before. And so I think we'll see that diversity for a long time.

Like, you know, I think for the next, it's hard for me personally to look much more than three or four years out in this space. But it's certainly over that time period, I think the core diversity will still be there. I think the bottom layers are settling out. I think what also tends to happen is you look at, there's different personalities

of how people adopt technology. And for the bulk of the market, for most companies, certainly here at IBM, I spend a lot of my time talking about to kind of larger enterprise customers, they're gonna look for a partner that helps them assemble these pieces, right? And public cloud is one form of that, right? Where we, you know, whether it's IBM or Azure or Microsoft,

you know, Google, we're all kind of delivering a cross section of this whole picture as a service on our cloud. So that's one way to deal with the diversity is you just, you know, you have a platform provider who does it for you. I think the same thing happens on the kind of software distribution side. You know, you just, you know, most people won't do choose your own adventure.

Most people will, you know, pick a partner who can pull some of these pieces together. And one of our questions in the list here is like, you know, why does service mesh, like, why is it suddenly so interesting? And it seems like maybe because of this roadmap now, that question seems to be answered to some degree, because like you said, Docker is the container. That's step one. Step two is CICD. Those are, for the most part, there's different platforms for it. But if you're doing a public cloud, you have your own ways of doing that. And you got orchestration, that's Kubernetes, Helm, those are involved there. Prometheus is a graduated project, so that's a clear winner. And then five is the next step, which is Envoy, Corda, DNS, Linkerd

and then obviously Istio here. So it seems like, as you mentioned in layers, these are beginning to be solved. That's just the next step is obviously to pick a winner. And I would assume that you pick a winner by integration. So you mentioned that Google's involved in this, IBM's involved in this, obviously. Azure has their own thing, something called, what is it called again? Service Fabric Mesh. So one more level of obscurity in there.

Now it's Fabric too. Is that a competing thing, Jason, or is it just by name? Because I know sometimes Microsoft and names don't necessarily jive. Yeah, I don't know actually how directly aligned that is. What I see happening here, though, is that at each level, as we start to pick away the layers

of cloud native, is essentially the agreeability. Okay, we all agree that Kubernetes wins. We all agree that Docker wins. We all agree that Prometheus wins. The next might be, you know, which do the public clouds choose and that's the winner is that is that fair to say jason i i certainly i do think it's fair and i certainly think that um public clouds have been a big influence on kind of um who the winners are um you know not always you know like one of the like watching kubernetes kind of become the

de facto was quite interesting where um you know it seems very obvious sitting here right now that cube is the winner if you went back in time 12 months or maybe 15 months it wasn't as obvious right and there were some big players who you know hadn't made that choice yet you know amazon hadn't made that choice docker hadn't made that choice yet. Amazon hadn't made that choice. Docker hadn't made that choice yet. But there was so much momentum around the developer community, around these technologies, that eventually everybody

had to acknowledge they needed to support it. They all put their weight behind it, and that kind of takes the air out of the room on other stuff. Tell me if you remember this term, Jared and Jason, too. DCOS. Data Center Operating System. I'm going back a couple years in our history of 2015, we talked about Mesos and Mesosphere,

and this kind of goes back to the days Jason's talking about. Yeah, I mean, they had a moment there where DCOS could have become what Kubernetes is now, and I think that didn't happen for a variety of reasons, and now all that energy has moved to Kube.

Istio is interesting, i think because you also i always try to look at like like i've been in it for i'm not that old but i've been in it for a long time you know and to some level how long you've been in it come on it's like yeah spill it spill it 20 20 uh two years 22 22 years, 23 years. You know, on some levels, it's the same problems, right? You know, like we're not solving, maybe with the exception of like machine learning and AI, to a large extent,

we're not solving fundamentally different problems. We're just solving them in better, more efficient ways. And so you got to look at a technology like Istio and go like, why now? Like, why was this approach possible now? And I look at it and I go, okay, to me, there's like two factors. One is the adoption of new approaches like microservices is generating a need. Like we now have these very distributed systems that are made up of lots and lots of components. you need something like an istio to help you manage that and then the second component is like the the approach like let's run 1200 independent proxies to route all the traffic would have been like a ludicrous thing to do five years ago if that meant like do all that yourself like install them all yourself configure them all yourself like so the fact that we all agreed on containers and Kubernetes and the

capabilities that that base platform provided made it feasible, right? Maybe combine that with public cloud, which kind of brings as a service and infrastructure into the picture. It makes it like feasible to run an architecture like that. So you have like a need and you have a technology foundation that we've built up that makes it possible to take that approach. And so, you know, Istio kind of falls into that gap in a really nice way. And it's funny, I haven't looked at the numbers in

six or seven months, but when we launched last year, like within a few months of launching the project last year, we were watching the kind of open source stats on project adoption. And it was tracking ahead of where Kubernetes was at the same point in its lifecycle. You know, so like four months in of Kube versus four months in of Istio. Istio was actually tracking higher than Kube. And I think, you know, that's a sign to me not that Istio was more popular than Kube, but that Istio was building on top of the momentum. You know, it's like when the iPad came out, like the iPad outsold the original iPhone. Well, that only happened because we all had iPhones already.

Right. So we understood the model and you could see that kind of additive value. Right. And I think that's what's happening with Istio too. It's interesting too that like Docker sparked the conversation of what it obviously did. And then when Kubernetes won in this landscape, it seems like they enabled, they were the keys to enable the next level. Like you mentioned, you wouldn't want to do 1,200 proxies by hand.

You'd want something to automate it for you, because otherwise you're just silly and silly. Yeah, exactly. It's interesting how all of these these layers kind of build on each other and enable you know the next round and and you know even if you look at istio um how it's built internally um it really built is built very deeply on top of kubernetes not not just from the standpoint of like how sidecars work and everything, but the actual implementation of Istio's control plane itself extends Kubernetes data model extends the Kubernetes API server.

I think on some levels, that's a little bit the untold story of Kubernetes. Like most users think of it as a container orchestration platform for running their apps, but Kubernetes is also becoming kind of the de facto control plane implementation. And you see lots of projects who are extending Kubernetes with new kinds of resources. So instead of having just pods and deployments, which are kind of Kubernetes concepts, you see builds and you see

an Istio, you see virtual services and gateways. but those are all built into the Kubernetes API server. And so it's really interesting to watch how, as we introduce these new technologies, you're able to build on the momentum of the layer below in a pretty interesting way. Whereas in the past, we would have had to build all that infrastructure out again, and you solve all the same problems. How do we protect the STO APIs? How do you control access? Like how do you run it and scale it? But all that's already solved. So we're just leveraging it.

This episode is brought to you by our friends at GoCD. GoCD is an open source continuous delivery server built by ThoughtWorks. Check them out at GoCD.org or on GitHub at github.com slash GoCD. GoCD provides continuous delivery out of the box with its built-in pipelines, advanced traceability, and value stream visualization. With GoCD, you can easily model, orchestrate, and visualize complex workflows from end to end with no problem. They support Kubernetes and modern infrastructure with Elastic on-demand agents and cloud deployments. To learn more about GoCD, visit gocd.org slash changelog. It's free to use, and they have professional support and enterprise add-ons available from

ThoughtWorks. Once again, go cd.org slash changelog. so jason before i put this landscape picture away because i still have it pulled up i'm still looking at it uh one thing when you see all these icons and businesses and you see all these open source projects it's interesting because it is kind of like, here's some CNCF stuff. Here's a company, right? Here's a open source project. And besides the sheer number of players involved, the other thing that strikes me, specifically in the cloud native area of open source, is you have a bunch of for-profit, really competitors in business collaborating on open source projects. istio is no exception to that like you said ibm google lyft red hat surely if you look at the full contributor list there's probably you know

players from or people from any company so tell us about that i don't know that milieu that relationship where you're basically i don't want to say sleeping with the enemy because that's probably too harsh a term, but like you're collaborating with people who, I mean, IBM cloud's a direct competitor with AWS, with Google cloud, with these things. So what's that like? Yeah, it's, uh, uh, it's interesting. Uh, it's, uh, full of conflict sometimes, but, uh, it's also just like like it's just how the industry works uh and it's been getting more and more like that um over the years you know um i started the show talking about my 20 year old app server experiences and back then we collaborated on like specification documents and we all agreed on some spec and then we went off in our corners and tried to create the best

implementation of that um that was the old way to collaborate um for a long time now the way we collaborate is we create open source projects and then you know you um maybe you extend on top with additional capability or you differentiate on quality of service or integration or how you deliver it you know user experience um so so i think you know on one hand it's like maybe i'm so used to it it doesn't even phase me anymore because you're always having this conversation in your head about for any problem space what's the stuff that um really should be open collaborative technology that we all agree on together because like the one defining behavior of both developers and companies today is for the most part nobody wants to

build a tie to some proprietary technology right that's always been true on some levels but it is uh exceptionally strong today and so you always have to have this debate about like when you're solving a problem, what should I do in the open with others? Right. And what should I productize? What I actually think is really different right now is like the VC, the availability of venture capital is frankly so easy right now that you

also get this just tremendous explosion of independent companies, small companies who are trying to build a business. And like Istio is a great example of that. Like when, when we started this two years ago, it was just the two of us, Google and IBM and Lyft, um,

three of us. Uh, and now I think there's like four to six VC funded startups in the Valley just around Istio. Like competing with Istio or building on top of Istio? Like building products around Istio. Okay. Contributing to the project and then building product around Istio.

So what would be a product, like, don't name a competitor, you don't have to like say here's one, for example, but like what would be a business model or a product that would say it's Istio plus our own stuff and so we're going to contribute back? What does that look like? Yeah, so an example of what it might look like is, I mean, the simplistic business model is you're a distributor. Like you provide a supported distribution of the code that's, you know, can install and run and be supported by you.

So, you know, a company isn't, they have a throat to choke when something goes wrong. There's not much value out there. There's not much value out there. The only company that seems to have like made that model really work for them is Red Hat over the years. Yeah. More interesting. I think if you look at Istio,

like the way I think about Istio is Istio is fundamentally building blocks that you like, they're kind of low level primitive controls that you can use to do these fancy things we've been talking about. So there's a tremendous opportunity to build UI and capability around those controls. You know, you can build a CICD system that automatically used Istio to do version rollouts or canary testing or rollbacks by kind of controlling the flow of traffic. Or, you know, one of the things we pioneered at ibm and istio was um a set of resiliency testing features um so i'll get a little bit gory for a second but um

if you think about most distributed systems where services are calling each other over the network one common technique is everyone has timeouts like you make a call you wait for some amount of time before you give up and assume that the other guy is never going to come back to you. In a distributed system, one of the common problems is like everyone independently decided how long they would wait. So, you know, A calls B and B calls C, and the guy in the middle is not willing to wait as long as he should for the answer. So you get failures that aren't really failures, right? And you can spend a lot of time, especially when things go bad, like the network slows down or something.

All of a sudden, things all fall apart because of all these kind of incorrect timeouts. Well, in Istio, you can actually artificially introduce delays between any in ibm exploring this exact problem you could actually write a tool that took a network of microservices and went through and programmatically changed all the delays and measured the results of the system and calculated for you how to tune the system so it behaved properly in the event of failure so like set all the timeouts automatically like So that's not Istio itself. Istio is just giving you the knobs to turn, right? Yeah. But you could build a product on top

that was like a testing tool or a performance tuning tool that used those knobs to actually do some higher level things. So there's tons of opportunity like that across security and all kinds of dimensions of Istio where you could imagine building product

that took those controls and actually solved some end problem. security in all kinds of dimensions of a steel where you could imagine building product that took those controls and actually solve some end problem, you know, that people have. And so that's what you're seeing in the, in the VC community. And the, frankly, that's what you're seeing in companies like IBM and others that are building a steel into their products and, you know, um, figuring out how to take these new tools that we have and, and solve some high value problems for people. So what are some heuristics that you use inside of IBM to make these decisions, like where you draw those lines and, uh,

between this is an SDO open source core contribution versus this is a value ad that's proprietary to IBM and we're going to sell it versus this is a service or whatever. I'm sure there's probably hundreds of those a week, um, some bigger than others, but do you have a guidelines or is it just your own personal gut? Like this, this is going to go in open source and this is not. Yeah, there's a blend of, of guidelines and gut feel for sure. And some of these things you don't know the answer to till, you know, you make some guesses and you know, you, you adapt,

you know, sometimes it's better to just optimize for being wrong and changing instead of trying to get it right at the beginning. But one of the kind of core measurements I use is anything that's going to touch the application itself. Meaning, of course, APIs are the obvious example that you would program to in your code. But also any files or artifacts that you would put into your application and into your deployment pipeline like things that actually touch the app should be part of an open community right like that's the kind of core contract you're trying to create for people is whatever you're writing could run somewhere else or could run on another provider and so i use that a lot. Like if it looks like, you know,

to solve this problem, I need an API that neither that API needs to be an open source or I need to find a different way to do it. Right. So that the application can become portable or press different environments. You know, there's others, of course. I mean, you know, you can you can do the feature chasing game. You know, if you add a feature that nobody else has, like, you know, that's possible. That's a tough game. I personally think that's a really hard game. Like if you create a feature that's an add on on top of an open source project, one of two things usually happens is either not that useful of a feature. And so nobody cares and nobody copies you.

So you you have this unique feature but it's not very high value or it's really valuable and useful and eventually it'll just show up in the open source project whether you want it to or not like somebody will go oh yeah that's a great idea and they will add it into your into the open source project so you wind up having like a window of time like where maybe you're ahead that's a hard game sometimes that's an interesting game to play but it's a lot better to kind of especially in in the cloud world to differentiate on one being the thought leader and defining these projects you know if you look at istio um lots of people will have istio at the end of the day ibm and google are the kind of

key thought leaders behind what's going on in that project. And we're, you know, have a lot of influence over kind of where we think it evolves to. And so you can, you can lead through leadership in open communities. And then some of that comes down to like, how do you deliver, you know, do you deliver it in, in public cloud, you deliver it as a service, you've integrated it with everything else you're doing. So it just becomes super easy to use. So you're not making a decision on a single piece of technology you're making a decision on this full platform that you want to go use right it's really a very complex game you know the old test the old the old best of breed versus you know right you know best in class kind of decision so on the other side of that going not i agree with you. Istio moving as a project in a specific direction that plays to Google's,

you know, core competency or their strategic advantage in business, who makes the calls and how are they made on like what stuff goes in, in the case of IBM is like, nah, we don't really think that should be happening. Well now what happens? Oh yeah. So, so what you're talking about is basically how does open source project governance work? In this case, yes.

You know, in this case, you know, so every project has its own governance structures. Google has been a great partner with us on this project. You know, we have it set up where we have essentially an executive steering committee and a technical steering committee, which have representation from across the community, from all the major vendors involved, from elected positions, from the community itself. And those committees are basically there for exactly the purpose you talked about, which is like, how do we make sure we're making project decisions in the interest of all the parties involved and not decisions that only benefit one party. And it's a sign of health of an open source project if it has a good governance structure. party. And it's a sign of health of an open source project if it has a good governance structure.

You know, it's funny, IBM, you know, 20, 30 years ago, IBM was maybe the pinnacle example of proprietary. But over the last quarter century, we've been doing open source deeply in our DNA for a long time. And one of the things that often happens behind the scenes at IBM is when we join open source projects, we do a lot of work to try to make sure there's the right governance structure in place for that project Google and Red Hat and others worked together to create the Open Container Initiative and to create CNCF and, you know, form those projects and foundations explicitly to make sure that, you know, if you looked at container tech, you know, at first that was an open source project. It was all run by Docker. And Docker has been a great partner. But, like, we all knew that if this was going to play the role, we thought it was going to play as the kind of foundation for software for the next 20 years.

Like it needed to run in a project that had the right open governance structure. So we created OCI together and we created the ContainerD project and we kind of set up a structure that allowed those decisions to be made more fairly. I think what's interesting, though, when you mentioned governance and maybe why Jared asked that question is because like one, it's not clear by going to the Istio website, but you know, like here's how this project is governed for one to be invitational to say, Hey,

you know, I know that the thought leaders are, you know, Google, IBM. However, you know, if you would like to get involved,

there's certain ways you can technically influence the direction of this project. And from what I understand about CNCF and maybe potentially like why you may want to become an incubator project is that they provide that for you, that they're essentially neutrality. And when Jared asked that question, like, how do you interact with Google and, you know, how is that, you know, I forget what the term you used, Jared, but one of the benefits of being part of CNCF is that neutrality and sort of the let your guard down when you come here because we're not trying to steal your bacon, we're trying to make it possible to make it. I think CNCF is interesting in that way. I do agree that

that's the role of CNCF. CNCF is a little bit different than some other communities. Like Apache is maybe the opposite example where Apache has a particular process, governance structure, and all projects under Apache follow the one governance structure. With CNCF, CNCF itself doesn't impose a particular governance structure in each project. They can make their own decisions. The only thing they say is you have to be neutral.

That's the only boundary they have. You can run it how you want governance-wise, but you understand that if you join CNCF, you need to be neutral. Exactly. So CNCF's philosophy is like, we all agree on what the results should be. How you get to the result can be project by project.

Whereas Apache's approach was, here's the process to get to the result, right? In the case of Istio, like, the process is documented. There's, you know, links on the community page on like how to contribute to different working groups. There's links in there about the technical oversight committee working group, which is the kind of overarching technical group um and then you know part of kind of moving forward is and obviously this has evolved over time but um you know how do we get to you know voting procedures and like what's the formal governance rules like the the charter if you will and that's been that's been kind of evolving over time but you know i think it's you know that's been kind of evolving over time, but you know, I think Istio,

there's no, um, there's no confusion about like, this thing is going to run the right way. We already use the CNCF code of conduct, you know, like it's, it's, uh, it's going to have the right, um, blend of, you know, who gets to participate. So that's, that's pretty important to us at IBM for sure. So we're not going to run a project where somehow we have undue influence. The challenge, the challenge sometimes is getting projects over the hump, you know, like getting them birthed into the world. You have to. Needed or usable. Yeah. Yeah, exactly. And then, so those early decisions,

like you have to keep this kind of, before you open it up too broadly, you have to kind of get it to a baseline. Otherwise, it becomes like the Wild West. And so that's what I think people have seen happen over the last 12 months is Istio kind of going from 0.1 to 1.0. And we have a pretty good baseline. So back in July, you mentioned, I think it was in either the second segment of this last segment. There's a lot of excitement around the direction and future of sdo you know what's what's in the future what's next yeah so um you know one of the big challenges we had over the last four months was just like

there's so many interesting things happening in a steel we had to kind of pull back and say we got to get what we have out the door you know out to 1.0 and stop working on new things. So we're pretty excited to have done that. And now we can start working on the new things again. There's lots of interesting spaces. One, I think that has a lot of excitement right now is kind of multi-cluster meshes or mesh extensions. So like if you think of the idea of the service mesh, obviously it's about connecting different

services together. Services live in all kinds of environments obviously it's about connecting different services together. Services live in all kinds of environments. They might be on different cloud providers. They might be, you have some stuff in a public cloud and some stuff in your data center on-prem, and you logically want one service mesh that spans all those environments. Maybe some stuff's running in Kubernetes and some's on Cloud Foundry and some of it's in virtual machines. And so you have this notion of how do you configure and manage a service mesh that spans these different environments. And so there's a bunch of alpha work in the project already to allow you to set up these kind of more distributed mesh environments where you connect different systems together. And I think that will be really, really valuable.

You know, I actually was in Australia a couple weeks ago for a week, kind of visiting clients and talking to them about the space and spent a lot of time talking about Istio. And that, you know, the notions of security in Istio and being able to do like application level security combined with this emerging idea of, you know, hybrid, a hybrid mesh that spans their data center in the cloud. That was super interesting to every customer I talked to, because, you know, it's one of the challenges they have, they want to adopt the public cloud, they want to enable all their teams to go fast and build microservices and, you know, use all these new approaches. But all those apps are

talking back to their existing systems. So they need some way to kind of govern over the top of all that. And Istio gives them a way to do that. So one thing I see on the Istio homepage is a learn more, but not a getting started. So what's the first step to adoption? I know you mentioned that you don't have to adopt all of it early in the call. So you mentioned that you can just sort of like

take little features of it as you need it. And at the same time, you don't even have to change your application. So what's a good first step to adopting Istio? Basically, do the setup, like follow the instructions on the website for setting up the quick start with Kubernetes. And then there's actually some great examples

built into the Istio project. There's an app called Bookinfo. So the first way to get going is just get it installed and try it and learn the concepts. And to help make that happen, we built a sample microservices app that has Istio applied to it that is pretty extensively documented. It's part of the project that can help you deploy an actual app that has three or four microservices and then explore how intelligent routing works or how to get telemetry data or how to secure it. So there's some pretty good guides in the documentation section of the website. They'll kind of walk people step by step through how to make use of Istio. And so that's where I always tell people to start. Don't apply this to your own thing first. Go try it out. You can do it in any

Kube environment. You can do it on MiniKube on your laptop or in public cloud service, wherever you have Kubernetes, you can get going pretty fast. Cool. Alright, we'll link up to this quick start with Kubernetes documentation. I found that just had to dig a little bit and I'm always, you know, one thing we try to cover at least in the last portion is like, you know, where, where's the project heading? How can people get involved? And what's a good on-ramp so that when we, when a listener close out, they can easily go somewhere and we have links in the show notes for that. Well, yeah. And that's good feedback too. And you know, we, we spend lots of time, you know, part of an open source project,

you also have working groups who work on the website, right. And we have lots of arguments about how to organize the information and make sure people have an obvious path to getting. I would say if there's two people in the world that have good insights into what are core ideas you should have on homepages, Jared and I'd be those people. So. That's good. I'm going to pick your brains later. Well, I said your name first, Jared, of course.

Well, thank you. I appreciate being included in the list. You're welcome, sir. But Jason, hey, man, thank you so much for taking the time to cover this deal with us. Congrats on the 1.0. Congrats on all the work being done here. Clearly, great, great progress, especially on the security side. Having considered that as a core piece of it and how it moved from the network up to this layer is certainly an important piece. But thank you so much for coming on the show. Yeah, thank you.

Thanks for having me. All right. Thank you for tuning into the show today. Love that you listened to this show. Do us a favor. If you enjoy the show, tweet about it, blog about it go into your favorite podcast app and favorite it, share it with a friend

tell somebody you know how much you love this show and we'll keep doing this and we'll keep producing awesome shows for you. I want to thank our sponsors Indeed, Linode and GoCD. Also thanks to Fastly, our bandwidth partner head to fastly.com to learn more and we catch our airs before our users do here at Changelog because of Rollbar.

Check them out at rollbar.com slash changelog. We're hosted on Leno Cloud servers. Check them out at leno.com slash changelog. And the host for this show was myself, Adam Stukowiak, and Jared Santo. The mix and master is by Tim Smith. Music is by Breakmaster Cylinder. And you can find more shows just like this at changelog.com slash podcasts. While you're there, subscribe to Master.

Get all of our shows in one feed at the changelog.com slash master. Thanks for tuning in. We'll see you next week.