Screaming in the Cloud - Multi-Cloud is the Future with Tobi Knaup

Episode Date: March 11, 2020

About Tobi KnaupTobi Knaup is a Co-Founder & the Chief Technology Officer of D2iQ. Knaup is an experienced software engineer focusing on large scale systems and machine learning. Previously, he helped scale Airbnb to millions of users worldwide as technical lead. Tobi’s research work is on Internet-scale sentiment analysis using online knowledge, linguistic analysis, and machine learning. Tobi also co-founded his first company at the age of 15.HeadshotLinks ReferencedTwitter Username: superguenterLinkedIn URL: https://www.linkedin.com/in/tobiasknaup/Personal site: https://tobi.knaup.me/Company site: https://d2iq.com/

Hello and welcome to Screaming in the Cloud with your host, cloud economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud. And this is important to me. No billing surprises. With simple, predictable pricing that's flat across 12 global data center regions and a UX developers around the world love, you can control your cloud infrastructure costs and have more time for your team to focus on growing your business.

See what businesses are building on DigitalOcean and get started for free at do.co slash screaming. That's do.co slash screaming. And my thanks to DigitalOcean for their continuing support of this ridiculous podcast. This episode is also sponsored in part by Logs.io. Logs.io hears it loud and clear from many engineers, specifically that they prefer using open source tools for observability. They get an easier onboarding experience, a built-in community ready to go, and a lot less vendor lock-in. But what about the enterprise grade scale, support, and security you need to improve the performance of any given cloud

environment? That's where Logs.io comes in. They offer a fully managed observability service, log management and cloud SIEM based on ELK, and infrastructure monitoring based on Grafana, the open source you love at the scale you need. Sign up today for a 14-day free trial at logs.io slash screaming and for your chance to receive a free Logs.io t-shirt. Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week by Toby Knope, the co-founder and CTO of D2IQ, which you probably have not heard of, and what used to be called Mesosphere, which you most assuredly have. Toby, welcome to the show. Thank you for having me. Of course. So let's start at the, I guess, burning question that at least is on my mind,

if not a bunch of other folks. Mesosphere was a company that everyone in the infrastructure space at least had a vague awareness that there was that thing over there. And last year, I think it was last year, time is speeding up, the company rebranded. What was behind that is, in hindsight, it wasn't a very good idea to put a technology name into our company name, to be honest, because technologies change over time. And we obviously started the company Mesosphere in 2013 around Apache Mesos. That was the core open source project that we had been using, my co-founders and I had been using at Airbnb and Twitter. And we wanted to start a company around that to help every enterprise out there to adopt Apache Mesos. But very quickly, we actually started helping people with other technologies from the cloud native ecosystem.

We helped folks automate things like Kafka and Cassandra and Spark and build these data pipelines on it. And very quickly actually got involved in Kubernetes as well, actually in the first year when it was announced. And so, you know, over time, the name Mesosphere as a company name became sort of a stumbling block for us because we always had to explain that, yes, we are the Mesos company, but we also do all these other things, right? We help you build data pipelines and we help you with kubernetes too and so it kind of became this anchor and we decided you know it's maybe not a good idea to have a specific technology in our company name and so we decided to rebrand and we wanted to pick a name that um kind of expresses what we really do what we help our customers with and that is we help them on day two, we help them be successful on day two, and be smart about their day two operations. So day two, in the sense of,

you know, the DevOps concept of day two, so the ongoing operations and maintenance of production systems. That's what's behind that. You always could have gone down the path that I did, where I started with a newsletter last week in AWS, a consulting company that had no bearing on any of it, and this podcast, Screaming in the Cloud. There were three brands instead of one, which means that whenever anyone asks me, so what do you do? My answer is always, well, it depends. Can you contextualize that question for me a bit more? It winds up effectively having to lead down this weird path of branding things very differently. And then, of course, I started another podcast with a completely separate name on top of that called the AWS Morning Brief.

And at this point, I just sound like I'm professionally confused. Naming is hard, especially once you have a name that is no longer accurate in some ways, but it's something that people have a definite affinity for. You have brand recognition. I mean, we had a guest on previously from Palantir.net, which predates the terrifying Palantir in the Valley by about 10 years. And it seems like their tagline has become, we're Palantir, no, not that one. That was lovely. Yeah. You know, obviously, like you said, naming things is hard and renaming a company is hard too. You know, we've built up a lot of brand equity over the years. And so what was important to us is actually that we don't give that up.

And so the name Mesosphere actually lives on. It's now the name of our product family around Mesos. So name lives on, just the company has a different name. So are you finding that, again, obviously from the time that you started Mesosphere back when, when was that? 2013. Okay, forever ago, internet time. There have been some, let's say, upheavals in the infrastructure space. Back then, I would have, frankly, bet the farm on Mesos. It seemed like the right answer. A lot of the big shops were doing that. And today, whenever you suggest that to people, they look at you a bit strangely and say, yeah, if we're doing anything net new, it's probably going to be on top of

Kubernetes, which I have a laundry list of complaints about. But I'm curious to get your take. How have you seen Mesos' rise and fall through the eyes of what you do for customers right so you know i think what we're seeing with with kubernetes is really the power of community when i talk to folks and ask them you know why kubernetes that's the thing that people most commonly mention it's the community in the broadest sense meaning there's a place online where i can go to learn about kubernetes and related technologies there's a place I can recruit talent from. There is people that want to have that on their resume. So, and obviously the community is so much bigger than any single vendor could ever be. And so, you know, that's where a lot of innovation happens and innovation happens much faster

in that community. So that's really the most common reason we hear. Mesos started as a abstraction layer for large compute clusters. And while we do a lot with Kubernetes now, and we have an entire product line around it, we also still have our Mesos product line, and it is still the platform of choice for those large scale deployments. So we have customers with, you know,

hundreds of thousands of nodes in production and they're running Mesos and they will be running Mesos for a while. So it's really a best tool for the job kind of situation, right? If you're a small shop, you're, you're getting started with cloud native. You have maybe a 10, 20, 30 node cluster. That's 20, 30 nodes is where we see most clusters out there in the industry. Mesos may be not the right choice because it is built for scale. And so what we said is, hey, let's offer our customers what they want.

Let's give them Kubernetes. Developers want that. And we still keep the Mesos platform for those large scale deployments. Are you seeing net new activity around Mesos in 2020? We do actually. So one thing that we built, that we invest a lot of time in over the years is helping customers automate data services, right? Building end-to-end data pipelines with Kafka, Spark, technologies like that. And the experience that they get around that on Mesos doesn't exist the same way yet on top of Kubernetes. We're working on making that happen. And obviously there's a lot of activity around building Kubernetes operators. There's various different approaches to building operators.

We started an open source project about a year and a half ago called Kudo that aims to make building operators very easy. It's based on our learnings on top of Mesos. And so, you know, the ecosystem is going to get there, but it's not quite there. And so we're actually seeing a lot of people still start new projects around these data infrastructure projects on top of Mesos. It's interesting you bring up releasing open source offerings around, I guess, anything in the infrastructure world. I mean, lately, it seems that there's been a bit of a pretty persistent narrative around the danger of open source as a business model, because then someone like AWS comes in and launches effectively what you do as a managed service.

Is that something that's currently on your threat radar? Is that something that you don't see as being particularly credible? Or am I missing something entirely? So it's definitely on our radar. And I think there is, you know, while while this is a threat that everyone's facing, there are also opportunities to build differentiated product for maybe a different, different use case for different customer demographic. So what we see a lot these days is folks wanting to run any combination of hybrid or multi-cloud scenarios, right? So they want a public cloud like experience, like they can get from AWS,

but they want it on the infrastructure that they choose. So we see a lot of activity. We work with a lot of customers that have industrial IoT use cases, right? So let's say they have a manufacturing plant, a factory where they have thousands or tens of thousands of sensors that produce data in real time that they need to process and do things like predictive maintenance, finding outliers in the sensor data and things like that. Those factories are often in areas where they don't have a high quality connection to the cloud. So it's not feasible to send all that data in real time to a public cloud. You have to kind of process it locally.

And so essentially what those customers need is they need a mini edge cloud, right? They obviously don't have highly skilled cloud native engineers in every one of their manufacturing plants. And some of those people have over a hundred of these plants. So what they need is really a public cloud like experience sort of in a box that they can deploy on the edge. Now, they also want to run a bunch of infrastructure on the public cloud, and they also want to run a bunch of infrastructure on the existing data centers. So how do you do that? How do you operate, you know, pick Kafka as an example, or Spark in a consistent way across

all of these platforms? That's, you That's one thing we're focusing on. And yes, you can go to AWS and you can get a managed Kafka, but you can't get it in a manufacturing plant or in an air gaps case where you don't have any internet connection. So there's still a lot of these use cases out there and that's how we differentiate or it's one way we we differentiate i've always said that one of the most effective attack ads that you could come up with about running kubernetes would be to send someone who's considering it to a three-day kubernetes workshop and by the time they come back they will understand that here be dragons and that has sort of continued to be the case as far as talking to anyone who's doing anything at significant scale in the Kubernetes ecosystem is the sheer level of abstraction built upon

abstraction that fundamentally turns into something that is incredibly difficult and opaque to understand what's going on underneath the hood. So it's not the day one experience, day two experience, as you alluded to earlier in the recording, that once you have something running and then you see a degradation or an intermittent failure, it becomes super challenging to figure out what's causing that issue and why. That's absolutely right. And the typical journey that we see a lot of people go through is something like they decide to do cloud native,

they decide to do containers or their boss tells them to. They go on the internet, they go on Stack Overflow or wherever, and they find Kubernetes. They try it out, they download it onto their laptop and have a great experience. The first touch experience with Kubernetes is really great. You can get a container up and running quickly or get your guestbook example up and running quickly.

And so too many people assume that putting it in production is going to be a similar experience. And the first common mistake we see is that people assume that Kubernetes is all they need, that Kubernetes gives them all the tools that they need to put a container stack into production at an enterprise. And that's just not the case, right? You need a bunch of other tools from the cloud native ecosystem around Kubernetes. You need a monitoring stack, you need logging, you need networking, load balancing, all of those things. And because people kind of take this fairly agile approach where they try it out and then when they hit a wall, they figure it out, right? Let's say I start a container and a stateless container and that's a great experience.

Now I need to add state to it. How do I do that? How do I get volumes? They kind of take it step by step. That's where we see a lot of cloud-native projects failing is because, like you said, at some point they face the complexity and they're like, oh, wow, there's actually a lot of things that I hadn't thought about. What we like to do there is make sure that people are educated about that. So that we say, hey, when you need to go to production,

these are all the things you should pay attention to. Make sure you have proper monitoring. Make sure you have proper logging. Make sure you have proper logging. You need a networking layer and so on. And that's part of what we teach in our Kubernetes trainings too. So we do these free trainings in the field in various different cities in the world

to just highlight these problems because like you said, a lot of people just aren't aware of those. This episode is sponsored by ExtraHop. ExtraHop provides threat detection and response for the enterprise, not the starship. On-prem security doesn't translate well to cloud or multi-cloud environments, and that's not even counting IoT. ExtraHop automatically discovers everything inside the perimeter, including your cloud workloads and IoT devices, detects these threats up to 35% faster,

and helps you act immediately. Ask for a free trial of detection and response for AWS today at extrahop.com slash trial. One of the, I guess, arguments in favor of Kubernetes historically has been the hybrid story, which I'm sympathetic to, and the multi-cloud story, to which I'm slightly less sympathetic. In that, on paper, it looks fantastic. In practice, it means that you're not just dealing with one cloud provider's deficiencies, you're dealing with all of them. And that's been a recurring subject of some debate on this show

for a while now. Where do you stand on the idea of multi-cloud as a best practice? Yeah, it's one of my favorite topics. So I think multi-cloud is where everything is going to move. To me, multi-cloud, it also includes hybrid because every large enterprise has massive workloads that they want to keep on-prem for various reasons, whether they want to protect their data or whatever. And at a certain scale, too, actually running your own gear becomes more cost-effective, too. So I think ultimately every enterprise is going to be there. Now, the reasons for why they want to do multi-cloud vary. And I think a lot of folks, when they hear multi-cloud, the first thing they go to is,

oh, you know, I'm going to have this abstraction layer, Kubernetes, or whatever it may be. And I'm going to dynamically move my workloads around. And I'm going to look at where the costs are optimal. And or, you know, I'm going to optimize for other things. That's typically not the main reason why people do that although we are working with some customers um that are fairly sophisticated that are literally doing that they're watching the spot instance price market on all the different cloud providers and then you know hour by hour decide where things should go um but that's only a handful and they're you know sort of

ahead of the pack they They're fairly sophisticated customers. For most folks, the reasons are something different. We work with a lot of companies that work globally, that work in a lot of different countries and jurisdictions. And so they need to take a look at data privacy laws and regulations around that. So when the infrastructure they stand up in China, that data that they process there for the Chinese customers can often not leave the country. So they need to run on a Chinese cloud provider. They may be operating in Europe, so they need to run on European infrastructure and within Europe in each country. And so in this case, when we say multi-cloud,

it's often not actually one of the big three cloud providers that they're thinking about. This may be some fairly small infrastructure as a service provider in one specific country that they need to run on top of for these data privacy reasons. And so in this scenario, multi-cloud makes a lot of sense, right? Because you want to architect your stack once, you want to build it on top of an abstraction layer

like Kubernetes, and then be able to stand that up in multiple countries on different IaaS for those reasons. That's a common one that we see. Obviously, that's with companies that act globally that are working in a lot of different jurisdictions. Another reason we see for multi cloud often is that they they want to handpick certain cloud provider services that they like. Right. So they may want to go to provider A for their machine learning stack and they want want to go to provider B because they have

the better managed databases. So it's more of those reasons, I think, not so much what most people go to immediately, which is dynamically moving the workloads around. And the dynamic movement of those workloads seems to be what people put up as, oh, it would be great to be able to magically deploy our entire application anywhere we need to at any point in time. Yeah, except data gravity always makes that

a bit of a challenge. The joy of trying to get even that baseline, fundamental, consistent experience working between two providers, even when one of them is on-prem and you control virtually every aspect of it, is non-trivial. An argument I've enjoyed for a while now has been,

great, take your provider, your primary cloud provider, whichever one it happens to be, I don't care, you probably care, I don't care, and try and go multi-region. Be able to span to multiple regions of the same provider and see what breaks. It's a good baseline story for the things you're going to have to start thinking about and then some when you start going multi-cloud. Now, there are workloads that justify that level of work and experience and stress, but it's certainly not, I'd say, worth an awful lot of companies' time and effort to do it.

Yeah, you're absolutely right. That experience is very similar to what you're going to have to do in multi-cloud. And there's one more use case I forgot to mention earlier, and that is people in certain industries that are regulated, they actually have to go with multiple vendors. They have to, for regulatory reasons, pick two or more cloud providers. And so they're kind of forced to do that. One of the main things you're going to have to build your own, or it's actually something we help our customers with is replicating your data, right? Like you said, data has gravity. And so the people that we see that are successful at doing multi-cloud or multi-region,

they do things like using, you know, Kafka to replicate their data or using Cassandra to replicate the data asynchronously between different infrastructures. So that's not something the cloud provider offers and um but you know we help folks manage cassandra manage kafka so it makes that a little easier so tell me a little bit about where you came from most people don't decide to spring fully formed from the forehead of some ancient god in the forms of a co-founder of a company in the infrastructure space everyone has heard of. Where were you before Mesosphere, if there can be said to be a time before the Mesospheric era? There is definitely a time before the Mesospheric era, yeah. My exposure to the internet and infrastructure and HA basically started as a teenager. So

my co-founder Flo Liebert and I, we grew up in Germany in the same town. And when we were teenagers, we started building websites, and we started building some adventure games and things like that. So we knew how to build websites. And we grew up in a fairly small town, 50,000 people, and this was the late 90s. And even in that neck of the woods, companies started to hear about the internet. And so they're basically wondering what this thing is. Someone told them, hey, you need to be on the internet. You need to have a website and you need to have an email address as a business.

But they had no idea how to think about this and how to approach it. And at the time, it was really hard to get a website because you basically had to work with three different companies. You had to find someone to design it for you. You had to find someone to program it and then someone to host it. Those were typically three different companies. And so what Flo and I did is we said, hey, you know, we know how to build websites and we know how to run Linux servers. We just did, you know, dabbled with that on the side. And so we actually convinced my mom to register a company.

And so we could program websites for people and host them. So that was sort of our first experience with infrastructure. And, you know, even back then we did HA things. We bought two servers, not one. One would have been enough to host all of our clients, but we wanted it to be highly available. So got some experience with that, running Linux servers, running production infrastructure. And then when you grow up in Germany or anywhere outside of Silicon Valley and you're in tech, then you hear the stories, right? You hear about Silicon Valley. And

I've always imagined it to be the super futuristic place. And I kind of wanted to check it out at some point. And so in college, I found an internship at the startup down in Redwood City and joined them for three months and helped them build their website in PHP and Ruby on Rails at the time. So that was my first exposure to silicon valley and i just loved the you know the energy the people that are full of ideas and the speed at which things get built and so after i finished college i um joined that same company where that that internship worked full-time and built the infrastructure there built built the website. And then my next job was Airbnb. I joined them pretty early on as engineer four.

And so, you know, wore a lot of different hats there. And one of the things I did there is also design and build the infrastructure for their massive growth. Hired the engineering team and did some machine learning work there too. That's my other passion besides infrastructure. And at Airbnb, that's when we started using Apache Mesos. We built data infrastructure there based on Apache Mesos, which my third founder, Ben, was working on at Berkeley at the time. I should mention Ben and Flo and I, the three founders, we've known each other for a long time. Flo and I grew up together and Flo did a student exchange and stayed with Ben's family in high school too. So we all love computers. We talked about Mesos and that's

how Twitter and Airbnb ended up using Mesos. And to us as the people running the infrastructure there and the people with the pagers that would go off at three in the morning sometimes, Mesos really felt like magic. It was a 10 times better solution because we could automate a lot more things and the pager wouldn't go off as much in the middle of the night. And so that's when we decided, hey, this is a great opportunity to start a company because the problems we were solving there with automation, they were not unique to Twitter or Airbnb or any Silicon Valley tech company. They were infrastructure challenges that every company would face at some point. And this was around a time

when this idea of software is eating the world that Marc Andreessen wrote about, I think in 2009, that was still fairly new. But we saw that every company, whether it's a bank or an insurance company or a car manufacturer, will have to run large scale cloud infrastructure at some point. And in fact, in order to stay competitive in the future, they're going to have to use some of the same technologies that the best software companies in the world are using. And so that's where we saw the opportunities. We saw that we had this tool that automated a lot more things and made infrastructure more robust and scalable and cost-effective. The only challenge at the time was, you know, it was an open source project. There was only a few people in the world that knew how to use it. And so we decided, let's form a company around it. Let's build an enterprise

product around the open source core. And that's how Mesosphere was formed in 2013. I have vague recollections back in the dawn of my version of the era of computing. We would configure a core switch at the office I worked at. Then we rented a van and a few of us on the tech ops team drove it down to the data center about 30 miles away and did the installation. And we learned a few things. One, we are super crappy movers. Two, it is vaguely disturbing the company decided not to spring for professional insured crappy movers. Two, it is vaguely disturbing that the company decided not to spring for professional, insured, bonded movers for this. And thirdly,

there's something very surreal about loading a piece of computer equipment that fits in a rack, two or three of you can lift it up, into the back of a van, and that van costs less than the Switch does. That, it was still such a strange and surreal experience.

You don't get to experience that in the world of cloud in quite the same way, but it's more than made up for it with the other hilarious and sarcastically disturbing things that it has exposed for us. Yeah, absolutely. I think interacting with real hardware and a real data center, it's an experience that really, really shaped how I think about stuff.

One big way is that always expecting failure, right? Because I mean, I could tell so many funny and partially funny, partially painful stories of things that went wrong in the data center, in the physical world. I think what that taught me is to just expect failure always. Everything can fail at any point in time. And then when you build software, even if you build it on top of a bunch of layers of cloud computing, you have to expect that. And even in the cloud, machines will fail and you get that email from AWS that says, you know, this instance is now broken. And I think if you don't have that experience, racking and

stacking gear and seeing a bunch of physical failures, you know, you might not think about it the same way. So I wouldn't want to miss that experience. But yeah, like you said, there's all kinds of other funny behavior that happens in the cloud. It's just abstracted away by a bunch of layers. But I've lived through some funny cloud outages there too where packets went in a circle

and then that caused EBS to go crazy and all kinds of fun stuff oh the cascade independencies are always the story the stuff of legend after the fact and it makes sense in hindsight every failure does to some extent but when you're in the middle of it you're wondering if you've lost your mind if the old rules no longer hold this behavior is completely inexplicable what happened and i guess figuring that out and living through that a few times is really, I think, the best way to learn to approach those things in a more methodical way. But oof, some of those early failures were not fun. Seeing aspects of that manifest in cloud environments is absolutely something that is definitely reminding me that old things are new again.

That's true. And one thing we shouldn shouldn't forget too, is that by using these cloud services, you give up a lot of control too, right? Because when things do fail, there's only so many things you can do, right? APIs may all of a sudden be read-only and, you know, you cannot restore your database from a backup all of a sudden, or you cannot promote your read-only database to a master instance all of a sudden. So there cannot promote your read-only database to a master instance all of a sudden. So there's definitely that aspect too, which that was much easier when we were running our old gear. You're in full control, right? You control the whole thing. And if you want to do something crazy to try and fix a problem, you can do that. Can't do that on the cloud.

So last question, I suppose, before we wrap this up, I made a prediction about a year ago. I said five years now. So we have four years to go where I argued that in four years now, nobody is going to care about Kubernetes. And my argument was not that it's going to dry up, blow away and be replaced with something else, but rather that it's it or something like it is going to slip below the surface of awareness. Just like we don't have to worry about what kernel version we're running on an operating system anymore, we won't care what's handling orchestration in our various data centers and cloud providers. Do you think that that is an accurate prediction or I'm going to be eating some crow?

No, I think that's absolutely accurate. It is a substrate. It's becoming a substrate. And unless you're directly involved in, you know, adding features to Kubernetes, or you know, you're using it in some other way where that requires you to make changes to it directly, you're probably going to use some other higher level API, right, you're probably going to be interfacing with a CI CD system as a developer. And, you know, maybe you know that it's Kubernetes under the hood, just like, you know, right now

that it's Linux under the hood. But, you know, you're not really interacting with it directly that much. Or, you know, if you're a data scientist, or you work in the data infrastructure world, you're much more likely to use a tool like Kudo to deploy that service versus, you know, trying to piece together your Kubernetes primitives in order to stand up that service. So I absolutely agree with that. I think that's a trend. It's sort of this wave, this abstraction layer wave that's always behind us, or the rising tide of abstractions.

And so I think the same thing will be true for Kubernetes. I think most folks out there, individual developers or end users of a platform, they'll know that it's there, but they're going to be talking to other APIs at different levels. And we're seeing a lot of activity around CICD right now. I think things like Argo and Tekton are super exciting. There's a lot of activity around that and people wanting to use GitOps approaches to deploy their software. So I think those are some of the signs that we're seeing of the abstraction layer rising and then, of course, serverless too. So if people want to hear more about your thoughts on these and other topics, where can they find you?

So they can find me on the usual places. I'm pretty active on Twitter. I'm on LinkedIn. I give talks at conferences sometimes. Those are some of the first places to find me. Excellent. Thank you so much for taking the time to speak with me today. Absolutely. Thank you so much for taking the time to speak with me today. Absolutely. Thank you so much for the opportunity. Toby Knaup, CTO and co-founder of D2IQ, formerly Mesosphere. I'm Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast,

please leave it a great rating on Apple Podcasts. If you hated this podcast, please leave it an even better rating on Apple Podcasts. this has been a humble pod production stay humble