In The Arena by TechArena - WebAssembly: Paving a Path to Distributed Computing with Fermyon

Episode Date: January 30, 2023

TechArena host Allyson Klein chats with Fermyon CEO Matt Butcher about WebAssembly and how it addresses many of the challenges that containers and virtual machines have brought to the cloud.

Welcome to the Tech Arena. My name is Allison Klein. Today, I'm delighted to be joined by Fermion's CEO, Matt Butcher. Welcome to the program, Matt. Yeah, thanks so much for having me. I'm excited to be here too. You know, we've been talking about cloud computing on the Tech Arena, and I was just delighted when I got the news that you were coming on the show, because you are mixing things up in cloud. And I wanted to talk to you about what you're innovating on. Why don't we just start with an introduction of you and Fermion? Okay.

So Fermion, we started about a year ago. Prior to that, I had really gotten interested in what we now call the cloud native ecosystem. At the time, it was really more the infrastructure as a service world, way back when OpenStack was first getting started. I did content management systems. I was a big Drupal developer. I started working at HP Cloud right when they were getting going. And you have these moments, right, where I'm saying to myself, right, yeah, I'm a big Drupal

developer. This is what I'm going to do for the rest of my career. I really like it. And then I experienced cloud and this idea that my preconceptions of how computing worked and what servers did and what workstations did had been so rocked to the core that I was like, okay, I need to do that. I want to do that every day, all the time. So I worked my way in HP cloud from a CMS team into the core infrastructure team, worked on their platform as a service offering, worked on a bunch of different projects on OpenStack, and I just absolutely loved it. So virtual machines really was what got me going in this whole cloud world.

Fast forward a few years, I ended up leaving HP, went to join a startup, ran the cloud team there, and that was when I really got my first taste at Revolve of the kind of container ecosystem. And containers were starting to challenge many of the assumptions that we were making about how virtualization needed to be done and what the shape of a compute workload looked like, right? Big, big, big things. And I got kind of in this headspace where I was thinking, okay, containers are just going to absolutely destroy the virtual machine industry. I've got to pivot really fast and I've got to get all in on containers. The startup that I was

working for at the time, Revolve was acquired by Google. And so I rolled into Google for a while. That was a really fun view sort of behind the curtain at Google and how they were doing at the time App Engine was a big product. It was prior to them getting into the virtual machine space. And so I got this look behind the curtain there and again was like, oh, this is what containers could be like. So when I left Google, I went and joined a startup in Boulder called Deus. And Deus had, what proved to be like the perfect nexus of my set of interests at that point. They were building a platform as a service, which again was the kind

of thing I was building at HP. But they were building it on the emerging container ecosystem. And so I was just all in on this. And we built this platform called Deus Workflow that was based on the idea that the developer pushes their code and it packages it into a container and deploys it out onto a server somewhere. That was when kind of the third piece of the great shakeup for me happened when Google released Kubernetes.

We had played with a number of orchestrators and I liked a number of them. But when Kubernetes was released, again, having been at Google, there's a sense in which you look at this and go, oh, this is the security blanket. I know how this works. And because it was very similar to Borg, it was based on the ideas Borg kind of brought to the fore. But also it was really exciting. And the way that Brendan Burns had designed it just very much appealed to me. And so at Deus, we pivoted from really focused very much on building a platform as a service to focusing more on this emerging container orchestration world. And that was kind of what got Microsoft's attention.

And so we ended up at Microsoft. My team was building the open source container ecosystem tooling for Kubernetes. We were working on Helm and Brigade and Draft and CNAB and all of these things that made their way into Linux Foundation or CNCF. And we were just having a great time. I would say, I say this frequently, I think the job I had at Microsoft was just the best job

anyone can have at Microsoft. Essentially, I got to build open source stuff. My team got to play around with the container ecosystem, try things out. If they failed, we wrote down what we learned from that and tried something new. If we succeeded, then we would open source it and push it ideally into somewhere like CNCF or Linux Foundation or other organizations like that, and then really work on fostering an open source community, which was a whole lot of fun. We loved that.

But there along the way, we started to run into challenges that we couldn't quite solve in the container ecosystem. The first one was this sort of, and it continues to this day, this feature of containers where when you're creating your container, when you're writing your Docker file, you need to know really at that point what the operating system and the system architecture of the deployment target are. And so if it's ARM64 in Windows, you use one set of libraries. And if it's Intel on Linux, you use a different set of libraries. And there are all kinds of permutations in there. And that problem boils up to the developer. So we identify that as

one of those cases where developers don't really want to spend their time writing four different versions of the same thing. Nor do operators want to spend their time explaining to developers, Hey, we're running on ARM now. So we need you to go back and rebuild all of this stuff that runs on ARM instead of Intel. So that was one of the first pain points where we identified it and said, Oh, this is interesting. So we started cataloging some of these and that was ultimately what led us to WebAssembly. We ended up with a group of, I guess you could call them missing features from the container ecosystem, or you could just call them attributes that container virtual machines didn't really have so performance, we really

wanted ultra fast startup time. We had this conception that the virtual machine is the big heavyweight class, right? You package everything from the kernel all the way up to your application, file system, drivers, utilities, everything gets bundled in this gigantic big image. And then containers are sort of like the slimmed down version of that. You don't need the kernel, you don't need the drivers. You really only need a little wedge of the file system

that has the thing your application needs. And so we were looking at that model and saying, all right, if we've got the big heavyweight class here and we've got a middleweight class, there's got to be something that we could execute really fast that we could package up in these really, really spelt little packages. But that would really only have the application code in a bare number of files that needed to operate. No file system, no utilities, no shell and the things we tended to package with containers. So we had this wish list, this set of features. And for a long time, we just kind of put them on the back burner and said, you know, we'll get to this stuff later, it's an interest.

These are interesting data points, but you know, we're loving the job we're doing here and we're building some cool open source stuff, but we had an all team. We were in Vancouver and the whole Deus Labs team, our team in Microsoft was named Deus Labs. We are all together in Vancouver having just finished a day of intense planning meetings. And you know how you get in that space where you're like, I'm tired. The analytic part of my brain kind of wore out a couple of hours ago, but also I just kind of want to talk about what's interesting to me, right? I just want to brainstorm. And we got into that kind of brainstorming creative space where we're just sitting around the table, you know, five or six of us and going, you know, what if we tried to solve

all of these? What if we really took a crack at this nearly instant startup time at cross-platform, cross-architecture at finding what this kind of flyweight class of cloud computing would look like. And we threw a couple of ideas around. We could try this. We could try this. And we left from that going, you know, this is something we should spend a little time on. So we were planning on spending our research time on Friday afternoons working on this.

And we did for a little while. And WebAssembly started, you know, it hit our radar right away. But then COVID hit. Here we are, a team of remote engineers, all kind of working on our own because our whole team, very few of us worked in an office. Everybody's kind of depressed and we're frustrated and also admittedly a little bit bored. And we said, okay, well, we got this web assembly thing that's looking promising. What if we just collectively as a team said, you know, this is going to be our Friday project. We're going to use it to get our energy level back up.

And that was when we really started looking at it. And as 2020 wore on and then 2021 started, we thought, you know, I think we've stumbled into something bigger than what we realized. And ultimately, it wasn't really Microsoft's thing to really push that project forward. Brendan Burns, my boss there, had encouraged us to ponder whether starting a startup would be a good way to do some exploration. And we ended up doing that. And so in November 2021, we started Fermion. And we've been kind of plowing along course ever since. That's an incredible story.

I was thinking and reflecting about my own journey in the cloud space while you were telling that story. You're like, oh, my gosh, I remember HP Cloud. That was amazing. And too bad it didn't work out. They were doing some amazing things in OpenStack. But you know, like a lot of the points along the way. I've been talking to folks about with all of the innovation and all of the bright people in cloud. And I think that this is what you've been talking about in your story, in your personal story, cloud has not reached that ultimate delivery of what the value proposition is. Full autonomy, single paint and glass management, total control over the workloads. And we've seen a number of workloads at every IT organization managing ballooning over

time. As you look at WebAssembly, and you've talked a little bit about the problems you were trying to solve, do you think that WebAssembly gives IT organizations the tools needed to overcome some of these issues? Or what is it that you think WebAssembly brings to the table to solve? And does it get us closer to reaching that full value proposition? To me, what we have been stumbling toward, and stumbling is probably a good word, and

we've made some really good progress in the last five years or so, but we've been stumbling toward how we do real distributed computing and do it in a way that humans can actually understand what's going on and build increasingly more distributed applications without making them increasingly complex. And what I mean by that is really sophistication does have a level of complexity in it, right? We want to be able to build systems that are better than and can do more than what we had in the past, but we don't want the cognitive burden of that sophistication to end up becoming the operational nightmare. To back up a little bit before I get to the WebAssembly part, I think there have been a couple of pieces of absolutely

critical breakthroughs that have been done. I mean, breakthrough is a funny sort of word to use in this case, but we've just learned how to do a couple of things better. One of them is observability, right? We went from, I mean, if I were to rewind 15 years of my career and you said, how do you debug a problem on the server? I would have said, well, I SSH into the server and I go to the log files and I start reading logs, right? And there are hundreds and hundreds of megs and I'm going to the syslog and I'm going to the, you know, and that was a terribly inefficient waste of time. And we spent, I remember in my days of CMS development, spending hours and hours sifting through logs and trying to find exactly the right line. Observability as a technology has really changed that, right? Not only is it not a matter of having to scan that much data in a text

terminal, but it's turning the data into actionable information and saying, okay, your CPU is now starting to move in the direction of being exhausted in four more hours. And here are patterns across your cluster of servers that don't require you to look at each individual server. And then that was really kind of the bare bones version of it, right? And we've continued to figure out how to look at that. So that's one piece. The second piece, I think, has really been starting to understand how orchestration in the abstract needs to work

if we're going to do distributed computing. And while I'd like to point to Kubernetes and say it was the watershed moment, really, probably even technologies like Mesos and Fleet from CoreOS and Swarm and all of those, each one of those as it debuted made a contribution into us better understanding what it means to orchestrate a complex multi-part application. And I guess I should have thrown in there the microservice design pattern as a way of building these applications has helped a lot too. But we started to learn how to chunk an application, right? The big thing into services, microservices that could together compose that.

And that was really, I I think the second part, I think where WebAssembly is really contributing to this story now is that we've started to learn how to run these kinds of distributed applications in a very specific constrained environment. And that is, you know, the data center inside of the Bespoke cluster, but we're kind of stuck there, right? Because we don't have cross-platform performance. Even while containers are quite good for performing in this kind of data center role, it would be hard to start breaking out of the data center and moving these things out to the edge dynamically, and then out to the far edge,

and then all the way out to the user's cell phone when they can do it. Our vision of the way that distributed computing is going to go is that we're going to stop looking at the cluster as a centralized thing that lives inside of a data center. And we're going to start being able to look at it as the sum total of compute resources that I as a user, and however you want to flesh that term out, I as a user can rely upon, right? That means there's got to be a security constraint there. I have to have a sense of ownership or rights to use it, but it doesn't matter if it's in the data center. It

doesn't matter if it's on the edge. It doesn't matter if it's my personal or mobile devices. We can just begin to spread out our application so that the parts that benefit most when they're close to me can be close to me. And the parts that benefit most when they're close to my data and the data center can be out there. I think an easy way to sort of visualize this is to think about, I'm not a big fan of this whole metaverse thing. I will be honest there, but it is the perfect example of distributed computing and where we could go if we do this well. Because the visual aspects of it, you know, human eyes are highly attuned to our surroundings.

We don't tolerate lag when we're turning our head left and right. We need to see the image match the velocity with which we turn our head, which means the compute processing to do that part of it has to be as close as possible to my eyes, right? Whereas synchronizing between hundreds of thousands of users who are each participating in the same virtual space means a lot of synchronous work done, some on the edge, maybe some in the data center. And I think for gaming, for that kind of thing, all of those sorts of interactive environments and interactive here, not even being necessarily the visual aspect, but the way in which we have to synchronize a lot of data moving back and forth in a way that feels natural to all of us as end users. That requires a different way of doing distributed

computing. And I feel like WebAssembly is really an unlocking technology that's going to get us at least substantially further down that path, where when we talk about distributing an application, we're not saying I'm going to load it into a data center. And in my tendency, and it's going to split it along five worker nodes, we're talking about, I deploy an application, and then it just moves so that the various pieces and parts of the application run in their kind of optimal environment. So I know it sounds a little sci-fi-ish and I suppose it should because it's groundbreaking. It's farther than where we are today. But I think WebAssembly is an important step there. WebAssembly is nascent enough that

even where we're sitting on the kind of cutting edge of this, we can't quite see what else there might be between here and there. And so there may be more technologies that we just don't know need to exist yet. But we feel like this one is really very much an unlocking technology steps to getting to that kind of sophisticated distributed application. One thing that I think was really interesting that I read from you before is that it's not an either or question. It's an and question. You were talking to an IT manager

that's managing an environment that has probably a combination of VMs and containers today. How would you say the future data center looks when you integrate WebAssembly? Does anything go away? I mean, we're still running mainframe, so maybe nothing ever goes away. But- Does anything ever goes away.

Does anything ever go away? Yeah, exactly. But does anything become obsolete and is just there because people don't like getting rid of legacy? Or do the value propositions of all three of those solutions still persist in the long term? From my point of view, I think all three of them still do a very, very, very important job on its own. I think virtual machines really for the long haul are going to be sort of the bread and butter way of partitioning off the space where we need to be able to run things. It's really brilliant technology and I think it continues to work really well. Might be less and less visible to the average cloud developer or maybe be less and less visible to the average cloud developer,

or maybe even less and less visible to the average platform engineer, but I think they're going to be around for a long, long time. Containers, I think, occupy an interesting space because they're really well-suited for these kinds of long-running applications that need to run for minutes, hours, months, and longer. WebAssembly is really better suited for short-term applications. And by that, I mean a few milliseconds up through maybe a minute, maybe 15 minutes. We'll just say 15 minutes at the top end, even though it's kind of an arbitrary number. So I think what we're going to see is some of the workloads will shift around a bit,

right? Some things that are done with containers today will be done with WebAssembly, just like we saw shift around a little bit when some things that were done in virtual machines were moved to containers. So we'll see a little bit of shifting. And as soon as you start to see shifting like that, people jump to the conclusion that one of them is going to go away or something is obsolete. In this case, I just think we're going to see shifting until we find the right fit for each of these technologies. And then we'll see them stabilize the way that virtual machines and containers together have sort of stabilized today. I mentioned earlier that I took sort of a view that containers were going to destroy virtual machines and just obliterate the technology.

And obviously, youthful me was woefully incorrect on that one. Instead, we've seen virtual machine usage grow just as fast as container ecosystem has grown. And that, to me, is a really encouraging thing. It says we're learning to build the right levels of abstraction and that some of them don't need to change or change, have kind of found their fit and others are going to change for a period and then sort of solidify, which I think is what we're seeing in the container ecosystem, that solidification of the technology. And then emerging ones will augment where the previous ones don't do as well.

So to give an example of why I think WebAssembly is the right third category of compute here, it has been very hard to figure out a way to run, to execute workloads securely that really only need to execute for a few minutes or seconds in that kind of compressed timeframe, because the technology was we built for isolation for security isolation containers and virtual machines each take a long time to start a virtual machine takes minutes to start typical containers take 12 to 15 seconds to start but the workload that we're talking about executing in say a lambda-like function is milliseconds so we have to invent all these technologies where we pre-warm things, which means we're

sitting around spending electricity and exhausting compute resources that could be used better. WebAssembly gives you that kind of compute abstraction that says you can drop a workload on here and have it starting in under a millisecond. And then, you know, it lives for 30 milliseconds and then we clean up and we can do all of that in a very secure way. So I think what we see is WebAssembly is filling a niche that before we had kind of clunky solutions that we made work, but that were not working efficiently. And now we have something that can do that efficiently.

That doesn't encroach too much, I don't think, on the container ecosystem and definitely doesn't on the virtual machine ecosystem. So I think that's one of those earmarks when you find a technology that solves a group of problems, but isn't competing with an existing technology. It means you're adding kind of a new right layer of abstraction, because you've already found the strengths of the previous ones. And now you're just addressing the weaknesses and solving for those. So yeah, I think all three of them will be around. I wouldn't even be surprised if we, you know, found maybe a fourth, though I have no idea what needs it would be solving yet

because I don't think we've seen those needs emerge. I hope it doesn't take another pandemic for you to figure out what that fourth is. So you've released Spin, your open source project, and you've released Fermi and Cloud. Tell me about what the response has been and what you're hearing from customers in terms of their plans for adoption? When we first launched Spin, and Spin came out, just to give a quick roadmap, right? Spin came out March 31st, the day before April Fool's Day. We're like, we're not releasing on April Fool's

Day. So it came out right before April Fool's Day of 2022. And then Fermion Cloud, we launched on October 24th of last year. And Spin is the developer tool and Fermion Cloud is the cloud environment where you can run these things. With Spin, our hope was at this early stage in the company, we would be able to have a tool that could articulate to people, here's the value of using web as something. Here are the kinds of things you can build. Here's this kind of serverless function model and a great runtime that we hope you enjoy using. We were shocked at how quickly people started looking at this. I think we got a

thousand GitHub stars in a couple of days and we're kind of looking at this going, hey, GitHub is broken, I think. We didn't expect that. The uptake on Helm took us two years to get to a thousand stars. And so seeing Spin shoot up that fast meant, oh, so we've kind of hit on a topic that people recognize right away is interesting to them. Hearing developers say they're interested is always a good thing when you're a startup going, is the spaghetti going to stick to the wall, right? So that was really encouraging. And as we initially started reaching out to these developers, largely what we were hearing was, okay, well, here's the set of features that we would like to see. And here's the kinds of things we'd like to do. A lot of people wanted to run their blogs. A lot of people

were exploring, you know, building microservices. And so as we started designing Fermion Cloud, we said, okay, we want to design a place where people can run this so that they can get from this spin local development experience to having something executing very quickly. So we had this core user story we laid out for 2022. We're big on user stories. And it was, as a developer, I can go from blinking cursor to deployed application in two minutes or less. And the idea was, if we can just give the developer that experience that Web Assembly is not a scary new technology, it's something that you can get going on right away. You open up VS Code or Vim or whatever your favorite editor is, and within a few minutes you should have the hello world experience.

And then from there, once you're at the hello world, then you can start building something. That included needing the cloud, right? Because we needed a place for them to run it. It had to be from zero to deployed application in two minutes or less. So, Fermion Cloud for us was really fleshing out the last part of the story and saying, here is a place where you can just type in spin deploy and immediately go to your web browser and be able to see it running. You can show your friends and that'll be the foundation upon which we as Fermion build a production grade cloud environment. So, we were delighted to hear people giving us positive feedback in that early stage of spin saying, yeah, we like this. Yeah, this is fun. But we didn't hear a lot of people telling us at that point, these are the kinds of applications we're hoping to build for it in the

future. So we got some characteristics of it, but not enough to really paint for us a picture of how people envisioned themselves using it. So when we launched Fermion cloud, we're thinking it'll be interesting to see what people do. We've been just incredibly fortunate to have gotten a whole bunch of users very, very quickly building a whole bunch of different applications. In December of last year, we did a little advent of spin kind of coding exercise thing to get people trying out different features of the platform. And what we're seeing is everything from web page backends to games to API style servers, REST API style servers. And that has caused us to reach the conclusion that this is what we had hoped we would be able to build, a kind of general purpose platform where people can build things where their overriding concern is they want to be able to get it done very quickly and get that fast startup time and good performance. I think we would have been happy. Every product marketer is like, you know, it's a really good thing when your market segment declares itself and they're like, okay, you know, we're going to use this only for, you know, mobile backends. And then you can say,

okay, well, I focus all my documentation on how to write a mobile backend. We were actually really pleased to discover that people were kind of running the gamut of stuff to do. The one that really surprised us was that we have had a lot of requests, particularly from about December to now, with people saying, you know, this would be really great for ETL and data kinds of workloads where we need to watch on a queue for information to come in, process a chunk of it, and then feed it out to either the next step in a process or store it into a database. And we had kind of written this model off earlier as something that people would use Lambda for, right? And we thought there were already enough tools in the ecosystem that people wouldn't want to do it. But one person was explaining to me, but this one is faster and

it's more reliable and I can write in any language I want, again, within reason. But for this particular person, she was happy with the languages we had. And I realized, yeah, within reason, but for this particular person, she was happy with the languages we had. And I realized, yeah, okay, so if we can improve speed and we can improve reliability and we can improve the ability to debug and observe what's going on, then yeah, this is an attractive ETL style framework as well. We've been really happy just to see people using this. We get very good feedback. Of course, we have an increasingly longer to-do list as people say, I'd really like to be able to do this, but what we're discovering is the things they're asking for

are exactly the kinds of things we have hoped to build. You know, great data service layer, good security, good authentication and authorization story, secure vault style integration for storing credentials, all those things that we know, if we want to build real applications, we need these pieces. Those are the things people are asking for. And so it's a good time, right? It's great to hear that your vision and the things that your early user base

are asking for really start to align very well. And there are no huge disruptive surprises there. I was thinking about that meeting in Vancouver. Imagine if you could go back to that meeting and visit your former self and your team and say, hey, we're going to go on a ride and this is where we're going to be in 2023. What are the key things that you learned that you had told me we would do this, I would have said there is there is no way, right? There's no way. At the time, you know, you stumble into those moments where you say, yeah, this seems like a neat little innovation. And it's only later when you go, oh, that was a bigger moment than I thought, you know, thought we were having a basket of french fries and decompressing after a day, the chance to go back and say to the group of us who are sitting at that table, this is something worth doing.

Apply yourselves to this. That would have been great, right? And perhaps we would have been where we are today a little bit faster, though. Really, the ecosystem moves as a whole, right? Eight people might be able to shape it, but eight people can't move it on their own. We shape small parts of it and the ecosystem as a whole has to mobilize and move together. But certainly my biggest sources of anxiety starting Fermion and in the early phases of that project, I could have done without them, right? The idea now of looking around and saying, we were onto a good thing. I spent an awful lot of nights sleeplessly doubting my abilities and sleeplessly

challenging our vision. That's really what it boils down to, right? I could have gotten a lot more sleep. I could go back and tell myself, you can sleep a lot better. The idea is not a bad one. One thing that I think we did really well and that I'm eternally grateful for, back then, even all the way back to the days, we focused very hard on building a good, healthy culture that was about innovation, but it was also about teamwork, collaboration, in a sense, vulnerability with each other and trust.

Even looking back now, if you were to ask me what the single most important ingredient was in our getting from there to here, it's not the technology, right? It was the ability to work in a highly collaborative environment with a really high degree of trust in each other and know that that part of the endeavor didn't need to be relitigated, didn't need to be formed. I think you hear a lot about when startups get going, there's the old model, right? The form,

storm, norm, conform kind of model where you start out, everybody's super enthusiastic and then you hit the storming period and startups, many of them fall apart. What a huge thing to not have to go through the storming period because we knew each other, trust level was high, we'd work together and we could cooperate very effectively. So that's the one thing there, if I said I wouldn't change it, right? I wouldn't go back and tell myself you should ignore this part. No, that was actually the most important thing that I think has been our key ingredient to success. I've loved listening to your story, Matt, and I can't wait to see what you guys do in 2023. I think it's going to be a very bold year for you, and I can't wait to follow

the story. Thank you so much for being on today. One final question for you. If folks want to engage, learn more, get their hands on spin, et cetera, where would you send them to go engage with your team and start using WebAssembly? So if you head over to fermion.com at the bottom of the page, there's a link to join our discord server. Uh, lots of good lively discussion going on there. It's a great place to see what other people are doing and a great place to connect with the developers.

Also from there, you can get to the developer.fermion.com site, which will get you up and running. And as I said before, right, blinking cursor to deploy an application in two minutes or less. So those are great places. If you're interested in just kind of catching up on the thought that's gone behind this, the blog at fermion.com, you know, we do everything from very practically oriented ones to our ruminations on what the future of serverless is and performance and security and things like

that. So that's a great place to go if you're more reader than a doer. Fantastic. Well, thanks for being on the show today. It's been a real joy. It has been a lot of fun. Thank you so much for having me.