The Infra Pod - Modal is upping the game for serverless! Chat with Erik

Episode Date: December 4, 2023

Ian and Tim sat down with Erik (CEO of Modal Labs) that's gaining wide adoption of the serverless platform built for data teams. Come sit down and listen with us how the journey started, and what's Erik hot spicy future take on the serverless infra.

Welcome back to yet another Infra Deep Dive podcast. This is Tim from Essence VC. Let's go, Ian. This is Ian, doing a little angel investing, helping turn some security companies into platforms, namely Snyk. And I'm super excited today to be joined by Eric of Modal Labs. Eric, why don't you tell us a little bit about yourself and a little bit what you're working on at Modal?

It's great to be here. I am working on a company called Modal, which I started about two and a half years ago. And basically the genesis was that I've been spending most of my life working with data for at least most of my professional career. And really anything related to data, everything from business intelligence to like large-scale distributed training of large models and everything in between and so a couple years ago i started thinking about how do we make it easier to work with data that's like an enormous problem the amount of like data startups and different components of the data stack is

is like thousands now but what i realized was that there's a core layer at the bottom around just running code that stood out to me. That's hard. And data teams are wasting a lot of time doing that. And if I build that layer, then maybe I can build all these other layers on top of it later. So I started thinking about what does it look like to run code in the cloud and not have to think about everything around resource management and provisioning and infrastructure and stuff, specifically for the type of things that data teams need. So that's why I started working on modal. Now we have a few hundred paying users, a few thousand users running a lot of Gen AI

applications. That's been sort of where we found a lot of initial attractions is we have some people running very large scale serverless GPU models. It's like one big use case. But we also have other types of use cases like biotech, 3D rendering, and all kinds of other stuff. And that's a great introduction. You have this insight that one of the biggest inhibitors to a data team being successful, especially as new stacks emerge, new things come and you have to be successful,

is deploying code and just getting stuff to run in the cloud. What was your insight? Yeah, I think it's almost like easier to look at it backwards. I think one of the sort of trends that I've seen in the last few years is, first of all, data teams have grown a lot. I spent seven years at Spotify and I built the music recommendation system there. So that was one of my core experiences working with data. I was the data team at Spotify at some point.

It was like a one-person data team. Now there's probably hundreds of people doing data. Data used to be the weird side thing that no one was really doing. Now it's pretty commonplace. I think that's something that created a lot of demand for people working with data and a lot of demand for infrastructure. And now this Gen AI stuff is obviously throwing fuel on the fire. So that's one of the tailwinds we've seen.

I think in conjunction with that, like you look at like, you know, backend developers and frontend developers, and you think about like how their workflow has changed, right? Like, you know, especially like frontend development. Occasionally I do it, I'm like, it's like so nice. Like you're just like, you know,

you load up the code in one monitor and like you put, you know, a web browser in the other one and it just like reloads. And so I started thinking about like, how do you have these like fast feedback loops? And I started thinking a lot about like developer productivity and like what's like inhibiting people from like building things quickly.

And a lot of it came back to like, I want to have this like super fast feedback loop. And I looked at like what data teams are doing and realized like, oh, like data teams, like in order to run something in the cloud, they have to like, you know, build a Docker container and then like push the Docker container to the cloud and then like go and trigger it in AWS console

and then download logs. So you have this feedback loop that's several minutes long. It's this super janky developer experience. And there's many reasons why data teams often can't run things locally. They need GPUs or they need to run things on production data. So there's a lot of weird things about how data teams are operating with the fact that data teams didn't really have a stack that's built for them, with the fact that data teams are new and they're coming at this and need tools that don't really exist. Those are all the trends that I think are why we started this company and why I think there's such an enormous opportunity in the market to build infrastructure for data teams.

Many, many founders and people have been trying to reduce complexity or build a for data teams. Many, many founders and people have been trying to reduce complexity or build a better developer experience. We heard this probably thousands or millions of times from someone out there. But as you know, everyone's approach, what steps they take are drastically different. They have a different idea in their mind.

What does that actually mean? I found it most intriguing about Modo is it looks different. I don't think it looks the same as any other better, easier developer experience. You can argue there's so many of these in the past. They all kind of trying to make things faster.

Reading your back in 2021 software infra 2.0 blog post where it talks about feedback. Can we have no more YAMLs at all? There's a particular set of thoughts process you have going to this space. I think it's just different. So maybe talk about

what are the most important things you have in mind when it comes to building Modo that you think is just different than other developer, better experienced products out there? And why do you think this is so important? And maybe also why people

don't focus on those things? Because I don't think it's like common knowledge yeah exactly i think like a couple things like i strongly believe in is that in order to understand developer productivity you have to look at the developer feedback loop and i think of it as almost like a nested set of four loops when you're writing code like innermost code you're like writing some code and you get a syntax error but like slightly larger code would be like you run a unit test and it fails. For data teams, often that feedback loop has been very janky. Running things in production has been very hard.

And so the feedback loop has been very long. And one of the things why it's so complex for data teams is that because it's so hard to work with the cloud, they sort of leave that to the end and then iterate locally. And then you have this problem like getting into the cloud. So one of the things I started with is like, what if we put the cloud inside the feedback loop? What if we just run everything in the cloud? And so one of the things I believe in is that data teams often have to run everything in the cloud anyway, because that's where the data is.

Why don't we just do all the development in the cloud, essentially? So that's one principle I think made a lot of sense for Modo. I just wanted to build a hosted platform that does everything for you, that seems to get started, that runs everything. Like you don't have to install anything except the Modo client library. The other thing that is more like why no one else has done this is that you have to be a little crazy, I think, to like go down and like do all the infrastructure. I think in the past, like there's been a lot of like people trying to like wrap Kubernetes

or wrap Docker and like build something build something on top of existing infrastructure that does this. And that's often when you go and look at a lot of startups or a lot of bigger companies, you look at what they actually do for the data team. It's often some sort of wrapper around Kubernetes and a bunch of other stuff. And I just realized very early, this is not going to work. I wanted to have the ability to take code and run it in the cloud in less than a couple of seconds. In order to get that speed, I realized like I can't use existing primitives. And so we realized very early, like we're going to have to build our own container around that.

We're going to have to build our own file system. We're going to have to build our own, you know, container builder and a whole bunch of other stuff. I think you have to be a little like crazy to go down and like, you know, like build something like more like a foundational level. And I think a lot of people shy away from that. I mean, every startup is a wrapper to some extent. But I think there's a lot of thin wrapper startups that aren't going to be able to deliver the user experience that I wanted. So we realized we're going to have to go down and rebuild all of that stuff.

And that's why it took a couple of years. We have to spend a lot of time just building a lot of foundational infrastructure in order to get there. So when you talk to your customers today, traditionally, just building a lot of foundational infrastructure in order to get there. So when you talk to your customers today, traditionally, having started companies multiple times, I've thought of problems in this space. I'm like, well, in order to get a customer to say yes,

I'm going to have to apply it in their side-to-side infrastructure, which immediately, this is the chasm problem, right? This is your opportunity to do the craziest thing to get the alpha. I'm curious, how do you think about talking to people about the fact that, hey, it's not going to run on your cube, it's going to be inside my little world. We're building for sell for the data teams. I'm curious, how do you think about talking to people about the fact that, hey, it's not going to run on your kube, it's going to be inside my little world. We're building for sell for the data teams. I'm curious how your thought process in having that discussion with your customers or users are. The answer here might be, we're not there yet, and that's totally cool,

but I'm sure you have a plan to deal with this because it's a fundamental problem in this space. Yeah, I think for us, foundationally, in order to build the infrastructure we wanted, like, we kind of have to control it. And so, like, just structurally, it's, like, easier to own it end-to-end. I do think, though, that being said, today, in the era of the cloud, like, there is more and more of, like, an embrace of, like, oh, actually, like, you can run infrastructure in the cloud. Like, Snowflake has done that for many years, right? And I think there's, like, more and more of more of like an acceptance that that's the way to go. When I look back at like when the cloud came in 2007

or whatever, like that was my first encounter with the cloud. I remember like my first thought was like, that's nuts. Like, why would I want to take like my code and put it in someone else's computer? You're crazy. Five, 10 years later, that's like so normal. And like, we didn't even think about it. Like that's just like a normal part of how we operate.

And now like, I think there's like the exact same situation again, but other vendors in the cloud that are not the hyperscalers and having seen the cloud, like, I just kind of feel like, you know, it will take some time. We're an early stage startup. Like we may not be able to go out there and tell bank of America to move their like, you know, core data pipelines to model. But I do think if we keep pushing for people to move their core data pipelines to Moodle. But I do think if we keep pushing for people to move their stuff and the fact that we're actually running things for

them in our own multi-tenant environment, I think we're on the right side of that trend. The other thing is, of course, the monetization opportunity is much bigger. It's like running people's compute. It's more of a precedent for charging for it. So I think that also helps. The other thing is also, when I look at many startups like i think they often take it for granted because they go out and like they do sort of customer discovery and they talk to a lot of bigger companies and of course like bigger companies they're always going to say that they absolutely need to run things in their own environment but i don't know if that's actually true when people tell me that like i don't actually hear what they say. What I hear is that we don't need the product desperately enough.

So then I'm like, okay, well, how do I make them need the product so much that they're willing to waive this condition? Because I already did that for Snowflake at some point already, probably in many cases. So maybe if I make the product even better, eventually they'll just give in and use modal, even though it's not running in their own custom environment. That's where things are going. And I think a lot of startup founders actually would benefit from pushing harder for that. So I think you used the term data teams. We all have a different assumption of what that means.

What does data teams really mean to you? Is it every developer building any backend? Or there's actually some particular kind of devs you group into as a data team audience as the strongest audience you're designed for? There's a reason I use the term data team and it's because it's sufficiently nebulous. And also the fact that what people actually do with data

and the titles that people use and the roles keeps changing. For a while it was data science and data engineer and then it was analytics engineer and now it's prompt engineer, ML engineer, AI engineer. Like, I don't know. There's so many different things. What they all have in common is like they're working with large data set or like doing large amounts of compute.

They often do things like, you know, more like on a batch way. They often need like to scale things up and down. They need GPUs sometimes. There's like a set of like common characteristics. They're often not super latency sensitive when you're building infrastructure that's actually a very useful thing to slice by because if you say actually we can live with like a second latency for a lot of stuff now you actually like have a freedom to build it in like a very different way than if you say i need millisecond latency for

everything so i think there's like a set of characteristics that they all have in common but i'm a little i'm a little nebulous when it comes to actually defining the customer user base. But there's clearly a lot of people working with data out there. And I think eventually they'll hopefully consider using Modal. Modal is really interesting. I looked at it, played around with it. Something that's very different is

very opinionated. And that opinion, I think, is what you baked into it so that you could create the experience you wanted to create, which is make it really fast and easy to set up, and get going and test the production. You don't have to configure cloud stuff because you define how your code is supposed to work, and then under the hood, it configures itself. Can you describe a little bit how you came to that conclusion? Was that naturally as a part of pulling on this thread? Or is this insight that came from getting rid of the YAML? Where did that come from? And how have

you seen people respond to it? When I look at it, I'm like, this is fantastic. I hate writing Kubernetes YAML. I have managed data teams. We're finding data engineers that can do both the low-level cloud stuff, kubes, Terraform, but also deal with the complexity of actually building at-scale data pipelines. Not often a common skill sets the end of this divergence in terms of people.

So it's very interesting. So can you talk through the thought process behind all of it? Yeah, I don't know if it's super opinionated. I think it's different. One of the things I think is maybe different with modal is that we start to look at what data teams are actually doing, like 95% of the time or 99% of the time, it's all Python. So that kind of presents a natural opportunity. You actually don't have to build for like arbitrary programming languages and i think the approach has often been traditionally with the cloud is like because you don't want

to build for like every language at the same time you just like take a docker container and then you know run that container but i think if you actually like look at data teams since they're all using python you can actually build a much tighter integration directly with the Python code. And the benefit of doing that is like now you can think about like actually Python functions, calling other Python function as a foundational unit of work, not like necessarily like containers calling containers and like pushing to the user to deal with all the like marshalling and serialization, that kind of stuff. I think that's maybe one weird difference between how modal operates and other.

I'm not necessarily long-term hung up on that. I think it's just the right place to start right now because you can actually deliver an end-to-end self-provisioning runtime to borrow Swix's terminology using a single language. In the future, we might add other languages and have very different SDKs for those. I think the other thing, you're talking about the

code defining the infrastructure. I think that comes from more around I just want to have a fast feedback loop. And it turns out that having multiple layers, you have to first build a container and then push that. It's easier if the code just defines it itself. And that's also my experience working

with DSLs and other languages like configuration is kind of annoying like modal has like zero configurations potentially there's like nothing you can configure like it's all code that was someone inspired actually by palumi seeing like kind of infrastructure as code for the first time instead of using terraform you like define your stack in like python or like whatever they actually support a lot of different languages i think we borrowed maybe from that to some extent, but also just like my general observation.

I built a workflow engine in the past called Luigi. Nobody uses it today, but like, that's also my like sort of experience is like configuration often ends up being very limiting. And like at the end of the day, why not just like make an SDK instead? And then like people can just like, you know, knock themselves out like programming things.

And so like making the cloud programmable, I don't know if I consider that opinionated. To me, it's like the right way to programming things. And so making the cloud programmable, I don't know if I consider that opinionated. To me, it's the right way to do things. I think people want a programmable interface to use the cloud. I say opinionated, and then in two years to five years' time, I might say the way that you should do it, in the same way

that Kubernetes was initially relatively esoteric, and now it's the accepted way. It's sort of like, this is the next step. And many of what I see, and Modal is a great example. And there's some others that are kind of like infrastructure from code companies that are out there, like AMP is one that comes to my mind that I've seen. It's sort of the next stage, which is really taking us back in many ways to the good old days of the year 2000 with PHP, fast CGI, and MySQL and PHP iBadmin. It's like, you didn't have configuration because everything lived in a box. And thenadmin. It's like, you didn't have configuration

because everything lived in a box. And then we had to unplug the box, and we opened up this huge wide world of configurability, this huge wide world of Lego blocks you can bring together in the cloud. And now we're kind of putting those Lego blocks back in, but different, but better. I think so too.

And the way I think about it is like code as infrastructure, but then you actually put the app code into the code as well. So now you have the app code also doing the infrastructure at the same time. If you think about it, like that's like the programming model, like I think makes sense for cloud development. Maybe you're retouching it a little bit, but I actually really just curious. I feel like all of these new paradigms or new way of doing things, let's say Kubernetes or Docker, they're all like

a new religion, right? Somebody has to go and be so zealous about it, right? And then go spread the love. Because we see many, many approaches and many, many things never got wide adoption, but Kubernetes did. There's definitely certain maybe key moments or key elements and key things that need to happen to kind of push forward where like everybody's finally accepted to be almost like the standard. I feel like Modo is probably in many other relative comparisons, more opinionated. To adopt it, you have to program it with the SDK. You can't just take any random code without any changes. What do you think is the necessary things for people to fully accept it to be the wide adoption? Is it just reliability of the system? Everybody can do everything? What do you think needs to happen

so that, wow, this is like Kubernetes now? Kind of a side note, by the way, but I feel like Kubernetes in a way, the approach of Kubernetes was kind of accepted when Kubernetes came out. If you look at a lot of previous attempts like Docker Swarm or Mesosphere, like you worked on. I think they were just like very hard to use. And then Kubernetes just like made it slightly easier to use. But I don't know like what it would take to make modal accepted. Like to me, it's just pushing for adoption

and like getting people to use it, like proves that it's like viable. And to be clear, like, I don't know, like I think it's the right way to do things, but like maybe I'm wrong. Hopefully I won't be wrong. So let's jump into what we always love. It's called a spicy future.

Spicy futures. So spicy future, as it sounds like, is Mr. Eric and we all chime in as well. We kind of talk about what you believe in the next three to five years. What should happen in infrastructure? What do you see the state will be? I think you already kind of stated it, but maybe we can even state it more clearly. Whatever paradigm shift or state change you want to see happen,

or you believe what should happen, and what necessary things it needs to do to get there. I think we're still very early with the adoption of the cloud. And what I mean with that is like, we're still sort of like taking pre-cloud technology and kind of putting it in the cloud. In a lot of cases, like, you know, the whole notion

of like a cluster. Why does that make sense in the cloud? Like, why do you need to think about what's a cluster? Like, it should just be like things, you know, scaling up and down, like magically, right?

Like you shouldn't have to think about that. So that's like one kind of like anachronism that I think clearly shows that like we have a long way of going. I think a lot about developer productivity and like one thing I believe in is like we have a long way of going. I think a lot about developer productivity.

And like one thing I believe in is like, we shouldn't even write code locally. Like why are we doing that? And I think part of that is just like the cloud is like annoying to work with because there's a lot of things missing and like moving the developer workflow into the cloud. But it's like kind of absurd when you think about it, like almost every profession has moved their entire workflow to the cloud.

Right. If you're like a journalist, you're probably writing articles in like a cloud editor. Or if you're like a event manager, you're probably like using some like, you know, cloud-based CRM and like emailing stuff. But then you go to like developer, they're actually like probably one of the few professions that are actually doing everything locally almost. And to me, that's like absurd. One of the things I definitely see is that obviously we've moved a lot of app code to the cloud.

I think we're also going to move the entire developer environment to the cloud. And it's going to look very different, I think, too. So in a large organization, you kind of have your local dev, and your CICD, and your pull request merge, and you have your deployment step to the Kubernetes pod. How does that change in your vision of Cloud 2.0, that end-to-end workflow? And you kind of have that feedback loop, right?

Which is like the developer sitting there watching the code migrate along this pipeline to play a game of Mario, where you're trying to get to the end of the level and hit the flag, right? How does this change over time for the application? I mean, you're very focused on the nebulous idea of the data team, but I'm kind of curious.

I mean, all developers kind of work in that workflow. How does that workflow change in your purview? I think initially, and we've seen this a lot, it's going to be kind of lift and shift. People are just going to stick things into a VM and put it in the cloud. And so I think we're going to see that with like cloud development too.

Like next five years, a lot of it's just going to be like, oh, it's not your laptop. It's a virtualized environment in the cloud. It's extremely identical. But I think if you actually like kind of think about like what is the cloud like there's so many things like we should you know clearly like you can do like completely different things like one of the things like for instance like one example i think a lot of it is like the cloud can scale almost infinitely so like why am i running one unit test at a time why can i take like every fucking unit test in my entire code base and just run in a Lambda and spin up 10,000 Lambdas

and run all of them in parallel? What if you can do those things? And suddenly, when you think through that, I think there's so many things that we can do to dramatically change the entire development workflow. We could run enormously massive test suites in parallel or CI-CD pipelines. I think we can dramatically rethink how those work.

We don't have the like programming paradigm to do that quite yet. That's like, I think like something that we're going to have to invent over the next few years. I think the other thing that doesn't quite exist is like the sort of idea of like ephemeral resources in the cloud. Like it's like annoying to like test your code against a database in the cloud because like now you have to like create a database in the cloud and Because now you have to create a database in the cloud and then run tests against it and then tear down that database. So that's why I think a lot of people, instead of using RDS, they end up running Postgres in a Docker container locally.

But what if you had an ability to create ephemeral databases or whether it's Postgres or Redis or whatever? I think once you rethink what the cloud offers and once you start playing around with the idea of creating ephemeral resources that are anonymous and having the ability to start containers in a couple of seconds or less, there's so many different things you can think about the development workflow that would look very, very different. I'm curious to get your perspective on this. Today we have three hyperscaler clouds.

You've got Azure, you've got GCP, you've got AWS. In the world where the cloud is the IDE, the cloud is your CICD, your cloud is your deployment, do you think we end up in a world where it's like the hyperscalers own everything end-to-end? Like you pick your AWS and you're there to go? Or do you think we're going to end up with a world

that's maybe more of a mosaic of services? Which is true for some organizations today, where they have these mosaic of third parties and they have their primary cloud provider. And you have other organizations that have multiple different cloud providers. And then you have other organizations that are just all in on AWS. If you're not an AWS service, pound sand. I'm kind of curious to think, how do you think the ecosystem evolves and changes to this new workflow? Because it will change buying behavior.

Totally. To me, I think the direction things are going is more along the lines of Snowflake. And what I mean with that is Snowflake runs in the cloud, right? But it runs typically on AWS, GCP, or Azure, or something else. But that's sort of abstracted

in a way. So I think a lot of people will use all these cloud services that may, in turn, run on AWS and the hyperscalers, but they may not realize it or they may not have to think about it in the same way. So I think to a large extent, you won't interact with cloud consoles as much as today. And look at, for instance, what Vercel has done for a lot of front-end development. If you're using Vercel, you don't really think about the cloud provider. You think about Vercel as like where you host your front end, similar to like what Snowflake is doing for a data warehouse.

And so I think there's a sort of layerization where I think there's like the tier one, like there's a sort of bottom tier of like, you know, not in a bad way, like AWS, GCP, and Azure, like the big hyperscalers, they benefit from enormous like economies of scale. And I think there's a very stable oligopoly. And actually, I think they make a lot of money doing that because people think of that as a commodity, but margins are pretty good. It's like 60% of these two or something like that. And then I think there's going to be a layer above

of vendors that own more end-to-end workflows and user experiences. And they maybe cater to different user groups, just like Modal focuses on building data apps and abstract the way they underline clouds. Like I think there's going to be other ones too. I already mentioned

like Vercel or Snowflake. I think there's many more examples like Fly or Railway or whatever. I think that's like the sort of world we're slowly moving towards. I'm curious, like what are the trends

you think that will accelerate the shift to the, I use the word model, but sort of the thought process of infrastructure from code and a lot of the things that you've bundled in with this like really tight development loop.

And I see it at Cloud IDE. If you've done Investor Decks, a lot of our listeners have done Investor Decks, looked at Investor Decks. There's always the why now slide. Why is this a good idea now versus all the other times? Are there key trends you see as this accelerator through drivers for why this will happen? Moving away from local to other areas and all this other stuff?

I think all the trends are there. The clouds are here. Fast networks are here. Relatively cheap internet bandwidth is here. Those things weren't here 10 years ago. Those things would have been a lot harder to be done. I think to me now it's more like a software and a workflow problem. And we've seen this. Changing how developers work takes a lot of time.

Developer workflow is not dramatically different today than it was like 10 years ago. We do a little bit more CICD, you know, there's a little bit more like unit testing, better IDEs, there's better, I don't know, you know, syntax highlight, like whatever, right? But like fundamentally, it's like these things like change very slowly. And I think there's also because it's sort of like a whole ecosystem of sort of vendors and products that kind of move in lockstep with each other right like you know you have to build a in order to build b and then like once b is built then you can move from a to c and then once c is built then you can move from b to d like there's like often like sort of like a lot of interdependencies between a lot of different products and that's why these things tend to move pretty slowly but i think all the things are there To me, it's just a matter of time before developer environments move to the cloud.

So I want to maybe ask a spicy question and don't have to name particular company names. But to me, one good example, when I see different philosophies that are kind of like in contrast is like the Vercel versus Netlify kind of play, where Vercel starts with the Next.js. Everybody adopts Next.js and that becomes the cloud where Netlify is like, okay, I'm not going to change your code. Take anyone's approach. And the next few years or so, I think you're going to see everything exists, right? To me, like when you talk about the next generation of clouds, I think there's actually opposing or maybe like different philosophies. There's a philosophy of I'm going to build like a universal control plane on top of all the clouds

and whatever you do behind me, do the same. But there's like some middleware here. You know, we can look at databases, layers and stuff like that. You're much more at the forefront, right? Change your code right into modal and everything behind the scenes work beautifully well but that does require like a lift and shift that you mentioned obviously that's your belief right this is the place we should go this is where the best experience people will be going for do you think maybe in the next five to ten years some other kind of vendors shouldn't exist anymore or shouldn't that those layers doesn't make any sense to you? It might be Kubernetes,

but you know, what is it in the middle you think will not be there anymore? Like people are not even talking about it anymore. It's like a time and space kind of thing. Yeah, I think Kubernetes

makes a lot of sense today, but I don't know if it makes sense in like 10 years. I think Kubernetes is not cloud native in a certain sense. You know, so the question is like,

what is a cloud native Kubernetes? And I would love to see some sort of open source project doing that. But my feeling is the first version of that is not going to be open source. It's going to be some sort of proprietary. And similar to Modal in that sense, we're building an end-to-end thing. We're building a proprietary. It might be very opinionated. But I think in the long run, to me, the spirit of Modal

is very much where I think data development will go. And hopefully by the time that data development moves there, we're also abstracted away modal in a certain sense that we can also offer other programming paradigms that at least spiritually align with how modal works. I do think there's a lot of startup vendors out there that are too thin wrappers around kubernetes and i think that's going to be difficult if there's something newer and better that makes everything easier on the other hand i do think kubernetes is realistically going to be around for another 10 years right but i think like betting on kubernetes right now is like betting on like hadoop in like 2008 like that was the right. Hadoop was the least bad way to do big data in 2008.

But by year 2016, I already felt like Hadoop was kind of legacy software. So you focus a lot on generative AI. You've got tools for fine-tuning. You've got tools for deploying generative AI models,

open-source generative models. You've got this whole suite. So clearly you're making a bet because you've spent a lot of product time investing in that part of the stack and enabling these workflows. Help us understand your thought process on if you're making a decent-sized bet on generative AI as a business

and how you think it will impact the type of products people build and why you made that bet. There's clearly a yin and a yang to your thought process. And we'd love to understand where you think this will be from three years from now in terms of the impact on the type of products we build in the five years. And then what the developer experience is

to enable people to actually create those products. Yeah, totally. First of all, we actually started the company before all this Gen AI stuff started happening. And I wouldn't call it as bad as much as just like, suddenly like a lot of people started asking how do i run jnai apps on modal and we just like realized like actually it kind of makes sense for us to do this uh so we started leaning into that and just like doubling down and you know a year later you know it's a very significant amount of our revenue coming from that sector but it was actually

a little bit backwards like we didn't necessarily bet on it i think jnai right now like it's like weird but like i didn't come up with this analogy but i think a lot of it reminds me of like when iphone came in like the app store the first like year or so it was like kind of like gizmos and like remember i don't remember but it's like ibeer app you drink like a beer like like with your phone and there's like a lot of like funny like consumery stuff and similarly i think back then i remember there was a lot of like funny like consumery stuff and similarly i think back then i remember there was a lot of companies that like we need to have a mobile strategy and so they would like pay people a lot of money to make a mobile app that in many cases sucked because it was like kind of like a tool looking for a solution people were like oh the mobile like mobile is the

future we need to have a mobile app but people didn't really like think about like okay like what problem does that actually solve and it wasn wasn't until a few years in, you started seeing more of a mobile and native type companies. And those were things like Uber or Spotify, I guess, to some extent, or I don't know, Foursquare or whatever, actually taking advantage of this platform and rethinking the workflow and building a very different type of product that actually truly needed this thing. And that that's where i think we're not quite there yet but i think we're starting to see some of it right so like some of the customers we have in the jnai space the initial customers it was like somewhat like kind of gimmicky stuff like which was cool but now we're starting to see sort of a second wave of customers and those tend to be people building more like end-to-end

video processing pipelines or like audio processing pipelines in many cases like you know maybe building tools for creators like podcasts you know make it easy to edit podcasts you know do like post-production right using gen ai because like it turns out like a lot of like speech synthesis or transcription or like video editing or lip syncing or translation like those are actually really valuable things but you have to put them into like an end-to-end workflow in order for it to like really make sense. So to me, that's like the next set of exciting things is going to be like, well, it's like more like end-to-end things. And then I think the third way is going to be like completely new, like business models that were like fundamentally like depend on Gen AI to exist. And I don't know if we quite yet see that.

I don't think there's like, maybe OpenAI is like the only one, but like that to me, it's like going to take a little longer to sort of shake up. So, you know, there's like a lot of also LLM, infra, middleware, like discussions right now, right?

There's so many kind of primitives people are kind of proposing. And I think Modo definitely, I think in my mind, fits into like the inference side, you know, calling GPU, able to actually call models. Where do you see Modo fits maybe in the more midterm-ish? Where do you think the primitives you want to provide? Because maybe you've seen applications moving towards, I don't know, providing something even more upstream than just inference only.

You can also go into training. There's a lot of different potential things you can use with data and GPUs. And so I just don't know what you think aligns with the platform and fits well to what you're almost like roadmap or vision-wise. I think as it pertains to modal,

I always wanted to build a very general-purpose compute platform. My end goal is to take over all of compute and then all this stuff up the stack too. And so to me, that general purpose compute platform. My end goal is to take over all of compute and then like, you know, all this stuff up to stack two. And so to me, that's not just inference. Like we do really well with inference right now because it turns out there's a lot of gap for that. And Model makes that particularly easy.

But we're starting to see a lot more people using Model for fine tuning, to some extent, a little bit training. Like we're very interested in long-term also doing things like pre-processing data pipelines, like dealing with, you know, ML observability, feature stores, workflows, scheduling, building query engines, real-time streaming. There's just so many different things, many of them decomposed data stack that we want to take over long-term. But right now, we're very focused on just the compute, and in particular, inference is where like we have kind of a wedge into the market. Looking at the larger market as a whole, it's still very early.

And you mentioned LLMs. I actually don't know in the long run how it's going to shake out. Clearly, there's a lot of startups building higher level services. There are a lot of companies right now building, for instance, LLM APIs. And I've talked to a lot of VCs and everyone's like, yeah, but that's going to be a super thin margin. Very few people are actually going to make money. I struggled myself to understand long-term what the layers and the

boxes are and what the supply chain is going to look like and who's actually creating value. To me, that's very early to understand. I do think it's relatively clear that at least focusing on compute and building a lot of different things, supporting a lot of different types of use cases

that people need within compute. To me, that seems clear. People are going to need compute. So that's where we want to focus. It's relatively far down in the stack, actually. It's far down the stack in terms of pieces of fundamental building blocks, primitives, right? But the way that you've built it is you've removed

so much of the middle part so that you're actually presenting this very high-level concept, which is very unique to what you're doing. I think a lot of the thought process around LLM and compute for LLM is, while we're looking at what are other comparable markets that we can look at to say,

well, this is how this market unfolded, so this market will unfold here. And we know that at the end of the day, that GPUs, while being tightly constrained today, won't be tightly constrained forever. And just like hosted compute was in the 1990s and the early 1000s and such, is that it's how computers drastically

commoditized. It's still a good margin, but you have to build all this ancillary stuff around it to capture it. Yeah, which is like, you know, the AWS market, right, like no one comprises in IAM for AWS, but like so much of their margin is driven by the fact that IAM is actually pretty good, right? Like, you think IAM is pretty good?

Well, in comparison to what we had before, you know what I mean? In terms of what you get out of the box, it's definitely better than like IP firewall management and all the other nonsense we were doing before all that. So yeah, yeah, that's absolutely true. The idea of security moving away from the network layer to the app layer, 100% agree that that's an obvious trend.

Yeah, I think you were probably inventing Layer 7-11. I've never heard that term before, but it sticks in my mind now. Isn't it Layer 7? That's usually the highest stack, right? But you could come to 11. You had to come up with 9, 10, 11. No, 8, whatever, it doesn't matter. That's usually the highest stack, right? But you could come to 11. You had to come up with 9, 10, 11. No, 8, 9, 10 yourself too. So it doesn't matter.

Yeah, yeah. I think we really touched on what we wanted to. Anything you want to maybe talk about? Like something to highlight about Modo, maybe it's an upcoming launch or some kind of like recent milestone you think is worth kind of plotting here? Well, we're generally available since last week. So that's exciting. We announced our A round too.

We're out there. We definitely encourage everyone to try it out and looking forward, like to me, next few months, like we're going to focus just a lot on like scale and performance. But we have some really exciting features on the roadmap. We're thinking a lot about beyond the pure compute layer, like where do we go? And there's a lot of really exciting stuff that we're working on that we hope to gradually release to the public over the next, I don't know, six months or so. Cool. And so where should

people go and learn more or start to use Modo? Modo.com. M-O-D-A-L.com. Cool. Well, I think this is a blast. We got all the spicy takes we wanted to. So thanks for being on the pod. And obviously we'll continue to use Moodal

and see all the successes coming from there. Awesome. Thanks for having me. Thanks for joining. It was great fun.