In The Arena by TechArena - Privacy, AI, and the Future of Cloud Computing with Rita Kozlov

Episode Date: September 5, 2024

Join Allyson Klein and Jeniece Wnorowski as they chat with Rita Kozlov from Cloudflare about their innovative cloud solutions, AI integration, and commitment to privacy and sustainability.

Welcome to the Tech Arena, featuring authentic discussions between tech's leading innovators and our host, Alison Klein. Now, let's step into the arena. Welcome to the Tech Arena Data Insights Series. My name's Allison Klein, and I'd love to welcome back my co-host, Janice Narowski from Solidigm. Welcome to the program, Janice. How are you doing? Hi, Allison. Thank you so much. I'm doing great, and it's so good to be back. So, Janice, I know that we've got a fantastic interview today. Do you want to

introduce our guest? Yes, I am so excited today. Today we have special guest Rita Kozloff, who is with Cloudflare, and she is their VP of product. So we're going to learn a lot about Cloudflare today. Welcome, Rita. Thank you. Thank you. Thanks for having me. So I've been looking forward to this interview all week, and we've had Cloudflare on the program before, but this is your first time. Do you just want to introduce your role at the company and how it relates to Cloudflare's larger mission?

Absolutely. So I've been at Cloudflare for eight years, and my role has changed a few times throughout that period. But the way that I like to think about it is there's three sets of services that really Cloudflare offers to our customers, right? So what most people know us for is our services around CDN, WAF, DDoS, and we like to think that as application services, right? So how do I protect, secure, and accelerate my application? Then we have our zero trust set of services that is more around how do I protect employees, devices, and the ways that really you connect to your network. So more internally facing,

how do you secure that side of the house? And then we started to think about, okay, so we've built out this network and we're already doing these things with it. How do we open it up to developers for them to be able to build directly on it? And so I help lead the developer product side of the house, which is around really enabling developers to build applications on top of this infrastructure that's really meant to empower them not to have to worry about scaling, maintaining, or managing any infrastructure's writing code

and being able to, you know, go from zero to MVP really easily and beyond that, of course. Awesome. Now, on that note, Rita, you do run product for Cloudflare. So can you tell us a little bit about what that means in the context of the solutions you're discussing? Yeah, absolutely. So I lead the product organization for our developer platform. So that really means managing a team of PMs that help define what our solutions look like. This can mean everything from how do developers interact with our platform? How do they write code? How do they test it? How do they onboard onto our developer platform? And how do we make them successful in writing their

whole world? But then also, what does it look like to give them the tools to release their code gradually to the world and defining our product strategy, both in that sense and also in the sense of what are all of the tools then that we need to give to developers to allow them to build a full stack application? When you think about that, one of the things that Cloudflare is really associated with is a distributed network of computing. Why is it unique from a typical cloud?

And what does this mean for your technology prioritization? Yeah, so if you've used any of the other cloud providers before, one of the very first things that they'll ask you to do is select a region. So if you use AWS, you have this dropdown and you go, okay, I'm going to choose US East 1 or maybe if I'm in Europe, I'll choose Europe. And at the end of the day, I think as a developer, that's first of all, a really big decision that you have to make. That's your very first thing that you're doing right because really what you're choosing is which customers do I care the most about right because if I choose the U.S. anyone else in the world is going to have a much greater latency when they access my application so I think that's one part of it in terms of what it means to

build on top of this network as opposed to building on top of a cloud provider that's built more around regional services that then when you want to expand them, you have to end up with rather complex setups, including load balancing and replicating everything across several regions. So all of that is non-straightforward. I think that's where there's the end user experience part of it, which is great. The latency is better. But I think that's where there's the end user experience part of it, which is great. The latency is better. But I think that the other part of it is really the developer experience and the way in which by having everything run at the network level, developers are able to deal with much higher level abstractions that are not just capering over the infrastructure, if that makes sense. Yeah, it absolutely makes sense.

And it makes me think about, obviously, today, AI is the center of our focus. But I'd be curious, Rita, if you could tell us a little bit more about what you're hearing from your customers about how AI workload requirements and, you know, what are those and what does this mean for the need for your team to continue to innovate? I think in terms of AI workload requirements, we're hearing a few different things, right? And every customer's journey, I think, is a little bit different. So the first is what I like to think of as operational AI. So teams looking more internally at, okay, how do I use AI to empower my own employees, my own developers, my own team to go faster. And that's where a lot of, I think, also the experimentation begins. So I think one of the most common introductory use cases into AI

is with code generation. And that's where also developers are able to see the benefits of it really quickly because all of a sudden you're able to move a lot faster. So that's an area that we're investing in and trying to figure out, okay, how do we make it easy for developers to use our developer platforms if they are using these types of products? But I think the second area is, okay, once you graduate from operational AI or even within operational AI, there's kind of, okay, how do I bring things to production or introduce them to customers? I was talking before about enabling developers to build full stack applications. And we're seeing AI really become a part of the stack from that developer expectation. Because previously,

if you look at the components that make up an application, you have a front end, you have a back end, which is an API and a database. And obviously that's a very simplified kind of way to look at it, but you have compute storage and data. But now every single application has some sort of AI component in it. Maybe it's that it's trying to predict your next actions or help assist you in what you're going to do next. And so you need to run a model in order to do that. Maybe it's that you want an automated chatbot that answers questions on behalf of your team or is able to replace search with something much more semantic where you can ask it like, hey, I'm looking for this type of dress for this type of occasion instead of checking a bunch of boxes and stuff. As AI is becoming more and more integrated into products in that way, I think it's really changing developers'

expectations. And as a result, we've launched our set of AI products to help support that. Now, obviously, Cloudflare is not working in a vacuum. You have an entire ecosystem that you work with to deliver these unique capabilities to market. How do you approach the ecosystem to deliver solutions that customers are really seeking, and especially in this time of inflection with AI? And what does this mean for the type of infrastructure and software you're targeting? In terms of providers or in terms of the ecosystem, first of all, we're very lucky to be working on AI in a time where open source AI is really booming. And that's where we've been really excited about the work that Hugging Face has been doing and incorporating their models into our model catalog. And in general, we've always been big believers in open source.

And so that's why it made sense to look in that direction really, really quickly. I would say the other thing too, is where you see the ecosystem moving the fastest is in terms of the models and how quickly the models themselves are changing. And so that's where we look to the model providers like Meta and making sure that we're partnering with them such that developers have access to these tools as soon as they come out. So when you look at AI deployments from an entire, you know, data pipeline perspective, everything from pre-tech of data to training, fine-tuning, and then ultimately inference, how do you see Cloudflare emerging as a prioritized partner? And how has that impacted your product strategy?

Yeah. So when we started looking at where does it make sense for Cloudflare to play a role with AI, one of the first things that we looked at was, okay, there's training and there's inference. And really, from a compute standpoint, at least, it doesn't really make sense for us to play in the inference space. That is something that's actually much better set up for kind of the more traditional hyperscalers to plan because you do want just a really, really massive data center with a lot of GP resources that are co-located to run that type of workload. But I will say even on the training side where we've seen a lot of developers leaning into using Cloudflare is for storing data. So we saw a lot of interest

in R2, which is our object storage solution, specifically because at the time, especially there was a really big GPU shortage and developers had to grow across several providers. And so that's where the egress fees really started to add up and having an egress free solution made a really, really big difference. The second, but then on the inference side, that's where we see the really, really big opportunity for Cloudflare. And the way that we think about it is that there's three places where it makes sense to run AI. And the two more obvious ones maybe are, first of all, the hyperscalers who we just talked about for training, right? But the problem there is that especially as AI becomes more and more ubiquitous,

these workloads are going to be more and more demanding in terms of the performance, right? Same thing happened with the web, where if you're interacting with AI several times a day, you want that real-time feedback. You want everything to feel really instantaneous. And so having that happen so far away, it feels non-ideal. Then you have devices which are going to run a subset of AI, right? You saw Apple's

announcement where there's so much that's getting baked into the device, but ultimately devices are going to be limited by their hardware capabilities. And so that's where I think we view ourselves as that perfect place that's able to run really close to the user without having to run on the device itself. And we see ourselves really well positioned as model training starts to wind down and inference becomes the primary workload. Really well positioned to power a lot of that. When you talk about this, one of the things that I think about is that you're operating across so many different countries and you're using a lot of customer data when it comes to AI. How do things like data security, privacy, and data sovereignty enter into the equation?

And how is Cloudflare addressing this with their customers? I think that's a really important question. And from an inference standpoint, actually, that's what we see as one of our big differentiators is that we don't use customer data to train models. That's not what we do at the end of the day. We're not opening AI. We're not training a foundation model. And so every single inference is completely stateless unless you opt into using a product like our vector database where you have total control over the indexes and the data that you store there. But otherwise, it's completely ephemeral. And so in that way, we see

ourselves as a really privacy first provider. And to your question specifically about data sovereignty as well, that's where we see our network is such a superpower because we're able to scope down where specifically AI is going to run if you have those regional restrictions based on certain laws or if you want the inference to stay in region. Yeah, Rita, I think you're not only a privacy-first provider, but Cloudflare is really known for being a leader on sustainable compute as well. I'd be interested to hear your perspective on how this is so. There's a couple of things here.

One is we actually view being able to run close to the user as a really big advantage here because you're not having to run all of that data back and forth across the globe. And that's where we see really large savings. And the second part of it is actually in terms of how you think about provisioning of resources. And so a lot of other solutions that we see for AI in the market treat AI resources as VMs that you have to pre-provision. And so as a developer, you're having to think upfront about this is how many instances I'm going to need, right? And that's a pretty inflexible way to think about it because it means that you have to think upfront about what's your peak traffic going to look like, which is a very inefficient way to utilize resources, right? So what you're going to end up with is a lot of resources that are sitting around idle when you're not having a bunch of

traffic and then traffic spikes happen. Maybe you're still under provision, so you provision even more. By contrast, workers AI and the way that we look at our whole developer platform really is very serverless, right? So we'll scale up and down based on your needs. We're able to manage that across all of our different tenants, which is much, much more sustainable way to scale in the long term. Rita, I'm so glad that you came on the show. I got a great education on Cloudflare and really what it's like to try to deliver services to customers right now. It's such a crazy time and you guys are doing a great job of keeping ahead of it. I am sure that people want to talk to you more. Where can folks find out more about the services that you've been talking

about today and engage with your teams? We love for folks to engage with us. I think if you want to talk with our engineers and with our team, I definitely recommend our developer Discord. So discord.gg slash Cloudflare dev. And then for our AI services specifically, ai.cloudflare.com lists all of them out and talks about them in great detail. So those are a couple of the places that you can find us. Thanks so much for being on the show today, Janice. It those are a couple of the places that you can find us. Thanks so much for being on the show today, Janice. It was another fantastic episode of Tech Arena Data Insights. Thanks for being here with me and sharing these great conversations with industry leaders. Oh,

thank you, Allison. And thank you again, Rita, as well. It was great. Thank you guys again for having me. Thanks for joining the Tech Arena. Subscribe and engage at our website, thetecharena.net. All content is copyright by the Tech Arena.