In The Arena by TechArena - Rafay’s Haseeb Budhani on Building the Future of AI Infrastructure

Episode Date: October 30, 2025

Haseeb Budhani, Co-Founder of Rafay, shares how his team is helping enterprises scale AI infrastructure across the globe, and why he believes we’re still in the early innings of adoption.

Welcome to Tech Arena, featuring authentic discussions between tech's leading innovators and our host, Allison Klein. Now, let's step into the arena. Welcome in the arena. My name's Allison Klein, and we're coming to you from AI Infrasummit in Santa Clara. It's a Data Insights episode, which means Janice Sorowski is with me again. Welcome, Janice. Thank you, Allison. It's great to be back. So, Janice, we are having the best week in Santa Clara, and we have been interviewing such interesting folks from across practitioners to those who are building the infrastructure.

Tell me who we're with today. It's quite an honor. We actually have Haseeb Bhutani, who is the co-founder of Raffei. Welcome. Thank you for having me. Nice to be here. Paseeb, why don't you introduce yourself? Rifé has never been on the show before,

and you are the CEO of the company. Obviously, a co-founder. Talk a little bit about Rifé. Why do you founded the company and what position does Reifay play in the AI arena? Absolutely. Thank you again for having me. I really appreciate it. My name is Hasebe.

I work at Ralfi. So Rase fundamentally was founded to make it easy for developers to consume compute. That was our idea a long time ago. It's just really hard for developers to do their jobs when it comes to infrastructure. Cloud on-prem doesn't matter. With the last six, seven years, we've seen this market shift tremendously. For the last many years, we were focused on Kubernetes management because that was a big thing for a long time. We spent a lot of time trying to make it easy to manage Kubernetes and deploy applications on top.

But as it relates to AI, right, so the use case here is we are finding so many large organizations invest hundreds of millions of dollars buying very expensive infrastructure. And then it takes a really long time to make it consumable. And even when they do that in some cases, the experience of the developers expect because they're all spoiled now. they're used to what Amazon can provide. They want the same thing. And they don't get it. It's a problem. So the question is, how do you deliver a CSP-like experience

on infrastructure that you've already purchased, perhaps, or purchase and operate? Because there are very good reasons to do so solid reason. Sometimes it's just cheaper to do this on frame. Our software helps those customers make it easy for developers to come and consume this infrastructure so that you have high utilization, develop happiness, better R.W. why everybody does.

So can you expand on that a little bit? Can you share your perspective on the current state of AI and overall infrastructure adoption? And where are some of the biggest gaps that you see in enterprise today? Yeah, absolutely. So I think that when enterprises took on the cloud migration task, right, so many of us did this, right? There's a specific set of skills that we don't need anymore. We don't need to be data centers, right?

We don't need to maintain its infrastructure because the public does we're doing this. But because of AI, there's a number of companies that, for all the right reasons, are repatriating specifically air workyards, data sound tree reasons, et cetera. And those companies find themselves in a position where maybe they don't have their talent to do this. So they look for regional GPU cloud providers who can help them solve this problem. But then going back to the point earlier, the developers, they want the experience they get from GCP and Azure and AWS and others. they want the same. So I want to be able to go to a quarter of an impressive button and start using deep seek.

Why can't do that? Well, if that infrastructure, but I can do that in Amazon. Do that for me, I will use it. And there's enough data that shows now that if you're investing in this infrastructure, in many cases, it actually ends up being cheaper per hour. If you deploy on prem yourselves, or if you work at the GPU clock provider, then going to a public cloud. Right.

So it's sovereignty reasons-wise it's a better bet. It ends on being significantly more cost-effective. So the only gap is the developer experience. Let's solve that. And developer experience, by the way, isn't this not about a website, right? There's not that. The issue is infrastructure has to be composable.

And the two examples I always use, I say, imagine if you're two developers, they show up at the same time. One says, I'm going to do some training job. I need a hundred one eight years. The other one says, I'm just starting my journey. I need one GPU.

You have to service both of them at the same time in a self-service way. If you can do that, you are CSP. If you cannot do this, you're not CSB. Rafi helps you become CSP grid on-P for your own users or for your cloud so you can sell to companies in your region. Now, this sounds great, but we know that people are pouring billions into GPUs, and yet their utilization rates are very low.

Tell me about why this is such a persistent problem in today's infrastructure, in today's cloud stacks, for lack of a better term. And what is missing in terms of orchestration tools or even developer access capabilities? So I think that it's amazing. All these things are tied together now, right? So if developers don't have a great experience, they're not going to use it, which means you're only going to get the kind of customer who's looking for a team. So who's that? These are large start-up sometimes or companies or building models. They come for a window of time and then they take off.

And they come and they'll use 2,000, like, massive numbers, GPUs, but they can't need it. So now you have cyclicality in your business, which nobody wants, right? We want people to use it forever. Now you compare that to how ABS works, right? So they have enterprise customers who do multi-a deals at times, right? So they call them the enterprise discount program of GDP. Right. So what is the EDP equivalent for each year? I should be able to go to an enterprise and say, well, today you spend

extra dollar dollars on a public chart for your AI use cases. I'll give you a better deal. In your region, it's going to be sovereign, it's going to meet the GDP or whatever your funds, wherever local compliances need to be. But you need to come into it, if we can help customers do that, do that, that is no longer a cyclical business, and that's going to end up a high realization. Yeah, that makes sense. My customers can do that because they deliver the right experience for the developer so

that they have the opportunity, talk to enterprise it and say, you're going to get exactly what you get in a cyberscaler, here, better price. That's a pitch. And it works. That's amazing. It's amazing. Now, you've said that the economics of AI are pretty much a margin scheme. Right. But with raw GPU capacity becoming more available, what makes profitability dependent on the orchestration and management?

So on Thursday, I have a talk here where I'm going to basically share the following story, okay? I had this conversation with Jad GPT. No shame to say I've been saying these things anymore, right? Everybody does this, right? So I asked a question, hey, so let's say I'm burning an app, travel agency. And I want to use some model back in to where I can ask some questions. Should I use? And so, well, you could use this. You could use this. something called BBC Gar-Wani, you could use that as well. Okay, I like it. What does that cost?

Well, if you use it in Amazon. This is a conversation, by the way. Okay, if you use it in Amazon with Bedrock, you could pay four bucks or eight bucks or, I don't know how much I should pay. So let's say I have a hundred requests per second. You're going to pay something between six and eight bucks in now. Okay, but how much does a GPU cost?

I have no idea. Says it, well, gpunist. A.I says that you can rent a GPU per hour for a buck 60 or a buck 80 depending on them. Okay, so the thing is like a buck 60, I can go get somebody to give me a GPU with powered up, not like a GPU, powered up, capacity, cool, everything taking care of a buck 60. Amazon charging me eight bucks and change for a modern dollar I consume and I don't even own

the GPU for a D hour. That doesn't make sense. So far. The difference is Amazon is delivering an experience, not a GPU. You can get a GPU too, but I'm looking for an experience where my application wants to talk to an API endpoint security so that I can do what I need to do in my app. What a different conversation. The conversation changes the price point.

So if everybody's selling a modern service use case, as one example, there are many other examples such as this. You can make more money. So why don't we do that? And the thing is, everybody understands this. So most GPU slough or GP provider shops that I talk to, they all understand that they need to go up the stack and deliver valued services. The question is how?

The big hypers, they have tens and thousands of, I don't know how many people work there. There are lots of people, they burn lots of software of 15 years. What are you going to do? Do you have 15 years? Do you have 10,000 developers? No, no, I think. Okay, so then there have to be alternatives, such as maybe you can buy software that can

get this up by then. So we are one such company. We do a very good job of this. We have lots of great customers globally now who are GPU providers and now sell software alongside the GPUs and are able to make more margins. And I believe this is where the market is going. Obviously, this is an early market. takes time, nothing happens overnight.

So people started with the fundamental, which was, let's sell some GPUs. Great, that market is here now. Okay, now let's go up the stack so we can go after the enterprise customers at the end of the day. They're the ones who pay money. They have the money.

Okay, let's service them. What do they need? If we can address that gap, they'll make money. Right, but I see it now. I see that in every region, there's going to be a few viable alternatives available. When I say a region, I don't mean Europe. Right.

Right. I mean like a part of France, there's going to be Two, three companies are going to become competitors of each other, and they would all provide great value to their customers. And then in the next region, there will be another set of people. There's going to be many of these. It's clear. This is something. Plus enterprises, these are the big ones.

They are making their own similar investments because they can afford it. And over time, actually, it's cheaper. Going back to the same mass, right? If I'm going to pay $8 an hour on Amazon and I already on the GPU, over time, that adds up. When you have two or three developers, it's okay. But at scale, I'll do it myself. That makes a little sense.

Now, you said that you do it well with your customers. Talk me through how you engage in enterprise to get this stood up. And what have you seen in terms of developer productivity once you get these solutions deployed in your customer sites? My favorite customer, should I call them favorite, they pay us the most money for any customer. Maybe that makes it our favorite customer. So they, before Rafi, used to have an army of 100 plus people who, maintain a platform. It's a very large company, so 100 is not that big of a number in their work.

And now they don't. They have three-ish people who operate the RAPA platform, and they operate 6,000 environments. Wow. So a developer effectively, or for the most part, a developer equals an environment in the world, right, who use the platform on a daily basis. They don't need to talk to anybody. They show up, they press a buckle, probably get a coffee, environments are ready, they do their tests.

So if you use an example, I want to use AI as a substrate for this example, if I just write Java code. I write code. I check into a Git repo. I want to test it. What needs to happen for me to do the test? All I should need to do is tell some system my code is here. My code is zero. Everything else to figure out because everything else is a template. In my company, we write, I don't know, spring code, and we have a React front end, and we have a MySQL backend. And most applications work this way. Okay. Everything should be matched. And before my environment is created, there's a service card ticket that goes to our manager and they's approved. If you can do this for non-AI workloads. Of course, AI workloads. We can do this. So sovereign AI.

Sovereign AI is getting a lot of attention as governments and enterprises really focus on resilience and competitiveness. How do you see sovereignty really shaping kind of the next wave of AI growth and what role does usability play in making it really meaningful? So I think there's push and pull for any such thing, right? So sovereignty is driving people towards this notion of regional clouds. Of course, because of AI,

people don't want their data which becomes IP for them. They don't want the data to leave the sovereign borders, so it must be here, which means the compute must be here as well. So that becomes an impetus for their to be GPU plants.

So sovereign clouds, to me, are no different than somebody who used to give Bitcoin money, but there's a lot of them out there. It makes a lot of sense, right? There's a little money to be made here. But they face the same problem. Uses, right?

The usability, I should say. Users will sometimes come because they may force people in their territory to do whatever they should do, but usability is really critical. When these things start up, I've seen this now over the last year and I have time and time again, but don't think about the developer experience and they learn about the underbellies too late. Right. And they don't have a realization, right? They look back over the last six months and look, I'm running at 20, 25, 30% utilization.

What a terrible investment I've been. And you have to start thinking about this on day one. So one of the things that we do with our customers is sovereign or otherwise is we obsessed with them over the user story. Was your customer? Internal X-Rater doesn't matter. What are they going to do? How are they going to strike?

Where do they go? How do they sign up? I'll be simple examples. Like many countries, when you go to a website and you sign up for an account, first you need to program entity. Sure. So it's called a K-by-C process or your customer process, right? So how do you have a system that's going to block all requests to the K-by-C process is approved

in all the system? So all these little little things add up to deliver a good experience so that people will keep coming. And then you have growth. Like what I'm measuring is, do my customers buy more six months from when they start working with us? They always do. That's a fantastic metric. Now, you've talked about your heritage. You talked about where you are today.

We have a vastly changing landscape with AI, both from a standpoint of what the large providers are doing to train algorithms to how. enterprises around the world are trying to adopt AI and transform their businesses. How do you see the fundamental platforms being challenged and the solutions that you're delivering to the market being challenged and where are you pointed next from an innovation student? It's a forever changing market, new hardware from Nvidia. I can't sometimes keep track. And of course, customers keep driving us for sure towards this newer infrastructure. So for us, there's always going to be this race to make sure that we can meet the requirements, for the latest, whatever technology happens to be.

So this is not going to stop for a while because this is an early market. So in an early market, a lot of sins are forgiven because it's such a dearth of technology, right? So if you're mostly there, people are actually pretty happy with progress. And so many of our early customers, they see it as a badge of honor to partner with us on your technology. So this has happened consistently. We found some great, very large global partners who say, okay, I'm going to go experiment. I don't tell you a specific story on this.

So we have a partner who, when they hear this, they will know who they are. they have access to an amazingly large network of undergraduates in a very large country globally where they figured out a very specific use case for data songs that is taught in universities. And they have come up with a beautiful user story for how to make those students successful. And they came to us and said, let's put this together. Sure. And we did. Let's. Seems like it's okay. Okay. So we do this because right now is not the time for prioritization. Right now is the time to say yes to everything.

What would that mean? He's going to say yes, right? We all figure out together because so long as I am committing to my partner that we are in it with you so that you can make money and then, of course, as a result, we will make money in everybody that's right. But the thing is, look, for that specific set of users, these students, the customer has obsessed over what is the experience going to be like? And they have now forced us to understand that and we have to work with them to implement

that is part of attendance. So long as that is the number one target, which is clarity on the user experience, delivering that with composable infrastructure. The premise is quite simple and clearly sound. It's just really hard to actually do. And that's the beauty of the best technologies, the best companies they work out, because it's such an obvious problem, which is hard to build. You were wrong with a formical day every day, right? Yeah, maybe. You've been super clear in kind of your overall mission and vision. But can you tell us a little bit more about what your ultimate goal is for race, say, in terms of AI infrastructure and the overall ecosystem. And how do you see this evolving over the next few years?

Seven years in, I say this consciously, but then as I say it, I'm thinking, oh, my God. So it's super early. If you keep meeting clouds globally, aren't res globally. You're just starting a journey. And so what I'm thinking about is, okay, so my customer who just invested a couple hundred million bucks, so we're working with a large SI company in Africa who's investing. a couple of million bucks to deploy our infrastructure. Wow, they just started their journey.

We're going to be with them for the next three, five years. Where is they going to end up, right? Bigger. Of course, right? We want them to do successful. So this journey is just starting. Again, we're a seven-year-old company, and I kind of feel like we're just starting.

But it's been amazing. We're very blessed. We're very blessed to be where we are at this point in time to have the solution that we have, right? It's crazy times for everybody. Nobody sleeps with our company right now. Nobody sleeps. Everybody's not being here and there, but what a great time.

to be working on a problem like this. I love your enthusiasm and your optimism. And I share it. I work telling the stories of the industry and I feel the same way. I'm sure that your story has inspired our audience to want to learn more about the solutions you're delivering and engaged with your team.

Where can folks find out more information? berafi.c.0.c.0.c.0.c.c.com.com. too expensive. Do expensive. So definitely haven't bought that yet. We're on Twitter, on LinkedIn. Find me on LinkedIn. I'm happy to talk to anybody about what we do. Janice, that works the end of another Data Insights episode.

Thank you so much for your partnership. And thank you so much for being all. Thank you for having me. We appreciate it. Thank you, Allison. This was fascinating. I can't wait to learn more. Thanks for joining Tech Arena.

Subscribe and engage at our website, Techorina.ai. All content is copyright by Tech Arena. Thank you. Thank you.