In The Arena by TechArena - Advancing Sustainable Infrastructure to Fuel Hungry Workloads with CircleB's Matty Bakkeren

Episode Date: April 24, 2024

TechArena host Allyson Klein chats with CircleB’s Matty Bakkeren about how his organization is leveraging OCP specifications to deliver innovative and sustainable solutions to data center customers, how AI is re-shaping operator requirements, and how he sees the market shaping in 2024.

Welcome to the Tech Arena, my name's Allison Klein, and we're coming to you from Lisbon at the Open Compute Platform Conference. I'm so excited to kick off this week with Maddy Baccarin from Circle B. Welcome to the program, Maddy. Hey, thank you, Allison. Good to be here. I'm looking forward to this chat. So Circle B has been a huge name in data center infrastructure, but they've never been on the tech arena before. So can you just provide a little bit of background on the company and what is your relationship to OCP? Yeah, sure. So Circle B has a long relationship with OCP.

So actually Circle B was the first solution provider, OCP solution provider in Europe. They were founded in 2012. We are based in Amsterdam. And over the years, we've built up a long history with the OCP community. And there's just a lot of experience and knowledge around OCP in our company. So basically what CircleBee does is we co-design and develop OCP solutions. And we innovate with our customers as well. We're really about that customer relationship and making sure that they get the infrastructure

that they need. All of our taglines is revolutionary IT infrastructures, which under the covers basically means OCP. And yeah, we're just looking to create long lasting opportunities with our customers. And so far, so good. And we're excited to be here. We've got a few things to show as well. There's a bit of a sneak peek going on right now. So we have a bit of a premiere at our booth. So yeah, a lot of good energy here. That's fantastic. You guys are talking on a topic

that is near and dear to my heart this week, which is actual deployments. Can you give some highlights on what you're sharing to the audience? Right, I will, but the presentation is not till later, so I'm not going to spill all the beans yet. But basically, one of the big OCP benefits is the time to deployment, right? Because you're so agile in deploying these RECs. So we wanted to focus on that topic because we wanted to demonstrate that technological OCP benefits, but we also wanted to talk a little bit about our skills and our capabilities in deploying those racks and in doing so. So really the story is about two customer deployments in which we had different challenges and how we handled those,

anything from testing to transport to deployment. And I'm actually presenting that class. So I'll share some learnings and we'll also share some in-house developed tools that we use to deliver these deployments successfully. One of them is actually a software tool, which you'll download.

So yeah, I hope we see lots of people in there and we'll have some nice surprises for them. That's fantastic. Now, one thing that I love about your company is that you're squarely focused

on compute sustainability. We wrote a compute sustainability report in partnership with OCP last year. Why is there such an emphasis and what are customers' perspectives that you're hearing when you talk to them about valuing a sustainable approach? Okay, that's a big question with a long answer. So if I go too long, you stop me. This is really a big one so right now the data center space is going through what we call a twin transition right so there on the one hand side there's this need for digitization in society and on the other hand you know there is

this increased urgency around sustainability in the data center so it's kind of a split right because you need to build this should need to fill this big capacity demand. And on the other part, the more infrastructure you put in, the more power hungry, the bigger your footprint will be. The data centers have already done a lot of good things around sustainability, and they've really increased their efficiencies. I mean, you know, we can talk about PUE, you know, we can talk about that all day long. But bottom line is more is needed. On top of that, we're seeing some really big challenges come up. National power grid providers, there's limited availability of power, increased legislation

on CO2 emissions. And again, capacity demand is on the rise. So it's a real challenge. Now, what's interesting is that OCP is very power efficient. It's a great design from an efficiency standpoint of view. So it's not going to fix all those problems. I think it's definitely going to help alleviate some of those power and cooling challenges that we're seeing in data centers right now worldwide,

which is why I think OCP right now has a lot of wind in its sail when you talk about sustainability, because here's actually something that's a fix. And in all these years, I know you've been active in the data center space for a long time. We've talked about optimizations around the infrastructure. This actually goes to the heart of it, the infrastructure itself. That's what we call the pizza box that we had in the 19-inch rack. It's now an optimized server in the 21-inch rack. It's now an optimized server in the 21-inch rack. Yeah, we think OCP

right now is a very important topic in the sustainability lens. And don't forget, right, the compounding factor is AI, which is right there on the hype cycle right now. And it's driving a lot of innovation, but it's also driving increased power and cooling demand on the data centers. That's fantastic. I love this space. I've been working in data center sustainability before it was cool. I think that... I see what you did there.

Yeah, exactly. That was nice, wasn't it? I think that it's going to be so interesting in 2024 to see how this topic tracks, and especially across the SCP events, because SCP has taken such a front-foot-forward approach to sustainability. Now, I want to go on to our next topic because I can't do a podcast without talking about it. We're at the point where the industry is really propelling innovation to fuel AI based on customer demand. And at a speed, I think, you know, we're both industry veterans.

I've never seen this kind of speed of innovation before. How do infrastructure providers like Circle B continue to innovate, keeping platforms in balance with that sustainability vision and delivering the performance customers require? Yeah, so this is a huge challenge we have in front of us right now. And it's coming to us at light speed. You're absolutely right. And this is like warp speed.

It's like the next level. AI is right there on the top of the high curve. And, I mean, you know, the high curve, right? It goes up, it's down, and you go to what is a trough of disillusionment, disillusionment, and then it kind of bottoms out. So I think we're at that point where we need to see, you know, what are the limitations and capabilities also on the infrastructure side of AI? I mean, the talk of the town right now is large language models, right? There are a lot of possibilities there and a lot of innovation that's happening.

But as you well know, to train one of these LLMs, it's massive infrastructure and it's a big investment, right? And that's definitely there for those who can afford it and who have that capacity and ability to monetize it. But I think there will be many other use cases that will be based on AI, but not necessarily on a large language model where we'll see enterprises train their own models.

Because remember, a lot of these enterprises sit on a mountain of data, right? The precursor to AI, big data was all about data and data lakes. And what do we do with the data? That data can now be used to train models and quality models because you know the data you have in house. So it's going to be interesting to see how different industries and enterprises will build and consume AI for their own vertical and internal purposes.

With regards to OCP, look, one of the beauties, I think, of OCP is once we see OCP products come to market, they're actually trialed and tested already by the hyperscapers. And mainly Meta because they are basically the founding father of OCP products come to market, they're actually trialed and tested already by the hyperscapers. And mainly Meta, because they are basically the founding father of OCP. So these configurations will be a good fit for those enterprise customers that want to standardize and democratize AIs in a balanced way. Again, it's not going to fix the problem of GPUs being so power hungry as they are, but having power optimized at a rack level is going to help. A lot of discussions around GPU

cards, right? PCI Express, but there's also the OEM modules. I don't hear a lot of talk about that one, but it's actually more power efficient. So let me give you a sneak peek. We're actually demonstrating an IA training unit on our booth. We have a live unit here. So we're actually going to not just talk about this, we can show you this as well. And that's what Circle B is about, driving that innovation on OCP. I would love to see the power differences to include in my recap blog today. Now, we're in Lisbon. As we're talking, literally the experts from the industry are streaming into the building behind us. Who are you most excited and what are you most excited to see this week in Lisbon? And how does that relate to the broader trends in the market that we've been discussing? Okay, so first of all, this is my first time at an actual OCP summit. So that's super

exciting just to have the whole ecosystem in one place and, you know, meeting up with people that I've been talking to, virtual speaking or even yourself, right? We've known about each other, but never met. So it's just a great place to meet up and network. I'm looking at the schedule. There are so many cool topics here. Liquid cooling is, you know, right up there. Open RAN, CXO, OEMs, as we just discussed. Actually, the data center regulations, I think that's one to watch as well, because they're going to become more stringent and really have an impact on how we define data center architecture. Unfortunately, I don't have time to attend them all. And there's a bunch of customers here as well, which is, you know, the number one priority for us, as you can imagine. But I do have two favorite topics that I want to carve out some time for here. One of them is DCMHS, because I think that's going to be a big topic.

This talks about modularity at a board level, right? So now we're taking the OCP principle and we're bringing it into the node. I think that's super exciting. Like Lego blocks, just applying what type of server you want to build and you kind of pull it together. That just makes so much sense. And there's a session on blind mate cooling that I definitely want to well, I just hope I can sit in on that

one. Otherwise, I'll do the replay. I think that's a big one. As we head into this AI space, we see DLC being the main sort of trend now for liquid cooling. And the fact that in OCP or ORV3 racks you have this standard basically for liquid

cooling with blind mate I think it's going to attract the attention of a lot of data centers because right now there's a lot of different vendors that have different solutions if you can standardize that's always a win so I think that's definitely one to watch so yeah I think those two for me are kind of like the big ones.

Yeah, you named all of the hot topics and I love the ones that you chose. It's just a matter of finding the time for the sessions. It's a great reminder that they're going to be available on replay. One final question for you, Maddy, before we say goodbye. How can the audience learn more about the solutions that you discussed today, especially those new technologies that you're demoing and engage your team? Yeah, so first of all, I mean, this is an easy one, right? We have a booth exactly for that reason to meet with customers, industry, and vendors.

822, so if you're here, swing by. You're welcome to visit and meet the team. We're here with a team of five speakers. We recently updated our website, which you can visit, right? So circleb, one word,.eu. Have a look there, contact us there. I would also highly encourage you to subscribe to the newsletter. LinkedIn is probably the best way to do it.

Whenever there's news, we push it out there through social media. And it's a great way to stay in touch. And lastly, especially for the folks here in Europe, right? OCP is a community. So join those community calls. We have an EMEA OCP community call every month. Circle B always sits in. There's a number of other vendors and solution providers there, but hey, we're all a community.

And I think the bigger this community becomes, the more powerful and the faster we'll adopt this technology in the community. And I think the bigger this community becomes, the more powerful and, you know, the faster we'll adopt this technology in the industry. So I would, you know, just invite everybody to join those calls and be part of the gang. Maddie, it was such a pleasure to have you on the show. Thanks so much. We need to do this again sometime. I'd love to. Thank you for having me. Thanks for joining the Tech Arena. Subscribe and engage at our website, thetecharena.net. All content is copyright by the Tech Arena.