In The Arena by TechArena - CircleB: Optimizing Data Centers for AI Workloads

Episode Date: October 14, 2024

Tune in as Jason Maselino of Circle B discusses the role of Open Compute Project in revolutionizing data centers with energy-efficient solutions, modular designs, and AI-ready infrastructure.

Welcome to the Tech Arena, featuring authentic discussions between tech's leading innovators and our host, Alison Klein. Now, let's step into the arena. Welcome in the arena. My name is Alison Klein, and I am so excited to be joined today by Jason Mazzolino, GM of Circle B and co-founder of the Taurus Group. Welcome to the program, Jason. How are you doing? Hi, Alison. So I'm super excited to be in your podcast,

and thanks for having me. So Jason, you've had a really interesting history in the tech sector, co-founding the Taurus Group, and that's an incredible story unto itself, and then making the decision to acquire Circle B. Can you talk a little bit about that journey and how you've come into the position of leading Circle B? Yeah, sure. So actually the journey's been almost 30 years now that I started in IT and have been in Taurus group or Taurus for almost 20 years. So it's our 20th year anniversary next year. I got basically into IT through some friends. I've always been passionate about technology.

So I remember when I was young, my father, I think when I was six or seven years old, he bought one of these Sinclair ZX81 computers. So it's always been something that I've been really into. I remember programming BASIC and teaching myself to program BASIC when I was young. So it was definitely something that was in my core. In terms of studying, so I studied both computer science at a university and English literature. So I was a little bit torn between either being a writer of some kind or a journalist or being in technology. After university, on doing a few months of some work in retail, I managed to get into technology through some friends of mine.

And I've been here ever since. Started originally in sales and then sales in IT hardware components distribution for a small UK organization like a reseller. And I did that for five years. And then after that, I moved to the Netherlands for a year. And with one of my current partners and co-founders, I was working for a company where he was actually my boss at the time. So it was a large distribution company. It was a really exciting international business. So at one

point we were probably the largest components distributor for hard drives. So this was back in 2002 and worked in that company really happily for three years. But I think he primarily and I, to some extent, we got a little bit restless. He floated the idea about starting a company of our own. And before we knew it, in 2005, we took the decision to leave that company and start Taurus. It was him and I, and then later on, a third colleague joined. And it was the three of us that were running this business all the way from 2005 until 2016 as three people. I remember our first office, a small office that we used to sit, the three of us working hard, trying to make deals and obviously trying to make money in the IT hardware. And it was a really interesting journey because it was very much that kind

of guerrilla feel, just trying to go through day to day and try to do better and just adding to our business growth. And we did really well. At one point before we started to grow in terms of people and in the acquisition, so we were doing around 30 million annually. So that was really impressive for just three people. But given the kind of people that we were in terms of our nature, it wasn't enough. And we got some opportunities in 2016 to be able to expand our business. And we acquired two companies, two other distribution companies in Germany and Belgium. And that helped us to grow in terms of our revenue,

but also in terms of our strength. And then we started to acquire official distribution contracts and that helped us to really lay the foundation for what Tours Group is today. Super exciting time. That's amazing. Yes. I heard so many things about my own journey with Coding Basic

and also starting a reseller channel, actually, that we have in common. I didn't know we had those things in common. But Circle B, also an interesting history as an infrastructure and service provider in Europe. Tell me about the company and why you guys decided to acquire it. Yeah, so the acquisition of Circle B happened in 2019. And actually, we were super excited about the synergy that we felt that Circle B would have with our existing portfolio of companies. At that time, we had Taurus in the Netherlands, the German operations, which we had acquired, and the Belgian operations, which were called 2x2. And Circle B was actually the first acquisition that we made in 2019. We made two. And what attracted us about Circle B is that the founder, he was

basically a one person outfit at that time, but he was really passionate about what he was doing in the IT world. And we were captured by his enthusiasm for it. We saw that we could provide a kind of platform to be able to grow that business. And it seemed at that time to be a perfect fit. That's awesome. Circle B has long been involved in the Open Compete Project, an organization that's focused on defining open hardware

for the data center. And this week is OCP Summit. It's an exciting week to see what is going on in the Open Compute Project community. But what spurred the engagement going back in time? What spurred the engagement in OCP? And why does OCP hardware provide you an advantage today in the solutions that you're delivering to customers? Okay, so first of all, Circle B was Europe's first open compute solution provider. That's amazing. The fact that we've had a long-term collaboration with the open compute community has obviously meant that we've seen the developments over several years. What we like about the Open Compute community is the fact that there is this collaboration

and this open collaboration between all of these different technology companies and they're looking to really drive efficiencies and innovation across the whole data center. It's very different from, let's say, the traditional, let's say, server business, whereby you have a lot of competing companies which are providing their own solutions. Typically, as a customer, you end up being locked into one kind of ecosystem. And with the OpenCompute projects, it breaks down all of these barriers. And the fact that it is looking to

really disrupt the whole data center ecosystem by making sure that the solutions are open, so anyone can build on top of the solutions that are there, that there's really a collaboration between companies from all over the technology sphere, it really gives a different dynamic. So this is what really spurred Circle B to join. And the original founder of Circle B really saw the value in being part of that community so that you can engage with companies of huge size, of course, the likes of Meta, of Intel, and to be able to be a partner alongside in that ecosystem delivering cutting edge solutions to customers across Europe and across the globe.

Now, I know that there's been an incredible emphasis on OCB configuration design for efficiency and scale. I've been thinking about this. How is that playing out with today's workload requirements? And what do you see as a new opportunity to meet customers' emerging demands as you speak to them about what they want to do with their compute? Yeah. The last few years have been absolutely crazy technology-wise.

We've seen huge growth spurred on, one, by the innovations that hardware leaders in the space have done. If you take, for example, what has been done on the processor side by AMD and Intel, so many more cores in the processors than were previously possible, which has meant that much more computing power can be leveraged by these server systems. And then, of course, the advent of GPU has given birth to the craze of AI. And we are now in the situation where the possibilities are completely endless. What we see today and what has been really critical for OCP

is that the growth of these type of technologies has meant that there is a strain in terms of the power requirements on the current, let's say, power infrastructure. You see now that you have processors, for example, that are coming up to 400 or to 500 kilowatts per processor, and you have GPUs, which are now a thousand kilowatts. It means that you can't really rely on the traditional methods of being able to deliver this infrastructure in the data center and a new way of thinking needs to be there. And of course, the OpenCPU project has been already for many years looking about how to make this infrastructure more power efficient,

to bring more maintenance efficiency to the equation, how to be able to make sure that all of this infrastructure is able to operate seamlessly with each other. It's very well placed, in my opinion, now to deal with this major disruption, which is largely driven by AI, but generally driven by the innovation which has happened across these hardware vendors. And we feel that customers now need to think much more clearly about the underlying infrastructure on which their workloads are running. It's no longer a possibility to say, okay, if I need to run an AI workload that we can just not think about the whole infrastructure inside the data center because it becomes costly

and sometimes not even possible to be able to deliver the right amount of power or cooling or to make that happen in a sustainable or cost-efficient way. So the OpenComputer projects and OCP is really looking now to bring those efficiencies into the market. And we see also that very prominent technology companies, for example, NVIDIA, recognize this also and are beginning to build platforms which are OCP-based in order to be able to leverage those efficiencies that OCP brings. Now, I know that you've also been engaged in some interesting POCs of late, including one involving an immersion cooling solution that's been making some waves in media. What is driving the move to immersion liquid cooling? And what role did you play with the trial?

Immersion cooling is one of, let's say, three main ways of cooling infrastructure. For the last decades, it's all been primarily done with air cooling, with fans, with the advent, of course, of these powerful chips and the GPUs. It's not always possible just to rely on air cooling. So if you want to go a different type of route, you either have what is called direct liquid cooling or you have immersion cooling. So with direct liquid cooling, you're basically providing pipes that connect directly to servers,

but these pipes are obviously closed off and sealed and water is pumped into the system and the heat is then taken away by that water. The challenge with those solutions is that you do need to have a data center, one that has that infrastructure. And second of all, no matter how slight, there is always a potential risk that there can be leakage and of course water and electricity don't mix and that can be potentially a problem. Now with immersion cooling, it's looking at things from a different perspective. So immersion cooling is all about putting all of the infrastructure within liquid. So we're working with a really innovative and

cool company called Submer. And they have a solution whereby you can literally have a data center rack in a pod. And in that, let's say, tank, you can place all of the components that you would need for a rack. And it's in a special thermally conductive solution, but it doesn't obviously conduct electricity. And there you can run very comfortably both OCP infrastructure and you can run a traditional 19-inch infrastructure. We see a lot of value in this. Of course, it's not without its challenges and things are still being worked out in terms of, let's say, the wider ecosystem and the support of, let's say, the components when they're sitting in that liquid.

However, we do see that it is going to be somewhat of the future. One of the other advantages that immersion cooling has is that because it's a pod, it's a kind of self-contained system. So you have the cooling system also within the pod. It means that you can think differently about where infrastructure is placed. And in the most extreme cases, you can really place that infrastructure where the data is actually being used and consumed. So it's not necessary to have a large data center outfit.

You just need to be able to have access to power and, of course, network connectivity, and you can contain everything within that pod. So what we're doing in this POC is that we're working together with an ecosystem. So we have a data center, Stellium, which is in the northeast of the UK, and we're working with a server chassis manufacturer, MyTech. And then we are also leveraging the relationships that we have with other technology companies such as AMD to provide the components. And what in this POC is one, the whole ecosystem. So data center,

technology providers and solution providers working together to provide a complete solution for the customer. And we're able to provide some kind of live environment for people to see how immersion calling really works. We feel that it's important to be able to demonstrate to the customers about the different ways in which we, Circle B as solution providers, are working with this ecosystem to provide the most innovative solutions to customers, depending on the kind of needs and use cases that they have. Awesome. Now, I'm really excited to learn more about that as you guys go,

but I want to turn the conversation into this week and something very near term, which is what I mentioned before. We're at the OCP Summit in San Jose. What are you looking forward to as progress from the group and broader community this week in OCP? First of all, it's great that the Open Compute Project is hosting these kind of technology summits. So they do, of course, the global one, which is happening next week, and they have a regional one in Europe, which they also do on an annual basis. What we're looking to be able to see from the summit in San Jose is more news about some of the really cool innovations which are on the horizons. So I've talked already about some of the topical conversations around the technology space. So for example, what is

happening with cooling and cooling is a major topic because as I said, we're on the cusp now where air cooling is becoming an unsustainable solution. So we need to think about something else. The challenge has been that there has not yet been a clear standardization when it comes to cooling. And we're looking to see what are going to be the main standards which will emerge either in the immersion cooling space or in the direct liquid cooling space. So that's something I'm really looking forward to seeing what is going to come from this conference. The other thing and the other super exciting area is in the area of the modularity of server designs. Historically, OCP has been modular from the perspective of you can change major parts of the server and change them slightly depending on if you, for example, need to have in the past SATA storage or NVMe storage. There was a little bit of modularity there.

But essentially, the systems were quite limited in terms of range. There was not anywhere near as many solutions as you would see in the traditional 19-inch space in terms of different configurations. So what the community has been working on for the last few years is about creating modularity actually at the motherboard level. And I feel that this is a really exciting development because you can literally mix and match what is sitting on the motherboard level. So if you need a different type of network connectivity or you need a different type of storage connectivity, and I think eventually it might be that you might even be able to change the CPU platform. These are the kind of innovations which will give so much more

choice and power to customers so that they don't have to try to fit their workloads in a kind of one-size-fits-all situation and that you can have truly modular and custom-built designs. So I think this will be really exciting. And of course, AI. So AI is the thing that is driving technology at this moment. I'm super excited to be able to hear what solutions the community are coming for to support AI workloads, primarily GPU workloads. And I'm looking forward to seeing some announcements on some of the hardware that will be coming to support this.

I love that you talked about the modularity aspects of what OCP is driving. I think it's one of the most interesting things that's coming out of OCP right now. And we cover it in our Tech Arena compute efficiency report that we published for 2024. One thing that I'm thinking about, Jason, is you look at everything that's going on in OCP and you think about what you're delivering for solutions for your customers. As we head into the second half of the decade, you're operating across 10 countries in Europe. How do you plan to continue differentiating your offerings to customers given all of this OCP innovation that we've discussed? Yeah, absolutely. Of course, we are privileged to be very much embedded in the OCP world. I think that already gives us somewhat of a leading edge when

it comes to solutions providers, particularly in Europe. I think the adoption, of course, in the US and in Asia is somewhat higher. So I think that we are going to play a significant role in being able to help evangelize and bring these solutions to the wider market. We're doing a lot of work in trying to energize the wider ecosystem, working, of course, with multiple data centers and speaking to more data centers about the, let's say, the challenges ahead and preparing the ecosystem to transition to this innovative infrastructure. So I think this is somewhere where we will definitely have a differentiator in comparison to some of our competitors. Of course, we know the more adoption that will happen, the more competitive the marketplace.

Our main kind of goal is to bring simplification to this whole discussion for our customers. And I think this is where our kind of center is. So in talking about the ecosystem and having to work with different partners to provide the solution, it's something that the typical customers that we have, so we have customers, for example, in the MarTech space or in software as a service or in the cloud service provider space. Trying to enable and leverage that ecosystem to provide solutions can be complex. And what we see that Circle B's place in the market is to make that simplified. So we will do the kind of heavy lifting when it comes to providing an end-to-end solution for our customers and the customers can engage with us in deciding what the infrastructure should look like. We will help them to design it. We will help them to assemble

and eventually to manage all of that infrastructure. So that will be one of the key growth areas that we see for Circle B. And then we're also working on leveraging our own cloud. So rather than just bare metal infrastructure to have a cloud environment, because we feel that we're in the world now where we talk about the multi-cloud or hybrid workloads. It's a kind of very complex situation for companies to be able to navigate. And we want to help simplify that. So we would offer a combination of on-prem solutions for customers, but also to be able to connect to, let's say, our own private cloud or maybe even to public cloud as and

when necessary. So we're going to put a lot of focus and attention there. It means that we will be looking to augment our team because it's an evolving market and it requires different skill sets to the ones that we currently have. So it'll be exciting in terms of investment and in development. Jason, it was so wonderful talking to you. And second time that Circle B has been on the show, but first time for you. So welcome to the tech arena. Where can folks find out more who've been listening online about the solutions

that we've discussed today and engage with your team? Yeah, so we're pretty easy to find. So we're on obviously on the web. So if you go to httpscircleb.eu, you'll find multiple ways to be able to connect to our team, either by email or by phone or socials. We're a small dynamic team, but we're very open to people who want to engage with us. We're a technology-driven team, so it's very easy to have conversations about technology with our team. And we will be able to help anyone who would like to explore not only the world of OCP, but of infrastructure in general.

Awesome. Thank you so much. It was wonderful talking to you. Have a great rest of your day. Yeah. Thank you very much, Alison. It was a pleasure as well. Thank you. Thanks for joining The Tech Arena. Subscribe and engage at our website, thetecharena.net. All content is copyright by The Tech Arena.