In The Arena by TechArena - Exploring AI Innovations: Chen Lee of Gigabyte at OCP Summit

Episode Date: December 2, 2024

Join host Allyson Klein and co-host Jeniece Wnorowski in this episode of Data Insights as they chat with Gigabyte's Chen Lee about AI innovations and the future of server technology at OCP Summit.

Welcome to the Tech Arena, featuring authentic discussions between tech's leading innovators and our host, Alison Klein. Now let's step into the arena. Welcome in the arena. My name is Alison Klein, and we're coming to you from the OCP Summit in San Jose. And it's another Data Insights episode. So I am joined by my co-host, Janice Hrowski. Welcome to the program, Janice. Oh, thank you, Alison. It's great to be back. So Janice, we've been podcasting all week from most CP Summit. What is the highlight of today?

Oh, gosh. Today, there's been so many different highlights. We just had our panel breakfast this morning. That was one of our big highlights. But I'm really excited to talk with Gigabyte. I think this is going to be our next exciting point today. All right. To be honest with you.

Well, let's get to it. Let's do it. I am super excited to have here with us today Chen Li, who is the VP of Sales at America for Gigabyte. So welcome, Chen. Thank you so much. And I'm honored to be here with you to do this podcast. And I'm very excited to share some of my understanding of the OCP and AI business with you. That's awesome. Thank you so much for being on. This is the first time that Gigabyte has been on the tech arena, and so we're so happy to have you here. Gigabyte is known for being a major player in delivering quality,

innovative solutions to the marketplace. For those of our listeners who are just getting introduced to you, can you please introduce the company? You're engaging with OCP this week. Okay. Well, probably for a lot of the listeners who are gamers, they know who Gibbivy is. We're great at making gaming motherboards and video cards and all the cool stuff that you can play all the video games with. But it's very little known that we actually make servers. So 1989, our founder decided that, you know what, this is a business worthy of some investigation. So he created a small

team to investigate how the server is going to affect Gigabyte's future. And then in 2000, we started Network Communication Business Unit, which is our server division. And today we are about 26 years in the play. And interestingly, you asked about innovation. What do we do as far as OCS, please write. Around 2004, this very, very little known company came to us. We got a search engine, and we want to build this motherboard and this thing called OpenRank. We go, oh, so who are you guys?

Google. Our company, their salespeople have no idea who they were. Right. Sure. Why not? Let's talk. So that's how we got into OCP. We were actually the first company that helped Google develop the open-to-the-foot.

That's amazing. Of course, Kittify doesn't have the manufacturing capability to continue with large scale. So they went through in our larger stamps, but that's many years down the road. So we did support people for quite a long time with this OCP. And a lot of our Google OCP development team went to Facebook. And that's how OCP got started. And so we've been doing this OCP things way back since 2004. And that's very little known by a lot of people in the industry. It's very unknown. It's very unknown. So, yes, we do a lot of the innovative stuff. And today, we don't only build servers for AMD and Intel, we do Ampere for Onslaught.

I mean, we don't do just regular, we have immersion, we have a look at the right to a chip and look like so yeah we we had a good growth of the past 26 years. I came to give up I want in 2006 and today the North America team we are somewhere on that one at once. Wow. Yeah. That's incredible. Well, incredible that this Jensen Allen and videos, you know, develop. Everyone involved us. Yeah. And speaking of that, right, there's been so much innovation and incredible demand with AI.

Right. You know, specifically within the data center. Can you share a little bit more about how Gigabyte has taken on the AI challenge? AI challenge? Yeah. So for the first four years of my career, I did it by Google Zero. And we weren't concentrated and focused in the channel market. But our CEO decided, you know what, we need to do something for the channel. So around 2010, we developed the server. It's a 2U server, but we can hold eight. The whole while building GPU inside. It's the highest density server on the market. From there on, we were known for GPU servers and we built quite a few GPU servers. So when people

started working with AI angle, right, Gigabyte was already in the game, right? We can supply a GPU server starting from this PCIe card all the way to A100, right? And then now we're in H100, you know, and H200 and go on to BlackRock. So our focus on the AI part of it really started from 2010, built in 10 years ago, virtually. Yeah. Lucky enough to make many engagements with a lot of different companies. And then, again, something like a little-known company called Cornweave came to call. Oh, wow.

So we're now one of their suppliers. And it has been very good a couple of years for them. You know I think that one of the things that you've talked a lot about your server technology but a lot of people don't know that you are playing in liquid cooling. Yes we are. And so tell me about how you approach liquid cooling and how you settled on delivering direct liquid for your systems. So we do both direct liquid and also immersion. And then we've been in the immersion game with several oil and gas companies for probably

over six years. So we gather a lot of information and experience when it comes to immersion. And this foundry that makes chips in Taiwan came to us asking to co-develop a bank for them. So that big foundry that makes chips for like this half the world and most of the world, now using immersion technology built by DigiBot. That's incredible. Yeah. And I can't, I'm sorry, I can't say the name of it, by DigiBot. That's incredible. I'm sorry I can't say the name of it,

but I think that's my number one probably. Yeah, for sure. As immersion cooling is cooling down a tremendous amount of compute, the compute we're talking about requires not only a tremendous amount of data to feed the training and then

drive the tuning and inference, but what do you see as infrastructure changes to manage this data across different environments? Well, that's an interesting question, Flat. Everybody today are talking about, oh, wow, we H100, H200 to do all this training, and they're also using those systems for inferencing. But sooner, and I'm not saying later, sooner,

this inferencing is going to explode. It's going to be on the edge, right? It's magic. A convenience when you walk in, right? The ad will know you walked in and position the ad towards you, right? You can watch where the customer turn, which aisle, what aisle to look at. All that is the machine in the back of the room closet,

feeding it into the headquarter. So this is something that has been talking to us about at a convenience store. So the training right now, it's just exploding. But I just don't see a lot of people moving towards the inferencing. But more and more as I talk to customers or so on and so forth, they begin to understand, now, OK, we need data center space. We need to do this and back in order to

see all the stuff that we've been training all the stuff that we've been working on to be able to deploy to the world so of course a lot of the the inference you've done on this egg machine but sooner or later right it's not everybody going to need to be driving a ferrari or horse ship right somebody's going to have to drive a toyota or a Porsche. Exactly. Right? Somebody's going to have to drive a Toyota or a Honda. And that is the mass market. And that's where the growth, I think, for the compute side of AI is going to come in. When you think about that, and I'm going to ask some questions, a follow-on to that. Yeah.

Where do you see the industries that are moving fastest in terms of inference? Do you have any idea about where we're going to see this at the edge first? Okay. In adequate. Okay. And heal. And every day type of shit and multi-tuneeling store. That's what I see well now. Are you? Yeah.

Medical is the biggest thing, and it's something that makes me very excited as well. There's going to be so much breakthrough, hopefully, with this AI development. That's going to benefit us human beings to be healthier and to be stronger. How about that? But I think the most important thing is that everybody joked about AI, right? Like

Skynet and Terminators and this and that. But I think AI is going to benefit us to make us more efficient, make us to be able to do things right, to do things better versus you like, oh, we are scared yeah right yeah there's nothing i don't think there's anything to be afraid of with this ai thing it just it's going to be the future it's not going away it's not people say oh there is the boom it's like a dot com i said uh-uh this thing happens to be a botcon but then this is this is going to be an ongoing trend, an ongoing way. The USP company will come and go, but AI is here to stay. And we better get ready to accept this new world that's coming.

And it's going to be exciting. That's awesome. So how do you work with storage leaders like SolidEye to deliver solutions like what we've been talking about today? Well, Solidigm has been making some great drives, great storage. So when it comes to these AI inferential training, all those type of drives and disks are being installed into our server in order to feed the customer needs. And then the bigger capacity as we're developing of course it's expensive but right now it's a it's almost in a uh you know expense is not an issue kind of a game that we're playing and the solid is delivering drives that are just you know opposite right and they're helping us getting even better business because Solidaire also delivers on very good price in Splash 2.

So that is a big advantage, especially the company like Digibyte or any of my competitors. We relied on the company like Solidaire to be able to provide us with the support and the drive that's needed and off of the market. That's amazing. Really cool. and a drive you know but that's needed and off of the market that's amazing really cool so you you mentioned chad you've been a gigabyte since 2006. oh six yes thank you okay long time so of all these years and you kind of look back what are you most excited about in 2025 per gigabyte you don't have to say anything secret either. Nothing secret. I'll tell you what. The new upcoming NVIDIA Blackwell is going to be very exciting.

And to be able to work this business along with SolidVine and Intel and AMD and so on and so forth, I think it's going to be the next level. So with all your experience here at Gigabyte, right, since 2006, right, what are you most excited about in 2025 for Gigabyte? Yes. So the Blackwell launch is going to be exciting for us because as the Haswell sales

slows down, affordable, I mean sales, so we're going to talk about revenue and that's going to be an exciting couple of months

to get Blackwell other running and getting orders from our customer to start shipping. Right. Bidding sales, of course, you know, I look at numbers Blackwell other than running and getting orders from our customer to start shipping, bidding sales, of course, I look at numbers and the number is very exciting. Especially not just when I say Blackwell, we're also talking about the GB200s in higher rack.

That the revenue and the business that could bring two gigabytes or any of the people within our competitive realm, it's unprecedented. Yeah. It's an incredible opportunity. It's an incredible opportunity for all of you. And I think that what's really important, Chen, is it's an incredible opportunity for end customers too. Because what you talked about with all of the applications that are going to be delivered from AI, yes, it's a great moment for the industry.

Yes. But the transformation of this technology will be delivered is incredible, right? Yeah, a little scary too, isn't it? How fast we're developing, how quickly we're getting this, isn't it? I mean, Tesla just announced their robot, right? $30,000, you can have something like that in your house. Robotech's taxi.

I mean, all that stuff, it's going to just come one after the other. Well, I am so glad that we spent some time with you today. You know, it's incredible to hear the story about how Gigabyte started, how you started in servers, and where you are today. It's been an incredible journey for you guys, and I'm glad that you brought it to the tech arena. One more question for you. Yeah.

Where can folks find out about all the awesome tech we talked about today? So, www.gigabyte.com and go to the Enterprise site and all of our product line will be there.

Fantastic. Well, thank you so much for being here. Thank you. Thanks for joining the Tech Arena. Subscribe and engage

at our website, thetecharena.net. All content is copyright by The Tech Arena. Subscribe and engage at our website, thetecharena.net. All content is copyright by The Tech Arena.