In The Arena by TechArena - NVIDIA Extends AI Supremacy in the Network

Episode Date: February 26, 2024

TechArena host Allyson Klein chats with NVIDIA VP Rajesh Gadiyar about how his company is riding the AI wave into the heart of the network and edge with innovative platforms like the company’s powerful Ariel VRAN stack.

Welcome to the Tech Arena, featuring authentic discussions between tech's leading innovators and our host, Alison Klein. Now, let's step into the arena. Welcome to the Tech Arena. My name is Alison Klein, and I am so delighted to have Rajesh Gadiyar, VP of Engineering for Telco and Edge at NVIDIA back on the program with us. Hi, Rajesh. How are you doing? Hello, Alison. It's great to be talking to you again after a year, thanks for the opportunity. So, and it's been quite a year. It's an action-packed year since we last talked. And NVIDIA has been at the center of the industry's attention with everyone discussing advances in AI

and the importance of GPUs to unleashing that powerful workload. And we're going to talk about that. But I want to just start today with a brief introduction of your role at NVIDIA and a look back at our talk last year on the advances in RAN technology as one of the final frontiers of network virtualization. Can you just introduce yourself and then talk a little bit about RAN? Yeah, sounds good, actually. So I lead telco engineering at NVIDIA. I joined NVIDIA in September 2022. So it's been about 18 months. And it's been a fantastic ride. I've been mostly focused on our wireless efforts and working at the confluence of 5G, cloud and AI. During the course of this podcast, I think we'll cover hopefully a lot of ground, and I'll talk about some of the things that I've been focused on.

Fantastic. Now, we talked about RAN last year as really the final frontier for virtualization of the network. What did 2023 bring in terms of vRAN and the deployments of virtualized RAN around the world in your mind? Yeah, so 2023 was a great year for us. It was a breakout year for generative AI. It's early days, but use of Gen AI is already widespread and almost every industry vertical is adopting Gen AI and LLM applications for various use cases, including many telco operators around the globe. So in many ways, 2023 was the year that people began to understand what AI really is and what it can do. So now coming to VRAN, we made some great progress in 2023. At MWC Las Vegas,

NTT Docomo announced that they have deployed NVIDIA Aerial Accelerated VRAN in their 5G network. And at this point, we are in a live deployment, which is fantastic. Yeah, it's great. This network that I'm talking about at NTT Docomo, it uses NVIDIA's Aerial VRAN stack, converged accelerators, the Wind River Cloud platform, and Fujitsu's 5G VRAN software. Nice. Now, compared to its existing 5G network deployments, Docomo has said that the solution reduces total cost by up to 30%, the time required for network design by up to 50%,

and power consumption at base stations by up to 50%. Wow. So these are some huge improvements for our first commercial deployment. And also, Alison, last time we met, you will recall that I had told you about our RAN in the cloud efforts, which is a distributed edge cloud running AI and 5G services together

on the same accelerated cloud infrastructure. And the idea is that typically in most operator networks, RAN is significantly underutilized. And if you can use the same infrastructure to host both RAN and AI applications, such as generative AI and LLM applications, you can increase the utilization, reduce the total cost of ownership, improve the quality of service for users, and increase the monetization opportunity for telcos. So at Computex last year, like in May timeframe, NVIDIA announced a product called Grace Hopper, which is really state of the art. We also announced a platform called MGX-GS200, which is a modular and scalable architecture for data centers. So combined with NVIDIA DPUs,

our Bluefield 3 data processing unit, MGX-GS200 offers the best design for distributed edge data centers. So at Computex, we also announced a collaboration with SoftBank to deploy MGX-GS200-based data centers across Japan, hosting 5G and AI services together on the same infrastructure.

So as you can see, we've been busy. We have continued to build on our vision of a software-defined radio access network built in the cloud and with AI. And we have great momentum. NVIDIA is a performance leader in VRAM, and we have demonstrated best-in-class cell density and leadership throughput and spectral efficiency. This is the bits per second per hertz per watt KPI using our aerial platform. So really proud of the efforts and where we are and the work that we've done in 2023.

And you have the best codename ever in Grace Hopper. That was such a great choice by the company. And what a cool product. You and I have been talking about 5G for a long time. And I think we can now look at the network landscape and say that we've got broad adoption of 5G. And you've been involved in this since it was a twinkle in your eye and some other architect's eyes in terms of what was going to be the capability. What has the industry been able to deliver in terms of that envisioned value

proposition? And what's left to tackle in your mind? Yeah, twinkling the eye. I really love that. So you're absolutely right. I mean, at this point, 5G as a technology is broadly deployed in operator networks. And there has been some good progress towards virtualizing the RAN and adoption of cloud-native technologies in RAN. Additionally, Open RAN Consortium has also made some really good progress in disaggregating RAN with well-defined interfaces and enabling a larger ecosystem. But I do think that there are still many challenges that we need to address. First, as you know, 5G was supposed to be delivering three things. Enhanced mobile broadband, massive machine-to-machine communications, and ultra-reliable low-latency communications or URLNC. Now, in particular, network slicing as a technology was supposed to enable some of the use cases around the massive machine-to-machine communications and URLNC.

But this really has not taken off that much. Another key area is private 5G. This is a huge opportunity, but it's yet to see large-scale adoption in the enterprises. So I think some of these challenges will be addressed in the coming years, and AI will play a critical role in not only solving the technical challenges, but will also become a business driver and a key monetization opportunity.

So a great example is GenAI and how every telco around the globe is beginning to use GenAI and LLM applications for various use cases. So Alison, I also emphasize one key element that I believe is critical to accelerate the 5G vision and create a bigger impact for 6G. It is really important that the future network infrastructure is built on a software-defined foundation with a modern accelerated compute infrastructure. Because if you look at current VRAN deployments, they are running into a performance wall because the CPU performance is just not keeping up with the network applications

in the 5G and AI era, right? So as a result, some in the industry are integrating fixed function accelerators in the CPU. And this is like going back in time and building appliances all over again. So aside from an inefficient architecture, it leads to wasted single function resources, which is a complete antithesis of cloud computing, if you can think of it that way, right? So instead, I strongly believe what network infrastructure really needs is a fully software-defined and programmable infrastructure, such as an NVIDIA GPU-accelerated aerial platform. And the beauty of this platform is that it can accelerate many workloads. So various RAN configurations, such as 4T4R, massive MIMO 32T32R and 64T64R,

AI in RAN, generative AI and LLM applications at the edge, and other Mac applications. So this software-defined and accelerated cloud is our core belief, and you're working with the Telco ecosystem to modernize the network infrastructure with a software-defined and accelerated cloud infrastructure. You know, it's really interesting. When you were talking, you talked about network slicing. You talked about the cloudification of the network. And one question that I have for you, Rajesh, I know that you work really closely with the Telcos.

Obviously, comms infrastructure was not born as cloud-native infrastructure. What do you think it's going to take to get folks completely on board with that software-defined vision and accelerated vision? Is it the workloads that they're looking to run and being a forcing function to move to that model? Or are there other things in play? Yeah, that's a great question, Alison. So if you look at how things have evolved, we used to be in an appliance world back in the 3G and 4G days, right? And along came the realization that we cannot sustain an appliance-based infrastructure because RAN is the most expensive thing for telco operators, right? That's where they spend most of their capex. And not being able to use that precious resource for more than just wireless networking

and a resource that is typically significantly underutilized, that wasn't going to sustain. And so I think that led to NFV. And the initial efforts were with a CPU-based infrastructure, naturally because I think CPUs gave a lot of programmability and portability of applications. You could disaggregate hardware from software and could use general purpose platforms for building network applications.

And I think that was okay at that point in time. But as things have evolved, as you see, like every generation of wireless actually results in 10 to 100x more performance, 100x lower latency, and many new use cases. And particularly now with the unleashing of AI capabilities for every vertical, I think what you're finding is that the CPU performance is just not keeping pace. I think if you look at the typical sort of like CPU performance increase, generation over generation, I think we are seeing 15%, 20% kind of performance improvements, which are not going to be good enough.

And as a result, like I was explaining earlier, I think there is this desire to now solve the problem by putting fixed function accelerators, which is what I was saying actually is like going back in time. And so this is where we have stepped in as NVIDIA. Our vision is full stack acceleration on an accelerated computing platform where, like I was explaining earlier, you can actually run many applications. The result is that you can actually deliver many X improvements in various workloads.

Now with our efforts with what we've done with Ariel, we've been able to actually show that the same thing is possible for the radio access network and wireless as well. And I think this is going to become more and more critical as we march towards 5G advance and 6G. Now, you've talked about AI and you've talked about where you see it being a core capability, and it seems like nowhere in the tech sector can you have a conversation in 2024 without it centering on AI.

When you look at the opportunity to accelerate network workloads, you've talked about the RAN. You know, there's an opportunity to drive a lot of automation of work utilizing AI and comms networks. Where do you think the best opportunities are? And where are telcos leaning in right now to take advantage of technologies like generative AI?

Yeah, that's a great question, Allison. Accelerating network workloads and driving automation through AI presents many opportunities across various domains. And let me actually share some key areas where these opportunities may be particularly promising. In 5G, the deployments of 5G networks require efficient management of increased data traffic and low latency communications. And AI can optimize network resources, predict failures, and dynamically allocate bandwidth to meet varying demands. Now in edge computing, AI at the edge allows for faster decision-making by processing data locally rather than relying solely on centralized cloud servers.

This is really crucial for applications like IoT devices, autonomous vehicles, and smart infrastructure. Enhancing edge infrastructure with AI can lead to improved resource allocation, load balancing, and predictive maintenance. AI can optimize data storage, processing, and delivery, resulting in more efficient and cost-effective cloud services. In network security, AI can play a vital role in enhancing cybersecurity by identifying and responding to threats in real time. So machine learning algorithms can detect anomalies, predict potential security breaches, and automate responses to mitigate risks. Another example is like software-defined networking. So SDN allows for programmable and flexible network management. AI can really optimize traffic routing, automatically adjust

network configurations based on demand, and predict network failures to ensure continuous service availability. So we have an opportunity to go from reacting to failures based on policies to predicting potential failures and taking proactive action. I'm also actually seeing a lot of interest in what is called intent-based networking. So intent-based networking uses AI to interpret high-level business objectives and translates them into network configurations. And this really simplifies network management, reduces manual intervention, and ensures that network policies align with business goals.

It's like other examples too, right? The automation of network operations. So AI-driven automation can streamline routine network operations such as provisioning, configuration management, and troubleshooting. This is where I've seen actually a lot of LNs being used. In fact, AT&T had actually showcased QoP in their collaboration with NVIDIA, which gave them significant operational efficiency, reduced downtime, and faster response to network issues. So just to summarize, the convergence of AI and networking technologies opens up opportunities to enhance performance, security, and operational efficiency across a wide range of applications and industries.

Now, the key is to identify specific use cases and tailor these AI solutions to address the unique challenges and requirements of each scenario. And this is exactly what we at NVIDIA are doing. We're working with Telcos and the AI software ecosystem around the globe to unleash the power of AI for Telcos. Now, this positions NVIDIA in a very interesting place in the network. And so I guess what question that would follow for me is, what can we expect from NVIDIA in terms of solution differentiation? You've talked about Grace Hopper and also, you know, how you're dealing with working with telcos in the ecosystem to deliver solutions on top of NVIDIA. At MWC this year, you will see several demos showcasing our work, right?

So first, Juniper is demonstrating a RIC, a RAN intelligent controller, and AI ML-based XApps and RApps for service level agreement or SLA assurance. And this use case is a pre-trained AI model to assist slice optimization based on SLA and network loads. Another demo you're going to see is with a company called Arna Networks, where we show an NVIDIA cluster agent that can orchestrate both 5G network functions and AI applications on the same cloud infrastructure. And then at GTC next month, we're going to be talking a lot more about AI in RAN. We'll be discussing some new developer tools and software. And I'll keep the suspense for now and ask the audience to join us at GTC. It really promises to be a great event and good learning opportunity. And it will be an in-person

event in the Bay Area. Wow, that's impressive. And I can't wait to see more, both at MWC and at GTC. Now, you talked about 6G a couple times in the interview, Rajesh, and I need to ask you, you know, 5G was incredible in terms of being a transformative force in the network, and it continues to be. What do you think is going to be the main differentiation of 6G? And how far off is this technology? Yeah, first, as I noted earlier, although 5G was supposed to be transformative, it has not fulfilled the entire promise yet. I think it's fair to say that the jury is still out on some of these promises, because most early 5G deployments are all about speeds and feeds and

mobile broadband. So the real question is why, right? What has prevented us from achieving the full vision of 5G? There are likely many reasons, but one of those reasons in my mind is the sheer complexity of building, deploying, operationalizing, and managing these networks. In my mind, the only way forward and how we solve this problem is to utilize AI in networking, in RAM, in core, in OSS, BSS solutions. So we at NVIDIA are firm believers that AI will be an integral part

of next-generation wireless, that is 6G. And additionally, 6G will need to be fully software-defined, deployed in the cloud, and on an accelerated cloud infrastructure. So 6G research work has already started. Standardization effort in 3G PPE is going to come in soon. And NVIDIA will be a key part of the journey towards 6G. And we are looking forward to partnering with the rest of the ecosystem to not only drive the transformative vision of 6G, but bring that vision to reality with our platforms,

tools, and software. That's fantastic. When you think about all of the stuff that you just talked about, how is NVIDIA engaging the ecosystem to deliver solutions based on your GPUs? NVIDIA has a very large ecosystem for AI in many verticals, such as financial services, gaming, healthcare, robotics, transportation, and so on. Now, in 5G, we have several publicly announced ecosystem partners. For the VRAN software stack, we work closely with Fujitsu, Capgemini, Radices, Maveneer. These are software partners.

For RAN intelligent controllers and XApps and RApps, we are working with companies like Juniper and HCL Technologies. In addition, we are working with many co-operators. We're also actually seeing tremendous telco interest in building AI plus 5G data centers and provide the infrastructure also for hosting AI applications at the edge. Very cool.

I learned so much, like I always do when I talk to you. So thank you, Rajesh, for the time. Where can folks find out more information about the solutions you talked about today, the deployments you, Rajesh, for the time. Where can folks find out more information about the solutions you talked about today, the deployments you talked about today, and what you're delivering with the rest of the industry? And talk to your team. We have a lot of information at developer.nvidia.com.

In particular, if you're looking for 5G resources, developer.nvidia.com slash aerial-sdk. Or you can just connect with me on LinkedIn, and I'll be happy to help you and provide you more information. Fantastic. Well, Rajesh, thank you so much for being on the show. It's always a pleasure. Thank you, Alison.

It was great talking to you. And I look forward to meeting you and all our partners at MWC in a week's time and at GTC next month. Thank you. Thanks for joining The Tech Arena. Subscribe and engage at our website, thetecharena.net. All content is copyrighted by The Tech Arena.