Utilizing Tech - Season 7: AI Data Infrastructure Presented by Solidigm - 2x21: The Intersection of 5G and AI with EdgeQ

Episode Date: May 25, 2021

You might think that 5G and AI are completely unrelated, but these new technologies support each other. Both are expressions of information theory, and both use similar mathematics under the hood. Both 5G and AI are also disruptive to existing business models and enable new applications. EdgeQ develops processors that leverage machine learning to improve customer experience in 5G and enable customers to develop their own AI solutions on-chip. 5G is bringing the edge closer to the cloud and it enables seamless deployment of AI across the network. Three Questions When will we see a full self-driving car that can drive anywhere, any time? When will we have video-focused ML in the home that operates like the audio-based AI assistants like Siri or Alexa? How small can ML get? Will we have ML-powered household appliances? Toys? Disposable devices?Guest Information Guests and Hosts Vinay Ravuri, CEO / Founder at EdgeQ Inc. Connect with Vinay on LinkedIn or Twitter at @EdgeQ_Inc Frederic Van Haren, Founder at HighFens Inc., Consultancy & Services. Connect with Frederic on Highfens.com or on Twitter at @FredericVHaren Stephen Foskett, Publisher of Gestalt IT and Organizer of Tech Field Day. Find Stephen’s writing at GestaltIT.com and on Twitter at @SFoskett.   Date: 5/2/2021 Tags: @SFoskett, @EdgeQ_Inc, @FredericVHaren

Welcome to Utilizing AI, the podcast about enterprise applications for machine learning, deep learning, and other artificial intelligence topics. Each episode brings experts in enterprise infrastructure together to discuss practical applications of AI in today's data center and beyond. Today, we're discussing the intersection of 5G and AI and how both of these things drive to each other. First, let's meet our guest, Vinay Ravuri. Hi. Yes, I'm Vinay Ravuri, CEO of EdgeQ. You can find me on LinkedIn and you can find the company on Twitter at EdgeQ underscore Inc. I'm excited to be here.

So my name is Frederik van Heren. I'm the founder of Hyphens, which is a company active in consulting and services, specifically for the HPC and AI markets. And my background is in telecom and speech technologies. I can be found on Twitter under Frederick V. Heron. And I'm Stephen Foskett, organizer of Tech Field Day and publisher of Gestalt IT. You can find me on Twitter at sfoskett,

and you'll find me this week hosting the AI Field Day event at techfieldday.com. We've talked quite a lot on this podcast over the last few years about the many exciting ways that artificial intelligence is impacting our world in pretty much every aspect. And one of the things that we've discovered is that almost no application, almost no corner of computing is untouched by AI and specifically by machine learning. Now, one of the things that is also out there on everyone's lips right now is 5G. And it might seem that these two things are just a pair of buzzwords that have nothing to do with each other, but that really couldn't be further from the truth. Considering that it's

a brand new technology, everything in 5G is being designed with machine learning and other artificial intelligence capabilities in mind. And of course, it is an enabler for AI as well. That's one reason that we wanted to invite Vinay here onto the podcast so that he could explain just what it is about 5G and AI that work together. 5G and AI are, you know, a lot may think that they're completely separate technologies and they don't have an intersection, but they actually do have a lot in common. technology comes from an existing theory of mathematics called information theory, which is what 5G is all about actually. This was the fundamental mathematical operations

that one is used to in AI. A lot of people talk about things like matrix multiplication. These are low level operations that are done. They fundamentally do come from information theory. And so, therefore, AI really came out of a branch of technology that existed prior in communications. Now, looking at AI and 5G today, I really like to categorize these things as two things. One is called 5G enabled by AI, where AI makes 5G better. And we can talk about specific elements there.

But then there's also what I call AI enabled by 5G, where AI is the main player there and 5G is sort of there to help AI itself. And as you know, just today, it is pretty much any application, anything you can think of, whether it be on a phone or car, really AI is everywhere, and it is disrupting a lot of things. And as you said, it is a buzzword in some sense, but also 5G is also another buzzword. These two coming together is somewhat new. But for those of us that are in this field, we are seeing this as coming together and working in a uniform manner very nicely. And I think that in the next few years,

you're going to see technologies that are rolled out, whether it be on a phone or it's a base station or a car or whatever it is, you're going to see those things intertwined and overall make the experience better. And that's the overall goal here. Right. So you talk a little bit about 5G and over the wire.

How does that compare to, for example, wired technologies today, right? There's obviously the flexibility that you don't need a wire, but what are the benefits of using 5G in situations where, for example, you could use a wired networking? Do you have kind of an overview of why people would go for 5G over wired, you know, the kind of the pluses and the minuses? Yeah, I mean, just wires in general,

right? They take a while to deploy, for example, as a simple matter of time. You know, if you're in like a building environment, you need to install cable. That's, you know, Ethernet cable, for example, to build out a network. And that takes time. Sometimes you need people to get permits and things like that, where if it's wireless, you can literally build it out within minutes versus this could take days, months sometimes. So that's one aspect of it. Another aspect of it is also just the time it takes to deploy something and not just the

installation, but configuration, these things where if it's wireless, they tend to, you know, the protocols are designed in such a way that they can dynamically network together. They can form networks. Like you think like Wi-Fi, for example, you know, when you come into a Wi-Fi environment, your laptop directly connects and figures out the addressing it figures out a lot of things as long as you have credentials for it in a wired world you you know typically it's a very static thing somebody has to have provisioned this ahead of time uh and and then that has to get done all across so it just it's a much more dynamic uh ease of use is a lot better when it comes to this wireless world. And there's other benefits with 5G, particularly here that are related to latencies and related to

mission criticality that it's built in. Some of those exist in wired as well. In fact, wired worlds are pretty, you know, reliable, But wireless sort of brings that kind of reliability, particularly 5G, without having to deal with wires. And that's really the key thing about 5G. Specifically, what areas will, how does artificial intelligence support the deployment of 5G networks specifically? Why are these things, you know, why is AI enabling 5G? Yeah. So let's, this is how is 5G being made better by AI essentially, right? So, you know, much like I like to kind of take an example of what you've seen already in the past. For example, image processing or computer vision is a very big field with machine learning or AI, like object recognition, right? ImageNet competition that took place where the accuracy went from 70% to 80%, 90% now

in the high 90s. That kind of a thing, what did it really replace? What did AI replace? It replaced a well-known mathematical equations that were used to do this recognition. Whereas machine learning, you are sort of throwing a neural network at it and it sort of learns what to recognize and how to recognize it.

That aspect of it is what machine learning did. And that happened in natural language processing and it's happening in other fields. Now, bringing that to 5G, all the processing, as it's called, baseband processing, particularly this is of an area for us, is where your wireless signals are essentially converted into bits and eventually into packets.

Then there's different mathematical operations that are done, which are they use traditional mathematical equations to solve those problems. They use matrices and multiple inversing them. As an example, there's something called channel estimation, which means that when I'm transmitting a bit over the air, that thing comes back with all garbled up. How do I figure out what that real bit is? So there's famous mathematics that you apply to that, and then you get that. That's the traditional approach, and you figure out what that bit is. But the thinking nowadays is perhaps what we should do is use machine learning, AI,

learning that channel dynamically without actually having to send any pilots, as they call them, reference signals. Maybe there's a way to do this by just learning the channel just from data. And in doing so, I potentially have higher performance that I can achieve with that. So that would be an example. Another example, maybe a much more simpler example would be, let's say you as a user, like a cell phone user, I may learn your behavior. Like this time of day, I'm watching YouTube videos or another time of day, I tend to be on a phone call. Knowing this behavior, I can potentially reserve the channel or the bandwidth and provision things in a certain way so that I can schedule users automatically. In doing so,

the network itself might be a lot more balanced and utilized in a better way. And that learning mechanism can also use AI. And the interesting thing about this here is that unlike images and those things where you're looking for data here, you are constantly getting that data because you are the network. So you can learn this very dynamically. And that's the nice thing about 5G is that you don't necessarily have to get data from somewhere else. You are part of that data and you can dynamically learn and adjust the network. We've heard similar stories in the networking space and the data center networks and so on. So I think that that's very interesting to think that, you know, machine learning can solve some problems quicker than conventional math. Fred? Yeah, I was going to ask. So HQ really delivers a product on a chip. So are you then saying that you're doing AI on the chip or you're using your chip to enable customers to do AI or maybe both? of those. What we do is 5G enabled AI, meaning 5G being made better with AI. Those I call first party applications for 5G. So that's essentially implementing 5G algorithms in a more potentially

efficient and high-performance manner. Those we see as our own applications, meaning we implement that for ourselves and use our own 5G aspects, rather AI aspects of the chip to make 5G better. So essentially you get a silicon from us that's largely not known how the implementation is to a user, but it really is a better solution. That part is not so much seen by customers, but potentially there are customers that have the wherewithal to implement their own algorithms, can replace one of ours and implement their own. And that's the nice thing about what we do at EdgeQ is that this is a very software-driven silicon, so you can replace aspects or portions of the chip or the algorithms with your own as a customer. When I say customer, I think like an OEM, a system box maker.

On the other hand, AI-enabled 5G, use an example. Think like there's a camera somewhere that's recording some video or surveilling something, and these images are sent to its next hop. Let's say it's a base station and at the base station it can recognize rather you could use that base station double up double it up as a machine learning edge computing device so where it's actually trying to recognize objects or you know anomalies or something it could be like a manufacturing plant with a camera and it's detecting anomalies in that plant on an assembly line.

In that case, most likely customers would probably bring their own models and run this on our chip. So in that case, we are sort of the chip that's providing the link, the 5G link to that camera. But the actual machine learning model that runs to recognize those images would probably come from customers because that's their value add and that they probably do a better job there than we can.

We will provide the underlying hardware and software layers to enable that framework. And these customers that you speak of would be network service providers, right? I mean, the people that are actually rolling out 5G. They come in multiple flavors. The traditional network providers, which you may call operators in this case, the traditional telco kind of companies, but there's two other up and coming. You can call them service providers or you can call them consumers of such 5G technology. And I'm actually very excited about these other two.

One of them is traditional enterprise. Think where Wi-Fi goes today, right? Corporations might go into a manufacturing plant. These are private networks, really, right? They're enclosed, they're. So in that case, this would be deployed and managed not so much by operators. However, operators may also play a role in servicing that market. So it's not necessarily either or. And you're talking about CBRS there, right? Yeah. CBRS is a band in US that essentially placed there. That's a very good example of that. But then there's also one more segment of this market that's really in the nascent part right now, which is hyperscalers. Traditionally, hyperscalers were not known to get into the connectivity world unless it's their own

data center. But one of the things that's really happening that's very interesting is these hyperscalers have scale. They have reach, they have both their centralized data centers, but they also have edge data centers. And more importantly, they have the efficiencies of their software. They can manage large scale networks users and provision them and orchestrate them. And 5G networks as they're being deployed now with especially this thing called Open RAN,

which is a very up-and-coming standard, where this sort of a network potentially can be serviced, and it looks like they may be, by a hyperscaler. So Amazon and Dish recently made an announcement about that partnership where Amazon would provide a, you know, it's called a DU, which is a distributed unit that's part of the 5G stack, which is an O-RAN terminology. And they would sell that or partner with an operator. So essentially, operator says, okay, I'll bring what I'm good at.

I own spectrum. I have the reach to the end customer. And I will do that part. And maybe I'll partner with a'm good at. I own Spectrum. I have the reach to the end customer. And I will do that part. And maybe I'll partner with a hyperscaler. And the hyperscaler will say, well, I'll bring the rest of it, which is the software part and the management aspect of it, and then the hardware itself. And you as a customer choose which ones you want. And you sort of kind of choose from a menu.

And then voila, you have a service that stands up very fast, because that's what these guys are good at. So that's a newer market, and that again can be CBRS, it could be non-CBRS, and the way I see these hyperscalers is that their motivation is not just to provide connectivity, but to really get their data into their cloud because they can monetize that data. And these end networks, these private networks are rich in data. And that is really the exciting part of all this. I agree with that. I think it used to be that in the early days of AI, everything was done, all the processing and the computing and storing the data was all centralized.

And then we started talking about edge, where we kind of pushed it to the edge just for data gravity reasons and performance. I think what 5G is bringing to the table is that the hyperscalers suddenly become very interested in, again, trying to centralize the processing as much as they can, one, for scale, but also because there's enough bandwidth now to kind of not eliminate, but reduce the discussion around data gravity.

Do you see that as the number one use case in AI? I mean, from the customers I talk to, I do see that as definitely a top three, but is that within a top five of your 5G? Yeah, absolutely. I think you hit the nail on the head. So really, as far as AI is concerned, our intelligence is concerned, there's really two locations that intelligence can be handled. One is at the edge, and edge itself could be defined as the true edge, which is where the end device is, but unlikely that end end device would make that many intelligent choice decisions. But let's say some sort of an edge cloud. And in which case that edge cloud, these hyperscalers would want to play a big role there by bringing aspect or augmenting their centralized cloud into the edge. That's one. Second is, you know, it's not either or, meaning that it's not like, you know, Amazon has to move all of their AWS

technology to the edge. It's just that what needs to be done at the edge, they will do at the edge. What doesn't need to be done at the edge can be taken into the cloud where it is probably a lot more efficient in terms of money. But, you know, there's another element here, which is not just about cost economics. It's also security. And these centralized clouds, they, you know, they have regional impacts, like, you know, they could be an European data, which needs to stay in Europe or stay in the sovereign country that they're in and they can't cross borders, in which case you have to physically have your data there. It could be that it is a

factory that does not want that data to be going to any kind of cloud, meaning that it needs to be private, in which case that cloud might be in that building. But yet, hyperscalers want access to that business, right? So that isolation can also be provided with this sort of a framework. And whichever way they participate, whatever you call cloud here, is definitely up for grabs because this is new world. Right now, everything is really centralized, right? There's really no edge cloud or processing that is that big. And that's where the world is headed next. Yeah, I think that's the right message. I do think that 5G is actually bringing the edge and the hyperscalers closer to each other. I think that's the main argument. It

was always kind of difficult to figure out what is really edge, what isn't really edge. And the edge by itself is not that scalable, although most people will say that. But I definitely believe that 5G brings the hyperscaler closer. And that's also one of the reasons why AI is really important or 5G is really important to AI. it's because it provides a lot of opportunities. Yeah, yeah, I agree.

Yeah, I appreciate that. until we have ubiquitous and reasonably performing network infrastructure, and until we have really distributed computing infrastructure, which is another element of 5G that I think people overlook, the fact that it does move even data centers to the edge. There's computing devices all over the place in a 5G network. It really enables the deployment of applications seamlessly all the way from client devices to network edge to edge core, I guess you could call it, to hyp to see how the the the compute infrastructure has just totally exploded. And and 5G is sort of rioting that and AI is writing that. Is this how you see I mean, where do you see this going? Where will the ultimate where will we be looking at in five years from now in terms of edge and 5G and AI?

Yeah, that's a great question. I think there are several elements here. 5G, number one, cannot be in isolation, meaning that it isn't one thing that's just going to replace everything. You have to think about what happens to Wi-Fi in this context. That's one thing. Second is, how does 5G fit into existing, not just connectivity, but frameworks, software, and business models? How do people buy things?

That's a second part. And the second part. And the third part, which is what type of applications will 5G actually enable here at the end of the day? You know, Edge Cloud and in analytics, these are all interesting, but what does it all do enable at the end? So maybe let's take one at a time. So the Wi-Fi part, I think Wi-Fi and 5G will coexist. You know, there's this thing, nobody has won betting against Wi-Fi, okay? Many have come and

gone. So these two will have to play well together. And, and, and, and, you know, if Wi Fi keeps improving, it will look like 5g. If 5g keeps changing itself to, to, to operate like Wi Fi in terms of ease of use and deployment, it's going to look like Wi Fi. So essentially, those two will cooperate, meaning compete and cooperate in the beginning. Hence, like CBRS will be there in a private network and Wi-Fi will still be there, Wi-Fi AX and BE in the future. But over time, and I don't think this is in five years, maybe 10 years down the road, they might just become one thing and in which case the ease of use the cost points will sort of become the same and the protocol itself will look the same I mean 5G and Wi-Fi they both use OFDM today so there's already some commonality and eventually it will become one so that's one part the next thing is. The next thing is, it's not enough to just

have that. You also have the ease of use in terms of the management aspect of this, right? How does this get managed? Wi-Fi is managed over the top, IT managed. 5G is managed literally by an operator. You've got core network, you've got all kinds of stuff. You've got SIM cards. And 5G is trying to be more and more private. So that's trying to get into the world of Wi-Fi. So that's going to become also an interesting thing where I think the next five years, you're going to see a lot of interesting technology that make 5G have the use model of Wi-Fi. Okay? And I also think that the price points are also going to come down. Maybe not equal to Wi-Fi, but throwing distance from Wi-Fi. Because that needs to happen. Not just because you're competing with Wi-Fi in some places.

It's really that enterprise applications expect that for this to proliferate. Well, all this is nice, but at the end, who's going to use it? What is it for, right? So I think of these as two kinds of applications. One is the enterprise networks themselves. So, and I largely piece them into two buckets in enterprise. One is this very, very, you know, mission critical applications, like the robotics of the war, you know, factories that are, you know, I don't think of like automobile manufacturing, things like a Tesla plant, where you have a robot arm welding things. Those robots can move untethered, no cables. So in that case, that connectivity is 5G.

And they need to be mission critical, meaning very low latency, high reliability, things like that. So those are almost like captive 5G. There's nothing else that can work there. In a long term, that will be a large market, but short term, it's very small. It's sort of almost a niche market. But then there is another set of applications in enterprise. I think that is where CBRS can go.

It's essentially augmenting the capacity of Wi-Fi. So if you think of Wi-Fi from a spectrum perspective, right, you know, you've got 5 gigahertz, 2.4 gigahertz, you've got 6 gigahertz, but none of it is really that clean. You've got, because all these, you know, components are the clients on the network and access points, they're sort of colliding with each other. So essentially the efficiency isn't that great.

However, 5G or cellular in general is a coordinated network and it's much more efficient. But what's interesting about this is that if you think about 5G as just a capacity expander, I'm just going to give you more clean spectrum. So hence, I'm going to add more bandwidth to you. As long as the price points are brought within reasonable range, then 5G just becomes an extender of bandwidth in a corporate

environment. And CBRS is that. For example, iPhone already enables CBRS, right? So you think of a world where you sort of roam into your corporation and essentially that network is either free or close to being free. So that's another interesting application space. I would say that. And this is within the five years that you discussed. And then you have something that's where the hyperscalers would get into. The hyperscalers would serve as this enterprise world too. But then the consumer world is another interesting area, which is beyond the phones. And this is my own vision or view of things. And we'll see, I guess, how this pans

out. But if you think about Netflix and if you think about content today, content creators are king, right? Content is king, as they say. So, but if I am a content creator, I don't necessarily have the wherewithal to go build a network or a platform like Netflix, right? It's just too expensive for us, you know, two guys in a garage to do. But wouldn't it be good if someone creates that for me? So essentially gives me a Netflix as an API or something like that. Well, and then essentially controls

the entire chain of this thing all the way to the phone. And who has that ability? Hyperscalers have that ability. For example, Google has that ability. Why not, right? They have data center, they have Android, so they can create this experience all the way to the end. And that's just a phone. What about cars? What about TV at home? And that end-to-end experience control, as well as having that and giving a content creator ability to spin up that service in, let's say, hours or days is another very, very exciting thing. And I think you'll see that. I think in the next five years, you'll see a company like Netflix

pop up. And it's a simple, you know, like two guys in a garage that create a company like that. And I think that is where I sort of see the future of this. Interesting. So a service provider as a service almost. Yeah. Very interesting.

So, well, thank you so much for that. And it was a great way to bring this conversation full circle. As I warned you before, at the end of each podcast, I like to surprise our guests with three fun questions about the future of AI. And as you can hear, Vinay is somebody who's got a lot of thoughts about the future of AI. So let's see if we can knock him out of his comfort zone a little bit and ask him about some neighboring answers. Now,

one of the things that you've mentioned and hinted at a little bit is autonomous driving. When will we see a full self-driving car that truly can drive anywhere at any time? Wow. Yeah. Okay. So this is one of those crystal ball questions. I am not one of those guys that's going to say in the next two years. I think the semi-autonomous or assist kind of technology is already here and that'll be what we will see for the next five years. I would say that in the manner that you mentioned, 100% autonomous, it can go anywhere. I think you're at least 10 years away from that. All right.

And that actually jibes with some of the other answers that we've gotten, including a couple of weeks ago when we were talking to somebody who specifically works in the autonomous vehicle field. So you seem to be on target there. Next, one of the things that 5G brings, of course, is ubiquitous connectivity and lots and lots of bandwidth. So when are we going to have a video-focused ML, basically a video version of Siri or Alexa that's watching us and responding to our gestures? First, that scares me. A video, some camera always watching everything I do. And it's always scary when Google, my phone just picks up when I say Google, for'm not, I think it's nearby, like a year, two years away, it's not that far. But I don't believe people will adopt that anytime soon.

And the reason for this is, I think just security concerns, it's just like having a person watching you, right? Physically watching you almost feels that way when there's video in front of you. And I think that maybe an aspect of it might be there in five years, but I am very skeptical that will ever actually be ubiquitous. All right. And finally, as someone who's interested in developing chips that would enable machine learning at the edge, how small could this edge get? Will we have machine learning powered household appliances? Will I have an ML toaster or doorbell? How about toys? How about disposable machine learning devices that serve just one purpose? I think that's absolutely possible, and it will happen sooner than you think. I see some of these ML chips literally operate on sub one watt, and some of these things can stay on a battery for a year, two years. Some

of them even can harness RF frequencies for power, meaning there's really no power source. Essentially, the RF signal itself provides some of that power. So, and I think that, you know, we're going to see a lot of it. And I think within the next two years, you're going to see very, very interesting applications, like you said, toys and vacuum cleaners and all of that is definitely right. It's here and now to some level, but in the next two to three years, you're going to see a lot more of it.

Excellent. Well, thank you so much for joining us, Vinay. It's been really interesting. And I hope that our audience has enjoyed this too, because of course, we've been hinting at 5G and talking about the impact of machine learning at the edge. And now we've got somebody who's really working in that space. So Vinay Ravori. That's one way. And you can also find us on Twitter, edgeq at edgeq underscore inc. And yeah, and our website, of course, edgeq.io is another way. There's contact us there that you can contact us with any questions you may have. We'd love to hear from you. Well, thank you very much. And we're going to put those links in the show notes for this episode. Fred, how about you? What have you been up to lately? Yeah, so it's still HPC and AI, but I specifically work more and more in data management. It's where people have billions of billions of files and next year they're not going to have less files,

but more files. So that's really a market where I try to play. All my adventures and anything I write is on LinkedIn. So if people are interested, they can find me on LinkedIn or on my website. But data management is really my pet work, if you want, right now.

Well, thank you so much. And as for me, I'm going to be pretty excited to be hosting the AI Field Day event this week. I believe that, Fred, you'll also be joining us there. And hopefully we'll see more of Vinay and EdgeQ in the future as well. So go to techfieldday.com to learn more about that. You can also go to gestaltit.com for show notes for more coverage of the AI and enterprise IT infrastructure topics.

Thank you so much for listening to this episode of Utilizing AI. If you enjoyed this discussion, please do click subscribe. We would love to have you join us every Tuesday here for Utilizing AI. Also, maybe head over to iTunes and give us a rating and a review

since that does really help. And please do share it on LinkedIn or Twitter or whatever social media networks you prefer. This broadcast is brought to you by gestaltit.com, your home for IT coverage from across the enterprise. For show notes and more episodes, go to utilizing-ai.com or find us on Twitter at utilizing underscore AI.

Thanks a lot and we'll see you next week.