SemiWiki.com - Podcast EP298: How Hailo is Bringing Generative AI to the Edge with Avi Baum

Episode Date: July 18, 2025

Dan is joined by Avi Baum, Chief Technology Officer and Co-Founder of Hailo, an AI-focused chipmaker that develops specialized AI processors for enabling data-center-class performance on edge devices. Avi has over 17 years of experience in system engineering, signal processing, algorithms, and telecommunications while… Read More

Hello, my name is Daniel Nennie, founder of SemiWiki, the open forum for semiconductor professionals. Welcome to the Semiconductor Insiders podcast series. My guest today is Avi Baum, Chief Technology Officer and co-founder of Halo, an AI-focused chipmaker that develops specialized AI processors for enabling data center class performance on edge devices. Avi has over 17 years of experience in system engineering, signal processing, algorithms, and telecommunications

while focusing on wireless communication technologies for the past 10 years. Welcome to the podcast, Avi. Hi, Daniel. Thank you for having me. Great to be here. First, I'd like to ask, how did you get your start in semiconductors? Do you have a story you can share? Sure. Actually, my beginning of our introduction into the semiconductor industry was in the happy days of mobile communication when it was very common to have those big

chunky and chubby phones. And at that time, I was seeking for marching on my career in engineering and it was very interesting for me to see what's coming next. And part of this research or search after a new career opportunity I was looking for something that's intriguing enough and I found Texas Instruments back at the time which was making its way into the telecommunication industry with the big names back at the time like Nokia and it was very interesting for me to look at new mobile technologies that would change the world and make everything in our pocket connected everywhere and anywhere. Interesting. Yeah, TI is still a great

company. So let me ask you, how can organizations that use AI regularly incorporate AI at the edge into their strategies? I mean, how does your company support this? Yes, so in many ways this relates to my story because I think this is yet another wave of enabling technology becoming so common that one can start considering how it becomes so available in any end product and becoming something that people can harness to not replace the data center in the cloud but in contrast to enhance it and that's what happened back at the time with mobile technology and the ability to

have connectivity everywhere and basically make a continuum between edge and cloud. That's the way I see today AI at the edge becoming, getting out of the data center as the sole source of wisdom and putting it in the hands of everybody, anytime and everywhere. And in that sense, this is where Halo is focused. Halo is basically delivering devices that serve as an infrastructure to empower devices with AI technology that can run at the edge

the same way that they run in the cloud and give a continuous experience in the sense that it's affordable and also capable of delivering the experience with the constraints of edge devices in terms of power, area, and of course cost. And what role do you see generative AI at the edge playing in the broader evolution of the future of AI? So in a way, getting to the center of the stage, generative AI has become a sort of a revolution in revolution with the AI domain and it's basically making the AI not

business to business kind of opportunity but rather a consumer facing support or supporting solution that allows use cases that are no longer something that goes on behind the scenes, but actually you can directly interact with them. I guess many of you are using, as we speak, chat GPT and the likes, and this is basically something that is possible now to think how they can be part of any edge device and not just something that you leverage from a data center. And that makes this technology in a way, the killer app or a potential of becoming the killer app of AI in a consumer facing way and therefore at the edge.

Right. So where are edge AI and generative AI relevant in security applications today? You know, security is a huge concern. So what's next? Yes, so security can translate into in multiple ways. It can translate into how you leverage the ability to understand what your security devices are seeing and the other side of it is privacy. In the first aspect, the ability to have the human-like wisdom close to the source of the data makes it possible now to analyze what you see in a way that is not predefined. So if up till now you had to train your so to speak intelligent camera or intelligent provisions for securing your premise in a way that you need to upfront dial in

what you're looking for with the new coming Gen. AI applications, you can all of the sudden train or request from your endpoints to extract information, even information that has not been seen, like almost like a human brain. That's the first part and that finds its way to applications that allow to do much smarter and much more complex search and indexing through videos that are serving the security industry and of course to trigger events and anomalies as needed. The other part which I mentioned is the privacy aspect which is yet another catalyst for putting smart or advanced AI at the edge device

close to the source of data such that you can avoid delivering unnecessary and private or intimate information out of the premise of the source of the data and where it is being acquired. Right. So what are the emerging use cases for edge AI and generative AI, for example, automotive applications? So the automotive industry is a very vibrant in recent years. Many of us are using cars that have growing capabilities in terms of being closer to or having more autonomy in their capabilities.

And this is yet another industry which can enjoy a lot of understanding of the surroundings. Things are moving fast. It is mostly mission critical in the auto domain, driver assisting systems and driver supporting systems. And in that sense, AI in the car is becoming more demanding in order to assist better or to serve better the driver. Again, it is a twofold kind of support. One is for the driving function itself, basically, to be kind of your co-pilot, pun intended, in the sense of driving along with you and understanding the surroundings, alerting of things that are otherwise could be missed by the driver, and of course for some comfort features like guiding you as a companion of what can be seen throughout your route, how you can improve your route, and things that are more complementary

to the driving function itself. And can you share upcoming use cases for Edge AI and Edge generative AI in smart home applications? Sure. The smart home has become smarter in recent years, thanks to, again, the former technologies that I've mentioned like wireless connectivity. It's not uncommon anymore to have tens of connected devices from your lights, speakers, media center and doors and parking lot. Managing all this in a more intuitive way

this in a more intuitive way and in a more human-facing way is something that is now not a fantasy anymore with the ability of introducing smart endpoints that are now capable of native exchange with the owner and the members of the premise. That allows to basically talk natively to your local router and as mentioned earlier this is complementary of course to what the cloud can entertain so the cloud can always be complementary to what's done at the home but due to the same reasons that I mentioned earlier of privacy and security, one would definitely like that the more intimate aspects of these use cases be kept within your private home, and that's one aspect that allows for yet another domain for a generative AI to be

entertained in the smart home domain. Yeah, I agree completely. What about telecom companies? You know, how can they incorporate Edge AI and generative AI? So here, this is a more advanced trending adoption that is yet to be seen where the killer apps are basically residing. But one part of it is what I just mentioned about the smart home. So there are two contenders to get into your home. One is your personal devices that I mentioned like the white goods and the media center and the likes, but the other is basically the edge of the networks.

That's where the telecom players or the telecom companies are playing around. Your edge router, your gateway, that you get service from the mainstream, but they eventually find their way into the home. The other aspect is to serve you better with quality of service that is not accessible to the telecom company from remote because there are some aspects that are very specific to the deployment scenario that those boxes are seeing in the environment that is at the edge of

the network. For those aspects, telecom companies are at a very unique position. On one end, they have a huge data center, so to speak, where they can collect a lot of information about the end users, but to serve you better, they need to reside at the edge of the networks. And that's something that's complementing what they can serve today from their data centers and call centers in terms of improved services and more intuitive troubleshooting to lower the churn rate of the networks and their subscribers. You know, we could talk all day about this, but is there any application that is not going to be touched by AI? I mean, I see huge amounts of applications and how this is going to change the entire world? Definitely. I think the future is yet to be seen and I'm sure that there will be

new coming technologies that will be also enabling technologies in other domain but the way I see today it's yet another ingredient in the toolbox of tomorrow basic technology concepts. And like any enabling technology, some of the use cases that I mentioned will prevail and will be here to stay. And some of them will probably be left outside of actual plausibility and adoption. And this is yet to be seen. I think many of them will be adopted and become something that will be intuitive and be used in our day-to-day life.

Yeah, I agree completely. I've been covering AI and working on AI projects for the last three years and I've just seen amazing progress. And most of this is AI inside of the semiconductor design world. And it's just so incredible, and the opportunities seem unlimited. Indeed. Well, thank you for your time, Avi. It's a pleasure to meet you.

And please come back and keep us posted on your progress. Thank you very much time, Avi. It's a pleasure to meet you. Please come back and keep us posted on your progress. Thank you very much for having me. That concludes our podcast. Thank you all for listening and have a great day.