Everyday AI Podcast – An AI and ChatGPT Podcast - EP 234: Driving the Future - NVIDIA's Vision for AI-powered Transportation

Episode Date: March 22, 2024

Awesome Stuff From Our Partner, NVIDIA -Register for the FREE virtual NVIDIA GTC Conference or buy tickets to the in-person event and fill out this form here: https://www.youreverydayai.com/nvidia-giveaway/You might think the auto industry hasn’t changed since the Model T.  There’s a lot going on under the hood, though. Especially with AI revving its engine now louder than ever. So what’s next in the auto industry with so much new AI tech behind the wheel? That’s exactly what we’ll be discussing with Danny Shapiro, NVIDIA’s Vice President of Automotive.  Newsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageJoin the discussion: Ask Jordan and Danny questions on AI and automotiveUpcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: [e-mail] with Jordan on LinkedInTimestamps:00:00 About Danny and his role at NVIDIA04:29 NVIDIA transforms into automotive-grade company with innovation.08:48 Constant need for computing power for safety.12:09 Testing autonomous trucks in Texas using simulation.15:00 Collaboration in design and virtual simulations.16:44 Future of automotive technology.23:51 Phone-like software updates bring joy to vehicles.Topics Covered in This Episode:1.  NVIDIA's Role in the Automotive Space2. Impact of Increased Computational Power in Vehicles3. Impact of Generative AI in Vehicles4. Automotive Simulations and Safety Measures5. Future of TransportationKeywords:AI, transportation, future, self-driving, flying cars, automotive, NVIDIA, GTC conference, safety, supercomputing, autonomous vehicles, automotive industry, generative AI, connected cars, software updates, Blackwell architecture, Omniverse, digital twins, simulation, language models, perception algorithms, manufacturing, end-to-end approach, design, engineering, virtual test drive, service and maintenance, highway autopilot, urban autopilot, valet parkingSend Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info) Start Here ▶️Not sure where to start when it comes to AI? Start with our Start Here Series. You can listen to the first drop -- Episode 691 -- or get free access to our Inner Cricle community and all episodes: StartHereSeries.com Also, here's a link to the entire series on a Spotify playlist. 

This is the Everyday AI Show, the Everyday Podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life. Meet Firefly AI Assistant, now live in Adobe Firefly, the all-in-one creative AI studio. Just describe what you want to create and the assistant handles the rest, orchestrating multi-step workflows across Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome. The assistant accelerates execution. What does the future of transportation look like, especially when AI is getting more and more involved?

Well, you know, are we going to have flying cars? Is everything going to be self-driving? I don't know. I don't have all the answers. But today, I have a very special guest who is going to be able to tell us what the future of AI-powered transportation looks like. All right, what's going on? Welcome, everyone. Thanks for joining us. My name is Jordan Wilson, and I'm the host of Everyday AI. We're a daily live stream podcast and free daily news that are helping everyday people like you and me, not just learn what's going on in the world of generative

AI, but how we can all leverage it to understand the world around us and to grow our companies and to grow our careers. So if you're joining us on the podcast, thank you. If you're joining us on the live stream, it's great to have you at this special time, not our normal time, but we're doing double duty here because we are reporting live from NVIDIA's GTC conference here in San Jose. I'm extremely excited. I'm looking over the show floor. The future of AI is happening literally right outside this window at NVIDIA GTC. And if you feel like you missed out, don't worry.

You can still sign up right now and attend the conference for free virtually. If you missed out on some great sessions like CEO Jensen's keynote, you can catch that. well as so many other great sessions and workshops from world and industry leaders in all different aspects of artificial intelligence. And if you do sign up, make sure to check out the link in our show notes. So you can enter as well to win an Nvidia, G-Force GPU, as well as DLI learning credits. Like y'all, like there's literally the experts, leading experts in the world, like our guest today, that you can go learn from. So make sure to check that. out. But I'm excited now to talk about the future of transportation and AI. So please help me. Welcome to the show.

Let's bring them on. There we go. We have Danny Shapiro, the vice president of automotive at NVIDIA. Danny, thank you so much for joining the Everyday AI show. It's great to be here. All right. Hey, can you? Yeah. Can you tell us a little bit about what you do in your role as vice president of automotive at NVIDIA. And so our team is really focused. on revolutionizing the transportation industry. And so we're working with hundreds of car makers, truck makers, robotaxie companies, shuttle companies, to make transportation safer and more reliable,

we're bringing supercomputing into the vehicles and adding extra safety and convenience features through sensors and supercomputing. And it's just, it's really a lot of fun what we get to do. We work with a lot of different companies, but the reality is the roads are dangerous and AI is going to make us all safer. Yeah. And Danny, can you maybe explain this a little bit?

Because, you know, I'm thinking I'm like, hey, my, you know, 18 year old Honda probably doesn't have any Nvidia parts in it, right? But how, you know, specifically is Nvidia because a lot of people still think of Nvidia as a GPU company. And it's obviously so much more than that. But how specifically does Nvidia work in the? automotive space, you know, is it the newer cars, self-driving? How exactly does that work? That's a great question. So we started working with the auto industry more than two decades ago.

And it really is kind of the types of things you're alluding to originally is graphics, right? People would design cars using Nvidia graphics. The engineers would use their computer-aided design tools running on a workstation with Nvidia inside. And we would do all kinds of other, you know, crash tests and things like that in simulation. So that was really the early days. Then we started bringing our graphics inside the car. And so back in the early 2000s, you know, this push of consumer electronics was taking off the iPhone, tablets, but the electronics inside vehicles were pretty antiquated by comparison. So we helped pioneer bringing the touchscreens to vehicles, digital instrument clusters, rear seat entertainment, the head-up displays. Again, anywhere there were

pixels, you look to Nvidia to bring those into the car. Now, one of the things we had to do was really transform ourselves as a company into an automotive grade company, meaning if you have a mobile phone and you leave it on the dashboard of your car on a hot summer day, it won't operate, right? You get that little alert. But inside of a vehicle, whether it's super cold temperatures in the winter, hot summer days, it has to operate. And so there's also a lot of other harsh conditions inside of a vehicle of shock and vibration.

and dust. So we basically create a version of our products, and in this case it's the Nvidia Drive product that is a system on a chip, we call it SOC, that integrates a CPU, a GPU, and a number of other processors on a single die to handle all the different types of computation that you would want to do in a car. It's a little different, of course, than a phone or a laptop or a data center. But that's the flexibility of to have this architecture that can scale across many different platforms. Can you tell us, you know, what is, what is new, what is happening? You know, I'm here at the conference, you know, in San Jose, and I do see, you know,

vehicles all over here with, with the Nvidia signs. I mean, what is new and what has just been announced, you know, with invidia here at GTC in the automotive space? Again, another great question. So, yeah, we have vehicles all around the show floor. there's a semi-truck parked in front of the convention center powered by Nvidia. We have Mercedes-Benz in the lobby, their brand new CLA concept car, which is the precursor to their next generation of vehicles that will all have the video processors inside. Lucid is there,

Volvo, Polestar. These are all brand new cars that will start shipping later this year. And so what we have then is essentially this supercomputer, an AI brain, inside the vehicle, it's connected to different sensors, cameras, front facing, rear side cameras, radar, LiDAR, which is a laser scanner in many cases. And so all of that information is helping the vehicle understand what's around it. It can sense the lane markings, of course, see signs, streetlights, other cars, bicyclists, pedestrians, fixed objects. And it takes all that data, brings that into the NVIDIA drive system,

which then has to figure out where it's safe to drive and ultimately control the vehicle steering, accelerating, and braking. And what we announced. Go ahead. No, no, no, no. Yeah, sorry. Sorry, go ahead. Yeah, talk about, Danny, what you were just saying about announcements.

Yeah, so at the keynote, we announced that BYD, which is the world's largest EV maker, is adopting our newest product in Video Drive Thor. So that's our next generation supercomputer. It's four times more powerful than the one we have in production right now, be available next year. And this is going to enable all kinds of new AI capabilities for sensing outside the vehicle as well as inside the vehicle. So knowing who's in the car, are they paying attention or not, detecting if there's anything left behind in the car. So

there's just a whole new development going on with respect to software that will be able to control many different aspects of the car. Yeah. And hey, for any of our lives, stream audience like Douglas here who's interested in an automotive. Go ahead. Ask a question. You know, we have one of the leaders in the automotive industry joining us live today. But, you know, one thing, Danny, that you brought up there is, you know, a lot of these new advancements with Nvidia's chips. So you said the Thor is, you know, giving you about four acts of power. You know, we even saw on on the kind of consumer side with the Blackwell and, you know, how much more powerful, you know, even this generation of chips. Invita was already the leader.

And then you come out with chips that are, you know, exponentially more powerful. So, I mean, what does that even mean for, you know, the future of transportation, right? Like, is there a certain limit, right, where your consumer vehicles can't even use the power of these chips? Or does this just mean that we are going to be seeing new capabilities in our cars that maybe we just had no clue we're possible? No, you're absolutely right. I think if you look at the history of computing, whether it's personal computers, super, computers, mobile devices, there's never enough compute, right? The software gets developed to take on new features, new capabilities, but it always then hits that upper limit, and then the next

processor comes out. And so that's where we continue to innovate, create higher performance systems, lower energy consuming systems as well, because that's really key, especially in an electric car. But I think the thing to realize here is more computation equals greater safety. So as you have more cameras or more sensors on the car, higher resolution sensors, more complex algorithms, there's just a lot of diversity and redundancy required to ensure safety. So the more compute you have, the safer the vehicle can be. And so, for example, we can have a system that's detecting pedestrians, and that's going to take up a certain amount of computation. But then we also have a neural net that's looking for what we call free space, the absence of objects. And so these are two different algorithms, but together they ensure it's like a double check, right?

We make sure there's nothing in front of us, and if we see things, we know, to avoid it. And that just requires double the computation there because they're running two algorithms. Then multiply that by lane marking detection, sign recognition, pedestrian detection. So there's dozens and dozens of these neural networks that are running in the car to make sure that we're able to drive safely. You know, Danny, speaking of that, you know, adding, you know, this new functionality on the road that maybe wasn't possible before, it seems like the industry has really been working, you know, toward the full self-driving and autonomous driving for a while. And, you know, different companies are finding different levels of success. Do you think now that maybe, you know, compute is maybe less of an issue for, for what is capable? Do you think, you that over the coming months and year or two, are we finally going to see the point now where, you know, full self-driving and autonomous vehicles are more of the norm versus the exception? I think so I think that it still is going to take some time to ramp up. But the industry really

underestimated the complexity here. If you go back 10 years, let's say we're in 2014, 2015, all the automakers were predicting by 2020. We're going to have full self-driving on the road. So we were driving towards that goal. It's hard. It's really, really hard. This is perhaps one of the most complex AI challenges in the world. And lives are at stake here. So safety is key.

So we want to make sure that we get it right. And so we're really not pushing for a specific timeline. We're pushing for a level of safety that far exceeds what humans can achieve. And so there's a number of tests going on in different regions with car companies, with trucking companies, with robo-taxie companies, and we're seeing great success. And it's a global thing. We're showing Wii rides Robobus on the show floor.

And they're operating autonomously in Beijing, in Shanghai, in Abu Dhabi. So there's different trials going on around the world. Aurora, their truck is driving autonomously in Texas on the highways. In the Bay Area, there's a ton of tests going on from a wide range of companies here. It's all powered by InVidio. But I think that the key thing is to ensure that we can handle the basics. It's those things that almost never happen that will cause the system to trip up. So one of the things that we're doing is using simulation to be able to train as well as test these vehicles.

So it kind of goes back to our roots a little bit in video game technology, but we'll create virtual worlds that are digital twins of the environment. So the roads, the buildings, other objects that are fixed, plus the other cars, trucks, pedestrians, bicyclists, motorcycles are all in the simulation. And so we simulate the actual signals that the cameras, the radar, the lidar would be generating the real car. And we run that through the computer that's in the data center, but it's a digital twin of what would be in the real world. And so the computer that's in the data center doesn't know it's not on the road. It thinks it's driving on the road and making driving decisions. And so that way we can create all kinds of hazardous scenarios without putting anyone in harm's way.

We can test what happens in different weather conditions, different road conditions, you know, having a virtual child run out in front of the car at night and do we detect that. So it gives us amazing ability to ensure the safety of vehicles and the software before we put it on the way. Adobe just introduced an entirely new way to create, bringing the power and precision of its creative suite into one conversational experience. Meet Firefly AI Assistant, now live in the Adobe Firefly app,

the All In One Creative AI Studio. Powered by Adobe's Creative Agent, Firefly AI Assistant lets you start with your vision, just describe what you want, and shape the outcome as it takes form with the Assistant. The Assistant orchestrates multi-step workflows, drawing on 60-plus pro-grade tools across Adobe Creative Cloud apps, including Photoshop, Illustrator, Premiere, Lightroom Express, and more to help bring

your ideas to life. You can also get started with creative skills, a growing library of pre-built workflows for common creative tasks, like batch editing photos, creating mood boards, portrait retouching, and creating social variations. Every step the assistant takes is visible, so you can refine, redirect, or take over at any time. You stay in the driver's seat as the creative director. Adobe Firefly AI assistant now in public beta. See it today at firefly.adobie.

And Danny, can we talk about that a little more? Because I think at least even for me personally, my eyes are on the omniverse here, right? And in the keynote, seeing so many great, you know, real world applications for safety, you know, climate change. You know, so many different applications that are positive. But can you just talk a little bit about? for our audience that maybe isn't super familiar with the Omniverse and also now with increased compute, you know, because now I'm guessing you can run way, way more, you know, simulation simultaneously because you have, you know, more computing power. So can you talk a little bit about

what you're even capable to do on the safety side with, with this increased functionality of the Omniverse? Yeah, so Omniverse is our platform for digital twins and simulation. So we can make it, I mean, we do make it available basically to every single department in, let's just take an automaker, for example, the designers can be working in a variety of different tools and bring their designs into Omniverse and be able to collaborate. So somebody can be working on the wheels in one software application, someone else could be working on the body designer and others. Maybe someone who's working on the headlights. And so Omniverse lets them bring it all together and collaborate and see what everything looks like. So this is in photorealistic rendering,

and using real-time rate tracing so you get all the reflections and it just it looks beautiful the materials right and so that then can be shared with the engineers who are trying to figure out well how do you build this thing and then so they can have a common design and they can then go back and forth and collaborate we can bring that into all kinds of structural engineering and bring it into a virtual wind tunnel simulation so a designer can create something and And you can see, well, what's the coefficient of drag? How fuel efficient will this design be? And how easy is the manufacturer?

We're working with a number of automakers in BMW, Mercedes-Benz, who are creating factories. The entire factory is going into Omniverse now. And so we can model the layout of the factory before the factory is even built. We can see how the robots are all going to interact. We do a simulation of the car being built. And so, yeah, you can see. just a still here showing this entire factory.

It's a massive amount of data. Every robot here now is following its programmed path and we'll operate. And so we can make sure that there are two robots as they swing, they don't collide. So we do that in simulation first, as opposed to then realizing on the factory floor that there's a problem and then you have to move things around

or move a wall or make the ceiling taller or whatever it is. So this is just an amazing tool. And the amazing additional part is that that factory runs in simulation so we can see the vehicles being built in the actual simulation. And so then in the real world, it makes it very easy to bring the factory out. Yeah. And if you're listening on the podcast, we just have a photo here that kind of shows the concept of, you know, what, you know, this can look like, you know, in the omniverse, you know, kind of being able to replicate what is actually happening, you know, in a real world manufacturing plant. and being able to replicate that digitally in the omniverse. So I'm wondering, you know, Danny, like what's in,

we don't have to go into specifics, but you know, in general, how much of the decision making behind the future of automotive because, you know, obviously, you know, and video's not making cars, but you're working with, you know, some of the world's largest car manufacturers, but how much of the actual development of future, you know, safety, future AI technology,

How much of that is actually, you know, originating from the omniverse versus maybe, you know, kind of real world testing? No, it's a really good question. And it's moving in the direction of everything will be simulated first. That's the principles we're using at Nvidia and making our chips we simulate it all. Our new building, you were just at our headquarters that opened recently. That whole building, it's, you know, it's a smart building. of a living entity. It's software to find. There's all kinds of systems monitoring. We simulated everything in that building before it was built. The angle of the glass and the position of the

sun. It's designed to let maximum light in, but still not create a lot of heat inside the building. And so we're able to model the physics of systems now in Omniverse and use that to build better products. And hey, a great question here from Douglas. Thanks for this question. So he's asking, how does the AI for a vehicle differ for on-premise versus a connected vehicle? If I drive a vehicle in the mountains, will the AI stop? I wasn't even thinking about that. But yeah, Danny, how does that work, right? Like, is most of all the AI being on? Yeah, go ahead. It's a super good question. So I think it's maybe a little bit of misconception too. So connected cars, everyone talks connected cars and I think that's great, right? You want to have a cellular connection because you get your nav updates, you can check the weather,

you can talk to Siri if you want to, you can stream Spotify. For autonomous operation, though, and for safety, nothing goes to the cloud. It's all, it has to all be on board. So you can't rely on, you know, your pedestrian detection system having to go to the cloud to decide if that's a pedestrian or not, right? It takes too long, and the connection is too unreliable. So all the sensor data goes straight into the supercomputer, our Nvidia Drive system on board, and those decisions are made in a fraction of a second. We're looking at video, which is 30 frames in a second. So we have one 30 to analyze all the pixels in those frames of video and decide what's a pedestrian or what's a car, what's a lane. The connectivity piece then is software updates. There are cars parked overnight,

and you could get an update just like your phone, which would add new features and new capabilities. And then if you're trying to use a system and say, hey, I'm interested in finding a good Japanese restaurant, then it'll go to the cloud and get that data and come back. Yeah. It's something that I don't really think about a lot, Danny, but it sounds like, you know, for the most part,

especially if you have a newer connected car, you know, powered by Nvidia, that your car is probably smarter than your, you know, maybe your computer, if you're using a basic laptop or a standard cell phone, it sounds like, you know, whatever is the AI powering, your car might be just as powerful or maybe more powerful. Is that the case? Oh, absolutely. Again, it's tuned for specific applications. You know, it has the software is designed for, again, sensor processing in this case. But our new drive-thore, it's using our

Blackwell architecture. So those really powerful GPUs that are going into the data center, that same architecture is leveraged and goes into Drive Thor. And this will be a 1,000 tops processors. So that's 1,000 times a trillion operations per second. So it's a massive amount of compute for artificial intelligence and sensor processing. Yeah. And one thing that I'm, you know, thinking about now, Danny, because I've even heard it from, you know, certain people I've been talking out to on this exhibit floor and people are saying, you know, hey, now with Blackwell and, you know, all this additional compute, you know, some people are, are rethinking, you know, what's possible with, with their product or service. You know, I'm even curious for you. You've

obviously, you know, known about what's going on, you know, at InVidia. So you've had time to think and to process what all this additional compute that's been announced here at GDC actually means. But what do you think that it means outside of, you know, what's what's directly in front of you? You know, people are always saying, oh, are we going to have, you know, flying cars? Is every single vehicle going to be, you know, fully self-driving? But what does the future beyond tomorrow look like with all of these new AI powered announcements? So obviously the big trend and it's kind of taking over all industries is generative AI now, the ability to have artificial intelligence, not just do pattern recognition, which was kind of

essentially the past way, being able to identify things, but now take content and create new versions of it. So chat GPT, you type in text, and text comes out. We're going to see a lot of large language models running in the car. So you'll be able to speak naturally to your car, and your car will answer intelligently, and that will happen locally on the car. So we'll get trained on everything with your car, the brand, you could ask it any kind of question. It can also maybe tell you about things that it's sensing in the car. Maybe there's a new rattle or something. And so it can understand what's going on and maybe alert you or go ahead and just book

a service appointment. And if it is self-driving while you're sleeping, it would drive the car back to the garage and have that repair made and then it'll come back to your house in the morning. We do believe that everything ultimately will be automated and autonomous. And it's a process that's going to take a long time. There's lots of cars in the world and a lot of them don't get refreshed for many, many, many, many years. But we're working with a lot of automakers to bring this technology to market, bring it to consumers, and make it affordable. So, Danny, in your role at Nvidia, you know, you have your hands on really the future of transportation.

You're right, like, Nvidia is working with the largest, you know, hardware and other software manufacturers in the transportation. industry. So I'm curious, what are you most personally excited about in the future of transportation? I think you look at your phone and you get periodic software updates and all of a sudden the interface is better. And there's some new apps. And it's like you get a sense of joy from getting things getting better. And I think that's really what's going to happen with your car. It's not the old model where it was the best it's ever going to be the day you drive it off the lot. But moving forward, you have a software-defined vehicle. You have a Vivida drive inside, a very powerful computer,

and it will just get better and better with each software update. So your next car you may buy, and it doesn't have this fully autonomous capability, but during its life, as the software is enhanced, as regulation changes, and autonomy becomes the norm, your car can get a software update if it has the sensors and the compute on board to handle it. So maybe first it will be a highway autopilot, then maybe an urban autopilot. Maybe there's valet parking. You can go to the restaurant, go to the movies. You get out of the curve and the car's just going to go park itself.

You know, we've covered just about, you know, so many different things, you know, both from recent advancements that Nvidia has been making, you know, the future of full driving, autonomous vehicles. But, you know, Danny, as we wrap it up here, what's the one take home, you know, piece of advice or message that you really want people to know, specifically about how NVIDIA is powering the future of transportation with AI. Yeah, we call it an end-to-end approach. It really starts from that design of the vehicles. We're helping automakers and truckmakers create more innovative designs,

more efficient designs, and safety designs through the engineering and manufacturing, the testing, the simulation, and even retail, being able to take all that information through to the consumer and help them make purchasing decisions and streaming a car configurator in their web browser on their mobile device and even take a virtual test drive. So the buying experience and then even the service and maintenance,

Omniverse is going to play a large role there too. And so we're just really excited by the technology that we're bringing to the market and seeing that now our customers and partners build on it and really create cool products in the future. I think that many people are potentially, afraid of autonomous driving. There's a lot of sensationalized stories in the press, but the reality is that there's a lot of hazards and dangers on our roads and a lot of accidents, injuries, and

fatalities every day. And as a society, we've come to accept it. But I believe this technology is going to dramatically reduce those figures and make the world in our roads much safer. Danny, you've helped us so much better understand what's happening in the transportation, and automotive industry and even walked us through everything new here, all the new announcements with AI and at NVIDIA GTC. Thank you so much for your time and joining the Everyday AI show. We very much appreciate it. It's a pleasure.

Good. Thank you speaking with you guys. All right. And hey, there was a lot there. Don't worry if you missed it. If you're out, you know, on your walk and maybe your reception cut off for a second. We recap every single episode on our website. So make sure to go to Your EverydayAI.com.

We're still going to have more information in there about what's happening here at Nvidia GTC. There's so much we couldn't get to. So we're going to have more in each and every newsletter throughout the rest of this week and probably early next week. So if you haven't already, go to Your EverydayAI. Sign it for that free daily newsletter. And we'll see you back tomorrow and every day for more Everyday AI. Thanks, y'all.

Meet Firefly AI Assistant. Now live in Adobe Firefly, the Allman One Creative AI Studio. Just describe what you want to create in your own work. and the assistant handles the rest, orchestrating multi-step workflows across Adobe Creative Cloud apps, including Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome while the assistant accelerates execution. Stand control with the ability to step in and refine at any time.

See it today at firefly.adobie.com. And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going. For a little more AI magic, visit your everyday AI.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.