Everyday AI Podcast – An AI and ChatGPT Podcast - EP 184: On-Device AI: What it is and do we need it? What no one's talking about.

Starting point is 00:00:00 This is the Everyday AI Show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life. Meet Firefly AI Assistant, now live in Adobe Firefly, the all-in-one creative AI studio. Just describe what you want to create and the assistant handles the rest, orchestrating multi-step workflows across Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome. The assistant accelerates execution. Do we need AI on every single device we own?

Starting point is 00:00:51 Do we need an AI in our fridge or on our laptop or on our smartphone? It's something especially after the, all of the recent CES announcements where it looks like AI is coming to every single device. And I'm not just talking about software. I'm talking about the hardware. All right, so we're going to be talking about that and a lot more today on Everyday AI. Thanks for joining us. My name is Jordan Wilson, and Everyday AI is for you. I mean, that's why we're building it.

Starting point is 00:01:24 We're building it for you. So you can better understand what's going on in the world of generative AI and not just keep up with what's going on because that's hard. That's difficult. I try it every day and I feel like I'm always falling behind. But not just how we can all keep up, but how we can actually get ahead. right? Because if you're listening to this show, first of all, congratulations. I'm not saying that because you get to listen to me, right? That's not what this is about. I'm saying because you are still

Starting point is 00:01:51 an early adopter, right? And as AI becomes more and more intertwined in our lives, you are listening, so you are keeping up. You are getting ahead. All right. So if you're joining us on the podcast, thank you as always. Make sure to check your show notes. I leave my email in there. Drop me a note. Let me know what you want to see more or less of on the show. If you're joining us on the the live stream. Thank you as always. Let me know what your questions are about on-device AI. This is something to tell you the truth. I'm always learning about it, but I've noticed this trend. So thanks for our, you know, live stream audience. Hey, it's good to see Mike Forgey back in the house. What's going on on, Mike? Thanks for joining us. Brian tuning in from Minnesota. Hey, is it snowing

Starting point is 00:02:31 where everyone else is? Maybe not in Dallas where, you know, Josh is. Maybe so. But man, me and, me and Cecilia from Chicago here, we're actually getting hit with snow for like the first time and forever. Chrissy, thank you for joining us. So our live stream audience, let me know. What are your questions about on-device AI? I know it's kind of confusing. I'm even confused about it sometimes.

Starting point is 00:02:51 So we're going to dive into that today. And a lot more. But before we do, if you haven't already, why the heck not? Please go to your everyday AI.com. Yes, sign up for the daily newsletter. I was told yesterday was one of our best newsletters ever. So make sure to go read that, check it out. It was all about how to build and monetize on the GPT store.

Starting point is 00:03:11 But on our website, we have more than 180 back episodes of podcasts, more than 180 newsletters where we take deep dives into generative AI. We have different learning tracks on our website. So you can go and click, maybe you're in sales. You can see every single sales podcast we've ever had. So it is literally a free generative AI university. So you've got to go check it out. But let's talk about what's going on in the world of AI news for today.

Starting point is 00:03:39 There's a lot. Here's a good one. Any Swifties out there, be careful because there's a new Taylor Swift that's confusing fans. It's obviously an AI version. So an unauthorized ad for a cookware company featuring an AI generated Taylor Swift, offering a giveaway was recently removed from social media. So if you saw Taylor Swift giving away free cookware, not really a thing. So you're going to have to shake that one off.

Starting point is 00:04:02 This is not the first instance. Yeah, that was a joke. That wasn't AI written. That was just me ad living. But this isn't the first instance of AI generated ads using celebrity images and voices without permission with cases. You've seen these involving Tom Hanks, Scarlett, Johansson, etc. Also, this has led to propose legislation, at least here in the U.S., called the No Fakes Act aimed at protecting individuals' rights to control their image and voice. Y'all, we're going to be talking about deepfakes a lot. If you don't like it, sorry. because at least here in the U.S., we have the election cycle coming up.

Starting point is 00:04:37 I've been saying it since literally the very first episode of the show nine months ago, that deepfakes are going to ruin us in the 2024 election, at least here in the U.S. All right, next piece of AI news. The enterprise version of ChatGBTGPT is picking up some steam. So OpenAI just kind of announced or information was released about some usage for their ChatGBT GBT Enterprise version. So they said that they have 260 pay in companies with over a hundred and 150,000 registered users.

Starting point is 00:05:08 So this is big companies only, right? Because ChatGPT just released, you know, not even 48 hours, not even 48 hours ago, the ChatGPT teams plan, which is really geared for companies to 149 people deep in terms of size. And that one is, you know, pretty affordable by $30 a month. I haven't even seen the enterprise, you know, pricing per customer. I've seen so many different things. But, you know, Open AI coming out and saying they have 260, I think that's a pretty big deal. I think that means that, you know, more companies are at least experimenting with this enterprise version of chat GPT.

Starting point is 00:05:45 All right. The last piece of news for today is one of the biggest names in generative AI has teamed up with a tech giant. So Microsoft and Typeface have announced an integration that will see the startup's AI technology embedding into Microsoft's Dynamics 365 customer insights platform to simply. the campaign building process for marketers. So yeah, if you are an enterprise marketer, this is huge news for you. So this integration will allow marketers to create content aligned with the company's branding and style from a blanket canvas thanks to this tightface integration. So if you haven't heard of typeface, it is an enterprise grade gen AI product that helps

Starting point is 00:06:21 large companies use generative AI to create more personalized content for work. Woo, a lot going on in the AI news today. Hey, Josh is excited. Josh is going to see Taylor Swift soon. All right. Hey, thank you. Tara laughed at my Jordanism there. Hey, Val, thanks for joining the show.

Starting point is 00:06:42 Hey, it's good. Good to see you back, Val. All right, but I want to talk now about this trend, if you will. And I don't even know if it's a trend per se, or is this just the direction that generative AI is heading. I'd love to get your questions and your thoughts, your comments, your insights from our live audience. And also, if you're listening on the podcast, let me know as well. So let's talk just a little bit today. And yes, sorry, you're just stuck with me today.

Starting point is 00:07:08 I don't have a brilliant guest as I most days normally do here on the everyday AI show. So you're going to have to listen to my rambling. So here's what on-device AI is. And I want to talk about if we actually need it. All right. So I've noticed a huge trend over the last couple of months of trying to bring. bring all kinds of generative AI to a device. All right.

Starting point is 00:07:34 So let me first kind of hit rewind and say what that means, right? So right now, all of the generative AI platforms that most of us use on a day-to-day basis. So whether you're talking about chat GPT or you're talking about Bard or mid-Journey or runway or whatever it is, you know, whatever generative AI, you know, that you use, it's essentially in the cloud, right? Like we go on a company's website or we go on a Discord. server and we access, you know, generative AI that way. So the big picture, right, is, and this is where the trend is heading, is bringing these models, specifically large language models or small, large language models, whatever you want to call them, is bringing them

Starting point is 00:08:15 to an actual device, right, and not needing to connect to an external service or even connect to the internet to use AI, to use generative AI, to use large language models. So we've seen this a lot of recent announcements. And, hey, FYI, I'll tell you now, if you're hoping to get into the technical aspect of this, you know, and talk about like mistral, LLM and, you know, LMA, too, and all these, you know, small models, you know, that people are hacking and forking and putting on devices, that's not this. We're talking about the bigger picture here, right, this trend. And we're seeing it because you've seen, you know, even in the last week or two, big announcements, you know, Microsoft, Nvidia, you know, big announcements that they just had at CES, you know, the consumer

Starting point is 00:08:59 We're electronic show talking about releasing AI chips. Right. So this is where everything is heading. Even another one, Qualcomm and Meta, you know, talked about a huge partnership. And this should be releasing soon, I believe. They're bringing Lama 2. So Lama 2 is, you know, meta's large language model, a small yet very powerful model. So they're bringing that to PC chips to the Qualcomm chips.

Starting point is 00:09:25 You know, so that means that on your local computers, you know, without needing to connect to the internet, you will have Lama 2. You will have a large language model running locally on your machine. So what does this mean? Right. And this is also actually, before I dive into that rhetorical question, we have to talk about Apple too, right? Because if I'm being honest, I think the short-term success, right, so if we're looking at short-term as, you know, six to 18 months, I think the short-term success in markets, adoption rate really goes to what is Apple going to do, right? Apple's never first to the party. We already see the people who are first of the party here. You know, it's, it's companies like

Starting point is 00:10:10 Microsoft, you know, bringing AI to PCs. It's companies like, you know, Mattup, bringing their model. It's, it's companies like Qualcomm and Vida, right? They're first to the party. But it's usually what in, in how Apple does it to see if this is going to be an industry trend, right? example, Apple wasn't the first media player, right? But the iPod made that a trend. Apple wasn't the first, or the iPhone wasn't the first touchscreen phone, but that's what made it the norm. Right. So I think we can't overlook, even though we haven't heard a lot.

Starting point is 00:10:45 We've heard rumors, right? We've heard rumors that Apple is reportedly spending more than a million dollars a day building their next generative AI product, whatever that is, whether it's software-based, which is one thing, or whether it's hardware-based, even though technically, you know, it's if it's on the software, it's technically hardware-based, right? Because then it can be offline. So we really don't know, I think, or won't truly know the future of on-device AI until we see what is this next Apple announcement that they've been working on.

Starting point is 00:11:19 Is it going to be cloud-based? Are we going to need to be connected to the Internet? Or is whatever Apple is working on in the, you know, large-language, Model gen AI space, is it going to live offline on our phone? Right? Is it going to live offline on our computer? Personally, I would love that, right? Because what offline or on-device AI does is it brings this concept of personal AI to life.

Starting point is 00:11:45 Whether you like it or hate it, right? Offline AI is the marriage of your personal data and productivity, whatever productivity looks to you, whether it's being more social personally with your phone or whether it's accomplishing more on your PC. But that is essentially what on-device AI is really aimed toward. But it's going to be interesting to see what Apple actually does, you know, is whatever they're working on, this, you know, rumored Ajax is one of the names or some people call it Apple GPT. Is this going to be cloud-based? Or is it going to live offline on our phones? I would get to. the latter, but I have no clue. You know, all there is is speculations and report out there.

Starting point is 00:12:33 Yeah, like what Val is saying here. And again, love hearing from the live stream audience. Let me know what other questions or thoughts you have. You know, Val says not being dependent on the internet would be a game changer. Absolutely. You know, I think that is the big push to bring on device AI. And I actually have some thoughts because we're going to go over the pros and the cons of of this here in a minute. But I have some thoughts, Val, on what that actually means. And I love what Mabrid is saying here is, do you see AI coming to Alexa or Google Home to improve the responses further? Mabrit says hers is so sassy. Yeah, absolutely, right?

Starting point is 00:13:07 So this has been reported on for many months, you know, that Alexa, Google, and Apple are working on large language model integrations with their smart assistance. And we've already started to see that slowly roll out in different form factors. All right, but let's get back and kind of talk again about Apple is impacting this trend, right? Because I think that as we jump into, you know, not talking examples, but talking about how this is going to play out, I think it's going to be largely dictated by what we see from Apple and when, right? That's, I think, that's what brings so much new technology to the masses. You know, Apple's usually never first, you know, you always hear, you know, Android phone users, you know, whenever Apple releases something big and it becomes popularized,

Starting point is 00:13:55 it's, you know, Android users are like, oh, yeah, we've had that for, you know, two years or whatever. But it's, it's, you have to be keeping your eyes on what Apple is doing in this space. All right. So now we understand there's a trend. All the big companies are working on on device AI, making, you know, generative AI or large language models live, you know, on an actual chip, to live in an offline, to bring this personal AI to all of us, to, to, to, to, to, to, you know, to marry our data in a fast way, because that's ultimately what it's all about, not having to connect to an internet and waiting.

Starting point is 00:14:28 It brings that latency or the wait time, bringing personal AI to your device. It brings it from maybe a second or 15 seconds, depending on how complex the task is that you're working in a cloud base to maybe instant. Maybe we're talking milliseconds, right? And that is the big picture here. Right? Because not everyone needs a large language model like GPT4 with its 1.8 trillion parameters, right?

Starting point is 00:14:59 You can't really get the full version of that right now on a device. I know people are, you know, trying to fork it and to, you know, make versions of GPT4 run locally. And there's some success on that. But that's not even what I'm talking about because I don't even think for the most part, the average every person needs a model that big on their hardware. So I do think, right, even though I don't know, I do think this is where it's heading, even with Apple. I think we're going to see an on-device, fully capable, generative AI that works offline. So why? Why do we need this?

Starting point is 00:15:35 Do we need this? And what are the downsides? All right? So let's talk about the why. You know, we already talked on some, uh, hit on some of these things, but the offline access and the speed are huge. But the personal AI, that's what this is ultimately about making every aspect of our devices smarter and faster. So if you think of as an example, think of in your mind maybe your best use case of chat

Starting point is 00:16:04 GPT. Maybe it's very specific. But you found something that you can use a large language model in one very specific use case. And it's extremely efficient at it. It saves you time. You're extremely pleased with the outcome. Right.

Starting point is 00:16:16 So think of that feeling that you have. right. But think of applying that to your everyday interactions with your device, with your phone, with your computers, right? Because that's, I think, the bigger picture, right? Right now, our relationship, I think, with large language models or generative AI for the most part is a when we need it kind of thing, right? Like we have to go and seek that out. You don't just, you know, you can't just reap the benefits right now from generative AI at all times. But think of this. The average human checks their phone hundreds of times a day. So think if you had on-device AI a large language model on that phone, think of how much more productive maybe your life could be or your work could be or your

Starting point is 00:17:12 responses could be or how much time you could save, right? When you can have a large language model, being able to work and understand what's going on on every single program you're using. So whether you're on your phone and you're reading a message or you're typing an email out or you're scrolling social media or you're reading an ebook, right? To be able to have a large language model understand all that and work seamlessly between those apps, that's the future, right? And that is how I think large language models and generative AI and on-device AI eventually becomes seamlessly integrated into our lives to the point where it helps us so much that

Starting point is 00:17:47 we don't even know it, then it's also maybe, is it hard to live without it? Right? So let's quickly go over some of the downsides and the upsides, you know? I promise y'all, this isn't going to be one of those episodes where I accidentally talk and rant for 45 minutes. We're going to keep this to a somewhat succinct episode. All right. Yeah.

Starting point is 00:18:12 Okay. I like this. I like this comment from Tara here. You know, bicentennial man meets. Big Hero 6, having on-device AI. Yeah, I love that, right? Having your helpful assistant that's with you everywhere you go. And here's the other thing. And this isn't what I'm talking about, but I'm really just talking, at least in the context of this conversation, about our devices, our current devices, right? I'm not even necessarily talking about new devices, you know,

Starting point is 00:18:42 the humane pin, you know, the pin that you can wear around on you and it's, you know, always seeing and hearing and understanding what's going on and you can talk to it, right? Or the new rabbit, right? I think it's the rabbit R1. That device was just announced at CES. It's essentially a new piece of hardware. And, you know, a lot of people are like, oh, why can't this just be an app on your phone? But the rabbit R1 is a completely new piece of hardware.

Starting point is 00:19:06 We talked about it hours after it was released. But it's a new piece of hardware. Same thing. It has a camera. It's kind of like an assistant. You know, it has a large action model built into it. right? So we'll see if that whole terminology picks up steam and sticks around. Then you have other devices, right? These hardware devices, you know, the meta rayban glasses, right? That's not what we're

Starting point is 00:19:28 talking about in this show. I'm literally just talking about the current existing devices that we all use on a day-to-day basis, mainly our cell phones and our laptops, right? Or your desktop computer. So now let's let's talk about some of the downsides and the upsides. Because there's some things in here that I don't think many people are talking about or thinking of. You know, I'm sure maybe the extremely smart, brilliant people that are building this on a day-to-day, they're thinking of these things. But, hey, everyday AI is for everyday people. So I don't know if we're thinking about some of these downsides and upsides when it comes to on-device AI.

Starting point is 00:20:07 All right. Let's first talk. Yeah, and hey, side note, I agree with Brian here saying the rabbit R1 looks cool. but the last thing you want to do is carry around another device. I agree with that. But I mean, who knows? Maybe it'll be so awesome. You won't care.

Starting point is 00:20:24 So yeah, we're just talking about the devices we already have on us on on device AI. So first, a downside. A downside of on device AI. And for for this argument, let's just stick to phones. Let's stick to phones here. Okay, because there's two different things. Downside of on device AI. Some of these things are obvious.

Starting point is 00:20:47 Some of them aren't. But local AI, offline AI, on device AI, is so resource-heavy. It's so resource-heavy, right? Think of the compute and the tens of thousands of GPUs needed to power something like chat GPT, right? and think of just the sheer amount of power and resources and compute that is now going to have to be on our personal devices, right? Obviously, that's, you know, the chip companies have no miss for many years and they're

Starting point is 00:21:27 coming out with smaller, more powerful, just shockingly, like shocking advancements in chip technology and these new now AI power chips. But still, it's resource heavy. I can only imagine that we're going to see, you know, some of these first smartphones that have, you know, this offline large language model or this AI built in. I'm going to, I think some of them are going to be problematic, you know, how much testing can they actually do with these before they hit, you know, before they get these to market? There's always going to be a rush. You know, Google's already, you know, announced that some of their next, you know, Samsung phones with the Google Gemini Nano. model, you know, they're going to have this, right? So it's a rush. It's a rush to market. Everyone

Starting point is 00:22:15 wants to be first. They want to sell more. They want to, you know, be the first one with an offline, a large language model. You know, we're seeing that already. It's resource heavy. What that mean, what does that mean for your device, your personal devices? Well, it can kill your battery life. It can potentially kill your performance, right? Think of like when you're, you know, maybe on your laptop, you know, I was just doing this last night. I was working super late last night. I was working super late last night, you know, putting together some materials following up with all the amazing people out there who took our free PPP Prime Prompt Polish course. Side note, if you want to access, just send me PPP. But, you know, my computer was hot. It gets hot, right? I have a fairly expensive

Starting point is 00:22:58 MacBook Pro. And, you know, when you're using it, especially when it's not plugged in, I think it's hot. I mean, luckily, in full disclosure, it was also kind of keeping me warm because It's kind of cold in Chicago right now. But performance when it comes to on-device AI is huge. All right. Another downside is, is it going to be useful? Right? Because with on-device AI, you have to find the balance between it being small enough,

Starting point is 00:23:25 yet powerful enough. Right? And I think that's where, especially some of these early iterations, and again, I think that's why Apple usually waits, I think some of these early iterations, it's either some of them are either going to be too powerful and too resource sucking or some of them are just going to be not powerful enough and not useful enough. And then it's like, all right, there's millions of people, maybe unnecessarily, you know, either you had to pay extra, right? Because that's the other thing. On device, AI phones are going to be more expensive, right? They're going to cost us more.

Starting point is 00:24:02 And I'm guessing it's going to get to the point where we don't necessarily have an option. right? Like if you want a new iPhone, you can't opt out of the, you know, seven cameras that it has on the back, right? Or you can maybe get one with five. You know, it's not actually seven. I think it's like four, right? But you might not be able to opt out of these things when it, when this hardware comes to your device, this AI hardware. So you're probably going to be paying more. It's going to be more resource heavy. Battery life, I'm sure is going to suffer. It's actually probably going to go down to where things are now because we might be at, you know, kind of peak, you know, battery life. performance level. All right. And then there's one other downside. And I'm going to take a sip of this. You know, Tara is saying habit and life changing. Absolutely.

Starting point is 00:24:50 I'm going to take a sip of my water before I get to this downside because I have to throw out a little bit of a precursor here. Adobe just introduced an entirely new way to create, bringing the power and precision of its creative suite into one conversational experience. Meet Firefly AI Assistant, now live in the Adobe Firefly app, the all-in-one creative AI studio. Powered by Adobe's Creative Agent, Firefly AI Assistant lets you start with your vision, just describe what you want, and shape the outcome as it takes form with the Assistant. The Assistant orchestrates multi-step workflows, drawing on 60-plus pro-grade tools across Adobe Creative Cloud apps,

Starting point is 00:25:34 including Photoshop, Illustrator, Premier, Lightroom Express, and more to help bring your ideas to life. You can also get started with creative skills, a growing library of pre-built workflows for common creative tasks like batch editing photos, creating mood boards, portrait retouching, and creating social variations. Every step the assistant takes is visible so you can refine, redirect, or take over at any time. You stay in the driver's seat as the creative director. Adobe Firefly AI assistant now in public beta. See it today at firefly.adoply.com. So another thing that I think no one is talking about when it comes to the downsides of on-device AI is the, we'll just call it the SkyNet factor, right?

Starting point is 00:26:27 Is there going to be a kill switch? You know, no one wants to talk about that. If we have on-device AI, right? Whether it's in your computer chip or it's running, you can't, you know, you might not be able to opt out of it eventually. right? So think of this. Think of sometimes how, you know, Siri or Alexa, you ask them a short question and they get it wrong and then they keep on talking for like another 15 to 20 seconds, right? And it's kind of annoying. Think of like what that would look like if this is happening

Starting point is 00:27:06 all the time on your device. Will there be rogue AIs running wild? Again, because it's It's not like you're going to a website or opening an app or signing up to a service and putting a prompt in, right? This AI very soon, I'm guessing, is going to be running locally on just about every device. There's no kill switch. Again, I'm not saying, right? I'm not one of those people. Oh, my phone's going to, you know, take over my life and take over the world. But what's to stop it?

Starting point is 00:27:36 You know, will there be a kill switch if it is hardware based? If it's in the software, right? if I'm not opting into it when I use it. That's something we have to think about. I hope companies are keeping that in mind. But, you know, what happens if every single iPhone in three years has AI built him the hardware? What happens if I don't want it?

Starting point is 00:27:55 Will there be an opt-out? I hope so. But I don't know. That's a downside that we have to think about. Yeah. Yeah. And Sean, with a great comment here, which we're not even getting into, you know, when we're talking about things like Neurilink, right?

Starting point is 00:28:08 With the, you know, the AI in your brain. That's, yeah, that's definitely the next step. But here's the thing. On device AI is already here. It's already announced the chips are here, production. We're about to see it in mass pretty soon. You know, maybe the first big push is, you know, Google's Gemini Nano that's coming out to, you know, some of these new, you know, Android or, you know, Google smartphones. We're going to be seeing it soon.

Starting point is 00:28:34 Okay. Or I think it's coming to the new Samsung smartphone. Sorry. All right. So now let's talk about some of the upsides. You know, I don't want to end this episode on a downer like that. Let's talk about some of the upsides of personal AI. All right.

Starting point is 00:28:51 One example is, I think, the potential for your life to just be much easier, much more productive. You know, like I just said a couple of minutes ago, if you can picture in your mind that one task that you do in a system like chat GPT that is just like, you're just blown away at the, the amount of time it saves you, how impactful, you know, having that for that one specific task is. Imagine if this all goes well, if it goes swimmingly, having that on your device, having that, you know, let's just say on your phone, everywhere you go where your phone or your AI, your on-device AI knows you. It knows where you are. It knows what you like.

Starting point is 00:29:35 It knows what you're doing across multiple apps because it lives on your. or device. Yes, part of that is creepy. Part of that is extremely empowering. I think it's going to bring a lot of positive momentum and good things toward people that maybe have accessibility problems or setbacks, right? I think it's huge. I think it's potentially very powerful. You know, the upsides of having a personal AI of having on-device AI running locally, right? That's that's the thing that makes this possible, running this locally, you know, and getting that wait time or that lag time that normally you have to wait because, you know, you're chatting with a server, you know, like chat GPT and there's compute power and there's,

Starting point is 00:30:27 you know, downs and ups, you know, downtime, up times. It's not like that when it's on your device. It's instant. It knows everything. That's number one. You know, think of those like recommended replies. If you're ever using I message and, you know, someone texts you like, where are you? Right.

Starting point is 00:30:41 And then it pops up right there at the bottom and you can just click one button and say like, oh, sends your current location, right? Think of that across your entire cell phone or your entire laptop where it just knows everything about you. And it brings ease, flexibility to your fingertips. All right? the speed and always being connected is huge. But I have, yes, Tara.

Starting point is 00:31:10 Tara loves AI for accessibility, democratizing opportunities for us all. Same thing. So the last upside as we wrap up today's show. And if you do have a question, please get it in now. But the last upside. that I don't really think people are talking about is will on-device AI eventually take away this environmental black eye that generative AI has. You know, I've been trying to read more about this and to learn more.

Starting point is 00:31:47 So, hey, maybe if you know someone, have them reach out to me. We'll bring them on the show. But, you know, one thing about generative AI, as it is now, right? So working on it in the cloud is the environmental impact, right? It's huge. It's enormous. It's something that people aren't really talking about. You know, the compute power, you know, a lot of us don't understand it.

Starting point is 00:32:22 But the compute power needed to even just. For me to use chat GPT every day is crazy. It's crazy. I'll put the exact amount, but there was a, we talked about it many months ago in the show that, you know, for every 50 prompts that you run, you know, it was a certain, you know, gallon, certain number of gallons of water that means, right? Because the thing with, you know, all of these, you know, kind of cloud centers, data centers, again, I'm not the highly, like the most highly technical person. so I might not be using the right word, but essentially not just the compute power that is having a toll on the environment, but also you have to keep those data centers and in those chip centers cool, right? That's what all this water is being used for because they get hot and you have to pump this water to keep these centers cool.

Starting point is 00:33:17 The environmental impact of generative AI is huge. It's huge. The carbon footprint on this thing is enormous. So again, I don't know this, but here's my assumption. I think on-device AI can help with that, right? Let's just say as an example, let's just say as an example, I send 200 prompts a day to chat GPT. It's probably a lot more, right?

Starting point is 00:33:53 And let's say there's, you know, tens of millions of people that use that as much as me. Let's say there's tens of millions of people who do 200 prompts a day in inside chat GPD that obviously has a cost, not just a compute cost, but an environmental cost as well. So now let's say in a year or two, on device AI, instead of me now prompting chat GPD 200 times a day, I'm only prompting chat GPD 20 times a day. I reduce, you know, my cloud-based prompting by 90% because I have a local. on-device AI. Yes, I still have to power my phone every day, but presumably that has a much smaller environmental impact than always having to, you know, prompt out, you know, or, you know,

Starting point is 00:34:43 mid-jurney or runway, you know, these generative AI tools that so many of us use literally all the time. You know, companies say how much that they're even losing when people use generative AI, at least right now, because of not just the actual compute cost, but the environmental cost as well. Something has to change, right? And I think on-device AI might be that first step. So that's a little sneak upside for you as we wrap up here that I think a lot of people aren't talking about.

Starting point is 00:35:15 All right. I hope this was a fun one. I had a good time learning a little bit more about on-device AI. Also, whether you're listening on the live stream or on the podcast, let me know. Let me know what else that you want to hear about, right? So one thing that we do hear in everyday AI, aside from, yes, we're reading so much news it hurts. We're trying out so many generative AI programs.

Starting point is 00:35:39 I literally lose track. I can't tell you guys the amount of time that, you know, I'll see a generative AI program, you know, on social media or something and I'll Google it and I'll Google a tutorial. And, you know, the first result that comes up is, is me from six months ago. That's how many generative AI tools I use. But one thing that we try to do here at Everyday AI is to look at the trends, right? To not just look what happened, but to try to better understand what is happening and have an open conversation about it and to build a community of people together who want to learn about this to better grow their companies and to grow their careers. So thank you for tuning in.

Starting point is 00:36:18 As always, there's a ton more. So go to Your EverydayAI.com, sign up for the free daily newsletter. Maybe, you know, a lot of these statistics I was trying to rattle off the top of my head. I'm going to make sure that we get all of those specifically done for you and get them in the newsletter. So go sign up at Your EverydayAI.com. Shoot me a message. Shoot me a DM. You know, we always, you know, put my contact information in the podcast notes.

Starting point is 00:36:42 You know, if you're listening on the live stream, you have it. Let's continue to grow together, to learn generative AI, to grow our company. these growing careers. Thanks for tuning in to Everyday AI. We'll see you back for another one soon. Thanks y'all. Meet Firefly AI Assistant. Now live in Adobe Firefly, the Allman One Creative AI studio. Just describe what you want to create in your own words and the assistant handles the rest, orchestrating multi-step workflows across Adobe Creative Cloud apps, including Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome while the assistant accelerates execution. Stand control with the ability to step in and refine

Starting point is 00:37:26 at any time. See it today at firefly.adobie.com. And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going. For a little more AI magic, visit Your EverydayAI.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.

Everyday AI Podcast – An AI and ChatGPT Podcast - EP 184: On-Device AI: What it is and do we need it? What no one's talking about.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.