The Startup Ideas Podcast - Local AI Clearly Explained

Episode Date: September 8, 2026

I run this episode solo. I explain local AI in plain terms: the model runs on hardware I control, and a cloud model runs somewhere else. I map the four pieces of the local AI landscape — the model, the warehouse, the software, and the workflow — and I define the words that beginners meet first: parameters, tokens, context window, quantization, and GGUF. I walk through the Google open model stack (Gemma 4, Google AI Edge, LiteRT-LM, AI Edge Gallery), compare the other open model families, and show three ways to run a model today. I close with a first workflow you can copy and three startup ideas that use local AI as the wedge. And a special thank you to Google for supporting the podcast. Timestamps 00:00 – Intro 01:35 – The Open Model the Landscape 03:09 – Vocab Decoder 06:48 – Google Gemma Clearly Explained 10:29 – Other Open Model Families 14:20 – Path 1: Run Gemma in LM Studio 18:17 – Path 2: Ollama 20:15 – Path 3: Google AI Edge 21:07 – Hardware Cheat Sheet 21:52 – First Workflow to Build 22:47 – Workflows Before Fine-Tuning 25:06 – Local vs Cloud vs Hybrid Eval 26:33 – Framework for Local AI Startup Ideas 27:22 – Startup Idea 1: Home Health QA Reviewer 29:24 – Startup Idea 2: Offline Field Report Copilot 32:10 – Startup Idea 3: Pre-Send Reviewer for Professional Services 34:47 – Build Your Local AI Lab 37:55 – Closing Thoughts Key Points Ask whether the model is good enough for the job, and the business opportunities become clear. Local AI has four pieces: the model, the warehouse (Hugging Face), the software (LM Studio or Ollama), and the workflow you build around them. Gemma 4 E4B is my practical starting point; E2B fits phones and older machines. Hybrid architecture wins: local does the private first pass, cloud does the heavy reasoning, and a human approves anything important. Start with one repeated workflow — one folder, one model, one output — and run it 10 times. I see a 24-month window to build local-AI-native software for verticals that still run early-2000s tools. The #1 tool to find startup ideas/trends - https://www.ideabrowser.com LCA helps Fortune 500s and fast-growing startups build their future - from Warner Music to Fortnite to Dropbox. We turn 'what if' into reality with AI, apps, and next-gen products https://latecheckout.agency/ The Vibe Marketer - Resources for people into vibe marketing/marketing with AI: https://www.thevibemarketer.com/ FIND ME ON SOCIAL X/Twitter: https://twitter.com/gregisenberg Instagram: https://instagram.com/gregisenberg/ LinkedIn: https://www.linkedin.com/in/gisenberg/

I think local AI and open models are going to create a ridiculous number of business opportunities over the next 24 months, and I don't think most people actually have the map yet. They've used chat chit, they've used Claude, but when they hear local AI, hugging face, O Lama, L.M Studio, AI Edge, it sounds like it's for this developer world and that normal founders are just not supposed to touch it. And I think that's a mistake because the opportunity here is actually pretty endless. By the end of today's episode, you're going to understand what local AI is, when it matters, how to run open models at work, where Hugging Face fits in here, which Gemma model I'd start with, how I'd run a model locally with LM Studio or Olamma, and how this turns into real business ideas. And I'll give you three startup ideas I'd actually consider building using local AI, including who the customer is, what the first version does, why local matters, and how I'd sell it. Basically, this is going to be a masterclass around local AI, how to run models, how to build apps, how to make money from it. And I'm going to explain it for the average person who isn't technical. Quick shout out to Google for sponsoring today's episode and for caring about local AI and open models for entrepreneurs.

Today's episode, I'm going to use Gemma and Google AI Edge as the main examples, but the goal is to give you a full map so you can actually understand the space and build with it and use whatever model suits you. Okay, let's dive in. So put simply, local AI means the model runs on hardware you control. The hardware could be your MacBook. It could be your Windows laptop, an Android phone, an iPhone. It could be a browser, a Raspberry Pi. It could be in a workstation in your office. I just got a DGX Spark, which is like a high-end one.

But the important part to note is a phone now could run local AI. Cloud AI means the model runs somewhere else. and you access it through a website or an API, and that's the basic difference. The business question is, where should the intelligence live? If I'm doing deep research and strategy and hard reasoning or something where I want the strongest possible model,

I'm probably going to be using a frontier cloud model. If the work involves private files, like sensitive customer data, offline usage, field work, low latency, audio input or an internal workflow that runs again and again and again, local AI starts to make a lot of sense. A smaller model in the right place can actually be very valuable. That is the idea I want you to keep in your head.

The first question most people ask is, is this model smarter than the biggest model in the cloud? The actual more useful question to ask actually is, Is this model good enough for the job and does running it locally make the product better? Once you ask it that way, you start seeing these business opportunities which we'll go into. So there's four pieces to the local AI landscape. The model, which is the brain file, that could be something like Gemma, Lama, or Mistral. The warehouse, which is where you find the model, you might have heard of Hugging Face.

I think they're trying to get acquired right now at $13 billion. that's what they do. The software, which is what runs the model, that's something like LM Studio or O Lama, and then the workflow, which is the product you're building around all of it. And those are the real four pieces. The model is the brain file. Gemma is a model family. Lama is a model family. Quen or Mistral, you might have heard of Fai. These are model families too. Some of these are actually better at reasoning and some of them are better at coding. Some of them are smaller, some of them are faster, some of them are better for images, some are easier to run on your own machine. And then you need somewhere to find these models.

That's what Hugging Face is. That's the first place I would go. They're the biggest at it. The easiest way to explain Hugging Face is that it's a model warehouse. You go there and you can find model cards, licenses, file formats, examples, benchmarks, community versions, and sometimes versions that, have already been compressed so they're way easier to run locally. If you're new to local AI, one of the best exercises is actually just to open Hugging Face and read a model card really slowly. You're going to learn a lot. At first, though, you can't ignore half the scary looking details

and just look for a few basic things, in my opinion. What is the model for? How big is it? What license does it use? What hardware are people running it on? Does it support text, images, audio, tool use, or embeddings? Are there quantized files available? Once you can answer those questions, the space gets a lot more intimidating. Because I know when I first looked at these cards initially, I was like overwhelmed. So just those are the key questions to ask. Then you need software that runs the model. For most people, I would just start with LM Studio or or O LM Studio feels like a normal desktop app. You download it, you search for the model,

you click download, and then you can just chat with it. My opinion is it's probably one of the most friendly, first-time user experiences if you're non-technical. O-Lama is a little more builder-oriented or developer-oriented. You install it, you run a command like O-Lama, run, Gemma, 4-E4B. And now you have a model running locally,

with an API your apps could talk to. Then underneath those tools, you're going to start hearing about things like Lama.cp and MLX. And I'll explain what those two things are. Lama.cPP powers a lot of the local model inference. MLX matters if you're on Apple Silicon. And if you're thinking about shipping real on-device apps

in the Google ecosystem, that's where Google AI Edge and light RTLM come in. Basically, Google AI Edge is the broader on-device AI development world, and light RTLM is the runtime layer for the language models. This is what you study when you want to move from, I ran a model on my laptop to I want this model inside an iOS app or an Android app or a web app, desktop app, whatever it is.

We got to talk about some key vocabulary, just about the most important things you need to know about these words that come up time and time again in local AI. I'm just going to give you simple, clear definitions of what they are. By the end of this part, you'll know just the core basics of local AI vocab. So I'm sure you've heard this one before, parameters like $2 billion, $4 billion. These are what's called the internal weights of the model. And more parameters just usually means more capacity for hard. tasks, but it does require more memory. So parameters are the internal weights of the model.

The beginner shortcut is that more parameters usually means more capacity. And more capacity can help with the harder tasks. So the tradeoff is usually memory, speed, and hardware. So a 2 billion or 4 billion model is the kind of thing you might use for edge devices, phones, fast workflows, and smaller tasks. A 12 billion. parameter model is more of a middle ground and a 26 or 31 billion model is getting into the stronger workstation territory depending on your hardware and how the model is built. I recommend like not going out there and spending $5,000, $20,000 on a workstation just yet. By the end of this episode, you're going to understand how to just, you know, set up some of these things on your phone

or on a laptop, a spare laptop that you have from 2021. Then there are tokens. So tokens are the chunks of text that the model reads and writes. Locally, you care about speed and memory rather than the per token bill. Then there is the context window. The context window is basically how much information the model can work with at once. Then there is quantization. The word sounds more technical than it needs to.

Honestly, I can barely pronounce it. Quantization is the compression for models. It allows giant models to fill. on normal laptops. For example, you might have heard of Q4, Q8 formats. That's quantization. If the full model is the giant version, the quantized model is the version that can actually fit on a normal laptop. So you might lose a little quality, but suddenly this thing magically runs. You will see things like Q4 or Q8. And as a beginner rule, Q4 is just usually easier to run and Q8 keeps more quality, but it needs more memory. If you're just getting started,

Q4 is just a reasonable place to begin. So I would start there. Then there is G-GUF. It's a common file format for local models that make inference easier on normal machines like you and I have. And in the Google AI Edgeworld, you'll see something called the Light RTLM. This is the model format and runtime path you care about when building on-device apps with light RTM. So the simple map is this. Hugging Face helps you find and understand models. Gemma is Google's open model family and Google's a trusted brand. I run my business on top of Google, so it just makes sense. LM Studio helps you try models locally without much friction. Olamma helps you run models locally in a way that a bit more technical people can plug into apps. GGUF is a common local model format, and Google AI Edge and

LiteRTLM are the path towards shipping on-device AI products. That's what you need to know. So let's talk about Google's Open Model family, because I feel like there's a lot here. It's a bit overwhelming. And I'm just going to break it down so you understand what you need to know about the whole Google AI open model family. So Gemma is Google's family of open models, and Gemma 4 is built for the efficient, local, and on-device use. So they have Google 4E2B, which is the smaller edge model for phone workflows. You have a bigger E4B, Gemma 4E4B, which is pretty much the most practical starting point for most local tests. Then you have Gemma 412B, which is a middle ground with more capability.

for laptops. And then you have Gemma for 26B slash 31B, which is, you know, the stronger local workstation territory. That is the main model picker. Then, and a lot of people don't know this, there's specialized Gemma models that are just really useful to know. So you have things like embedding Gemma, which is just for search. So specifically, it helps you turn text into embeddings, which lets you search by meeting. If you want to search your own docs or customer notes or support tickets, sales calls, or knowledge-based localities,

embeddings matter a lot. Then they have something called function gemma, and that's a tool use and structured function calling. That means the model can help software take actions in a way more structured way. It is part of the path from the model gave me an answer to the model help the product do the next step. Then you have a few more like Pally Gemma, which is more vision focus.

You have Shield Gemma, which is more safety focus. Then you have Gemma scope, which is more understanding how models work under the hood. You can leave most of the family alone on day one. The practical path, like on day one, if you're a beginner, start with Gemma 4E4B, understand the workflow. Then you can move up or down or sideways, actually, depending on what you are building. So the way I understand the whole Google AI ecosystem is you have Gemma as the open model family. You have Google AI Edge, which is the on-device AI development ecosystem.

You have Light RTLM, which is the runtime for running languages, models across all the devices. You have AI Edge Gallery, which lets you try on-device models and see the experience just more directly. And if you need huge scale, things like strong managed infrastructure or frontier level cloud reasoning, you still have Gemini and Google Cloud that you can use or another Frontier LLM that you can use. The reality is a lot of big products and serious products are going to use a hybrid setup. They're going to use cloud for certain things and you're going to use local for other things. As an example, imagine a local AI tool for a professional service firm. So the local model is going to read the sensitive drafts, checking for the issues, it's going to strip or summarize all the private details,

and prepare a clean version of the problem. Then, when the customer wants deeper reasoning, a cloud model can help with the sanitized version. That to me feels like a more natural architecture than just putting everything into the cloud, which a lot of people don't want. You basically have local handling the private files as a first pass, and then cloud handles the heavy thinking when you need it. A human can improve the work before anything important goes out. This is how I'm starting to think about building a lot of these products. Beyond Google Gemma, I'll give you a quick primer on the other families or other open model families you'll hear about and some of the pros and cons. Lama is a meta's model.

And it's probably the default open model reference point for a lot of developers because it's a pretty big ecosystem. The upside is the community, the tooling, the examples, support. The downside is you still need to read the license and the model card, especially if you're building a serious commercial product. Quinn is Alibaba's model family and has become very strong, especially around coding multilingual work, long context, energetic tasks. The China thing is real. A lot of people use Quinn because it performs really well, but if you're in an enterprise, a government, healthcare, finance, or sensitive data environment, you need to separate running open weights locally from sending data to a hosted service. And you need to check what your company is comfortable with or what you're comfortable with. Deep Seek is similar

in the sense that it's made, it's made a lot of people. realize how strong Chinese-based open models really could be, especially for reasoning and coding. The upside is performance and costs. It's pretty cheap. The trade-off is that some buyers will have procurement, security, or geopolitical concerns. So I'd be thoughtful about where I'd use it, how I deploy it, even if you want to, even if you want to use it, and go down that path. There's also GLM or people know it as z.aI It's another one you'll see pop up a lot I actually did an episode on it

especially if you spend time on hugging face and Olama or just local model Twitter you're going to see it a lot The thing to know is that some of these models can be weirdly good for specific jobs So I wouldn't ignore them just because they're Not the obvious brand name You can test them, you can read the model card

You can check the license and just play with them. But what you might deploy in the sense of for your business or for what you're doing might be very different. There's also Mistral, which is the European model family. I think they're based in France. If you care about efficient models, and they do a lot of releasing,

a lot of practical developer use cases, they're pretty good. It's a strong model with a pretty builder-friendly posture. but the downside is the lineup is a little confusing. Some models are open, some are commercial, so some question marks there. Microsoft also has their open model family. It's called Phi.

I think it's interesting if you care about smaller, faster, lower latency models. But for a lot of use cases, I haven't seen it work very well. And honestly, there are new models. showing up all the time. It feels like every other day. And that's why hugging face matters. You're not going there just to find the big models. You're going to find these like weird specialist models, these like community fine tunes, quantized versions of stuff, these forks

and model cards that tell you whether something's actually usable for the workflow. So you don't know, you don't need to memorize all of this. But the takeaway basically is that there's these ecosystems and your job as a founder or just, you know, someone who's playing with these models is to pick a model family that fits your workflow, that you connect with that company, you like how they do things, and then go from there. You can play with a lot, learn a lot, and then, you know, pick a family. So how do we make this whole thing real? Like, if you actually want to run Gemma, here's how I would do it. I would start with LM Studio. I would Download LM Studio.

It's free to download. You open the app. You search for Gemma 4. If your machine is solid, try E4B. But if your machine is a bit slower, older. I would look for E2B. And then I would look for the quantized version if you're using the GGUF path because you want the model to run just a lot more comfortably. Once it downloads, open a chat and just ask it something really simple.

You know, to use like a business prompt because I want you to feel the value immediately. It's sort of an aha moment. Maybe it's something like read these customer notes and turn them into a one-page memo about what customers are struggling with. What has changed and what the business should fix this week. And then just paste like some customer notes or just fake customer notes, just if you want to see the value. The point of this exercise is just basically. The model is now running on your machine and you're using AI without sending that prompt to a cloud model. I believe everyone should try that and feel what that is because I do think that it's just going to be a lot more common and it's going to unlock your brain in a completely new way.

After that, go to LM Studios developer section and start the local server because that just gets a lot more interesting because other apps can talk to the model on your laptop. your computer becomes this little AI server. So you can have a script or a prototype or just an internal tool that can call the model through local host and you get an answer back. I think that's when you start to see how products are going to get built in the modern age. The second path is Olama. So install Olama and run Olama pole Gemma 4. then run Olamma run Gemma 4 colon E4B. Now you have Gemma running locally from a command line.

Olamma also gives you a local API port. I think it's on 111434. It is useful because you can connect your own app or script to it. If you want to test a larger model later, you can try the 12 billion, 26 billion, or 31 billion versions, assuming your hardware can handle it. And you can ask an LLM if your hardware can handle it, or you can do it yourself and just suffer through the slowness and the pain of it. The third path is Google AI Edge and LightRTLM. I would only use this path if I wanted to build an actual app and a model inside of it. For example, maybe I'm building a mobile app and the model is running on the phone.

Or could be like a browser app where the model runs locally. Or maybe it's a desktop app with a private workflow. Or something on an edge device. Light RTLM is designed for that world. Android, iOS, web, desktop, and edge environments. That is the path from local AI. as a demo to local AI as a product. So here's the hardware cheat sheet that I would use.

If you have 8 gigabytes of RAM, start small and keep the first test simple. But if you have something like 16 gigabytes of RAM, you can do some useful experiments with models like E4B and smaller quantized models. If you have 30 gigabytes, 32 gigabytes of RAM, you have way more room to do larger local workflows. If you have a strong GPU or a workstation, like a DGX Spark, the bigger models become just much more realistic. And for phones, I would think a lot less about model size and more about the job. So can the model understand a photo? Can it summarize audio? Can it classify something quickly?

Can it help a worker in the field? Can it run without a strong connection? Can it do something useful inside the app before the user even thinks? thinks to ask. Now, let's build the first workflow in our heads. So I would make a folder on your desktop called Customer Notes. And inside that folder, I'd put 10 support tickets for a specific business. Let's say it's a home health agency or a med spa or a water damage restoration company. The notes might say something like, I tried to reschedule but couldn't find the link, or the technician didn't explain what happens next,

or, hey, no one actually confirmed my appointment, or I was charged twice here. Then I would run a local model like Gemma and ask it to create a file called What Customers Are Telling Us.md, the markdown file. The output should include the repeated complaints, the exact customer language, the likely root cause, the part of the business that seems broken,

and the one thing the operator should test this week, the high priority stuff. This is a good first local AI workflow because it's useful and it's simple. What do you have here, right? You have this private messy data. The model runs next to it and the output is a memo someone could actually use. And then once you actually go and, you know, you're going to go and do this and get the output, you're going to, like the unlock I was talking before, like it's going to unlock something in your brain, you're going to see this pattern everywhere. A folder of customer calls become a market

research memo. A folder of support tickets become a product roadmap signal. A folder of PDFs become like a risk check list. A folder of drafts become a pre-send reviewer. This is why I always start with workflows before I'm fine-tuning anything. People here, you know, open model and immediately want to train their own model. And I get it. I get why. I was actually the same. way. It sounds really cool. But I feel like that's like an advanced move. The practical move, the beginner move, where you should start is just to find a repeated workflow first. You pick one folder, one model, one output, and you run it like 10 times. You see where it gets confused. You see where you can improve the prompt and add examples. You add a checklist and then you create

like a small e-val. You know, what's an e-val? And eval is just a small, it's just a test. that tells you whether the model did the job well enough. For this workflow, for example, the e-val could be like really simple. It could be like, you know, take the same 10 customer notes and run them through Gemma locally and then run them through a strong cloud model, a frontier model, and then just compare the outputs. And then you, you know, you ask, did Gemma, you know, catch the same complaints? Did Gemma pull the right quotes and did it follow the format? did it miss something? The comparison actually teaches you where locals are ready useful and where

you still want that stronger cloud model and how you should think about the hybrid model I was talking about. That's really how I think about local versus cloud decisions. Use local for private, repetitive, fast, offline, device native, and high volume workflow, stuff that you want to run all the time. You use cloud for deep reasoning, giant contacts, broad research, and cases where the strongest model changes the quality of the answer. So you use both when the product has sensitive data and hard reasoning. A lot of valuable products will work that way. You know, local first pass, you do the cloud escalation, human approval for anything important. I think that's the way works going to get done. So I want to give you three startup ideas.

where local AI actually matters. And these are the kind of businesses I would look for, niche, useful, cash-flowing businesses that you don't need to raise venture for, and tied to a painful workflow. The filter is pretty straightforward. So I look for a customer with sensitive data, repeated review work, bad software usually, expensive mistakes, like mistakes that will cost them a lot. And a workflow that happens close to the device. that combination is like the interesting zone for me.

So let's go through the three ideas. I want you to steal these ideas. And at the very least, it'll get your creative juices flowing with how you can use local AI to run model, build apps and make money. Idea number one is a local QA reviewer for home health agencies. So home health agencies have nurses and caregivers and they go into people's homes and they write, you know, visit notes and updating care plans and dealing with billing and compliance. The paperwork is a pain. It takes a lot of time if you ever witnessed it in person,

but it matters so, so much. Like a missing detail can create a billing delay and a vague note can create extra admin work. And a mismatch between the visit and the care plan can create a ton of risk and we don't want that. So the first version is a local desktop app for the agency. The agency drops in visit notes and care plans and dictated transcripts. And then the model is going to review them before the submission and should look for flags.

So it's going to flag things like this note mentions dizziness but vitals are missing. Or the caregiver described a medication change, but the follow-up instructions is pretty unclear. Or the note may not support the build service level. The buyer mostly cares about fewer documentation problems before. the billing or the audit or a supervisor review. So if you solve that, you have their attention. Now, I don't want to just give you the idea. I mean, how would you actually grow this? If I was starting this business, how would I grow this business? I would actually start it as a service. So I would find five small home health agencies and I would offer to review a batch of

notes. I would do the review with AI helping behind the scenes, with the local AI. And I would inspect everything manually with like human beings myself first. I'd write down the 20 issues that keep showing up. And those issues become the checklist. And then the checklist eventually becomes the product. So you have this wedge. It's pretty simple where you're catching documentation problems before they cost the agency time or money and then you build from there. I love this business and totally what started. The second startup idea is an offline field report copilot for restoration contractor. So think water damage or fire damage or mold remediation, things like that. Those teams are out there field taking photos, recording notes, documenting damage, and creating reports

for homeowners and insurance adjusters. I unfortunately had this, so I know a little bit about it. The job is actually pretty visual. It's also physical, right? It happens like away from a desk. And the report matters because the report becomes the handoff. between the technician, the customer, the office, and the insurance process. So how would we build a product here? The first version is a mobile app. So a technician walks through the property, takes photos, record voice notes, and the app drafts the report before they leave the site. So it can flag missing pieces while the technician is still there walking around. You mentioned the basement, but there are no basement photos. You took a photo of ceiling damage,

but there are no moisture meeting reading, things like that. Or maybe like the affected room is like missing. Could be the homeowner explanation is way too technical. Here's a clearer version they can understand. And the last part of that is underrated. In a stressful home damage situation, clear communication is part of the product, right? So if you had that, that would be key. How would I grow this business?

Well, I would pick one niche first. I wouldn't go after everything. So say I'm going after, you know, water damage restoration. I would talk to owner operators. I'd look at their current report templates, study the software they use, which is some old stack, and I'd build around the checklist that's already in their head. The demo is actually the easy part.

You know, send me three old jobs, and I'll show you how fast your techs could create reports. If that works, then the product could expand from there. That's just the wedge, right? it can go into QA and estimate and insurance packets, customer updates, and training new technicians. But I would start with the field report because it's specific and obviously super annoying.

And, you know, I just think that there's, when you look at some of these old softwares that, you know, these people are using, I recently had some water damage at my apartment. And I was seeing some of the software and it's antiquated. It's stuff from like the early 2000. So I think that there's just opportunity to create local AI native software and wedge now. And that's why I said in the beginning, like I think there's a 24 month window and opportunity to do some of these products.

Let's go into startup idea number three. So startup idea number three is a local pre-send reviewer for professional services. So every professional service firm, or 99.9% of them, has a version of this workflow. Someone writes a client email, a proposal, a memo, a contract summary, an investment note, an HR note, and then ask someone else to check it out before it goes out, like a review. And it happens constantly. Law firms, accounting firms, wealth advisors, recruiting firms, even consultants have a version of this. So the first version is you build a local desktop app that reviews outbound drafts before they leave the company.

So for a wealth advisor, it could be flagging language that sounds like a guarantee return, which is a definite no-no. For a law firm, it'll flag a sentence that sounds too definitive. For HR, it's going to flag sensitive employee information that should stay out of the threat. For an agency, it flags a promise that the scope does not support. And for an accountant, it flags a number that doesn't match the attached file. You'd be surprised how often that happens. The product is basically a second set of eyes for sensitive work. It's basically Schmuck Insurance is the way I think about it.

And maybe that would be the name, schmuckinsurance.com. Someone tell me if that's taken. How would I grow the business? I would start with one vertical and one dollar. document type. For example, I would do email review for independent wealth advisors. Not everyone, probably not the big banks to start, independent wealth advisors. I would interview 10 advisors and asked them which emails make them nervous. I would collect anonymized examples. I would turn their real concerns into a review checklist and I would build a local tool that checks drafts against

that checklist. Obviously, this is so sellable because the buyer, understands this behavior, and they already ask someone to check the jaft. So you're just basically giving them a faster first pass that lives closer to their client data and internal rules. I love this idea and hope a few of you take it. By the way, if you're not building one of these ideas tomorrow, I still think you should learn local AI because it does change how you work with your own files. So I think just like from a personal productivity perspective, it's still super, super helpful. So, you know, if you're working at a company, say, and you just want to be more productive, so you have more time to scroll TikTok or watch movies or hang with your family,

make a folder called Local AI Lab, and then put 10 files that matter to your work in that folder. It could be anything from sales calls or meeting transcripts, old tweets, ideas that you have, then run Gemma, whatever model you choose to make it produce one useful artifact. And then ask it to create a weekly business pulse or ask it to find what's changed in customer conversations or meeting notes. Ask it to group feature requests by the actual pain behind it. Ask it to review drafts and tell you what your audience keeps responding to. The key basically is to produce a file, a memo, a checklist, a brief, a report,

or a review that you can reuse. A chat answer is nice, but a useful artifact changes that workflow. This is the first rep I would recommend. A model reads the folder, the model writes the file, you inspect it, you improve the workflow, and then you run it again. If you do that a few times,

your brain really starts to connect the dots. You start noticing where private data is trapped in folders. You notice which reviews happen over and over again. And you notice which workflows depend on someone checking a form, reading a note, comparing two files,

cleaning up a report, or writing the same kind of memo week after week. Hopefully this episode got your creative juices flowing because once you see the pattern, you start spotting local AI businesses everywhere. You can learn enough of the map to spot where these models belong

without turning yourself into a local engineer overnight. I believe, Some local AI belongs in the cloud and some AI belongs in the device. And a lot of the best products of the next couple of years are going to combine them both. So if I was starting today, what I would do is I would run Gemma locally. I'd read model cards on Hugging Face. I'd learn the difference between L.M. Studio and Olamma.

I'd play with Google AI Edge and then look for one boring workflow where local AI actually makes the product better. Those categories are things like private data, offline work, camera audio contacts, or low latency, or if there's a high repeated API cost, if there's a buyer who feels better when the model is just close to them, a workflow where a small agent team could recheck, summarize, and prepare work every day. That's like the hunting ground. Local AI is just way easier to understand once you stop treating it like a model benchmark conversation and start treating it like a product conversation. You have to ask yourself, where is the work happening?

Where is the data? Where is the device? Where is the trust issue? Where is an annoying review loop? And then you answer those questions and you just start seeing the ideas. So overall, I hope you understand a little about the core things you need to understand about local AI, some of the models, some of the apps you need to download, some of the workflows that you can build and some of the business opportunities that exist.

I just don't see that many non-technical people playing with local AI. And over the last two months or so, I've gotten deeper and deeper into it. And it's just, like I said, it's been connecting the dots. And I'm grateful for it. I hope you have a creative day. I read every single comment on YouTube and respond to most. So I'll see you in there. Share this with a friend who you think could benefit.

from understanding local AI in a clear way. And I'll see you next time. Happy building.