The Vergecast - Inside the AI memory machine

Episode Date: July 14, 2024

Humans are terrible at remembering things. On this episode of The Vergecast, we talk to one of the people who has been working on this problem for a very long time: Dan Siroker, the CEO of Limitless. We talk about what it takes to build a great memory aid, how we might use them in the future, and why it’s so tricky to get right. We also talk about the human side of it all — what does it change about our lives when we stop forgetting things? Is remembering your friend’s birthday different when it’s actually an AI model doing the remembering? And will these tools ever really work outside of work? Tools like Limitless are coming fast and improving quickly, and we’re going to have to figure out how to live with them. Further reading: From The New York Times: Can’t See Pictures in Your Mind? You’re Not Alone. Limitless AI: a new wearable gadget, and app, for remembering your meetings Recall is Microsoft’s key to unlocking the future of PCs Microsoft’s all-knowing Recall AI feature is being delayed The Pixel 9’s ‘Google AI’ is like Microsoft Recall but a little less creepy Apple announces iOS 18 with new AI features and more customizable homescreen Notion AI can automatically write your notes, agendas, and blog posts for you Email us at [e-mail] or call us at 866-VERGE11, we love hearing from you. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Support for the show comes from Retool. Too many companies run critical operations on duct taped spreadsheets, Slack workflows, and whatever else they could cobble together. Not because they want to, but because building internal tools means weeks of waiting on someone else's backlog. That's where Retool comes in. Build custom internal tools just by describing what you need. Prompts something like,

Build me a revenue dashboard on our Salesforce data. And Retool actually builds it on your company's data, in your cloud with enterprise security built in. Go to retool.com slash vergecast. We all need to retool how we build software. Welcome to the Vergecast, the flagship podcast of Total Recall, both the concept and the movie, but only the first movie, not so much the con-feral one.

Anyway, I'm your friend David Pierce, and this is the first episode in our latest mini-series all about AI. We talk so much about AI these days, and so much of it is so abstract. AI is this big, messy concept, and it's easy to get lost in all the theories and consequences inside of it. Those things matter, of course, a lot.

But we also want to try and get a bit of an on-the-ground view of how AI is actually being used and whether it's actually useful. Today I want to talk about a use of AI that is suddenly everywhere. It goes by many names. Microsoft calls it Recall. With recall, we're going to leverage the power of AI and the new system performance to make it possible to access virtually anything you have ever seen on your PC. Apple calls it personal intelligence. It understands who's in your photo library, so you can just ask photos to create a movie about Leo learning to fish.

For Google, it's one of the most important features of Gemini. Do you remember where you saw my glasses? Yes, I do. Your glasses were on the desk near a red apple. There are also lots of startups, enterprise software companies, and everybody else you can think of working on something similar. Call this feature whatever you want. But the goal of them is all the same. To remember everything you need to remember and give it back to you at exactly the right moment and in exactly the right way. As an idea, I find this incredibly compelling.

I think I find it more compelling than most, actually. See, I have this thing called Afantasia, which essentially means I don't see pictures in my head. Like, if I tell you right now to picture an apple, there's a wide range of things that you might be picturing. Maybe you see a super crisp, detailed apple with colors and leaves on the stem, and there's a whole surrounding universe around it. Or maybe you just see sort of a cartoon-y-looking apple, like the apple-liest apple you've ever seen. Maybe you just see the rough shape of an apple in your mind's eye. If you're like me, you don't see anything. Nothing.

Blackness. I can tell you what an apple looks like. I know what an apple looks like, but I can't see it there. I honestly always thought that when people talked about visualizing stuff and their mind's eye, they didn't mean it literally. It breaks my brain still to this day every time I think about the fact that people can just conjure images in their brain and they see them. I don't get it, and people who see it don't get that I don't. Brains are weird. One thing that seems to be true about people with Afantasia like me is, is that we don't necessarily have particularly great visual memories. Like, I can't just conjure up a photo of my childhood home, you know?

I remember lots of things. And to be clear, I don't think Afantasia is like a disability so much as it is just a different way your brain works. But I just don't have that kind of visual memory. So over the years, I've leaned pretty hard on technology to help me fix that. I take a lot of photos, and I love all the on this day and automatically generated albums that I get. on my phone and in Google Photos, I've tried to journal much more over the years because going back to words and pictures and videos does put me back in a place better than just closing my eyes and trying to remember something. And anyone who knows me will tell you that I am an obsessive

note taker and list maker because it just helps me keep my world in order. It's how I remember things and go back to things. All of this is to say that a tool like the one these companies are describing is kind of my dream. Something that can remember every article I read, every message I receive, everything I need to do, person I meet, place I go, everything else that happens in my life, and then both store all of that for me to access, and then actually present it to me when I need it, that would literally change my life. There are obviously huge complex privacy and data security questions associated with how that all works, and I want to get to that. But this is one of the things AI is actually pretty good at. Just storing, sorting, and categorizing vast amounts of data.

What if that data could be my life? I wanted to get some perspective on what it takes to actually pull this off, so I called up one of the people who's been working on it the longest. I'm Dan Seroaker, the co-founder and CEO of Limitless. Here's Dan's story in a nutshell. He worked at Google out of school, then started a company called Optimize Lee, which he sold in 2020, and along the way, he started to lose his hearing. The way he described it to me, he had this really visceral experience the first time he tried on a hearing aid.

He told me it was like gaining a superpower. And he started to look for other ways to give himself and other people technological superpowers like that. And he landed on memory. We as humans are terrible at remembering things. We forget things over time. We forget things immediately

because we just weren't paying attention when we learned them. We remember things but misremember where they came from. we remember things selectively because we come to everything with specific biases. Some people have better memories than others, but nobody's memory is perfect. And Dan had this thought that maybe he could help fix that. It started with this idea, well, if there's a hearing aid for hearing and glasses for vision, what's the equivalent for memory?

The first version of Limitless was actually an app called Rewind, which you'd install onto your Mac, and it would immediately begin to capture audio through your microphone and speakers, and it would also take almost constant screenshots of whatever you were doing on your screen. that basic technology is pretty simple, right? Capture a bunch of audio, take a lot of screenshots, and it gets you pretty far in terms of trying to figure out what you're doing on your computer. The question then is what to do with all of that?

What does it actually mean to give someone a better memory? At the beginning, that was the big question for Rewind. It's good to think about it as an analogy. There are many things today that you do not remember, and you're very happy to have offloaded technology. A good example is phone numbers. I'm not sure the last time you had to type an hour, actual phone number in your phone, pretty rare. Now you just open up your phone and tap the person's

name. There's a period of time in my youth where I had to remember everyone's phone number, and that's the way I would call them. Similarly, with getting from point A to point B, there was a period of time in our life when we had to remember cross streets and we had to remember how to navigate the world. Now we just tell the computer, this is where we want to go and it gives us in time, real time, the most optimal route we can take. And like the memory of those things, that sort of, I'm using those as very basic examples, we don't miss at all. There are other things in our lives that are like that, that we don't even realize we're forgetting because our memory is flawed. You know, a good example is how I know a person or, you know, when you and I maybe reconnected

in a few months, how old was your young one? And, you know, those details that actually help build connection and rapport make us feel more human to one another, they may seem, you know, robotic to have your computer remember before you, but they also create this amazing connection that if you imagine a world where you, if you had perfect memory, what you could do, both in terms of connection, in terms of productivity. And the same way, could you imagine, if you wore glasses today or contact lenses, could you imagine going a day without them? Like, why would you live your life blurry?

And that same thing could be said of memory. Why would you live your life with a blurry memory? That's such an interesting, like, philosophy question, right? Because I feel like the way you think about that as opposed to something like hearing aids is an interesting one, because I think what you're saying is less kind of, let's solve for an issue that you're having, right? And then kind of get something back to the level at which you had it or, would like it to be. And more saying, like, how do we limitless pill this, right? And it's like,

what if instead of using 10% of your brain, you use 100% of your brain? And that's such an interesting, like, what does it mean to be a person when you have all of that stuff? It's like, have you ever seen the reaction people have to like dating spreadsheets when people have these, these really like intense databases with all the people they're going out with? On the one hand, makes total sense. But on the other hand, I think it gives people this sort of like human ick in a way that I've never really been able to describe. This is such a technological question. we're having to reckon with right now in so many ways that it's like if all of this stuff is just available to me, how do I use it in a way that makes it still feel like me and feel

human and feel honest and feel real? And I don't know, it's not even really a question. It's just, it just opens up these really complicated, like philosophical wanderings in a way that so much AI stuff does right now for me. Yeah, I often imagine how will the future think of us today. and one way to think about that question is to think about how we think about people 50 or 100 or 150 years ago. And that feeling you described that ick of using computers to do a thing that you thought should just be, want to just go to a bar and meet somebody. That ick actually, in hindsight, seems silly. You know, it's like the same way that people had an ick toward cars.

Well, I love my horse. And they had an ick toward telephones. You know, that feeling of newness, of novelty, of difference, people just as human beings are not wired well to respond to change. We are just not. There are some people who are very excited about new. They buy the newest gadget. I'm one of them, like, no matter what it is, no matter how bad it is. Just that it's new is good enough for me.

There's others for that would be the worst way to live life. They're just happy with the way things are, and the world is changing faster than they like, and it's changing faster than ever. And, you know, the way I think about it is five, 50, 100 years from now. People will look back at today and be shocked that we accepted the lives that we live, that we would forget 90% of what happens after a week. And that we thought that was okay, sure. Yeah, this brilliant device that we have of our minds.

mind that's so precious, this incredible machine, we just let it forget things. And that's okay, no big deal. That's just life. I think people will laugh at that. You know, my grandkids will ask me, really, Daddy, you just went through life forgetting most of it. And how's okay with you? I don't know that I buy that theory of the future completely for whatever it's worth, but it is a theory that I hear a lot in talking to AI people. And it's largely true over time that as we found more ways to augment our own capabilities, things have gotten better. I mean, look, what is a computer, if not a way to just do math faster than writing it down on paper, right? Offloading things that we don't do well to technology that does do it well

tends to be a pretty good outcome in a lot of ways. We have to take a really quick break, and then we're going to talk about what an app actually does when it tries to give you a better memory. We'll be right back. Support for this show comes from Shopify. Starting something new isn't just hard. It can be really scary, too.

So much work goes into this thing that you're not entirely sure, will even work. But here's a better thought. What if it did all work? What if your instincts were actually right all along? Shopify wants to help you get there. They're the commerce platform behind millions of businesses worldwide and nearly 10% of all e-commerce in the U.S. From established brands like Allbirds and Heinz to companies just getting started. Their design tools make it simple to create the exact online presence you're envisioning with hundreds of ready-to-use templates available. And with built-in marketing tools, you can launch full email and social campaigns in just a few clicks. So you can connect with customers wherever they are. It's time to turn those

what-ifs into with Shopify today. You can sign up for your $1 per month trial today at Shopify.com slash vergecast. You can go to shopify.com slash vergecast. That's Shopify.com slash vergecast. All right, we're back. So when Dan starts really working on his memory superpower app in earnest. It's 2020, early COVID times, and everyone's suddenly working from home and on Zoom calls all day. This is pre-chat GPT, by the way, and before everybody decided that AI was going to immediately change everything forever. Zoom, by the way, it was like the hottest company.

Do you remember these times? It was weird times. Dan has this big, huge idea about memory superpowers, but he ends up deciding to start his product on a much narrower path. We started with a meeting bot. Everyone was doing Zoom meetings. It was a bot that would join your meeting, capture, record it, transcribe it and make that shareable. When we started, that was novel. And then every week, a new startup came and did it. And now there's dozens of companies doing that. You just described my whole, like, newswriting life of 2020 is that that startup over and over again.

Exactly. So that became very crowded. And what we realized from our users is, and this is where any of our best ideas came from. They're all the seed in a problem that our users described. They love the idea of comprehensiveness. They really wanted to capture more of their lives, not just these, you know, Zoom meetings they're having. And that led us to ideate around how do you capture more than just meetings? And around that time, the enabling technology of Apple Silicon came around. Apple M1 came, a chip that allowed you do a lot more things locally to offload a lot of the things that would otherwise have to be done in the cloud. And that's when we moved from Scribe with meeting bot to rewind, the Mac app. That was, again, long before chat GPT, GPT335, which is the model that powered it. And I

actually say we got more lucky than good there, where the data that we were actually already capturing with Rewind just really blended itself well to rag, retrieval augmented generation, like the ability to use that data, you know, to ask a model like GPT35 or 4 or 40 now. If you asked it to draft me an email to Sam Altman, if you just ask it today without any context, it doesn't know okay job, it knows who Sam is. It certainly doesn't know who I am and doesn't know our relationship. But if you augment it with the context of our relationship, the fact that he invested four years ago and that, you know, we've been working tirelessly in this migration.

revolution to limitless, all of that context you provide a large language model, it drafts a perfect email. In fact, an email I sent to Sam that an AI could have drafted worked great and we met and we connected. So like these things that, you know, otherwise you have a blank piece of paper can easily be made easier by having AI augmented with your context of your past. What would you have done had this not happened? Like in this sort of parallel universe where like chat GPT doesn't happen and we don't get this kind of incredible run of stuff we've been on the last few years. It doesn't sound like you were counting on that. You weren't saying, like, we're going to bet on this foundational technology to be the thing. What is parallel universe, Dan,

building to make that stuff work? Yeah, I would have been doing more of the same. I mean, I would have, it was already valuable enough to do search over the things you've seen, said or heard in the past. The large things model just makes that search more useful and more actionable and more, it sort of takes that, you know, the task you would have had to do. Like, let me search through all my emails with Sam. Now we figure, okay, when did we meet, who introduced us? Instead of doing that manual task, now you do it in an audit. automatic way. And so it just made me an analogy draws. Like before, you know, we were on this evolution from horse and buggies to self-driving cars, our evolution before ChatsyBT was maybe

we added a car with manual transmission way better than a horse and buggy. But what Chatsybtee and the underlying API that we use but enabled us to do is go to automatic transmission. So we're not quite at self-driving cars. We're the, you know, autonomous AI simulations of your mind doing things for you. Yeah. But we are able to actually save you time and give you a lot of value through this sort of evolution. So that's what we would have been doing. And by the way, way now, our focus is basically just banking on the models getting better. Like, everything we do is under the premise of just that the models are going to get better. So like, just collect all the data in the best possible way to ride that wave. So our mindset has definitely shifted. We're not

sort of doggedly just pursuing what we've done before. Now we realize, okay, there's this amazing wave of change or these models just going to get better and better and extrably. Why not just ride that wave and build a product that just gets better on its own, gets cheaper, gets better on its own as the models get better. The car analogy there is a little messy, but it's an interesting way to think about where we are with AI. We spent two decades or so with products like Google and Facebook, which built very smart and sophisticated systems for looking at a huge amount of stuff and ranking it a million different ways. Those systems are by and large very good, but the promise of AI is that it can take all that stuff and actually come to understand it, not just find you the

thing you're looking for by putting it at the top of the list, but by finding the perfect thing that you're not looking for, or by using everything that you already know to help you do the next thing. We don't need AI to do Google searches. We really don't. Even Google is currently showing us how much we don't need AI as it tries to put more AI in to Google search. But if all the AI boosters are right, we can use AI to not just find things, but build new things on top of them. In a personal context, that brings up the central problem with all of these AI systems. In order for an app like Rewind to know everything about you, it needs to know everything about you. Do you want your computer to store and save everything you do, everything you click on,

all the words you type, all the TikToks you scroll through, all the pictures you look at, every single thing that you do while you're at your computer? Forget about the data security risks behind that for a second. Just like as a human, how does it feel to know that all of that stuff is being recorded and stored in perpetuity? And how useful does your computer need to become? What does it need to do with that data in order for it to be worth the trade? Dan has been thinking about this for a long time, and he calls it the personalized AI privacy paradox.

And it goes something like this. In order to build a more useful personalized AI, you want more context recorded. But that raises more privacy concerns, which requires more need for data protection, which makes it harder to build a more personalized AI. So the desire of a personalized AI, part of it is this, okay, we want to collect more things because that context is going to be useful. But inexorably, you go down this path, and at some point, that's going to make the original goal harder to do. And you can see this paradox playing out in the world in two very different ways. With Microsoft, you know, they were pretty cavalier on privacy when they launched Windows Recall.

Lots of things to say on that, but let's say they launched a product called Windows Recall, which looked very familiar. But they took a very, very cavalier approach to privacy, and that really hurt them. on the flip side, Apple intelligence seems to respect privacy, but it actually is limiting what Apple can actually do in terms of usefulness. So they're kind of on this both sides of this paradox. We've actually made many mistakes in this space. I think the most recent evolution of our thinking has really set us up well for the future. But it's not obvious. It's not straightforward. It's one of those things you have to kind of, you're like tightroping between landmines. You know, you're trying to find the right path that respects privacy and doesn't make it a choice between privacy and the same time makes the product useful enough because you're able to use the data. and a way to offer personalized AI.

I agree with that. But then you have to put that in front of users, right? And I think even there's something about the idea of just like, here is an app that shows a timeline of every web page I've ever been to. Feels instinctively weird to some people, right? And I think it's been very funny watching a lot of this because, like, yeah, of course your browser knows all the web pages you've been to. Like, that's your web, like, yes, that's how it works.

But I think a lot of what has been happening in these recent months is, people are slowly starting to understand kind of how much awareness their technology has of what they're doing with it in a way that everybody probably should have had before but didn't. And I feel like what you did, especially with Rewin in the early days, is like really speed that process up. You're like, this app knows everything and it's actually its job to know everything. And I wonder if to some extent it helps because it's like a thing you have to download. So by definition, you're going to get people who are more comfortable with it rather than building it into the operating system.

But you have to do this thing, I would think, right away where you're like, okay, this is asking a lot. It's going to know a lot about you. I have to sort of immediately telegraph to you why it's worth it. And I feel like that's a pretty big hurdle to clear right away. Yeah, I think we certainly paved the way there, but I wouldn't say we didn't make any mistakes. I think showing a timeline of everything you've seen is interesting. It's a cool party trick.

Like it helps create, you know, this magical moment of like, wow, I didn't realize. I could do it, and people love that, and that's partly why we named the first product in this space, rewind. But if I really had to be honest, like, people don't care about technology. They just care about their problems being solved. The technology is the means to the end. And this was too much of a cool, let's see what the technology can do part of the product, unless here's the problem we're trying to solve for you. So I do think the right user experience around personalized AI is to be very opinionated on the use cases. What are the problems in your life every day that we're trying to solve for you? How do we give you time back. How do we make it so that you're in time for dinner with your young kids at

night? Those are the kinds of problems people care about. The technology and how sophisticated is, they could care less. Like that data, you know, the architecture, you know, that is important to us because it's how we build the product. But to a user, that's just implementation detail. So I do think limitless does a much better job of this. This is why we've kind of evolved and even partly reason that rename the product, because the core experience of rewinding time isn't the core thing people want. The job to be done is give me back more time, make me me more productive. Take things off my plate. What are things I do every day that, frankly, I just don't need to do. And a machine can do a better job of, be more reliable, and just give me

back more time so I can do the things that I'm uniquely well-suited to do. So those are the kinds of things and the kinds of experiences that I think will ultimately win in this world of personalized AI. What are some of those things that you identified early on as solutions to those problems that you're describing? Yeah, I mean, big one is a blank piece of paper. Very often as knowledge workers, You start with a blank piece of paper. Maybe it's an article you're writing or an email you're sending, or even a simple text message you're sending to somebody. You know, writer's block is a big version of this problem, but it's this idea that starting from zero is much harder. And part of what a machine can do uniquely well is capture the context you might want when you're starting from zero.

A good example is drafting emails. You know, I gave an example of drafting email, Sam Olman. You know, starting from scratch is a much, you know, why spend all that time and energy when a machine can surface to you perfectly the thing you might need and want? A way you can think about is auto-complete for your life. Why so many times you have to start with a MT line and where a machine can provide an option? It doesn't do it autonomously. It just provides a draft, something you can edit and tweak and change and delete.

That's like a perfect win between, you know, like I said before, horse and buggy and self-driving cars. You know, there's going to be a day when AI can do things autonomously. We trust it. It can book you a trip for you and your wife to Italy in three months and you know it'll put the right seat and everything. Right now, the use cases I think the AI is well suited for are these semi-autonomous use cases, things like drafting notes.

So that's just one. I mean, there's many, many others. Is that memory, though? It is in the sense that the things that the context that's useful for those moments are memories. They're things from your past. There are details around your last conversation. If you think about this idea that we forget 90% of what happens after a week,

you have a weekly team meeting. Many folks probably listen to this have a weekly team meeting with their team at work. Maybe it's an hour-long meeting. At best, the people remember six minutes of that last meeting. So, you know, simple things like following up with a context of what was said in that decision in the last meeting, the set of decisions, those kinds of use cases. is meeting summaries, preparing you for meetings, live notes during meetings.

All of these things are incredibly well. Great sort of use cases that personalize AI can help you solve. It seems to me there's almost two different things going on there. Because on the one hand, there's the thing that's like, okay, I'm going to make it easier for you to remember at least all the important bits of your one hour long meeting last week, right? So that when you go to your next meeting, you can very quickly call up all the things you talked about last week.

That is, I feel like I think a lot of AI companies are pursuing, right? This like take a bunch of notes. or we'll take the notes for you, and then we're going to give you sort of quick recall of those notes. But I feel like you're also describing kind of a full step beyond that, which is just instead of helping you sort of actively remember something, we're going to use all the things that you forgot to help you do new things. Yes. Those feel like two different things to me.

They are two different things, but the analogy I'll draw is go back to GPS or go back to phone numbers. Like remembering somebody's phone number was the means to the end of a conversation with that person. If you don't have to remember the phone number and just go straight to the conversation, that's a win. Understanding cross streets, that was the means to the end

of getting to someplace else. If you can just get to that other place without having to think about it, that's the means of the end. When people think about, when we're solving memory and trying to help you capture memories, it's not about the sheer fact

that you have those memories and it's a nice thing to clutch onto. It's the goal of trying to solve a problem for you, like drafting you an email to somebody using the context of your relationship. So those are all the means to the end. The ends are the things we focus on. What are the problems we can solve for you?

What are things we can take off your plate that AI can do well? and the data and the capture that we capture in your memories is the means to that end. Do you think there's value in the means there? I think, you know, you mentioned people don't actually spend a lot of time like scrubbing back through their old stuff. But I think about like the whole sort of journaling community, right? There's this real belief that like having these artifacts and reviewing them periodically and having them come back to you is there's real value in just the process of revisiting the past. Do you think that matters in this context?

Yeah, I do. I do. I think that's just one of the many use cases. I think many people can get value from limitless without doing any of that. And then others will also get additional value. A great example is getting insights into your life. You know, when during the day are you the most excited? You know, who in your life gives you energy? Who in your life drains energy? Looking back at the conversations you have with people or is a great way to answer those questions. You know, how many times are you interrupting people? How many filler words are using? When are you using them? You know, all these things are sort of introspective quantitative self. You can think about, you know, eight sleep that sort of introduces a whole concept of sleep fitness, but there's a whole set of that, which is kind of mind fitness. Like, how do I show up in the world through my conversations I can get a window into and how do I improve and how to get better? So I absolutely believe in that. I don't know if it means you have to literally read every word. I think there's things again, AI can do well there in that space to help you reflect and improve

based off of your past. But that's all just one example of all the things we lose the moment we forget something. Ultimately, in talking to Dan and others about this stuff, it seems like there two huge challenges to solve with any kind of AI memory product. The first one is the AI one, which I actually thought would be the hard part, but am increasingly convinced is the easy part. It's just how do you figure out what's actually relevant in all the data people are collecting? Like, I sit down at my computer, I click 75 links, I watch 20 TikToks, I send 300 Slack messages, and I have four meetings. How does rewind or recall or anything else make sense of that for me? Most of the folks in this space don't actually think that's really a challenge, or at least won't be forever.

It's a big computation expense, but it's not an unsolvable technological problem. We have to take one more break, and then we'll get into the tech that makes that work and the much harder problem. We'll be right back. Support for the show comes from Upwork. The days of doing it all, all by yourself, are over. There's no romance and burning out while you're trying to scale. Instead, you can check out Upwork. Upwork helps grow your business by giving you fast access to specialize talent across more than 125 categories, so you can fill skill gaps, launch projects faster, and scale without committing to full-time headcount.

And finding the right talent is easy. You can browse profiles, review past work, and get help scoping the role so you can get started quickly. Seriously, you could connect with the right freelancer in just a few hours, especially. especially when you sign up with Business Plus. Their AI powered shortlisting pairs you with the top 1% of talent and under six hours. No endless searcher required. You can visit upwork.com right now to post your job for free. That's Upwork.com to connect with top talent ready to help your business grow.

That's UPWRK.com. Upwork.com. All right, we're back. So the whole makes sense of this giant database of stuff that you, you do on your computer problem that rewind, recall, and others have? The solution I keep hearing about is a technique called retrieval augmented generation, which everybody just calls RAG. Basically, in a RAG system, you're applying an AI model only to a set of data that you've

selected, that you know is good and relevant. That means the only information the model can access to answer your question is inside of that data set. Like, if you want to ask questions about a book, you don't want to ask questions about the book to the whole internet, you just want to ask it about the book. And just by giving it the book, that's RAG. There's some evidence that using RAG can make AI models less likely to find incorrect answers or make up new ones. And it's particularly useful, I think, right now in a business context where, like, a model can access your company's internal wiki and nothing else.

So it's more likely to find the right answer. It's a little harder to apply that idea to all the many varied things you do on a computer every day, but it's still a big step in the right direction. The much harder problem is getting all that data into the system in the first place. Screenshots and audio are a decent approximation of what you do on your computer all day, but what about when you're on your phone or in the car or watching TV or out in the real world, like where other humans are and not interacting with screens at all? If you buy the idea that more data is good and that the more you can collect, the more useful your AI tool can be, you need access to everything that a human sees,

touches, hears, tastes all their internal biometrics, and a million other things besides. We're not getting that technology anytime soon, no matter what anyone in tech will tell you. It's just not happening. And that's why most of these products are business-focused right now, by the way. They're designed to help you remember what happened in a meeting, which is a finite and containable thing that you can record and summarize,

and it has a beginning and an ending and outcomes. We're a long way away from applying these ideas to the rest of your life and the rest of your memory. The question in the meantime, though, is how much, much these tools need in order to be useful at all? Like, we can't do the whole thing, but how do you do the next thing? The Apple feature I mentioned up top is actually a good example here. It's not like your iPhone is automatically taking pictures every time it senses something interesting happening. It's just trying to take pictures that you already have and put

them together in the ways that you want. That's technically doable and still really useful. So how else can these tools do just a little more like that? That question is what led Dan and his team to rebrand their company from Rewind to Limitless and to build a gadget in addition to their app. I think in particular conversations are the ones we found has been hugely valuable where there's almost nothing being captured today. It's a very human thing. It is so much low-hanging fruit in terms of the value we can create, especially in-person

conversations. That's a big reason we built the pendant is to capture more than just Zoom meetings, but in-person conversations as well. The pendant he's talking about is called the Limitless Pendant, and it's a $99 round clip that you can either wear on a lanyard or attached to your clothes or backpack. It kind of looks like an old school fit bit to me. Whenever you activate the pendant, it starts recording audio. And for limitless, the hope is that by capturing more of what you say and hear, you'll capture more of everything else, too. I think in particular, the pendant is going to be amazing for relationships.

You know, when I first conceived of this idea of something you'd wear that might capture conversations, I was terrified to even mention it to my wife because I thought, oh, she's going to hate it, because now finally Dan's going to win an argument. And now the truth was the opposite. She said, oh, perfect. Now we have clarity on what was said and what wasn't. And a lot of conflict comes from miscommunication, from misunderstanding, misremembering.

So I actually think from like that from a single player perspective and maybe two player perspective, one-on-one relationships, spousal relationships are actually going to be better. Actually, it sounds weird. I have a couple friends who do this thing that when I describe what our product does, they mention this. I think a lot of people do this, they don't admit it because it sounds weird. What they do is at least two of my friends that I know,

of when they are about to get into a fight with their partner, they say, okay, let's start recording. They put out the phone and they start with a voice memo, they record the conversation because they know that things are going to get heated. They know they want that may need to go back to it. And it's, and it has been a gift, not a curse to the relationship to have kind of this impeccable memory of what was said. And I think those are the kinds of things that once people feel it, it's like, oh, actually,

you know what? You see somebody better. When in your mind, you were so sure you're right. Like, no, you did not tell me to get eggs before I went to go to Trader Joe's. Oh, you did. And I just, I was in my mind somewhere else. It sort of creates a sense of truth and authenticity and a connection in the world that I think today, because we forget so much, we don't know what we forget, we have conflict needlessly. That's such an interesting example.

And I love that because there is immediately part of me that just recoils at the idea of my wife being like, hey, I'm going to start recording as soon as I start being an ass. But then I can also totally see the point of that. And this goes back to that same sort of philosophical question of like, how much are we supposed to remember, right? And like, yeah, it would make my life easier if I had a pendant on that was like, yes, she did tell you to go get eggs. But actually just reminded me to get eggs while I was out. Like that's where we go, right? That's where we actually get to something useful is I don't forget the eggs anymore. Totally agree.

And by the way, I think that this is a product that is probably better suited for people who just want to do better and be better. Not everyone wants that. Some people just they want to watch football. They want to be left alone. You know, and they don't have this idea that their life could be better. But I actually think most people on this planet live their lives as zombies. They go through the routines and habits of their day. They never really ask themselves what they could be doing better for themselves, for their partners,

for their families, for their job. And this is a tool to give you a window into your past in a way that helps you do better, be better, show up better, and do more. And I think there's some people for whom that's what they want. And there's not. That's okay. Wait, let me put that slightly more charitably to some of those people, which is I think most people exist in both of those states some of the time.

Right. And I think part of what is interesting about products like this is, I think about it the same way I think about like AR glasses, right? Sometimes super useful and I want more information in my face and I want to know where I'm going and I want to know all the information about all the coffee shops around me. Sometimes that sounds awful. And the idea of having something on my face all the time that is showing me that stuff whether I like it or not is a problem even though sometimes it's useful. And I feel like with something like this too that is that kind of knows me and sees me and is recording everything and is giving it back to me. there are times I'm going to want that and get value from it and feel like it's useful. And there are times where that's going to feel intrusive to me potentially. And I guess what I wonder, both for you as a product maker and just for kind of us as people, is like, is this the kind of thing that we're either going to learn to live with being there all the time and we'll just ignore it? Or can these things be kind of episodic and we can use them as tools rather than these kind of ambient always on always aware things? Yeah, I think people will decide and choose that path on their own based off of what value they get. I'm going to wear the pendant where most place I go, but sometimes I'm going to take it off and put in my pocket.

And that's okay. We've designed it to work that way. You will turn off when you put it in your pocket. It won't, you know. And so like that's, I think, important. You give people control and choice and they will choose for themselves when it's valuable, when it's not. With the recognition that there's sometimes you may not know, it might be valuable later. You know, it's a conversation with a friend.

They're like, actually, they're going to give me some good advice. You mind if I wear this pendant just so I can capture it. I really want to remember what you're going to say. And I think those things will change slowly over time. And people will, through their behavior and through the tradeoffs they make, we're not forcing anyone to use this without wanting to use it. I think the ability to use and capture more of their life is a choice, just like wearing glasses of the morning is a choice.

If you want to see better for the day, you can. If for whatever reason you don't want to see the world, you don't have to. I think most people will wear glasses most of the time. They like that. They'll realize that's what they want to do, is they want to see the world for what it is, not the blurry version that gets mangled up because they're lent. aren't quite sharp. I'm sure you've noticed this by now, but Dan is 100% committed to the big memory AI theory here. And in the last couple of years, even really the last couple of months,

it has started to seem like the rest of the tech industry has bought in just as aggressively. A bunch of the biggest companies on Earth are now building products that sound an awful lot like rewind and limitless. And when I asked Dan about all this new stuff happening, he said what all CEOs say, which is what I guess what you're supposed to say, which is that, It's validating to have competition, and he's not worried, and limitless has the right business model, and imitation is the series' form of flattery, all that good stuff. But what I found myself really wondering about all this was whether all of Microsoft's problems with recall, which specifically, if you don't remember, was the fact that it was

keeping all of your data basically unprotected on your computer, which is just an incredible security disaster waiting to happen. If that might make people even more wary of a product that wants to store all of your data from everything forever. And he said he actually thinks it's a win for limitless. We don't have to spend as much time or energy evangelizing this concept and this problem. You know, if Sachi and Adele will do that and use my talking points, all for it. That's great. That's great free marketing for me. And ideally they do it and they stumble. And then people think, actually, that was a cool idea. But I really don't like how they implemented it.

Is there something else out there? And that just creates a bigger market for me. So honestly, I feel validated. I feel seen. I feel like it's a party in the desert. And now we're planting some trees and soon this will be a rainforest. I've heard from Dan and others that there's probably a bunch of billion dollar businesses to be built on AI memories, things that can collect and then make use of all of the data associated with your life. I think he's probably right. And I think you'll see many more companies try and convince you to store everything about yourself in their worlds. It's going to get weird and everybody should tread pretty carefully. But through this all, I'm still thinking about the idea of getting in a fight with my wife and starting a recording on my phone

as soon as things get heated. Or sitting down at lunch with a friend and asking them if I can record in case they say something memorable. Sure, those things might make it easier to remember things later. And while I could take notes or something at the end of the meal, the real value might genuinely come from being able to recall something I didn't think was important. But suddenly days or weeks later, I realize actually matters a lot. But would recording those things change those things? Would lunch with my friend or a fight with my wife be different with their recorder on? Knowing that they would be preserved forever and for me to access for action items or whatever else,

does that change the thing as it's happening? I think it does, even though I can't always explain exactly how. Maybe Dan and everyone else are right that in a few years or a few decades it won't feel different and we'll all be used to it just like we're used to phone cameras being totally ubiquitous now. But I can't help but feel like maybe having superpowers runs, the risk of making us a little less human in the process. Wait, is that what all the Marvel movies are actually about having superpowers and being human at the same time? Anyway, sorry, that's a whole other podcast. Moving on. I'm fascinated by tools like Limitless and Recall, and I think they're

going to be really useful to a lot of people. I'm going to use all of them, frankly, but I'm also increasingly excited about the less ambitious versions of these apps, where you get to decide what's important, but still let AI make sense of it. Like, I use this app called My Mind, and it's basically just a repository of things that I like. That's how I use it. A podcast episode, a TV show, a funny gift that I find, an article, a photo I take, anything that I like, I just save to my mind, and the app automatically categorizes them with AI so that I can then search for sad movies or articles about sports or things that are read,

and it'll show me stuff that I like. I've also really loved the new Photos app in iOS 18, which really emphasizes those automatically generated albums of people or things that you did or just particularly interesting days you've had recently. Google, similarly, recently announced a feature, I think, is super clever, where you just create a giant repository of screenshots that Google's AI will look through and try to make sense of for you. In all those cases, all you have to do is basically signal, this is something I'd like to

remember. This matters to me. and then the AI model does the rest. I feel like that's going to be a big win for my memories. I won't be able to see the pictures in my head, but at least I'll be able to find them again. That feels close enough. All right, that's it for the Vergecast today.

Thanks to Dan for chatting with me, and thank you, as always, for listening. As I said up top, this is the first episode in our three-part series on AI, so make sure you come back next Sunday for the next installment. It's very different. It's a fun one. This show is produced by Andrew Marino, Liam James, and Willpore.

The Vergecast is a Verge production, and part of the Vox Media podcast number. We'll be back on Tuesday and Friday. We've got lots of gadget news to talk about. We'll see you then. Rock and roll.