The Vergecast - How AI really works, and how the smart home broke

Episode Date: June 21, 2023

Today on the flagship podcast of open-source lightbulbs:  David Pierce chats with Verge investigations editor Josh Dzieza about his story detailing how humans matter far more to AI development than we may have thought. Inside the AI Factory: the humans that make tech seem human Later, smart home reviewer Jennifer Pattison Tuohy explains why we're probably getting the idea of a “smart home” all wrong. Smart homes for smart people How microgrids and smart homes are shaping our energy-independent future Every device that works with Matter What is a smart home, and do you need one? How to pick a smart home platform From brilliant to basic, here are our smart home setups Email us at [e-mail] or call us at 866-VERGE11, we love hearing from you. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Welcome to the Vergecast, the flagship podcast of open source light bulbs. I'm your friend David Pierce, and I am coming to you from my laundry room slash furnace room slash, I don't know what you call it. It's just that weird room in your house where you keep all your tools and toilet paper and laundry detergent and stuff. Anyway, I'm in here because, well, last week was smart home week on Theverge.com, and I was inspired to try and connect all of my stuff a little better together. So I'm trying to get a water center, up to tell me if my hot water heater is leaking, which sounds simple and is frankly exactly as boring as it sounds, but it will make me sleep better at night. Anyway, we have much better

things to talk about on the show today than water sensors. Josh Jezza is going to come on and tell us a very different kind of AI story than the ones we've been hearing over the last few months. And then Jen Patizantooey is going to come on and we're going to talk about smart stuff. And whether we've been approaching the whole idea of the smart home all wrong. Actually, come to think of it, we might end up talking about water sensors after all. So sorry in advance if that happens. All that is coming in just a sec, but first I have to see if I can shimmy behind the water heater over here to get this thing installed. So if you don't hear from me again, I'm probably stuck back here. This is the Vergecast. See in a sec.

Support for the show comes from Retool. Too many companies run critical operations on duct taped spreadsheets, Slack workflows, and whatever else they could cobble together. Not because they want to, but because building internal tools means weeks of waiting on someone else's backlog. That's where Retool comes in. Build custom internal tools just by describing what you need. Prompts something like, build me a revenue dashboard on our Salesforce data. And Retool actually builds it on your company's data, in your cloud, with Enterprise Security built in. Go to Retool.com slash Verchcast. We all need to retool how we build software. What's up, y'all? I'm Skyler Diggins, seven-time WMBA All-Star, Olympic gold medalist, and mom.

And I'm Cassidy Hubbard, host and reporter for nearly 20 years covering the biggest names and stories in sports and mom. And this is Am Mom, a community for athletes, game changers, and moms of all kinds. Dropping May 14th. Tap in with us. Welcome back. I made it. Everything's okay. Obviously, AI is the biggest story in tech right now. in the chat GPT, chatbot sense, and just in the sense that AI tech is suddenly showing up in every product and service we use to help us write, help us find things, help us make things, and

lots more. And we kind of understand how all the underlying tech there works, right? You feed a bunch of data into a large language model. It automatically black boxy trains itself to understand all that data, and you're done. But of course, it's not that simple. And it turns out it's not that automated either. The Verges Josh Jezza has spent the last few months looking into one particularly misunderstood part of the process, where humans matter far more to AI development than you might think. He even got some firsthand experience in the AI process.

So we brought him in to talk about it. Hey, Josh. Hey. So there's a lot to get to in this story. And I want to spend a lot of time talking about the part where you became a data annotator because I find that deeply fascinating. But I think just to like set the scene a little bit, we need to do a little bit of, I think, term defining here to make some of this make sense. We go back to like 2007, which is kind of the earliest date in your story at Princeton with a researcher named Fei Fei Lee. Tell me what's going on kind of there and then and why this is sort of the moment that starts what leads to your story.

Yeah. So this is really the moment where deep neural networks and machine learning really comes back. It was sort of a technique of machine learning that had been languishing a little bit. And then you have these fast processors. And then Fayfayle just realizes that you need way more data than anyone's been training anything on to get image recognition, which is really sort of the test bed of this at the time. So using millions of images instead of tens of thousands. But it was really impossible to label or categorize that many images. You need, you know, there's this unprecedented corpus of imagery now of the internet. But you don't no longer have grad students kind of like.

like page through encyclopedias or whatever and scan images, but you still need someone to go through and say, you know, cat, airplane, automobile, whatever, and sort these images into something that, you know, this neural network can suss out the patterns in and begin to identify stuff. So she creates this database called ImageNet using mechanical Turk, which was sort of recent at the time,

the Amazon work platform where you can have people from around the world do microtasks for pennies usually. And, you know, it worked and the machine learning revolution kicked off. People realized that this is actually super effective if you have enough data and enough annotated data to train these models on. The phrase annotated data is like the one that kept jumping out to me because I think the way that we understand machine learning so much is basically like develop a model, just throw the Internet's data at it, whatever you can find, like scrape Google images, scrape Reddits, whatever you can find, throw it at it, magical step three sort of black box nonsense and out of it comes AI. And I think the thing that really jumped out to me through your story is not just this realization that what we need is a huge amount of data, right, which I think now 16 years later is is relatively well known, right? Like,

to make these things great, you need outrageous amounts of training data. What I was surprised by is the amount of human interaction that there kind of was and still is in this process, that Not only is it like a handful of people at the beginning saying, this is what a cat looks like, this is what a dog looks like, this is what an airplane looks like, that this is like a massive, ongoing, never-ending process. And it seems like that was kind of the like insight that sent you down this road to some extent, that like this is a bigger and more human task than anyone realizes even now. Yeah.

And there's, there's two levels to that. So there's the sort of model creation training aspect that's super manual. And there is sort of a first round where you just kind of throw all your data in there and it finds its own patterns, the sort of unsupervised learning process. But then you have to go through and, you know, there's different methods, but you need humans to either sort of pick the best examples of what the model does and then sort of re-train it on that or, you know, write your own sort of like, here's the gold standard essay or, you know, Python program or something and then train it on that. and then kind of do more rounds of feedback. And so that's sort of state-to-the-art for the language models you see now. But then you have this other element which is like, all these programs are super brittle. You know, just because they can do one thing well,

doesn't mean they can do something else that seems quite similar well at all. You know, whenever they encounter something that's not in their dataset, they struggle. And so you have kind of this other side that I had not really realized existed of people who are just kind of, you know, minding automated programs that are out in the world. you know, they're like sorting through credit card data and saying like this came from 7-11 or whatever, their, you know, self-driving car gets stuck. And everyone's like, oh, yeah, we really need to, you know, get some more bird annotations or something. Like the sort of the world is changing and things are always encountering data that's not in their training data. And so they, they fall apart.

And you always need kind of a human there to catch them. Okay. So this thing, this, this sort of insight from 2007 birth to this, like, gigantic industry. And I want to talk about kind of the industry at a higher level and then sort of the specific job of what the people who work for these companies do. But the industry is totally fascinating to me because you uncovered this like gigantic billion dollar sort of sketchy undercover world of data training. I don't know. There were moments where it was like, oh, this is a big sort of booming industry. And then there were moments where it was like, this sounds like the way that sketchy billionaires try to hide their money by like setting up shell corporations all over the world. This industry does

not want you to know about it, but you learned a lot. Like, sort of paint me the picture of what this training industry looks like now. Yeah, it's really weird. So there's a, you know, a bunch of different super opaque elements to it. One component, it looks sort of like an outsourcing company, like a call center. You go to some vendor, you know, usually in the global south and they have a lot of people in an office labeling data. But then increasingly you have these things that are kind of, I think of them as like the sort of next generation of mechanical Turk. Like mechanical Turk is sort of a lot of work. to get anything out of, like, you need to go in there and, like, post a task and sort through a lot of garbage. You have these other companies, and sort of one of the ones I focus

on is called Scale AI, and their whole thing is, like, we have well-trained people. They're really experts in this stuff, and they are also being assessed by, you know, automated systems that are testing, you're doing sort of quality assurance and things like that, and they're trained. And it's also a platform like Mechanical Turk that, you know, anyone can cite. up for, but like, it's a job. You know, you, you'd spend hours sort of learning each new task and you can get fired or, you know, banned or whatever, the algorithmic platform equivalent of fire is. And that's sort of what you have now. It's so huge numbers of people, like hundreds of thousands of people who are doing this work. And because you don't want your in-development AI system leaking out, they're sort of these nested layers of opacity, you know, code names, scales, platform is called Remo Tasks. There's no sort of public disclosure that they're related. And then every sort of project on Remo Tasks is it also codenamed with sort of weird stuff like Pillbox Bratwurst or whatever. So no one knows what they're working on. It's all quite murky.

We're going to come back to Remo Tasks, by the way. I'm very excited to talk about your life in Remo Tasks. But the thing that kept jumping out to me is that it seems odd to me that this industry is so secretive because on the one hand, it sounds a lot like kind of the way content moderation works, right? Where it's, the job is a lot of sort of menial drudgery in a lot of ways. It is, it is sort of repetitive tasks and folks outsource that. But I think that's in part because content moderation is viewed by a lot of these companies as not that important. It's like a thing you have to do, but they're happy to have it done sort of serviceably at the cheapest way possible. But in this moment of AI, like this is the game now. These companies are raising gigantic amounts of money. They're

spending a ton to train these things, you would think that they would want to have as much quality control, as much privacy and secretive control over all of it as possible. So I guess it's weird to me that it's this big secretive industry that is so mission critical to what everybody says is the future. Like, why is it this third-party subcontracting world instead of like Google hiring a team of 10,000 people to do it for Google and Microsoft for Microsoft? I think this is changing a lot. little bit, but I do think there still is this perception that it's not that important, that like, well, we just, you know, AI is improving so quickly where we just need to get a bunch of data and then it's going to be able to do this and we can automate it and we won't need this anymore. So why

put a bunch of people in your payroll permanently for this? So it's viewed as like the, the human element is like a stop along the way where like we won't, yeah, I think that's wrong and I want to talk more about that in a minute. But the job of these annotators, you talk to a bunch of people who actually like sit down at a computer and do this kind of training work for a living. Like, what does the job look like? What do you do all day when you do this job? There's a lot of variation. So the image recognition stuff is very, like, this is, I think, the thing people are most familiar

with. It's like the self-driving car stuff. You get, like, a bunch of capture-looking things, and you outline the fire hydrants or whatever, or, like, a bunch of lightar, laser points, and you sort of say, like, this is the human, and that's a human there. That is kind of the classic stuff. And then the chatbot stuff is more, you're in this kind of weird sort of text edit kind of interface a lot of the time. And you have a bunch of things you're supposed to be reading for or something.

And you're given, honestly, it feels like a standardized test. It's like you have like an essay and you have like two attempts to summarize it and you like critique which one is better or write your own version of it. Or like, here's some marketing copy. Did this include all the information in the product? or did it make up a bunch of stuff, things like that. It's pretty repetitive, but it is also really hard. Yeah, what is hard about it? Like, I understand why it's repetitive.

You tell a story of a guy who has to go basically, like you're talking about the self-driving example, frame by frame, which just, it just sounds brutal. And it's like the fire hydrant was here in the last frame and it's here in this frame, but I have to do it because that's how you train the thing and it has to learn all the edge cases and so on. But I did get the sense that this is a like genuinely sort of difficult job. What is it about it that people find difficult? You have to be really precise.

So, like, that kind of light art task, you have to go through every frame from sort of every perception mode for a car and label it and say, this is a vehicle, it's moving, it's this type of vehicle, and sort of do that for each thing. And so you have, like, a few seconds of footage that ends up taking a, you know, eight-hour day to fully annotate. I mean, you have to be really precise. Like, I kept failing a task where you have to outline pallets for, like, a, autonomous forklift or something.

And it was just like the quality you're supposed to get was down to the pixel level. So it's like if I got one pixel off, I would fail. And it's this blurry photo in a dark warehouse. And it's like, I flunked it. Wow. The other element is just the things that you're doing are really alien. Like the way these machines process information is not human. And so you're having to do these things like categorized clothing, but also label all the clothing in mirrors because to a

AI system, it's the same thing. And then what is clothing? You know, are rollerblade shoes, like these sort of weird questions that you get into. Right. And like the engineers, they don't even know that they need this to make these distinctions, but then they, you know, put the data in and their model doesn't work. And they come back and say, like, oh, yeah, don't label rollerblades, but do label flippers or something like that. So you just get these kind of incredible, you know, 40-page instruction manuals that you have to follow. Right. So yeah. And this is where we get to. your specific adventures in this world. You, if I'm remembering correctly from your story, decided at one point in the reporting process to sign up for Remo tasks, right? Tell me about the

process. How did you get into this world? Yeah. So I was kind of beating my head against the wall. It's a hard world to get into because it's so opaque. You know, you can't go to an office. You can't sort of figure out people don't put it on their resume on LinkedIn very often. And so it was kind of the only way to figure out who was doing the work and what the work was like was to join the platform. And it's totally open. So this seemed like the obvious way to go. And one of the quirks of the system is you're only brought into a group on remote tasks of workers if you pass like a training program. Like you need to pass the test and then you're brought into a Slack channel with people.

And so it was like this video game where I kept having to pass these training programs. And I kept failing because they're really like, I don't know, you see a thing and it's like, leave all the clothes and the social media posts. And it seems self-explanatory. And so I would sort of click through the instructions, fail completely, have to go back and be like, there's like dozens of pages of distinctions that you need to follow. You need to keep it up on like a second monitor and always be checking it because it's just incredibly precise and these sorts of weird. Give me an example. So like I'm imagining I can think of a social media photo. and it would say, circle all the clothes.

I would see somebody's wearing pants and a shirt and a pair of shoes. I would circle all those things as exactly as I could and then just move on with my life. That doesn't sound that hard. Like, what were the problems you kept running into here? So first off was what is the definition of real here? Like the instructions are like, only circle the real clothes that can be worn by real people. And one of the first things I got was just like a magazine on a, like a photo of a magazine with some clothes on. I was like, well, that's not real.

That's a magazine. not correct. Like, those are real images of real people in a magazine. And so you circle them. And the same goes for mirrors or whatever else, because, like, the AI, it's all just pixels. It doesn't know the difference between a mirror and a magazine. And so that was kind of the first thing was sort of that you need to be looking at,

you know, someone's in the background and you don't think it's the thing that's relevant, but, you know, they're wearing a hat and you need to circle that. And then it was weird stuff where it would be like, don't label clothes on action figures, but do label them on mannequins or label costumes, but don't label armor. And so what is, you know, someone is dressed up in like a stormtrooper armor? Is that a costume or is that armor? Like these sort of weird things where people do these distinctions and you can't keep them all in your head. And so you just sort of look at through it.

I have no idea sort of what side of the line that falls on. And then you guess and then you fail out of it. So, and I guess the assumption then would be that this AI is trying to learn to make the same distinctions that you're learning how to make. And essentially, you're just trying to train it over and over. You're doing all of the thinking so that the machine can just learn what the answers are, basically. Yeah, I never really figured out what that one was about, but, you know, I could guess.

And there was a lot of e-commerce stuff on there, and it all sort of seemed geared around the same thing to be able to, you know, comb Instagram and say this kind of, you know, like this shirt is, you know, spiking in popularity or something like that. Or like this person is posting about this sort of, I don't know, clothing, like maybe we should market this to them. Like that sort of, it seemed like it was geared toward being able to make those kinds of calls. And so you had to teach it, you know, how to recognize a shirt in a sort of a social media environment, which turns out to be incredibly difficult.

Yeah. So you eventually passed clothes. Congratulations. Thank you. What did you then get thrust into? What was your first big project after that? So, yeah, I did a little bit of it. And then I, you know, failed pretty quickly.

And I think it was, I started getting some people in mirrors or like in windows, you know, reflected in building windows or something like that. And that was something you were supposed to label, but not if it was too blurry. And so I flunked out of it. And then it kicked me to something, I think it was called crab generation. It was like, it's quite humiliating. You know, you just get this pop up and there's like a cartoon crab that's like, your low quality has

booted you from this, but now you can train for this new task. And that was more of a chatbot thing. And that was sort of just like seeing whether it was paraphrasing news articles without making things up, basically. But, you know, it's quite eclectic. There's all kinds of stuff on there.

There was one where it was just sort of videos of people talking about their jobs. And I was supposed to rate whether they seem like emotionally stable, I think, was the criteria, one of the criteria, and sort of how outgoing they were. where no idea what that was about.

Whoa. It's just all over the map. So, okay, this makes me think two things. One, this is an enormously hard job to get good at and the kind of job that maybe isn't going to, like, last forever. Like, you train on this closed thing, you do it for a minute, and then all of that knowledge is useless to you for the rest of your life, which seems like a wild way to use that kind of tool and time. Memorize these, you know, 40 pages of rules just to look at pictures first.

an hour and then just kidding, we're going to fail you anyway. And then on the other side, that this stuff is so, we're trying to teach computers how to think like humans when maybe that's just impossible, that like the idea of quantifying what outgoing is, like I can't do that. I don't think you can do that if you figure out how to quantify outgoing. Yes, congratulations. But so it just makes me think there's this like fundamental disconnect between what people are being asked to do, which is basically teach computers how to think like humans. But then in order to do that, you as a human have to think like a robot thinking like a human. And it's just, I don't know how anybody would do this well. It seems insane. It is really kind of crazy making. And you start to question, you know,

how do I know things? I've been making all these calls about, you know, what clothes are all my life. But do I really know. And you do have to kind of think like a robot. Like the trick is sort of being so literal, just sort of, you know, way more literal than any human could be and operate in the world. But in terms of the unpredictability and sort of the stop and go nature of it, that was the big complaint. I heard from people, you know, there were complaints about the pay, like the pay can be quite low. But even more than that, people would complain about, you know, spending a day learning a task, doing a dozen of them and then having it end and have to go learn a new task. Or there's a bunch of work and it pays pretty well and you start doing it. And then like a week

later it's gone and there's nothing for weeks and you don't know, should I go find another job or what's happening here? It's quite stop and go, which, you know, talking to engineers and vendors, it has to do with the way machine learning development works. Like, you need a ton of data to train your model the first time and you want it really fast. And so you sort of get as many people working on it as you can. And then it's done for a little bit. And then you need to find, you know, something comes up. You need to sort of retrain it on something. You need like a smaller, more specialized group of people, it's very kind of peaky. And that means people are, you know, you always have to be on the lookout for like a new task to drop and then you have to learn it.

And then maybe that's going to be totally wasted because, you know, work for a day. And then the AI gets it. And then it's on to whatever else. Yeah, that's tough. What was the general sense you got from the folks you talk to about whether this is a good job? Obviously, like you're talking about the variability is tough. But did the folks you talk to, did they like the work? Was the, was it better? than other things you could do kind of sitting in a computer doing these kinds of tasks all day? Like, what was the vibe you got? It varied a lot.

You know, I talked to a lot of people where a lot of this work gets done and also where I happen to just sort of get placed in my remote task experience. And when remitask started out there in like 2018, 2019, it was a good job. Like, it paid pretty well for the area that people were making about $10 an hour. But then, you know, more people joined the place. platform. They started adjusting pay based on regional cost of living and then it dropped to like a dollar to three an hour. And even then, you know, people stopped thinking as a career, but it was kind of a backup. It was my sense of it. Like, this is better than nothing until I can find something else to do.

But it was still so stop and go that they felt jerked around by it. It's a full-time job for a little bit, but like you can't depend on it and then it's gone or then I'm banned for mysterious reasons. And then I did talk to people who, especially doing sort of the newer language model stuff, who enjoyed it even. Like a lot of this work is coming to the U.S. because you need English fluent people and people with certain cultural references and what have you or certain expertise. And, you know, they're making $20, $30, $40 an hour. And a lot of them, it's like one of these, you know, work from home thing. They let they enjoy it while they have it. No one knows how long it will last.

I think the people who are getting paid the best, they wanted it to be a full-time. job or dependable job, but like they don't know who they work for or, you know, when they're going to be done collecting data for it. So they're kind of just like, as long as the tasks are running, I'm going to do them. All right. We need to take a break and we will be back with more from Josh on humans and AI. Support for this show comes from Shopify. Starting something new isn't just hard. It can be really scary too. So much work goes into this thing that you're not entirely sure will even work. But here's a better thought. did all work? What if your instincts were actually right all along? Shopify wants to help you get

there. They're the commerce platform behind millions of businesses worldwide and nearly 10% of all e-commerce in the U.S. From established brands like Allbirds and Heinz to companies just getting started. Their design tools make it simple to create the exact online presence you're envisioning, with hundreds of of ready-to-use templates available. And with built-in marketing tools, you can launch full email and social campaigns in just a few clicks. So you can connect with customers wherever they are. It's time to turn those what-ifs into with Shopify today. You can sign up for your $1 per month trial today at Shopify.com slash vergecast.

You can go to Shopify.com slash vergecast. That's Shopify.com slash vergecast. Support for the show comes from LinkedIn. If you're a small business owner, you know that every hire counts. but time and resources are limited. Finding, connecting with, and screening the right candidates takes up valuable time you could be giving to your customers. That's where LinkedIn Hiring Pro comes in.

It's built to be your hiring partner, helping you find the right candidates faster. That way you can hire with confidence without turning it into another full-time job. Hiring Pro streamlines the entire process from drafting your job to short-listing candidates and conducting AI-powered interviews for initial screenings. Its updated conversational interface lets you describe what you need in plain language.

Nearly 60% of hirers find a candidate to interview within a week. With Hiring Pro, you spend less time searching and more time connecting with the right talent. And instead of getting buried in resumes, you get a focus shortlist that actually moves your hiring forward. Join the 2.7 million small businesses using LinkedIn. to hire. Get started by posting your job for free at LinkedIn.com slash track. Terms and conditions apply. We're back. The expert level stuff was one of the things in your story that I really jumped out at me as something I don't really know anything about. Walk me through kind of what that looks like. Because obviously, like you're saying, there's this huge amount of work

that is teach a self-driving car what a fire hydrant looks like in every imaginable situation. Or, you know, does this chatbot response make any sense? That's the kind of thing that is hard work, but I think a lot of people could learn how to do it. But there's also, it seems like, this other side of the industry that is super specific and expert-driven and really different. How does that side of it work? Yeah, this is sort of where I think things are headed. You know, I'll love you to see it just sort of how many people think ChatGBTBT is giving them good information and, you know, submit legal, attempt to submit like things before the court or something like that. like, it's just very fluent at sounding accurate in a bunch of different fields. And so you need,

like, you need an actual lawyer to go through it and see if it is making sense or you need, you know, an actual physicist to be, to see if it's just, you know, bullshitting you or whatever, or if it's actually being accurate. And so that has increased the demand for people who have specific backgrounds in whatever area you're trying to fine tune your system to do. And that can get quite technical. Like there's a lot of, you know, you need a lot of programming. if you want it to program well. And so that is kind of the trajectory that you see, sort of as these things get better.

You can foresee something similar to what happened to self-driving cars. Like you have a high-stakes area like driving a deadly vehicle around. You need to get everything exactly right. And so you have these fields where people are saying it's going to be great in medicine and health. You need that to stop hallucinating. And so you need medical professionals or people with some kind of medical training to go through. and make sure that it's actually recognizing the cases it's supposed to be recognizing and not doing something weird. The thing that's the most that really jumps out to me about that is this is kind of a random aside, but it comes back around, I promise.

I was doing a bunch of research on, do you remember Ask Jeeves, the search engine from like the early 2000s? Ask Jeeves was like the sort of natural language question answering service 20 years ago. We're kind of back to the same idea now. And everybody thought Ask Jeeves was this sort of magic of technology. what it actually was was a bunch of human editors who worked at the company going in and seeing the most popular queries and like literally just answering them. And so because a lot of people tend to search the same things, they were getting these very good answers. But what that doesn't do at all is scale. Right. And so it was like if you wanted to do one of the, let's say, 200 things

that everybody else on Asked Jeeves wanted to do, you were in great shape. You were going to get incredibly helpful information. It was going to feel like magic because there was a human on the other side. but if you went down this kind of long tail of new stuff, and as we've seen on the internet, that long tail is infinite. It's just going to get worse and worse over time. And part of me wonders, especially with the kind of stuff you're talking about, if we're going to land in the same place with a lot of these AI tools, because we're going to get to the point where like, if this question that you ask your chatbot has been seen and answered

by a person who has actual expertise and knowledge in this space, you're likely to get a good answer because they've they've picked the right answer or responded or even potentially like written some of the texts that you might see. But even as you start to get one degree away from that, it seems like it's going to start to get worse. And it seems like there's this bet that if I just show you enough things, eventually will have the right answer for everything. And I'm just not sure that that reasoning holds up. Is this just like a forever problem that these companies are going to be chasing with folks like you're talking about forever? I ask people that and I could never get, Obviously, they hope it isn't a forever problem.

People working on these systems hope that at some point, you know, you add enough handwritten, accurate examples and it sort of starts to learn something about accuracy or something like that. But, you know, I think I sort of as an outsider, from what I can see about how this stuff works, it seems like that's not going to happen right now. It's kind of in the whack-a-mole-asked-Jeeves phase where it's like, oh, someone asked it this thing and it did something totally weird. we need to have some people, you know, handwrite some examples to retrain it on so that now when you ask it this, it says something that is, you know, not dangerous or not crazy or whatever.

And even if you do like a lot and a lot of that sort of work, because it still doesn't really reason in any kind of human way, there's always the risk that it's going to get stuff really accurate seeming, really authoritative, sort of get all the formatting right and the language right. substitute, you know, wrong nouns or something. Right. Just something that just means it's like a little, it's off in some crucial way. Yeah. ChatGPT is very good at confidently lying to you, which is somewhere between delightfully fun and super dangerous, depending on the thing that you're asking at any given time. That was just the thing I came away from your story with is this feeling like what everybody

wants to happen is that we're just going to train, train, train, use humans, use humans, and then one day this switch will get flipped. and it will work. And what I came out of this thinking is that actually maybe this is going to be a more human process for longer than potentially anybody wants or has planned for, but that what's going to happen is just that the people are going to push it up ever so slightly every day as they keep working on this stuff, rather than feeding an engine that will eventually, as if by magic, figure out all the right answers. Yeah, that's my sense too. And it kind of reinforced by seeing how much stuff that I had already assumed was super heavily automated is still relying on people saying, you know, this TikTok video is heartwarming or this ad is too sexually provocative or something. Like it's not stuff that I had sort of assumed was sort of so we had so much data on already is still, you know, hand-tooled because norms change. Language is confusing and ambiguous and programs just haven't been able to sort of keep up with it. You always need kind of a human there.

giving it new examples, making sure that it's adjusted. And so that's sort of what it looks like with language models. You know, it could be really high-end stuff. You could have, I mean, you already have math PhDs and stuff working on annotations, but you're always going to need some of that because it doesn't, it just doesn't think in a human way. There's always going to be a little bit of an area that's not overlapping. Yeah. One of the questions you reckon with a bunch in the story is this debate over, is AI going to replace everybody's jobs? or is AI going to create a bunch of new kinds of jobs or some combination or something else entirely? Where do you land on, obviously, that's a huge question that we could spend many hours talking about,

but do you have any kind of new thoughts on where you land after reporting this story? I love feeling like it's going to both automate and create new jobs in sort of the same area that, you know, one of the data vendor CEOs that I spoke with had a very convincing comparison to industrialization. You know, you have this task that used to be all done by one person, and then you break it down. Machines are quite good at one sort of, you know, whatever it is, sort of tool punching component of it. You automate that, but that means you need to change what everyone else is doing and everyone becomes more specialized at their little task. And that's what you could see happening. I mean, it's what's already happened with remote tasks.

Like automated system tries to identify something. It can't. It goes to a human. They annotate the scene. That scene has then broken up into three things that go to other humans. They annotate it. You know, it's sort of this global assembly line with all of these little, little tasks that add up to doing some larger process. But it's hard to say what it is.

Yeah. Well, and I wonder how long that chaotic opacity stays useful to everybody. Because on the one hand, people are going to get more sophisticated about how these kinds of systems work, right? But on the other hand, we're still in this era where I think a lot of people feel like the best strategy is just to get as much of this data as you can, throw it at the machine, and then kind of see what happens. And, you know, with the people sort of endlessly refining the rules about clothes is such an interesting one to me, because what that means is they're still learning how their system works. And what is happening is it is just this constant back and forth

of trying something, learning new mistakes and then trying something else and then learning new mistakes. And at some point, we're going to hit a point where the stakes get high for this stuff. Right now, it's like people don't really expect chat GPT to be right. And when it's wrong and you, you know, use it in court like you were saying, it's like a hilarious joke. But at some point, this stuff is going to be the kind of thing that people rely on for really important high-stakes stuff. And I wonder if that's going to be the moment. where all of this stuff at every level of the process is going to have to start to be a lot more sort of transparent and understandable on all levels. Because otherwise, we're just running like

it's just black boxes all the way down, right? I think that's right. I think especially, you know, you could see in healthcare, you know, that's already heavily regulated some sort of, you know, supply chain transparency mandate. Like, you need to say who is annotating what in this system or something like that. Like, that would be one way you could see it unfolding and maybe having that sort of with other, I mean, it's a supply chain, right? So it's like you need to sort of say, what is your impact at these various places? What are people getting paid? What are they doing? What kind of like quality assurance do you have? I think something like that would probably be better for everybody. You would have to move more slowly. But the number of tasks I saw in my

brief time on there where it was clear that the engineers had no idea what was going on. And there was a sort of international game of telephone happening. It happened kind of every time. I remember this task where I was supposed to be labeling sort of, I think they're called them like rare obstacles in the street for self-driving cars. Okay. And it would be like traffic cones, traffic control directors, wires, cables, like the potholes of a certain size, things like that. And sort of every couple minutes I would be doing it and then like log back in and it would be like, oh, no, don't label traffic traffic if they're not directing traffic because I had just labeled someone, you know, on a sidewalk having lunch. Um, or like, okay, like a truck that has.

has a bunch of traffic cones in it and it's like, label it and then come back and be like, no, only label traffic cones if they're blocking an obstacle in this street or something. You know, it's just sort of like they don't realize sort of how their instructions are being interpreted and then what that does to the model. It's kind of this sort of horrible game of telephone that keeps happening that you would think would be cleared up if like the people working on the model, we're doing the annotating or in the same room and communicating so that people knew kind of what the intention was. What is the status of your remote asks career?

Did you ever figure it out? Are you a good annotator? How are we doing now? I'm a terrible annotator, and then I got suspended, I think, because I was using a VPN, and now it just gives me this sort of sad-faced approach I signed into my account. So my career is over for now. Which sounds like it's probably for the best on all parties, including those of us who want to use good AI.

It's probably good that you're not annotating it. Yes. I think it's good for me and good for the AI. Fair enough. All right, Josh, thank you. I really appreciate it. It was a really great story. Everybody should go read it.

Thanks. We need to take a break, and then we're going to call up Gen 2E and get deep in the weeds on how to build a smart home. And frankly, what that even means. We'll be right back. Support for the show comes from LinkedIn.

If you're a small business owner, you know that every hire counts, but time and resources are limited. Finding, connecting with, and screening the right candidates takes up valuable time you could be giving to your customers. That's where LinkedIn Hiring Pro comes in.

It's built to be your hiring partner, helping you find the right candidates faster. That way you can hire with confidence without turning it into another full-time job. Hiring Pro streamlines the entire process from drafting your job to shortlisting candidates and conducting AI-powered interviews for initial screenings. Its updated conversational interface

lets you describe what you need in plain language. Nearly 60% of hirers find a candidate to interview within a week. With Hiring Pro, you spend less time searching and more time connecting with the right talent. And instead of getting buried in resumes, you get a focused shortlist that actually moves your hiring forward. Join the 2.7 million small businesses using LinkedIn to hire.

Get started by posting your job for free at LinkedIn.com slash track. Terms and conditions apply. Support for the show comes from MongoDB. If you're tired of database limitations and architectures that break when you scale, it's time to think outside of rows and columns. Because let's be honest, you didn't get into tech to babysit a broken database. You got into it to actually build something. MongoDB lets you do that. It's flexible, developer first, asset compliant, enterprise ready, and built for the AI era.

Say goodbye to bottlenecks and legacy code. Start innovating with MongoDB. There's a reason it's trusted by so many of the Fortune 500. And that's because it's a platform built by developers for developers. MongoDB, it's a great freaking database. Start building at MongoDB.com slash build. Welcome back. Like I mentioned at the top of the show, last week was Smart Home Week on the verge.com.

And we spent the week doing lots of stories about how to make your smart home work. and the interesting stuff that people are doing with their smart homes and much more. It turns out, by the way, that the Verge staff has some wild ideas about smart home. There's a lot of good stuff on the site. Go check it out. But I haven't been able to stop thinking about this one story that the Verges Jen Patterson-Tooey wrote, in which she basically tried to start at the beginning and answer a simple question, what is a smart home and why do you need one?

The story's great. We'll link it in the show notes, but it really got me thinking. Isn't it weird that that's a question you have to answer and such a complicated one? How did we get to the point where a smart home is even a thing and should it be a thing? So after a few days strangely deep in my feelings about all of this, I figured I'd just grab Jen and we'd talk it all through. Hi, Jen. Hey, David. Always a pleasure to be here. So I was reading your stories and I had this weird existential crisis I did not expect to have in the middle of reading your story,

which is that I've never really thought about the question, do you need a smart home and what is a smart home and what makes a smart home a thing? But the more I think about it, the more I kind of have come to the conclusion that we all talk about this exactly the wrong way, that we think about it as sort of this big, holistic system that is like you have to have a smart home

and it's very complicated and involves buying a bunch of stuff and you have to pick a platform. And I kind of get to the end of this And I'm like, maybe the answer is just buy some things that connect to the internet. Let chaos rain and try to sort of figure it out over time. So I almost wonder if like, do we talk about the smart home in sort of the wrong way at this moment? Like should the smart home even be a thing as we think about it now? Or should you just have stuff in your house that's on the internet?

So you are completely correct. We should not be calling it the smart home and we should not be talking about it like that. However. We are and you kind of can't go back. It's like the cat's out of the bag. In one of my pieces that we published last week, we had a big Verge smart home week in case you've missed it, go check it all out.

I talked about this a little bit that really the smart home is a misnomer because it's just the natural evolution of our home. Our homes are just getting better. Just like, you know, we went from candlelight to electric light. We're going from non-connected to connected. And it doesn't mean that you need to go out and make a smart home. You're right. Like, it's great if you can.

I would love to. Maybe not everyone. But if I bought a brand new house that had just been built and it was all connected and all wired and, you know, wonderful. But ultimately, for most people, you know, that's not going to happen. So no, you don't need to go out one day and buy every single gadget and make sure you have everything working in your house. I think the most sort of natural approach to the smart home is adding devices as you need. them to solve problems, and then once you get to a sort of a point where you have something like critical mass, you may want to start making them all work together. And that could be

after you've bought two light bulbs, or that could be after you've bought seven gadgets. And suddenly you're like, oh, hang on, here I can do, you know, I have a smart garage door opener and I have smart lights. I can have my lights turn on when my garage door opens. So yeah, I think the problem with saying we all need a smart home is that that's a bit terrifying to people. It's like, oh, what am I doing wrong? What do I have to go out and do? No, you consider this natural. Something breaks.

You need something. Consider a smart replacement. See if it's going to fit in your home. And I talk to a lot of companies about this, and they all quite often say, yeah, I kind of wish we didn't call it the smart home because that's causing us problems now. And it's not always about the internet, too. You've got to remember the smart home isn't. It doesn't mean you're necessarily connecting your home to the internet. There are ways that you can run a smart home locally.

So, yeah, but we're stuck with the label. It's a label that people recognize. And, you know, we don't want to rebrand like, you know, what happened to Coca-Cola or Pepsi. True. I do appreciate that the same companies who, you know, five years ago were ramming the phrase smart home down everybody's throats at every available opportunity are now the ones being like, maybe we overshot and panicked customers about all of this. But that transition you're talking about, I think, is super interesting.

Because it makes me think of like if years ago when, you know, refrigerators suddenly got built in freezers, if everybody was like, I have a freezer home now. Like, it just doesn't make any sense. But there does come a time where you go from, I have, you know, one thing that connects to the internet. Like I have a garage door opener that I can control with my phone. Two, I have a home that can sort of be more than the sum of its parts and put some of this stuff together like you're talking about. And I feel like the key question for a lot of people is kind of where that might. moment happens and where you go from like like it my own experience is actually a good example of this right i have a couple of phillips hue lights around that i just control with an app on my phone

i have a honeywell thermostat that's connected to the internet that i control with a different app on my phone that's all fine i don't actually need more system than that but i feel like i'm probably one or two things away from needing more system than that and also being able to do a bunch more stuff if i put all those things together how do you identify that moment where it's like okay you go from just having a couple of things to like you should build a system because it will actually make it better for you. So there are two answers to that. In your case, I would say you're maybe one or two children away from wanting to do more. Because convenience is one of the big things. And for me, that was a tipping point, having children. Once my children were older and we had more need for

routines in our home, the smart home is brilliant for routines. And I think that's where that can really help parenting or aging in place or elder care or any kind of specific life use case where you could really benefit from something being automated rather than you having to do things yourself, saving you time. Working from home is another great example. You know, setting up a home office and adding smart sort of automations to your home office. I know one of our editors, Dan Seafit, recently set up a ingenious automation that turns on a do not disturb light outside his door when he's on a Zoom call, you know, things like those kind of solutions to problems that come up in your home. That's sort of one use case. And it's probably the primary reason to start looking

at routines and automations because that's when you need the platform is when you want things to work together, when you want more than just one light to turn on, when you want your light to turn on, your thermostat to adjust, your shades to open, your door to unlock. That's when you start needing these platforms and ecosystems and more of a cohesive smart home. The second answer would be, and this was a piece, this is one of the pieces that we had last week was looking at smart homes with energy, smart energy management. So basically building homes that are better, like as we've gone from building simple homes without a lot of insulation or, you know, electrification, you know, now we've started to move towards better build homes that are better for the environment.

They cost us lots of money to run. And now the next level is adding smart to that. We talked to people who lived in new smart community in California that it has been built from the ground up to use smart energy. It has like a smart electrical panel, has solar panels, but it's also been built to the point that insulation is so good that you don't actually need to use an awful lot of energy. So there's this kind of combination of looking after the environment, saving money and convenience kind of all comes together. And that's where I think the smart home sort of of the future has a lot of potential. And if you go back to the beginning of the current smart home, the DIY smart home, I know there have been smart homes for a long time, but, you know, the start of where we kind of think of it today, the nest thermostat was kind of the darling of the beginning of this current smart home.

And that was all about saving energy, saving money. You could feel good because you're helping the planet and feel really good because you were saving money. So I feel like that kind of a use case is one where we're really going to start to see not only individual people wanting to make their home smart, you know, retrofitting with a smart electrical panel, but we're also going to see society as a whole pushing us towards making our homes better. And, you know, there's government subsidies. There's all sorts of things going on in this space that are really interesting. And then the other use case there, I think I mentioned, touched on it earlier, is aging in place. And again, this is another huge sort of societal shift as, you know, the baby boomers reach their golden years. And, you know, we've got a huge influx of elder care that's either going to fall on the shoulders of the next generation or is going to fall on society, you know, the state. And aging in place is a really compelling use case for the smart home.

If you can keep someone in their home for longer and not have to move into a nursing home or have to go to assisted living, you're really, you know, helping everyone. It's a win-win for everyone. So there's these kind of larger societal pictures that I think are often left out of the conversation about the latest smart doorbell or the best smart light. I mean, those gadgets are great and fun, but there is a much bigger picture here. Totally. And I think one of the challenges that I know you deal with in covering a lot of this stuff is figuring out kind of what is true now, what's going to be true in a year and what's going to be true in sort of 10 years, right? Because I think that idea of having all of this stuff set up in a way that just kind of magically works

feels to me like it's just not where we are right now, right? Like, we're still in the kind of like you have to build all the blocks yourself to make some of this stuff work. And it's getting a little easier. And the platforms have made it easier to kind of build routines and stuff. But like you still have to say, when I get home, I want the thermostat to change. I want the garage door to open automatically. I want this light to turn on.

I want this light to turn off. And I want the oven to start preheating. the right way for all of this to happen is that that should just happen automatically, right? Or that some process of that should like learn how you live and adapt around you. And I feel like we're just at the beginning of that. And it's like starting to happen in just enough really cool ways that it's like I can see why that's going to be great. But it's not quite there yet. And so that's where I keep coming back to this idea of like, forget the idea of a smart home.

Like let's worry about a smart home in 2035 when AI makes all of this really great. For now, just like buy the coolest, smartest stuff you can find. that works for you and don't worry about like the big 30,000 foot system of it all until you absolutely have to. And I think that's probably bad advice because I think the thing that happens is you then kind of get out over your skis with too much stuff and then you have to like retrofit a system to it and that doesn't work that well either. But I don't know, we're in this like limbo position, especially with things like matter, which will solve some of this stuff, that it feels like the less you worry about building a quote unquote smart home for the moment, the saner

your life will be. Yes, exactly. I think you're right. I think there are great use cases for individual or a few devices. And if you have an issue you want to solve, you want to save money on your HVAC bill, you buy a smart thermostat, you keep losing your front door key, you get a smart door lock, you know, and that doesn't mean you need a smart home just because you bought a smart door lock. I think you're right, AI could provide a huge shift in terms of that learning. our homes actually starting to sort of think for themselves and learning what we want and how our homes should operate, I think ultimately our homes are going to become, you know,

like our cars have become, they are going to be computers. But I think we're a long way from that still. I feel like, you know, for individual people today, you're right, don't necessarily need to think the big picture. But if you have one of those use cases we discussed before, if you're scared of the whole process, because it is daunting. It is confusing.

And one of the worst parts about it is once you get it all set up and it's all working great and your routine is, you know, kicking off exactly as you wanted and then it just stopped. And then you're like, what happened? Like now you've become like a network cisadmin for your home and you spend your weekends troubleshooting while your smart sprinkler didn't go off and all your plants are dead or why you can't open your smart garage door right now, you know. And that's frustrating. And that's where the smart home really starts to break apart when it causes more problems than it was designed to solve.

I'm sure that's a purely hypothetical example for you and not something you deal with every single day all the time, every hour. Yes, yes. My house is, yeah. What's the most recent smart home chaos you've had personally? Oh, well, my husband not being able to open the front door because I put a new smart door lock on and he couldn't figure it out. And he was not happy. Yeah, this is why my wife just straight up doesn't allow most of these things. She's like, I'm not, I'm not worrying about this.

And when it breaks, it's not going to be my problem. Keys are great. It's going to be fine. And it's like, I'm like, yeah, you're right. But also, what if I put a cool door lock on? But on that point, though, and I should mention this, and it's not something we cover a lot, but there are integrators out there that will do all of this for you. And they have kind of mastered a lot of these problems and solved them for you if you're willing to pay the money.

Because, you know, it costs money to have these people come out and do them, Although it's gotten a lot less expensive. They've obviously facing a lot of competition from the DIY smart home. So you can have some of these experiences without having to deal with all the troubleshooting yourself. It's just going to cost you a bit more money. So I think it doesn't have to be frustrating. And as I said, you know, one of the things that I really, I think my biggest piece of advice for people when they're looking to embark on the smart home is, as you say, start small.

But when you scale, don't scale all at once. Scale small. Because if you're like, oh, I love this, I'm going to buy 50 smart switches and 50 smart bulbs and wire them all up and get it all set up. I'm like, make sure you realize that smart bulbs don't work with smart switches because otherwise you, you know. So add slowly, build slowly, you're less likely to run into some of those sort of more frustrating scenarios that we've talked about. Yeah, that's very good advice. So let's talk about the platforms, though, because if, let's say, you've either hit the point where you kind of need to build a system or you just. are the type of person who wants to kind of solve the system bit from the very beginning.

You basically land on one of four platforms, right? There's Apple Home, Google Home, Samsung Smart Things, and Amazon Alexa. Are those kind of the four? Is there any other one that even sort of belongs in that same category with those four? Well, there's Home Assistant is another one that is a very popular option for more advanced users. It's the open source one that like it's the one all the nerds really like. It's just true. Yes.

Okay. And, you know, and it also, it is more complicated. So, you know, I would generally recommend starting with one of the big four that you mentioned. I mean, there is a big sort of push and pull, you know, is it a platform? Is it an ecosystem? Is it a controller? But, you know, ultimately, those four are platforms that you can use to control your smart home. You can also tie those in to some of the other options out there, like home assistant

or like habitat or security systems like abode or Vivant. know, there are other sort of platforms out there, but they can also work with these four one of, or maybe more than one of, these four big platforms. So these four big platforms do they're going to be present in every conversation about any kind of smart home setup. So especially if you want voice control. True. And it seems like maybe the simplest kind of heuristic about which one to choose is just to sort of look at the devices you already own, right? That if you're a person who owns a lot of Apple stuff, there are good reasons to kind of go all in on Apple Home and same if you have a bunch of, you know, echoes around your house. Is it kind of

that simple that, like, the first question you should ask is what other devices do you already have? Yeah, I think it's what are the devices you have on the smart home side and also just which, which phone you use because these platforms are all pretty tightly tied to their smartphone, although, you know, some of them are cross-platform, but you generally are going to have a better experience with Samsung Smart Things on a Samsung Galaxy device with Google Home on a Google Pixel and definitely with Apple Home on an iPhone because you can't use Apple Home on anything else. Amazon obviously does not have a smartphone,

but that also makes it the broadest. So it works really well with any of the smartphone options. Plus, if you have any kind of fire TV or any kind of fire tablets, there's really good integration there. So yeah, I mean, ultimately, if you're like, I'm interested in the smart home, where do I start? If that's kind of your first question, then look at your phone and look at what you already have.

If you already have an Echo Smart Speaker, if you already have a Google Chromecast, even if it's just one or two devices, if you have a Ness thermostat, that's a good place to start. You're not limited in most cases by that. So if you've already bought a device, it doesn't mean it rules out the other platforms. It's just a good place to start because you're already probably familiar with that platform to some extent. Totally. But, okay, so let's throw all of that out then, right? Let's say I don't own, I don't own any devices. I have never in my life bought a gadget. And I don't know why you're listening to the Vergecast, but welcome. You were a hermit and you've just moved in from... And you're trying to decide between these four platforms. Are there sort of meaningful differences between them? It seems to me there are a lot of devices that work across two, three, even four of them. So the idea of like, I have to, you know, use this platform to use this light is less true than even it used to be. And it seems like in most ways they're very similar these four platforms in terms of like what they let you do and how they work.

Are there big differentiators between them at this point? So originally the biggest differentiator was which devices will work with which platform. And that has been a big issue. But that is what Matter was designed to solve because it is meant to bring in cross-platform compatibility. So the idea being any device that you buy, if it has the Matter logo, it will work with any of these platforms. We're not quite there yet. But that is the promise. So I would still recommend buy Matter devices if you're not.

sure which platform you want to stick with. They just aren't a lot to choose from yet. So that was a big differentiator. That was an issue, especially for people who chose Apple Home, because there weren't a lot of devices that worked with Apple Home and the ones that were were more expensive. They had, you know, the Apple Tax. So whereas everything worked with Amazon Alexa, like everything, although how it actually worked with Alexa would vary. So there was a bit more Wild West with Alexa than Apple Home was very tightly curated. But in terms of So platforms, like features, each platform does have different features and different things that you might consider choosing it for.

For example, Apple Home is very well known as its devices are for privacy. It keeps all your data local. It doesn't use it to sell ads to you. It also works locally if you have an Apple Home Hub. All processing will happen locally for things like video. So even though there is a cloud component to Apple Home, it does all work locally in your house. which a lot of people were more comfortable with. It also means it's a little faster.

Local control means that when you ask a light to turn on, it turns on almost as quickly as flipping the light switch. And you know, that's the ultimate sort of goal of the smart home. Let's make turning a light switch on just as fast as flipping the switch, but way cooler. And then, you know, Amazon has a great integration if you want to buy stuff, surprisingly enough. Very easy to buy yourself toilet paper when you're, you know, stuck in a crunch,

assuming you have a smart speaker in your bathroom. Alexa is actually one of the more innovative platforms. There's a lot of really kind of fun, unique features that it offers. One in particular is hunches, which is an AI generated kind of. It takes what it knows about how you use your smart home and makes suggestions. So, for example, if every night you normally lock your door at 9pm and then one night you forget to lock your door and you say, good night to your smart speaker and it will say, oh, by the way, did you realize you didn't lock your door? And that's it's hunch. and that can be useful.

But obviously there are privacy implications there. You are basically letting Amazon know everything you're doing in your house. So if you don't feel comfortable with that. Right. But that is, as we were talking about with that kind of broader, your house learns how you work thing, if that is the goal, then Amazon is kind of one full step closer to that than just about anybody else. Very much so. And this is the real kind of push and pull with the smart home right now is privacy and data.

You know, we want privacy in our homes, completely everyone. wants privacy in their homes. But if you want your home to be smarter, it needs data. It needs that context. That's the only way it's going to get smarter. So it's trying to figure out how we can provide data without providing intimate details of our life. And that's a real push and pull. And that's something that matter is looking at trying to help fix. But I think it's also something that AI could be really instrumental to also machine learning on the edge. So devices being able to process that kind of contextual data on device rather than having to send it to the cloud. So you're not sharing your data in the cloud.

And then with Google Home and Samsung Smart Things, they both have really interesting sort of, I think one of the things with Smart Things, if you're interested in energy management, that's a really good platform right now. It's working really hard and innovating really well in that space. Also, if you have Smart Things, Samsung Appliances, that's at the moment the only platform that really works well with integrating those appliances. It's, we're still a long way from smart appliances being super useful, though. But, you know, that's the future, I think.

We'll get more of that as we, as the smart home expands and develops. And then Google Home, there's been a lot of really good innovation recently there. It's taken a while. I think the biggest selling point of Google Home right now is its Nest products. The Nest hardware is good. It's some of the best in class. and it also has a lot of local component too. So it does the machine learning on the edge,

a lot of AI processing locally rather than sending it to the cloud. But there is also, you know, Google is a data giant. And, you know, you do have to weigh that if you feel comfortable with supplying Google with your data. And Google Home is also probably the better of the voice assistance. That's what I was going to say. That's what it's been for me is the one that is most likely to hear me say, turn on the office light correctly is Google Assistant by a mile. Yeah.

I don't know if it's just me, but Siri is, I mean, Siri is a dumps your fire. Alexa's good, but Google Assistant, at least for me in most of these cases, is sort of at least one order of magnitude better than everybody else. I think it depends also what your main use case is, because the biggest problem I find with Google Assistant for Smart Home Voice Control is the delay. It doesn't have a local processing that Siri and actually Alexa does have on some of its devices. So I can find it can be quite slow. But Google does have a really neat.

I love its presence sensing feature. I think its presence sensing is probably its best feature because it has any device, any Google Nest device in your home to determine if you're home, not just your phone. Most of the other platforms rely on phone geolocation, which does not work for me because I live in a cell phone. Dead Zone. I once was lying in bed. I had Apple Home, geolocation turned on, turned the lights on when I arrive home. The lights just kept turning on and off, on and off. And I was just sitting there with my phone and it's like, Jenny arrived home. Jenny left home. Jenny arrived home. I'm like, nope, nope, still here. That was not fun. So whereas Google Nest can use like your Nest protect smoke detectors, your nest hubs, your nest smart speakers, your nest thermostat to tell if there's anyone

in the home, not just you. That's the other thing. Like, if you have kids who don't have smartphones and you leave them at home alone because they're old enough, they may turn the heating off because it thinks you're gone. So I like that present sensing. That's one of the things that Google does a good job for. And we have actually had, we had four or five writers on the verge go and write about their personal experiences with each of these platforms. So if you kind of want to dive more and find out what they like about them, we've got lots of great content on Vorge.com. Totally. Yeah, it's all very good. And I do think it's interesting to hear it. They've all very much sort of converged on how they think a lot of this stuff should work. And like the apps all look increasingly the same, which I think is really interesting. But they do each still kind of have their own strengths and weaknesses. They do. And one of the nice things about Matter now is that we do have this multi-admin feature, which means that you can use any of the platforms with any of your devices. So if you have a Matter device, it will work with any of these platforms. So, you don't have to commit. If you do choose a platform and you decide you don't like it,

you should be able to move to a different platform relatively easily. As I mentioned, matter is still in its infancy. But it is coming. We are going to have more devices that work with matter and hopefully work out some of the bugs that we've had to this point. But yes, you don't, you know, you can chop and change. You can try out different platforms. And if you get really good and really enjoy some of these platforms and you start to hit some of the limitations of the platforms, which are largely based on more kind of complicated automations. Like if you want your lights to dim only if it's after sunset and it's 30 degrees outside and your wife isn't at home,

you know, if you want to add all these conditions and do some like really cool, we've got some crazy, neat examples of people's fun automations. We had a piece that our staff wrote about their favorite smart home setups. There's some really neat fun stuff you can do like we'll geek out. And you might want to move to a different platform, you know, from one of these four platforms, sort of graduate to the next level, so to speak. And you can do that. Like, there's the beauty of the smart home is you really aren't completely locked in. There is ecosystem lock into an extent, but the walled gardens are beginning to sort of crumble.

And we're beginning to be able to choose what we want and control it the way we want. So that's what makes me excited about the smart home space that, you know, we're getting to the point where you don't have to choose your. Apple Home device to work with Apple Home, you can choose what you want. Yeah, no, it's good. And I appreciate that, you know, you come on the Vergecast at least once a quarter and remind everybody that matter is great. It's going to be great. If everybody could just ship stuff with matter in it, it would make everybody's life better. Can we all please make this happen immediately? Love the Vergecast. I do feel that way. I am not. I am there are a lot of people and I'm with them that think that

matter has not done its job yet. I completely agree. But I don't believe that. that that means it never will. I think it's been hampered very largely by the platforms and some of the manufacturers starting to kind of shrink back a little bit and get a little nervous. And a lot of people kind of doing it, well, we're just going to wait and see if this is going to be good for our customers, whereas the customers are going, we want matter. Yeah, right. The answer is yes, everybody. But is it really, is it good for our customers or is it good for the bottom line? you know, we're entering the political stage of matter, and I'm just hoping that we're going to get through it. Because, you know, otherwise, we can all enjoy the alternative.

Which is pure chaos. Exactly. What is the alternative? Like, what else are we going to do? Are we just going to throw this in the bin and then what? Go back to these walled gardens. So, yeah, we don't want to do that. Agreed.

Awesome. All right. Well, we've got to take a break. But Jen, thank you, as always. We'll be back here to name and shame all of the people doing this wrong next time. Jen, thank you as always. We'll be back here to name and shame all of the people doing this wrong next time. All right. It's always a pleasure. Thanks so much, David. All right. Before we go, we have a hotline question. This one is less of a question, as you'll see, but just something I think is interesting. And I have some thoughts.

So let's just play it back. This comes from Didi. It's Dedy from Maplewood, New Jersey. Love your Verge cast. I would just look at your assessment of the Apple Vision Pro. So you were speculating about what to do with eventually augmented reality glasses, not being able to completely occlude your entire environment as the current Vision Pro can. So I just wanted to throw out there, well, what if the end game is simply contact lenses? Presumably they would cover your entire retina, and then you could also presumably reproduce the entire VR experience. So maybe they'll do all-encompassing goggles and make them smaller and smaller,

and then eventually go directly to contact lenses. Just putting it out there, erase your thoughts. Thanks again. So the funny thing about this is this has come up for me a couple of times in talking to people. And I think it is kind of the long-term trajectory of this that people talk about, right? You go from headsets to glasses and then, okay, what's the next step after glasses? Obviously, it's contact lenses. And then you can have a debate about is the thing after that brain implants or, you know, what neuralink is up to and stuff like that. But I think contact lenses is a really interesting kind of far end point.

for a lot of this tech that would solve a lot of problems. And I guess I have two thoughts. One is that I do think there are a lot of people in the tech industry who believe that. I think there are whiteboards that have the word contacts circled on it as the long-term goal. I truly cannot overstate how far away that is from a technical perspective. Obviously, things change quickly. Who knows what will have been invented a decade from now. but the number of essentially technical miracles that have to happen between now and then in power consumption,

in how these things are produced in the size of chips to make a contact lens possible is just staggering. Like we are several orders of magnitude of technical invention away from something like that being possible. Not to say we'll never get there. I just am not spending any time holding my breath. But I do think that is an interesting endpoint of this. think as a product becomes really fascinating. It also raises really complicated questions because they get harder to get off of your eye. That brings up different questions about, like, are you more of a cyborg if you're wearing contact lenses versus if you're just wearing glasses that you can more

easily take off? How do we power these things? How do they get software updates? How do they connect to the internet? A million open questions. And I think one of the things we're going to start to see is if you, to pardon upon, use contacts as a lens through which to look at a lot of technical advancement, you can start to see are we getting closer to that as things get smaller, as things get more power-efficient. Imagine a chip that gets hot that is sitting on your eye. Like, it's just totally incredibly implausible right now that that is going to happen anytime soon. But I do think there are a lot of people who are going to look at a pair of glasses and say, okay, we've done it.

They look like Raybans. What's next? So, D-D, I think you're right. I think that's a way we might be headed. I am unbelievably skeptical that we are even remotely close. close to being decades away from that. I think that is a really cool thing that still exists and pretty much only in science fiction. All right, that is it for the Vergecast today. Thanks to Jan and Josh for being here and thank you so much for listening. There's lots more on this conversation,

a big story that we did in collaboration with New York Mag that Josh is talking about all of Smart Homeweek on Theverge.com was amazing. We'll put some links in the show notes. But as always, readtheverge.com. Very good website. One of my favorites. If you have thoughts, questions, feelings, or crazy smart home hacks you want to send my way. You can always email us at Vergecast at the verge.com or call the hotline. 866, Verge11. We love hearing from you.

Send us all your thoughts and questions and ideas for what you want us to do on the show. We're going to have more hotline stuff every episode. Obviously keep it coming. This show is produced by Andrew Marino and Liam James. Brooke Minters is our editorial director of audio. The Vergecast is a Verge production and part of the Vox Media Podcast Network.

Nelai, Alex, and I will be back on Friday to talk about pixel tablets, Spotify, Suvids, maybe, Reddit, and lots more. We'll see you then. Rock and roll.