Reply All - #109 Is Facebook Spying on You?

Episode Date: November 2, 2017

This year we’ve gotten one question more than any other from listeners: is Facebook eavesdropping on my conversations and showing me ads based on the things that I say? This week, Alex investigates. Further Reading Our guide to keep Facebook from following you around the internet can be found at http://replyall.limo/donttrackme . Facebook's official statement that it is not listening to users. Facebook's Rob Goldman (no relation) denying the same thing. Learn more about your ad choices. Visit podcastchoices.com/adchoices

From Gimlet, this is Reply All. I'm Alex Goldman. And I'm PJ Vote. PJ. Yes. Hi. Hello. So over the course of maybe the past year,

we've gotten a fair amount of emails from people who think that Facebook and Instagram, which is owned by Facebook, are using the microphone on their phone to listen to their conversations and advertise stuff to them based on the things they're saying. Yeah, I have also gotten some of these messages. I basically have just felt like I don't, I think it would be such a risk. thing for Facebook to do. They've been like, oh, it's probably just coincidence and people are imagining stuff. Yeah, but I've been talking to this guy named JP, and some of the stories that he's been telling me are super hard to dismiss. He told me that he first started noticing this happening at the

beginning of the year. I was baking pizza dough. I was, you know, making pizza dough. I said this would be a lot easier if we had one of those fancy kitchen aid mixers. Ten minutes later, there's an ad for Amazon kitchen aid mixers on sale. Okay, wow. Not long after that, JP is in Target with his partner, Gary, and he yells down the aisle, hey, can you pick up some Red Bull? And then I opened Instagram on the way home. I wasn't driving. I was in the passenger seat.

And there was an ad for four new flavors of Red Bull. Try them now. And I was like, this is insane. This is crazy. You know, they just kept coming. I was like, let's try something funny. And we would say something ridiculous. Like, man, I could really use a pair of really sexy underwear and like these weird

mesh underwear ad started showing up in our feeds, and it was nonstop. And then JP told me a story that just felt really crazy. Let's call it the perfume story. The thing that really got us was my partner's mom came to visit from Oklahoma, a very nice lady, but, you know, doesn't travel that often. Her name's Debbie, and she was going to visit JP and Gary in San Francisco. And on her way there, she gets a bottle of perfume confiscated by the TSA. So when she arrives in San Francisco, she says,

hey, I want to go to the perfume store and get a new bottle of perfume. And my partner, we don't wear colognes. We've never bought perfume that I know of, never search for it. And within 30 minutes, he had opened his Facebook, and there was an ad for a women's perfume store in San Francisco. That's weird. Yes, it's weird. On a scale of like 1 to 10,

what would you say your belief is that this is actually your phone listening to you? Uh, 10. I'm convinced. Wow. Okay. And it just, it creeps me out. And I have no idea how to stop it. I actually, this week, removed the Facebook app and Facebook Messenger and Instagram from my phone. And now I just have links to the websites. I'm like, this close to just deleting my account.

So, naturally, the first thing I did was contact Facebook, ask them for an interview. They said no. They said, no, we won't do an interview. And also, listening to people's conversations via the microphone on their phone to target ads to them is not something Facebook does. They said that unequivocally. Right. Which, again, I don't think they do it. But if you're a person who does, you're like, well, of course they would say that.

Why are they going to tell you about how they're secretly spying on you? Right. But the thing is that Facebook wouldn't offer me a satisfactory explanation as to why these ads were. showing up in J.P.'s feed. So I looked into this, and after doing some reporting, I realized the reason they don't want to talk to me about this is probably because the technology they use to target people with ads is really invasive. I talked to the guy who first built that technology.

His name is Antonio Garcia-Martinez. He since left Facebook, but he started at the company back in 2011. So just to be clear, there was no targeted ads division before you, right? You use the term division, like the whole part of the company. It was literally me and three engineers. Okay. Remember, the only Facebook ads were those little postage stamp-sized little turds on the right-hand side of, on the right-hand bar. There was no commercial content in feed.

Antonio's big insight was that they would make way more money if they just squeezed more information out of their users. And the most obvious piece of information was location. people's devices were telling Facebook where they were, and the targeted ads team could weaponize that. All of location targeting was the responsibility of one guy. He's a friend of mine, a guy named Pierre. He's this kind of weird, quirky, idiosyncratic French dude. And, you know, he would basically take, you know, the lat long data off your mobile phone,

your check-ins, you know, IP address lookups if you're logging in from like a laptop or a desktop machine. and that would go kind of into like Pierre's magic location machine, and out would come a location. Location is important to Facebook because, A, just where you live tells them a ton about the kind of stuff you're probably interested in. And B, if you suddenly appear in a different location, a location that Facebook doesn't recognize, then it knows that you're traveling. So like with the perfume story, it's like she was telling them that she was traveling, even if she didn't realize that she was telling them that. Interesting. And so the next thing I learned was that in 2012, Antonio came up with what is probably his enduring legacy at Facebook, the thing he will be remembered forever for.

Which is what? So he wanted to figure out a way to keep tracking people after they left Facebook, like to be able to see what they were doing all across the internet. And so he developed this thing that's now called Facebook Pixel. and it's installed on millions of websites. So when you go to one of these sites with Facebook pixel on it, it watches what you do and reports that information back to Facebook. It can see how long you linger on a certain web page.

It can see if you purchase something. It can see if you put something in your cart on a website and decide not to buy it. It's kind of like an internet surveillance camera. So that's why like when you look at a pair of shoes or whatever, It follows you around the internet. Right. There's this app I use that's called ghostry that shows you if pixel is installed on a site that you're visiting.

And it'll also show you all the other ad trackers that are on that site. Like if you go to the New York Times website, there are maybe 30 or 40 of these trackers. Like as soon as there's an ad, you basically have to picture 30 or 40, like, helpful friendly sales associates, like following you around the store, trying to guess how much money is in your wallet, like, guessing your, like, guessing your, like, weight and age. and being like, oh, he looked at the hooded sweatshirt. Oh, my God. Okay, okay, okay. Write that down. Write that on.

He likes hudies. Right. So by 2012, the targeted ads division has figured out how to follow you all around the internet. They have all this info on everything you're purchasing, everything you think about purchasing. But once they figured out they could do this, they got, like, data hungry. They weren't just interested in the information that you could give them online. They wanted to know things about what you were doing offline.

And so they figured out a way to buy your personal history. So it's like I have a file on Alex Goldman. I'll go buy Alex Goldman's credit report. I mean, probably not that. No. Yeah. I mean, we don't know exactly what Facebook is buying because they're a black box. But we do know that they're buying from companies that sell credit reports.

Really? Yes. I talked to this reporter from ProPublica. Her name's Julia Anguon. She's investigated a lot of this stuff. Wait, where are they buying this stuff? Oh, you can buy this from these delightful places. One of them just had a big breach, Equifax.

You may refer to them. Experian Axiom, there's about, you know, there's tons of them. There's probably about seven or ten big ones out there who sell information about your income, the square footage of your house within 25 square feet. These companies sell information on whether you've been married, whether you've been divorced, your credit score, whether your name has showed up in a lawsuit. They know your income. And you know those loyalty programs that like supermarkets and pharmacies have?

The data brokers often run those programs. So they know how often you're buying diapers or cold medicine or birth control. So like if Debbie had like a loyalty card at like her local perfume store, they would know like this is the type of perfume she buys. And even like in theory it's like they would know like oh and she bought it like eight months ago. Like she's due for some new like oh to Debbie or whatever. Absolutely. They're basically learning everything they can about you. And then they break your personality and your interests down into all of these hyper-specific traits.

And Julia told me that there are a ton of these. So we were able to put together a big database of about 52,000 attributes that Facebook was collecting about its users. Right. 52,000. So they had some categories that were just mind-boggling. There was one that is my favorite called a person who likes to pretend to text in awkward situations. How did they even figure that out? I have no idea.

There's actually a page on Facebook where you can see. see how Facebook categorizes you. I want to check mine. Okay, so when you go on your Facebook, you go to settings, and then you click on ads. And then there's a section called Your Information, and under that, you can click your categories. Your categories. Close friends of men with a birthday in seven to 30 days. So that's like a reason somebody, that's a category where you're buying perfume for other people.

Who are they trying to hint that I need to buy a birthday present for? away from family Gmail user, millennial Housemate-based households People living in households where one or more people are not immediate or extended family Away from hometown

frequent traveler I don't know, it's weird It is weird And this is not a complete picture Of the information that they have on you Facebook knows so much more But they just keep a lot of it secret And honestly we only get a glimpse

of how much they know when they screw up For instance, I talked to Charles Duhigg. He used to work at the New York Times, and he's written a lot about how big companies track you. And he had this story about a friend of his who learned something really disturbing through Facebook. So his friend's this liberal guy lives on the East Coast. His brother-in-law lives in another state. They don't see him that often. But his brother's law is like one of these like kind of like, he's into guns and he's really conservative.

But my friend, he wants to have a relationship with his brother-in-law. So he, like, he friended him on Facebook and they'll, like, cross-posts. And he always tries to, like, like, like, the posts of his brother-in-law that aren't, like, totally crazy. But then this really weird thing started to happen on Charles's friend's Facebook feed. Which is that he started getting these, like, these right-wing political ads that were, like, a little white supremacist. Like, not really white supremacist, right? Because you can't put white supremacist stuff on Facebook. But it was, it used a lot of the code words.

Which freaked Charles's friend out. Because it's not like he had ever expressed any interest in white supremacy or anything like that. So Thanksgiving rolls around and he sees his brother-in-law and he's like, hey, I've been getting all of this sort of like stuff that feels disconcertingly to me like white nationalist or sort of like racist. And you're probably like the only conservative that I friend online. I'm wondering if you have any idea why this might be happening. And he was like, come outside. We need to talk. What?

And they go outside and he says, listen, I've disavowed this. I'm not into it anymore. But for a couple months last year, I was going to a lot of white pride, white nationalism meetups. Yeah. So one of the things that Facebook can do is if you like something, it can advertise that thing to your friends. So the brother-in-law obviously signaled to Facebook that he was into white supremacy somehow. And Charles's friend was liking a lot of the guy's posts. and they were friends on Facebook.

So Facebook was like, all right, well, why don't I advertise this white supremacist stuff to you? Wow. That's wild. It's like Facebook built a machine that just like as a side effect out white supremacists, but that's not even like the point of the machine. Like they don't care. Like the whole point of it is just like learn things about you to sell you crap.

Yeah, but think about all of the stuff that this thing can do. Like if you look at everything that I just talked about and you apply it to J.P.'s perfume story, I think it explains it. Oh, totally. So, like, Debbie's going through the TSA in Oklahoma. She gets her perfume confiscated, and she's like, oh, crap, now I've got to buy new perfume. She searches for it on her phone. She looks at it.

She's like, oh, it's kind of expensive. I'm not going to buy it right now. Right. But she goes to a page where Facebook has a pixel on it. Facebook knows this person is now in the market to buy perfume. And it knows that she's traveling. She's traveling. Right.

And they probably know that she's traveling to visit her son. Because, like, he's her son. He lives in San Francisco. She's logging it in San Francisco, so is he. So why not show the son a perfume ad? Because he could be like, oh, mom, isn't this the perfume you like? Exactly. Hi, this is Joe Paul.

Hey, this is Alex. How you doing? Hey, I'm great. So I called JP, and I told him that while I couldn't say with 100% certainty that Facebook wasn't listening to him, I had a lot of evidence that they just didn't need to. So then Facebook knows that she wants perfume, right? Yep. Knowing your relationship, Facebook might have given your partner that ad because it knows that she's nearby and it knows that she wants perfume.

Okay. I mean, so maybe they're not listening on the microphone, but I don't know. It just does. It feels like they are. it's just really like this is a weird thing we've signed up for and allowed you know thank you for the additional information i oh boy so i actually kind of thought my work was done here and then a couple days later, I got an email from JP with the subject line, I'm not sure I'm convinced. And he just told me another story, a story about talking about having a leg cramp and then

getting an ad on Instagram for cramp cream. I actually don't find that surprising. Like, I feel like I think you can learn as much as you want. I don't think you're ever going to convince anybody who already believes that Facebook is spying on them that they're not. And I think it's actually Facebook's fault. Like, they've created this problem because they're really good. collecting information about us. They won't be very transparent about what they collect or how. And so you're basically like forcing people to come up with the simplest possible solution for how Facebook knows stuff about them. And that's that they're listening in. I would be surprised if you could find literally one person in the world who thinks this is happening, who you could tell

them what you've learned. And they would be like, yeah, you're right. I could find one person. You cannot find one person. We can literally open the phone lines. We let people call in. you will not find a person. Okay. After the break, Alex takes some phone calls. Okay, so Alex. Yes. So I tweeted out and I just said like, hey, we're going to take calls.

If you believe this thing is true, call in. Before we even open the phone lines, I just want you to know how screwed you are. So like, let me just actually bring it up. Hold on. I saw. Like hundreds of people, I think. Like, everybody thinks this is true, including like, including tech journalists, who I respect a lot. Like, it's not just like a fringe.

believe. Like, everybody thinks this is true. Literally, like, the VP of Facebook's ad division jumped in and was like, we're not doing this. And there were all these people just being like, you're lying, you're lying. Like, you are, I think you were walking into something that's maybe a little bit bigger than either of us realized. And I'm really excited to watch you walk into it. I still think I can do it. Okay. Let's take a call. Let's open the phone lines. Sounds good. Hello. Hi. Hi. Hi, who's this? Hi, this is Monique. Hi, this is Alex. And PJ. Hi, guys.

So what's going on here is that we're talking to people who believe that Facebook is listening in on them using their microphones. And Alex, who's done a bunch of research. And as far as I can tell, believes it's not happening. He'll try to give you an alternate explanation. Okay. So I have a very quick story. And this is so funny. I was just telling my friend about this last night.

So a few months ago, I was on the phone talking to my friend. And she was telling me about this device that she had bought to help her open coconuts. This is this really weird thing, and she was trying to, she's explaining this tool, but she couldn't remember the name. And we get off the phone, and then that was it. Maybe 15, 20 minutes later, I'm scrolling on Facebook, and I see an ad for this device called the Coco Jack. The Coco Jack. I screenshot it. I was like, is this what you were talking about?

She's like, yes. And ever since then, I've been convinced that they're on to me. Okay. God, this is like watching. a conductor warm up. Okay. Is this person your friend on Facebook? Yes. Did she buy the Cocojack online? I don't know for sure, but I don't think she did. I just watched a balloon deflate in person. No, not necessarily. Okay. Do you know where she bought it? If I recall correctly, she was in Vegas at some weird little shop, like, as seen on TV shop, and she picked it up there.

Do you think that she was, like, frustrated by all her coconuts beforehand? And so she, She Googled, like, how to open coconuts? Perhaps, maybe, but why would I be seeing it on my, like, I saw it on my feed? So Facebook has the ability to follow you around the internet as you browse. When you're logged into Facebook and you go to a shopping site and you put something in your cart and you decide not to buy it, the site will then transmit that information back to Facebook saying, hey, this person's really interested in the cocoa jack, right? Uh-huh. So another thing that Facebook does is it allows advertisers to advertise certain products to the friends of people who have either purchased or shown interest in that product.

So your friend being really into the cocoa jack, her favorite new device, might have left some kind of digital trail. I think that it's a possibility. The way it happens so quick, as soon as we got off the phone, you know, not long after I saw it in my feed. So I was convinced, like, oh, they heard me talk about a coconut opener and now they're trying to sell me one. That's what it looked like. I understand. And that sounds creepy. And I empathize with being creeped out by it.

But given an alternate explanation that does not require Facebook to be clandestinely listening to you using your microphone, which one feels more likely to you? You know, I'm still kind of convinced that they might be listening to me. But I see that. Oh, Monique, you're killing me here. happening for you today. A coworker of mine had, is going to sound silly, but she had brought in a brand of cough drop that I had never heard of before. It was new. We were talking about how I had never heard of this cough drop before. And I didn't Google it. I didn't get on my phone and look it up. And I'm scrolling through Instagram like maybe an hour later. And all of a sudden

there's an ad for this cough drop. Alex Goldman, how do you explain that? Are you friends, on Facebook with this friend who brought in the cough drops? Yes. Okay. But can I add a caveat? Sure. I wasn't friends with them until after the cough drop incident. You're so screwed.

Oh, boy. Whether we prefer like a firm or a soft mattress, and a couple days later, I started getting served ads for a mattress company called Casper. So Casper may have decided that they have not penetrated the market in your town of people who are about to graduate college and are probably going to need a mattress when they move into their own apartments, it's very possible that they know how much money you make because they buy information about you from data brokers.

Facebook buys information. Sir, are you finding this convincing? It's a decent explanation. It just, this one sounds as plausible as my phone heard me, have this conversation with my friends, and immediately after that conversation, I'm being served as ad about my friend. mattresses. What could I possibly do to tip the scale in this situation? I, I, I, okay, let's try another person to see if you have any chance of Oh, I'm definitely going to convince someone. So you're now zero for four? I can't

remember three or four. I'll a startup idea of a milkshake. And then for the next week, I saw ads for Soiland in my Facebook scroll. And like, what was the lag time between you mentioning it and seeing Soylent dads? So I was visiting some family in a different town. And I remember the next week when I came back. So it was within a couple of days. Okay.

Where were you visiting? And where do you live? So I live in Des Moines, Iowa. And I was visiting my wife's cousins in Kansas City. Okay. Both hotbeds of soylent consumption. Do you have any friends who, uh, consume soylent?

I had a coworker who was trying it for a while. But I don't think I'm friends with him on Facebook. Do you have any friends who live in San Francisco? No. Can you...

PJ has to keep turning away from me because he's laughing too hard about my futile attempts to convince people that... I've never watched anything to do something so bad. It's like watching... Someone in the Olympics just fall down. I didn't think this would go like perfectly,

but I did not think it would go this catastrophically so fast. What am I supposed to do? The problem here, which is the same problem with reporting out this story, is that Facebook not only is like a black box that tends to not want to tell you about how their stuff works. It is done using so many complex algorithms that they don't even know. If I was like, hey, tell me, how this ad got served to this gentleman. The people of Facebook would say, like, I don't know the answer to that.

I feel like you're reverse convincing me. What? I feel like I'm starting to go to the other side now. Hey, my name is Julia. This is Alex. How you doing? I'm great. How are you?

I'm good. I'm going to try and convince you that Facebook is not listening to you. Is Facebook listening to you? Oh, 1,000 percent. So I was at a friend's house a few weeks ago. We were talking about a guy that she went to high school with, and I went to college with. We did not look him up.

We did not Google him. We did not go on his Instagram or his Facebook. Okay. And the next day, both of us got him as a recommended follow on Instagram. Huh. And this is like not somebody who I had interacted with online literally in any capacity for like a good many years. And it was both of us that got the recommended follow and you don't get the recommended follows that often. So it's definitely listening to me.

PJ's smirking at me because he thinks that I can't answer this. No, I'm just smiling because your face is covered in flop sweat. And you're absolutely right that I can't answer this one. Because ad targeting and the people you may know, data sets are totally separate. I haven't been researching this. I have no idea. I can't answer this one.

I do think it tells you, though, that the microphone is definitely listening. What is they mean for? We'll see. PJ can't. PJ can't keep it together. He's losing his mind. He thinks this is so funny. I just think it's funny because Alex had a lot of confidence.

He would have all the answers. And be able to explain to me. Why is it? I'm sorry, Julia. I can't answer this one. I'm going to have to let you go. That maybe they're listening to you and suggesting friends based on that,

but I haven't been paying attention to that. You're coming up. You need to follow this. I do say it. It is like irrefutably the microphone is on. Okay. I think that this is terrible. So.

You're not even not arguing. Why would I argue? He's like, why aren't you arguing? Why aren't you arguing? I don't know anything about how they, how they decide who they should suggest you as friends. They could be. But don't you think this seems that like the microphone is on and is listening and is recording information?

I have no idea. Oh, my God. BJ can't take it. You need to pull yourself together. I've got to convince someone before the end of the day, and I'm definitely not going to do it with Julia. Well, I think your argument that maybe the microphone is listening to you. It's not going to convince anybody.

It might. It might. Julia, thank you so much. So I wasn't able to convince anybody. But whether you think Facebook is listening to you or not, we are going to put a bunch of information up on our way. website about how to prevent them from tracking you as much as they do. And if you do believe that Facebook is listening to you, we'll also have instructions on there for how to disable the

microphone privileges for your Facebook app. You can find it at replyall.com.com slash don't track me. Reply all is PJ Vote and me, Alex Holdman. We were produced this week by Shruthy Pinaminen, Fia Bennon, and Damiano Marquetti. Production help from John Hanrahan. Our editor is Tim Howard. Our intern is Anna Foley. Fact-checking by Michelle Harris. The show is mixed by Rick Kwan. Special thanks this week to Zoe Kleinman, Christine McClellan, and Emily Taylor. Matt Lieber is sitting on your couch and looking around at how nice your house looks after you just cleaned it. Our theme song is by the mysterious Breakmaster Cylinder, and our ad music is by Build Buildings. You can listen to the show on Apple

Podcasts, Spotify, or wherever you get your podcasts. Thanks for listening. We'll see you in two weeks. Is that a moth? Oh, hey, that reminds me. I have this thing. There are these empty compartments in the back of the helmet. I don't know what they're for, but I hide things in there sometimes. And then I forget about them. This thing. It's basically a computer chip with a tiny rubber band. You power it on.

And it triggers musical notes as you wave it around in three-dimensional space. Rimsky Korsakov and I were tying these things to bees in the late 90s. I mean, yeah, obviously. He was off its face. If we can catch that moth. Nice, here tie it on. Okay, let it go. Except these at least can predict our own flight patterns. Mobs are stupid.