Science Vs - DNA Kits: Can You Trust Them?

Episode Date: April 12, 2019

Millions of people are sending off their DNA to companies like Ancestry.com and 23andme to find out where they come from, and what diseases they might get. But how much can you trust these DNA kits? To find out, we speak to anthropologist Prof. Jonathan Marks and geneticist Dr Adam Rutherford. Check out the full transcript here: http://bit.ly/2OSICOD Selected references: This academic paper on genetic ancestry testingAncestry.com’s white paper The genetics of Alzheimer DiseaseA perspective piece on genetic privacy Credits: This episode was produced by Rose Rimler, with senior producer Kaitlyn Sawrey… with help from Wendy Zukerman, Meryl Horn and Michelle Dang. We’re edited by Blythe Terrell. Fact checking by Michelle Harris and Michelle Dang. Mix and sound design by Peter Leonard. Music by Peter Leonard, Frank Lopez, Emma Munger and Bobby Lord. Recording assistance from Cole del Charco, Madeline Taylor, Carmen Baskauf, Ian Cross and [Mareek] Marijke Peters. A huge thanks to everyone who spat in a tube for us, especially Toni Magyar and Alex Blumberg, and to all the researchers we got in touch with for this episode, including Dr. Wendy Roth, Dr Deborah Bolnick, Dr Celeste Karch, Professor Nancy Wexler, Dr. Robert Green, Dr Catharine Wang, and others. Thanks also to the teams at Ancestry.com, 23andMe, and MyHeritage. Thanks to the Zukerman Family and Joseph Lavelle Wilson.  Learn more about your ad choices. Visit podcastchoices.com/adchoices

Bumble knows it's hard to start conversations. Hey. No, too basic. Hi there. Still no. What about hello, handsome? Who knew you could give yourself the ick? That's why Bumble is changing how you start conversations.

You can now make the first move or not. With opening moves, you simply choose a question to be automatically sent to your matches. Then sit back and let your matches start the chat. Download Bumble and try it for yourself. What does the AI revolution mean for jobs, for getting things done? Who are the people creating this technology and what do they think? I'm Rana El-Khelyoubi, an AI scientist, entrepreneur, investor, and now host of the new podcast, Pioneers of AI. Think of it as your guide for all things AI, with the most human issues at the center.

Join me every Wednesday for Pioneers of AI. And don't forget to subscribe wherever you tune in. It's season three of The Joy of Why, and I still have a lot of questions. wherever you tune in. So brace yourselves. Get ready to learn. I'm Jana Levin. I'm Steve Strogatz. And this is... Quantum Magazine's podcast, The Joy of Why. New episodes drop every other Thursday, starting February 1st. Hi, I'm Wendy Zuckerman, and you're listening to Science Versus from Gimlet. This is the show that pits facts against phenotypes. Nerd alert!

Today on the show, DNA test kits. Can you trust what they're telling you? Millions of people are sending off their DNA to companies like Ancestry.com and 23andMe, and they're hoping to find out where they come from and what diseases they might end up with. These companies say for a mere $100 or so, they can reveal these amazing things about you and your family, which can feel really exciting, like it did for some of our colleagues at Gimlet. There's this, like, hole in my family tree.

I'd love to fill it. I'm just curious, you know, how I fit in to the rest of humanity. No one in our family actually seems to know for certain where we come from. Two of our colleagues in particular have a rather mysterious ancestry. There's Alice, who looks a lot like her Irish-American mom and nothing like her Indonesian dad. Like my hair is kind of red and I have green eyes and I like look like I'm from Ireland. Like I present completely white, but my dad does not. And then

there's Gabe, who knows he's got some Irish and Puerto Rican in his family. But then there are these big gaps that he is very curious about. We realize that we don't know a ton about our family history outside of, like, the stories that we tell each other. I do think that there's an element there that feels like I would have a stronger sense of myself knowing where I came from. And because there's blanks in his past, Gabe knows very little about what diseases might be lurking in his DNA. There's just, like, a lot of questions around what is our medical history.

Are you excited about knowing or is it scary? It is a little scary. It's like I would rather know than not know. The promise of DNA tests is that you can have answers to all these questions and more. And all you have to do is spit in a little tube. Science Versus producer Rose Rimla walked Gabe and Alice through this. Okay, should we get spitting?

Yeah, let's spit. Okay. So you're just going to watch me spit for a while? I'm just going to watch and record you spit. Is this something we want to get on audio? You're spitting very gracefully. You're doing a great job. All right, so I'll see you in a couple weeks.

Yeah, see you, Rose. I'm so excited. Yeah, see you when the results are in. Oh, my goodness. While some people can't wait to send off their spit and get some answers, others are more skeptical. They're wondering, how accurate are these results? And hang on a minute.

Is it okay to be giving something as personal as our DNA to a private company? It just feels weird to me. This makes me ask myself how, like, is it worth it? So that's what we're going to find out today. Is it worth it? By answering the following questions. One, can these tests really tell you where your family is from? Two, can they reveal diseases hiding in your DNA? And three, what can these companies do with your DNA once they have it? Can they sell it to the highest bidder? Perhaps your insurance company? When it comes to DNA kits, there's lots of... Spitting very gracefully.

But then there's science. Science vs DNA kits is coming up just after the break. Welcome back. Today we're tackling DNA testing kits to find out, can you trust them? We've come to expect a lot from DNA. When cops are chasing bad guys on TV, DNA evidence always seems to clinch the case. But when we send off our DNA to sites like 23andMe and Ancestry.com, they

come back with these curious results, and it's harder to know what to make of them. So let's start with one of the biggest reasons that people do these tests. To find out, where do I come from? How's it going? Hi. Hi. Our colleague Alice, who looks like she belongs on stage with Lord of the Dance, despite having an Indonesian dad,

and Gabe, who had some gaps in his family history, couldn't wait to bust open their results. So you got an email that was like, ding. Yeah. And it was, I have to tell you, it was so hard not to open it by myself. Let's open it up. Yay. Okay, flip through the story of your DNA. Oh

my god. Gabriel, welcome to you. Wow, they really dive right into that, don't they? Yeah, so 46.5% British and Irish with ties to 10 other populations. Sub-Saharan Africa, 11.1%. That's not something that you predicted. No, I never knew that. That's totally wild. Gabe was surprised that a chunk of his ancestry was from sub-Saharan Africa, while Alice found something surprising too. Even though her dad is Indonesian. I am a quarter East Asian, but only 2.8% of that is Indonesian. That's really interesting. I'm going to have to ask him about that because now I'm like, Dad, do you know who your family is? So there was a lot of excitement in the studio,

but are these results accurate? Okay, well, here we are. To find out, we talked to Jonathan Marks. They're going to edit this to make me sound intelligent. A professor of anthropology at the University of North Carolina at Charlotte. So yeah, so in between spitting into the tube and receiving your results, what exactly is happening there? And first off, he says that once these companies get your DNA, they don't scour through every single part of your genome. Instead, they zoom in on the chunks of DNA that make each of us unique. They then compare those chunks to this big DNA database that they have.

They've collected DNA from, you know, people all over the world. By recruiting people or using customers and even publicly available sets of DNA. So if you get a result saying you're 35% German. What they're telling you is that from their sample of DNA from different countries, you know, you have about a 35% match to their samples from Germany. So it's actually a measure of how similar you are to the DNAs that they have on file. John and about half a dozen experts that we spoke to said that what these companies are doing here is actually a pretty reasonable way

to take a stab at someone's genetic family tree. OK, great. So of the DNA they're looking at, 35% matches the Germans. But the big question is, who are these Germans? Like, whose DNA gets to represent German-ness in the database? Well, here's where things get messy.

Because it's not like there are tiny little bratwursts in the DNA of people who are German. So, sites like 23andMe and Ancestry.com have to decide who's German. And here's how they do it. Often to be considered German or French or Indonesian, at least both sets of grandparents had to be born in that country. And the thing is that sometimes even if they find someone who fits their granny criteria, all their grandparents born in Germany. But that person's DNA looks different to the other German people in their database,

then these companies will throw them out of the pile. They basically say, One day you're in, and the next day you're out. Which brings us to John's first big gripe about these ancestry tests. It's that these companies are basically creating a world of model Germans or model Frenchies. But there are no such people. Geneticists like to imagine the purity of the people that they're working with because it simplifies their calculations. John says that we're all mixed up. We don't fit into neat little boxes.

And that's for a really obvious reason. Sex. Humans have been travelling across the globe, having a lot of sex with a lot of different people for a very, very long time. And it means that all of our DNA is kind of mixed up. Take, for example, Europe. The Vandals and the Visigoths and the Huns and the Vikings, and, you know, there were all these migrations

and all of these movements of people. And where there's migrations and movements of people, there's going to be gene flow. And that's the polite term for anything from, you know, marrying your neighbors to mass rape. So there's always genetic contact between populations and there's always movement of populations. So the idea that people just sat still for prolonged periods of time is just a fallacy. On top of this frustrating fallacy, most of these DNA databases are filled with customers who have a European ancestry. So for a ton of countries, they don't have a big sample to work with.

And this can have real consequences for people's results. So take Indonesia. Alice's dad is Indonesian, but her results said that only a fraction of her DNA was from there. Well, it turns out that 23andMe has very few people in their database from Indonesia. In fact, they combine Indonesia with Cambodia, Thailand, Myanmar and Malaysia. And guess how many people are in this database for all five countries? A hundred and twenty-four people. A hundred and twenty-four! To represent a region with more than 400 million people.

Now, the big testing companies know this is a problem and they say they're working on it, trying to make their databases more diverse. But bottom line, maybe Alice's family is Indonesian, but with just 124 people in its database, the test can't tell her that yet. We talked to Alice about this. 124 people. Yes.

Whoa, that's it? That's it. That's it. That's crazy. Right? So who am I? What am I? Where do I come from? We don't know. We still don't know.

So it'll be interesting, like, as this database grows, I imagine your results are going to change quite a lot. Wild. Wow. So there are some problems with these databases, which might mean they get your ancestry wrong, particularly if your family's not European. But there's another place on their results page

where things might be a bit off, and it's in those precise numbers they give you, like Gabe finding out that he was 11.1% Sub-Saharan African. Rose asked John about this. Someone at Gimlet, one of our colleagues, he took a test from one of these companies, and it said that he is 11.1% Sub-Saharan African. What does that mean? The answer is, it's not really clear. They're giving you a number that's very precise,

but you don't know how accurate it is. Basically, don't take these precise numbers as gospel. And here's why. When these companies compare your DNA to their less-than-perfect database, they're basically using an algorithm that takes lots of chunks of DNA and asks, what's the chance that this chunk of DNA comes from Nigeria or, say, Germany? And what about this other piece of DNA? And, hey, you, DNA over there, you look a little French,

but you could be British. The algorithm then crunches the numbers, making its best guess on what the devil those chunks of DNA are, and it repeats this for thousands of pieces of DNA until it gives you these seemingly precise numbers. But because the world is messy and we've all got each other's DNA, every time that algorithm makes a call, there's a chance they get it wrong and put you in the wrong box. So you get these results that look really precise, but they're not. This is a very, very noisy system

that they're playing with here because human ancestry is noisy. And this nerd gripe becomes a real issue when you get results saying you're some tiny percentage of something. You know, 2% Italian, 1% Brazilian. Does that mean anything? Nope. Tell me more. I think I came out 1% Korean. Oh, are you 1% Korean?

No. I think what that means is that there are just some little regions of the DNA that they've misclassified. In fact, when the company later updated his report, that 1% Korean disappeared. But John says that at first, when we get results like this, it's so easy for us to start telling ourselves stories about who this Korean ancestor might have been. But that's what these companies are kind of doing. What they're selling you is a story. And it's up to you to decide whether you like that story,

whether you want to reject that story, whether there are other stories available that you like better. And John says there is something comforting about having a story about where you come from. He gets it. In fact, he's got one for you. Let me tell you a story. My dad took four machine gun bullets in the leg during World War II. He was in the army hospital and they wanted to amputate his leg.

But a Scottish surgeon managed to save it. And from then on, John's dad always wanted to be Scottish. A few years ago, John was in Scotland and passed by this shop that sold Scottish ancestry certificates for tourists. So he went in and talked to the shopkeeper. I told him a story about how we have this family legend about how our family began in the highlands of Scotland and then emigrated to Russia and then emigrated to the U.S. Did you make that story up?

Absolutely. It was a total lie? Yeah. Well, that's how I got my certificate. Does your dad know that? Yes. Yes. He was delighted to receive it.

Even though he knew it was based on nothing? Well, no, I mean, now he's Scottish. He'd always wanted to be Scottish. So when it comes to your ancestry, generally speaking, the science behind these tests makes sense, but they are far from perfect. If it comes back saying that half your DNA is Irish, yeah, you've probably got some Riverdance in you.

But for that exciting 1% Korean in your DNA, well, it's a bit like sweet-talking a Scottish shopkeeper into selling you a phony certificate. After the break, we'd raise the stakes because one DNA company promises to do more than tell you about your ancestry. They say they can peek into your DNA to reveal diseases that you might get. But can you trust them? Welcome back. Today we're tackling DNA kits.

You know, that Chrissy present still gathering dust under the dying Christmas tree. We are finding out whether you should finally spit in that tube and send it off. And so far we've learnt that when it comes to telling us information about our ancestry, these tests aren't bogus. But take them with a grain of salt.

Now we're diving into our health to find out what can these results tell you about whether you're going to get sick. There's only one of these DNA testing companies that is FDA approved to send you health info without going to a medical professional. And that's 23andMe. Now, this company says that they can tell you important information about your risk of getting conditions like Alzheimer's, type 2 diabetes, Parkinson's and even breast cancer. Our colleagues Gabe and Alice were nervous about these results, particularly Alice, who talked it through with our producer, Rose Rimler. Do you want to look at the health stuff?

I do. Okay. Yeah, I'm scared. Alice was so freaked out about what these results might tell her that she wouldn't look. Do I really want to know that about myself? But Gabe decided to open Pandora's box and read his genetic health report. And one thing stood out.

Alzheimer's. Oh. Okay, so one variant detected in the APOE gene. People with this variant have a slightly increased risk of developing late-onset Alzheimer's disease. Lifestyle, environment, and other factors can also affect your risk. So what do you think? I don't know. It is a little nerve-wracking, I'll say that. Gabe's 23andMe report said that he has a version of a gene that doubles his risk of getting Alzheimer's by the time he's 85, which sounds scary. So we took it to an expert who's really familiar with these tests. 23andMe take this

stuff really, really seriously, right? And they are, you know, they have proper scientists and proper clinical geneticists. This is Adam Rutherford. He's a British geneticist affiliated with University College London. And Adam told us that these results aren't rubbish. To get them, 23andMe scans your DNA, looking for genes that studies have found increase your risk of certain diseases. You look at a particular gene which is associated with breast cancer or, you know, a bunch of other stuff. So they only look at the positions in your genomes which are known to be interesting

for the types of characteristics that people are interested in. So that version of the Alzheimer's gene that Gabe has, well, it bumps up the risk of getting Alzheimer's by quite a bit. So for the best data we have, a man's risk of getting Alzheimer's by the time he's 85 is around 10%. For men with this version of the gene, it's a little over 20%. And studies in women have found that having it carries a little more risk. So Adam says at first glance, this seems like a big deal.

I'd imagine that would be absolutely terrifying to discover that you've got double the risk of getting Alzheimer's. Adam doesn't have to imagine. That is me, in fact, because 23andMe reports that I do, I'm in a higher risk category for Alzheimer's. Were you a little worried? I don't want to sound all macho about this, but I mean, the answer is just no. The answer, I went, huh, how about that?

It turns out that Adam is super chill about this result for a couple of reasons. The first is that he and Gabe only have one copy of this bung gene. So if you have two copies, your risk for Alzheimer's goes up even more. But more broadly, Adam says that even though having this version of the gene creates a real risk, it is far from a sure thing. The thing is, it's perfectly possible for me to have that version of the gene and not get Alzheimer's. I probably won't get Alzheimer's, regardless of whether I have that version of the gene and not get Alzheimer's. I probably won't get Alzheimer's regardless of whether I have that version of the gene.

And it's perfectly possible to not have that version of the gene and get Alzheimer's, right? So whatever the results are, they are not a fate sealed in your DNA. Basically, when it comes to the types of diseases that 23andMe can tell you about, like Alzheimer's or breast cancer, they're just showing you one piece of the puzzle here. And if you're going to get these diseases, there are so many pieces that have to fit together, including other genetic mutations that 23andMe doesn't test for, and even our environment, like our diet and how much exercise we do. And all these things come together in really complicated ways.

That is what most human genetics is like. It's a mess. It's a really interesting mess, but dear God, I wish I did physics because it's so much easier. All I'm saying here is this stuff is wickedly complex and we don't understand enough about it to be able to say definitive things. All of this kind of makes it feel like these tests are totally useless. But they're not. Because even though we don't know everything about the link between genes and these yucky diseases, these tests can still give you some information that might be useful. So you may find out that you have a version of a gene that increases your risk of breast cancer, and then you can go see a doctor about it.

Or you might find out, hey, you don't have a version of one risky gene, which is nice to know. There was an interesting thing that came out of those same results. So Adam's wife has a family history of emphysema, and Adam saw in his test that he didn't have a gene that would have upped his risk of getting it. It was reassuring to me to see that I wasn't a carrier on the grounds that that reduces the probability

that any of our children would have that disease. So in that sense, that was, you know, I found that slightly reassuring. So bottom line, Adam says these tests can give you a little bit of information, but you have to be really careful about how much you expect from them. My shrugging of my shoulder is based on 25 years of studying genetics. Most people who do those tests don't have that, and that bugs me. If you were a professor and someone gave you a site like 23andMe as an assignment, what grade would you give it? I would ask them to I would ask them to think about the question a bit more harder that's like c plus that's c plus I know I think I think I'd say listen it's an interesting project

we've done an interesting thing here but it doesn but it doesn't really qualify for my course. So you're going to have to do it again better. Okay, so it's a passing grade on the disease front, not a science slam dunk. Still, though, it feels like these DNA tests are a bit like a fun science adventure. You know, you get some fun mail with some pretty pie charts. So we had to wonder, what are the risks here? Well, when we asked Gimlet folks if they wanted to do these tests for this show, a lot of people said they were interested and they got on board. But then when the spit hit the fan, they were like, actually, maybe not. It just feels weird to me.

Yeah, it just feels like sensitive information to just give away to a private company. That's how they get you. Like, that's how they get you. What if that data ends up in the wrong hands, say, later on in life? Like, insurance company get a hold of that data. So you're going to do it? Hell no. I think I'm going to pass. Okay.

Part of the feeling here is that this is the most intimate data we have. It's our building blocks, our DNA. And once we send it out into the world, we can't get it back. So the big question that we wanted to know is what can these companies do with it? We're hearing stories about these companies handing over our DNA to Big Pharma and even the cops. So we looked into it. And it turns out that 23andMe basically lets you say, no, don't give my data to Big Pharma. And as for the fuzz, well, for now, the major DNA companies say they won't share your data with police unless there's a court order. But these folks aren't the only ones who might want to get their hands on your DNA.

What about employers and health insurance companies? If you find out that you have a gene that increases your risk of breast cancer. Can they get a hold of that? We talked to Adam about this. It's not a complete wild west. That would be to mischaracterise this situation. There are laws in place, especially in the States. And one of the big laws here is the Genetic Information Non-Discrimination Act, or GINA. And it basically says that health insurance companies can't use your genetic info to decide how much to charge you. It also says that employers

can't ask about that genetic information to make decisions like hiring or firing you. And this act, it should apply for the stuff that you learn from sites like 23andMe. Science Invest's producer, Rose Rimla, hey, Rose. Hey. Has been going down the rabbit hole on this one. So, Rose, what is going on here? How good is Gina at protecting us? So I didn't see any cases where an employer or health insurance company

tried to use information from 23andMe or from Ancestry.com or any of these other sites. There is this like one amazing case I did find that does show that there is definitely like this law seems to be holding up, at least when it comes to your employer. And I want to tell you about it. And just a heads up, if you've got kids listening, this does involve some bad words. FYI. What happened in this case? Okay, so it started in 2012. There was this warehouse company in Atlanta.

And the problem was that somebody at the company was doing a shit job. And I mean literally. They were like taking shits and leaving them in piles all over the workplace. In piles? Just like random shits and leaving them in piles all over the workplace. In piles? Just like random shits. Why would you do that? That's what the bosses wanted to know. Or more importantly, they wanted to know who was doing it.

The motivation wasn't so important. It's probably secondary. And so they got like cheek swabs from a bunch of employees to find the culprit. It was an ultimate crap down. Yeah, they're crapping down on the illicit pooping. And it's a very high tech CSI type investigation, if you will. Wow. Yeah. And so actually, so two of the people that they requested DNA from, they actually were exonerated. And shortly after that happened, these guys were like, you can't request my DNA. My employer cannot request my DNA. And they brought a case under

Gina and they won. And the judge is like, you're right. Like this case is what will now become precedent for the power of Gina. This case is brought up all the time in, you know, among scholars or journalists who write about this issue. They're like, well, there was the case of the devious defecator. The devious defecator. Which is what the judge called the case when she issued her decision. The mystery of the devious defecator. So thank God somebody felt the need to stink up the warehouse. You know, do you think that the case of the devious defecator and this law, does this protect us enough? So, I mean, Gina, there are definitely some protections under Gina.

Like if you're shitting in the warehouse and you don't want your boss to find out it was you. But there are definitely some limitations. And one big one that people bring up a lot is that Gina doesn't cover any, has no coverage for things like life insurance or disability insurance. So there are other things that might come up where your genetic information might matter, but Gina doesn't cover them. All right. So neither of us have gotten our DNA tested. But now that we've done this research, would you do it in the future? I probably wouldn't, mostly because I feel like the benefits are small and the risks are small. But if the benefits are small, why take any of those risks?

On the other hand, I have, you know, heard stories of people who are really glad that they did this. Maybe they found out something that was meaningful to them. And I know Gabe and Alice, like, they don't regret doing it. They had fun. But for me right now, I don't want to do it. Right. Is that how you feel? Yeah, I think so.

I would not do this. Because, I mean, I think I was swayed by geneticist Adam Rutherford, who we just heard from. And when we were talking about kind of the wider implications of where this might go, he quoted a great philosopher of our time. And I think that really convinced me that this maybe wasn't the right thing to do for me. Here he is.

Jeff Goldblum, Jurassic Park, saying, DNA is the most powerful tool in the known universe, and yet we wield it like a child who's found his dad's gun. That's just right. We talk about this stuff all the time. We wave it around in court cases and on ancestry tests. But we are at the beginning of understanding how our genomes work. And just so much caution is required. That's science vs DNA Kids. Next week, science versus race.

With all this talk about genetic differences, we had to ask, how does race fit into this? Like, biologically, does race exist? Human beings, regardless of race, are more than 99.9% the same. The problem, though, is that the 0.1% is a lot of genetic variation. It's the end of the episode as we know it. Which means it's time to find out how many citations in this week's episode, Rose Rimler. About 100. About 100?

You're going to do about 100 citations? You're not even going to give me specifics? There's 96 citations in this week's script. 96 citations? That's right. And if people want to read these citations, where should they go? They can check out our show notes. They can find that wherever they are listening to the podcast

and on our website, scienceversus.show. Thanks, Rose. You're welcome. This episode was produced by Rose Rimler with senior producer Caitlin Sori. Help from me, Wendy Zuckerman, Meryl Horne, and Michelle Dang. We're edited by Blythe Terrell. Fact-checking by Michelle Harris and Michelle Dang. Mix and sound design by Peter Leonard.

Music by Peter Leonard, Frank Lopez, Emma Munger and Bobby Lord. Recording assistance from Cole Del Charco, Madeline Taylor, Carmen Baskauf, Ian Cross and Marique Peters. A huge thanks to everyone who spat in a tube for us, especially Tony Maggia and Alex Bloomberg. An extra thanks to all the researchers we got in touch with for this episode, including Dr. Wendy Roth, Dr. Deborah Bolnik, Dr. Celeste Karsh, Professor Nancy Wexler, Dr. Robert Green, Dr. Catherine Wang and others. Also, thank you to the teams at Ancestry.com, 23andMe and MyHeritage. And finally,

thanks to the Zuckerman family and Joseph Lavelle Wilson. I'm Wendy Zuckerman. Back to you next time.