Dan Snow's History Hit - The Simulmatics Corporation

Episode Date: September 27, 2020

Jill Lepore joined me on the podcast to discuss The Simulmatics Corporation. Founded in 1959, it mined data, targeted voters, accelerated news, manipulated consumers, destabilized politics, and disordered knowledge—decades before Facebook, Google, Amazon, and Cambridge Analytica.Subscribe to History Hit and you'll get access to hundreds of history documentaries, as well as every single episode of this podcast from the beginning (400 extra episodes). We're running live podcasts on Zoom, we've got weekly quizzes where you can win prizes, and exclusive subscriber only articles. It's the ultimate history package. Just go to historyhit.tv to subscribe. Use code 'pod1' at checkout for your first month free and the following month for just £/€/$1.

Hi everybody, welcome to Dan Snow's History. It's such an important podcast today. We've got one of the most brilliant historians writing at the moment, or speaking at the moment, or being at the moment, Jill Lepore. She is a professor at Harvard University. She has her own podcast, The Last Archive. She's won prizes, and you'll see why. She's written a book about the Simulmatics Corporation, how they invent the future or certainly attempt to invent the future, how they use this new machine, a computer, back in the 1960s to try and analyse data about us, people, in order to sell us things, change the way we vote, understand us and change our behaviour. It is the history of technology that the tech giants do not want to talk about and refuse to acknowledge.

If you want to go and watch shows about American history, we've got plenty of them on History Hit TV. Use the code POD1, P-O-D-1, because you're a podcast listener. And then you get a month for free, and then your second month for just one pound a year or a dollar. So you can binge all of that historical content. We've got lots of US history on there. Lots and lots of podcasts about US history and electoral politics. So please, please go and check it out. But in the meantime, everyone, here is Jill Lepore.

Jill, thanks so much for coming on the podcast. Oh, happy to be here. Your book is so remarkable. Tell everyone about the Simulmatics Corp. So Simulmatics Corp was a pioneering data analytics company founded in 1959. It went bankrupt in 1970. But in those short 11 years, it tried out pretty much everything that is now done today by, you know, every major company that uses data, gathers data and uses algorithms to make predictions about human behavior. So they worked in all of the different realms that this kind of work is now done. They generally didn't succeed in any of those

realms because there just wasn't enough data to do the work. And also computing speed wasn't obviously anywhere near what it is now. And they didn't have machine learning. I mean, there were a lot of things they did not have. But the basic conception of gathering data, personal data, coming up with mathematical models to make predictions, and then selling those predictions as a commodity is the work that they did. What comes out from your book is we laugh at them because it was so unsuccessful. In fact, it was catastrophic, particularly when I go to Vietnam, which we might talk about in a second.

But I couldn't help thinking they were kind of right. They massively overpromised. Their election coverage in the New York Times, a catastrophe. You paint it so beautifully. I hate this, but they're on the right side of history. Their vision of the future looks like our present, doesn't it? I think it does. And, you know, maybe that's an error of looking at this era from such a greedy present. Do you know what I mean? Like we're so fascinated by the dystopia of the moment that maybe we see it earlier. But I think they really did establish so much of what characterizes our world today. And I'm not sure that I, I don't think the book is mocking. I mean, for one thing, I think all of these people were

extremely well-intentioned. The project really began with an attempt to try to get the Democratic Party in the United States to listen to black voters, to actually, you know, count black votes, to think about black voters as mattering. It still very much affects our elections. But what's weird to me as a historian is that the way they thought they could do that, convince the Democratic Party to move aside from segregationist conservatives, was to build a machine that could predict how black people would vote. You know, like that somehow you, like it really wasn't such a mystery in 1960 when these guys were building this machine. Like what were black?

Yeah. Like what did black voters want? I don't know. What do women want? I don't know. Like what? I'm sorry, but there's sit-ins across the South. There are people marching on the street.

It's not a mystery. You don't need an algorithm to solve it, but there's something to me so poignant and a little heartbreaking really about this crisis of mid-century american liberalism technocratic liberalism that really in some deep meaningful way it's a bunch of well-intentioned white male liberals puzzling over new people entering the electorate and new people making consumer choices and what what is to be done about that? Being kind of mystified by women and people of colour, and being convinced that answers to all of these mysteries could be found in machines. And the interesting thing is with this company that you chart, so in such

amazing detail, is they begin as these well-intentioned liberals, and then they pivot to kind of making money they realize there's enormous money and does that feel to me like i'm old enough to remember when the tech boom like in the 90s this was going to be a just an unstoppable liberal avalanche that was going to change our society for the better in every single way and now those same people are like well maybe that didn't work but we're making a ton of money is that something you see the gen the genesis of this of this revolution even as far back as the period you described and here's where it's that's actually a really great analogy.

And here's where, gosh, you do think the study of history might have been helpful there for the techno-utopians of the 1990s, you know, who did, you know, this sort of Stewart Brand sort of people, like, you know, they came from the counterculture and they had a vision, a kind of kumbaya, like the internet will end all divisions in society and will make us all connected to one another in a warm and loving way. And, you know, so like when Mark Zuckerberg says, well, Facebook, you know, our business model is to connect people to the people they love.

I'm sorry, I don't really believe that it started that way. But that's the rhetoric of the early Internet and early social media. But the actual business model seems to be to destroy democracy in any institution that might possibly support it for a profit. So yeah, I think that it does kind of follow that same path in a way. And then so then you wonder what were their lessons to be drawn from what happened to simulmatics? And surely there are. And the thing that struck me, the moments in the archives where I really was kind of knocked off my chair, or when I came across people in the 1960s, you know, as early as 1959, when the very first proposal for doing this work was being circulated as a top secret memo, there were

people who said, no, this would be, this could really be terrible. This, if we do this, then this will happen and then this will happen. And before you know it, we will have undermined democracy as we know it and all the institutions that support it. You know, there are people that had the capacity to see that. They were pretty much powerless to stop it. And they were certainly in the minority. But we now see all those costs and we still can't stop it. Well, that brings me on to the question about what might have been. I mean, you write history that makes us realise how contingent the present is, how nothing is inevitable. And during the book, I want to know, what stage could regulators, could politicians have intervened to create a different kind of present around this? Yeah, I got really fascinated. I mean,

the title of the book is If Then, which, you know, is a sort of syntax within Fortran, and how, you know, an infinite computer simulation works, right? If this, then that, okay, then if this, then that will happen. And that's how these early predictive programs worked. And in a way, that's how a counterfactual history would work, right? We could say, if then they had stopped this or put laws in place around that, then the following things would have happened differently. It's very hard for historians to adequately answer if-then propositions, right? And I think most of us are trained to not entertain counterfactuals for that reason. We can't really give a satisfying answer. There's no way to check your argument. There's, you know, we can't corroborate or verify any claim that is a

counterfactual. But there were a few moments where you could see, and I've spoken to people who were kind of compelled by these possibilities, the real moment came. So Lyndon Johnson, you know, who takes over the U.S. presidency in 1963 after the assassination of John F. Kennedy and institutes his Great Society program. The Great Society programs were really anti-poverty programs and equal rights programs

that required a lot of collection of data in order to provide federal assistance. You needed to know income levels in neighborhoods, you know, and compare them across cities. You needed to know voting rates in order to implement Voting Rights Act provisions. So the Johnson administration was really involved in the collection of data just at a time when the federal government had a lot of mainframe computers and was processing data. So in light of all of that, the Johnson administration proposed building what they called a national

data center that would be a sister institution to the Library of Congress, which collects books, and the National Archives, which collects manuscripts, that the national data center would collect data. And then it would be homogenized so that the Social Security Administration data could talk to the data in the Veterans Affairs Administration and the Fair Housing Division. And it seems, on the face of it, fairly straightforward. And it would have required a lot of rules and regulations about data in its anonymizing and whether it could be used in the aggregate

and in what format it could be collected, when it should be destroyed, after what period of time, would the public have access to it, et cetera. And there were proposals that kind of entertained all of these possibilities. But the public got crazy alarmed about the National Data Center, partly because people didn't understand computers especially well. And no one had one, right? Only giant businesses had computers at this time.

There are no personal computers. And because there's a lot of conservative concern about the overreach of Johnson's administration. So kind of from the left, people fearing a kind of Orwellian big brother violation of privacy, and from the right, people fearing federal government spending. So both the House and the Senate convened hearings and people came in and said,

you know, a lot of people came in and said, you shouldn't do this because this will violate people's privacy and then the government could know everything about you. The proposal was eventually defeated and so we don't have a national data center and never did. And so all those rules that were going to be proposed that would have been attached to it never got established.

But there's a moment at the hearings when this guy, Paul Buran from RAND, who's involved in the development of the predecessor to the Internet, ARPANET. He's the guy who works on one of the two important guys who works on packet switching, which is a necessary piece of the development of the Internet. And he kind of looks at the members of Congress and he's like, I don't know what you guys are talking about. Like a building? You don't actually need a building to hold data. Like, we're building an internet. Like, pretty soon, all the computers will be talking to one another. And data will live kind of in this ether, like almost in a cloud. Like, you don't need to build a building. Just stop worrying about, are you going to build this building? What you should be thinking about, you should absolutely collect the data

and homogenize it so that you could... We need you guys to make some rules here because we're on the eve of a huge explosion where computers will be transformed from storage devices and calculating devices to communications devices. When that happens, the horse is out of the barn.

Please make some rules about data and who can have it and what you can do with it. And can you sell it and can you buy it? And they're like, what are you talking about the computers can talk to one another? Like nobody, except for five guys at Stanford, understands that at the time. And the moment passes.

And I did this podcast called The Last Archive and I did an episode about the National Data Center for The Last Archive. And I talked to this guy, Arthur Miller, who was a very young law school professor at University of Michigan at the time. And he spoke before the Senate hearings into the National Data Center, and he opposed it on privacy grounds. And so I interviewed him. You know, he's an old guy now.

And he laughed and laughed and laughed. And he said, you know, within four years, we realized how wrong that was. Like, we should not. Like, by 1971, it was pretty clear that the data collecting that was going on was being done, not by the government, but by corporations. And we had no regulatory scheme in place. And, you know, even when we began to have rules about data collection, it was really only constraining the government. And it all happened too late. We just couldn't see far enough ahead. So, I mean, if then the National Data Center had gone through that, then I do think there would have been a regulatory scheme. But, you know, you can see, you know, I asked Miller, you know, could it have gone another way? And he said, no, it couldn't have gone another way. Nobody could have seen that clearly enough to convince Congress to act differently.

to convince Congress to act differently. I'm so struck by the fact that we still have these election day regulations here in the UK, and I know you do in the States as well. And they get down to details like, you know, national broadcasters not allowed to talk about politics on the day of the election. And they're all like Marquess of Queensbury rules over here. Meanwhile, like Twitter is like, blasting out fake charts and videos and like lies about unemployment numbers to people while they're like in the ballot. But like, it's, it's just, it's, I mean, it's, it's a wild west. It's crazy. Yeah, no, I mean, we have, we have the same, you know, the descendants of the Australian ballot reforms that, you know, Parliament adopted in 1872, my state adopted in 1888. You know, when I walk to my polling place around the corner, like when I get within a block of the building, no one can come up and talk to me about politics,

like they have to be on the other side of the road. It's like, and it's police. Like people are there, someone with a sign that says, you know, Trump, Pence or Biden, Harris comes, approaches me. The police will push them away and say, you can't interfere with this voter. Meanwhile, I could be getting a text at that moment that says your polling place is closed

due to an electrical outage. Please go home. And who's paying attention to that? No one. The other thing that your book brings out, the problem in the 60s was not just in computing power and available data, but also in entering the wrong kinds of data. And your Vietnam section is very interesting to me because, again, they attempt to use computing

power to build a giant psychological landscape, Atlas, encyclopedia of the Vietnamese people. But it fails not because of computing power. It fails because it's a stupid thing to do. Like, you've got all these guys in suits going out who are armed. They're armed, right? And they're talking to Vietnamese villagers about, like, their preferences. Like,'ve got all these guys in suits going out who are armed, they're armed, right? And they're talking to Vietnamese villagers about like their preferences, like the inputs are all wrong as well. And that seems to be a parallel, but similar problem. Yeah. And here too, I mean, I think these guys, like, you know, the young American graduate students who are out doing that work, they're guys who, you know, didn't want to fight in the war, have, you know, have gone to graduate school instead. They think that this is, you know,

winning the hearts and minds of the Vietnamese people is a better route than bombing North Vietnam. Like their intentions are in that sense good. They can't really see that you're not just going to, you know, going out with an armed military patrol to sit in a little tiny impoverished hamlet and ask people what they would watch on television if they had a television is not the same thing, you know, as the George Gallup polling company, you know, calling up a housewife in Milwaukee and asking her, you know, is she going to bake a frozen dinner that night? Like, it's just, it's preposterous and it's absurd. But the thing that was helpful to me is to understand that all of that work really comes in a deep way from the psychological warfare that lies at the heart of

the Cold War. I mean, we spend a lot of time thinking about the Cold War as having been an arms race, but it was also, it was called at the time, a minds race, right? Like it's a battle for the loyalties of ordinary people, especially in newly independent republics, post-colonial societies that are kind of choosing sides between the U.S. and the USSR. And, you know, we have all these methods that have been funded by the federal government that involve bringing together behavioral science in the 1950s and early computer science to try to figure out, you know, how to collect enough data about a population and its opinions and the means by which people's opinions can be changed,

that you could figure out what is the right set of messages to direct to which people to change their views. And that, you know, that's the basic work of psychological warfare. But in the U.S., it also becomes the basic work of, you know, kind of computer campaign consulted, political battles. It's how mass advertising works as aided by predictive analytics. For me, that sort of creepy, exhausted feeling you get at the end of the day when you've been dealing with your phone all day, you haven't been able to think straight, all these things have kind of been coming at you. That actually is, those are the wages of psychological warfare. Land a Viking longship on island shores. Scramble over the dunes

of ancient Egypt and avoid the Poisoner's Cup in Renaissance Florence. Each week on Echoes of History, we uncover the epic stories that inspire Assassin's Creed. We're stepping into feudal Japan in our special series

Chasing Shadows, where samurai warlords and shinobi spies teach us the tactics and skills needed not only to survive, but to conquer. Whether you're preparing for Assassin's Creed Shadows or fascinated by history and great stories, listen to Echoes of History, a Ubisoft podcast brought to you by History Hits. There are new episodes every week. Can you let me talk about these truths? It's one of my favourite history books. You're so good at capturing those moments,

whether it's Truman's defeat over healthcare or the narrow defeat of socialism in the 19th century, of capturing those moments that would have resulted in quite a different present. So the American journey, not as a sort of preordained and unstoppable super tanker, but as one that was far more delicate, if you like. And I think of Hillary Clinton losing in 2016 in those terms as well. What are some of the ones that you feel have been overlooked? What are some of the turning points of American history where it didn't quite turn? Well, thanks for those words about these truths. I really care about, I cared a lot about working on that book and bringing it to the public and hope that, you know, people have used

it as an opportunity to rethink the course of American history and in that contingency to see possibility, right? That if things as they are now, none of them are inevitable, then the future that someone's trying to sell you is also are inevitable, then the future that someone's trying to sell you is also not inevitable, that we have an enormous amount of control over the course of events. Moments that I found really compelling, the debates over the 14th and the 15th Amendment, the decision to set the right to vote for women aside, to not have those amendments to the Constitution bar discrimination by sex, that wasn't inevitable.

I think that, you know, it played out in a particular way. I think it could have played out differently, but that would have really changed the course of American history in a really interesting and dramatic way. I think the origins of our long, century-long culture war in the United States, kind of between science and religion, that kind of begins with the Scopes trial in 1925, where it comes to the public eye. John Scopes was a high school biology teacher who was charged with the crime of teaching evolution. I don't think that trial itself could have gone differently. I mean, Scopes was convicted and then, you know, later, you know, paid a nominal fine. I think that if people had made less hay of getting a lot of national attention for their cause,

they might have been able to see that fundamentalists and progressives actually shared a great deal in terms of their political commitments around aid to the poor and a more just and equal society. I think we are still really trapped by that divide. We have this weird divide between evangelical Christians and Catholics now, who are now on the right, and generally, and progressives on the left, when there should be so much common cause about justice and inequality across those groups. That's the legacy of that culture war. I think that could have gone differently. So I might point to those two as really signal moments. I do also think the ERA battle in the U.S. could have gone differently.

The Equal Rights Amendment was introduced to Congress in 1923. It finally passes in 1972 and then is defeated because the opponents to the ERA, I tell the story in great detail in these truths, attach equal rights to the cause of the emerging pro-life movement and they're not actually in any meaningful way this attached but they're kind of buggyed together by the conservative mastermind Phyllis Schlafly I think the women's rights movement got completely played there and is still being completely played

honestly yeah Now it seems like most of my moments involve failures of the left. Yeah. And then in the 19th century as well, with some of the more popular candidates just failing to seize power. But when I read your history, I find it sort of poetic and inspiring, but there's something that's, I think, sometimes peculiar to American academics. There's still a patriotism there I think there's still an obsession with that founding moment and still a reverence for what that flawed and compromising generation tried to achieve at the end of the 18th century that I think we foreigners still have a problem well understanding it in depth and it's odd because I've listened so many interviews with you and I've read everything else you've written, and there's, you know, you're struggling, but you still

clearly believe in the proposition that was laid out 250 odd years ago. You know, I guess nearly all of it is entirely genuine, in the sense that I do believe in the consent of the governed. I do believe in liberal democracy as the best form of government that is available to us at the moment. I very much believe that the challenges that liberal democracy as the best form of government that is available to us at the moment. I very much believe that the challenges that liberal democracy faces are challenges that could be met by public-spirited people of great energy and the spirit of cooperation and talent. And I don't see that happening right now, but I certainly believe that things could turn very quickly in a positive direction. But part of it is, I think it's important to, as someone speaking to the public in a time when there's a lot of ammunition available for people on the right. I mean,

our president gave a speech last week, maybe the week before, about the need to replace history education in schools and colleges with patriotic education on the back of an argument that those of us who teach high school and college history are teaching Americans to hate the country. So Frederick Douglass gave this speech near the end of his life. He died in 1895. It was 1894. He's an old, old man. He was not well. He went to a school in Manassas, Virginia, a black school. So think of Douglass.

He was born into slavery in Maryland, ran away, saw and suffered, witnessed extraordinarily brutality and atrocity, ran away, having taught himself to read, becomes a writer and a speaker, one of the most important statesmen of the 19th century, urges the United States towards a position of emancipation, leads his people through the Civil War, only to see in the 1870s and 1880s the rise of the Ku Klux Klan and the spread of Jim Crow laws across the South, which basically re-enslave black people. So now it's 1894.

He's been through all of these battles, and in many ways the country is back where it was when he was born. And he gives a speech to these young black children about the importance of hope. Why do the world need historians? What should historians be doing? I think it's interesting having spent some time with the Simulmatics Corporation and seeing the kind of great shuttering

that the Academy does upon realizing how implicated social science has been in the U.S. campaign in Vietnam. You know, the support that academic historians and more other kinds of social scientists gave to this immoral war leads American historians and not so much the social science fields that were involved, to really pull out of public life, to say, you know, it's indefensible to be part of public life,

to engage in conversations about policy and what's going on in the world today, because the next thing you know, we'll all be complicit in another Vietnam. And so there's this incredible retreat of intellectuals from public life in the U.S., you know, at the end of the 1960s. I understand that. I can see how that came to pass. I probably would have made the same call. But I do think that public discourse really, really, really needs informed historians who are scholars, whose work is accountable to evidence, who are engaged in the method of a humanistic discipline and who are not toadies and flacks and frauds, which is a lot of what you see standing up and offering accounts of the past. So I think historians need to be, not all of us, but more of us than currently do, need to be willing to bring their work to a wider public or to do their best to

try to and to not demand that the public come to them, but to figure out a way to go to the public. Just one more question. The election in November, some people are ringing the alarm bells. Some people are saying this is not a drill. This is the big Ben Franklin moment. It's a republic if you can keep it. Other people are saying, oh, you know, we've been through civil wars.

We've been through all kinds of, we've had elections bought and sold in the past. We've had British intervention in presidential elections. We've had foreign involvement. Where are the past. We've had British intervention in presidential elections. We've had foreign involvement. Where are you on what is at stake in this election and how precedented or unprecedented, if that's the right expression, this is? So, you know, up until just a couple of years ago, when people asked me the precedented question,

I would just kind of wave it aside. Oh, everything has a precedent. But something turned for me around 2018, you know, before the impeachment, but, you know, with the separation of children from their families at the border, new revelations about detention facilities. I think we can't responsibly as historians say that this is run of the mill. This is a return to something we've already experienced. This is, you know, that Trump is, you know, sometimes people will say he's Joe McCarthy meets George Wallace.

He's none of those guys. Those guys weren't the president. I hate, hate, hate the whole language of this is an existential election. What the hell does that even mean? I hate the false panic. I hate the dualism. I am enough of an optimist that I am still expecting there to be a bunch of prominent Americans who stand up and offer up. Obviously, they're not offering up a unity ticket, but more people who stand up and say, what I'm here to do is not to tell anyone how to vote, but I'm here to talk about the integrity of our elections. I think if enough of that happens, the election could actually go off without leading to civil war. But I would say I have some pretty grave

doubts that this election won't involve an unprecedented amount of political violence on the days before, day of, and days after. I've lost a lot of sleep thinking about that, wondering what could forestall that. And with each passing day, there just seem to be more indicators that a lot of people seem to want that to happen. Jill Lepore, thank you very much indeed. The book that we're allegedly talking about is called If Then. You've also got These Truths and then the short one you wrote after that. This America, which is an argument about liberal democracy.

Thank you very much for coming on this podcast and good luck with it. Hi everyone, it's me, Dan Snow. Just a quick request. It's so annoying, and I hate it when other podcasts do this, but now I'm doing it, and I hate myself. Please, please go onto iTunes, wherever you get your podcasts, and give us a five-star rating and a review. It really helps, and basically boosts up the chart,

which is good, and then more people listen, which is nice. So if you could do that, I'd be very grateful. I understand if you don't want to subscribe to my TV channel. I understand if you don't subscribe to my TV channel. I understand if you don't buy my calendar, but this is free. Come on, do me a favour. Thanks.