Invest Like the Best with Patrick O'Shaughnessy - Brian Christian – How to Live with Computers - [Invest Like the Best, EP.140]

Episode Date: July 30, 2019

My guest this week is Brian Christian, the author of two of my favorite recent books: Algorithms to Live By and The Most Human Human. Our conversation covers the present and future of how humans interact with and use computers. Brian’s thoughts on the nature of intelligence and what it means to be human continue to make me think about what works, and life, will be like in the future. I hope you enjoy our conversation. For more episodes go to InvestorFieldGuide.com/podcast. Sign up for the book club, where you’ll get a full investor curriculum and then 3-4 suggestions every month at InvestorFieldGuide.com/bookclub. Follow Patrick on Twitter at @patrick_oshag   Show Notes 1:11 - (First Question) – Summarizing his collection of interests that led to his three books 2:59 – Biggest questions in AI 3:43 – Defining AGI (Artificial General Intelligence) and its history             5:18 – Computing Machinery and Intelligence 7:54 – The idea of the most human human 9:59 – Tactics that have changed the most in learning to be the most human human 16:10 –Tests for measuring AGI and updates made to them 20:12 – Concerns for once we have AGI 26:06 – Self-awareness as a threshold for AGI 31:58 – Skeptics’ take on AGI 37:14 – Advice for people building careers and how AGI will impact work 38:16 – Explore/Exploit trade-off 44:57 – How to explore/exploit applies to business concepts 49:16 – Impacts of AGI on the economy 52:40 – Highlights from his second book 57:39 – Kindest thing anyone has done for Brian   Learn More For more episodes go to InvestorFieldGuide.com/podcast.  Sign up for the book club, where you’ll get a full investor curriculum and then 3-4 suggestions every month at InvestorFieldGuide.com/bookclub Follow Patrick on twitter at @patrick_oshag

Hello and welcome, everyone. I'm Patrick O'Shaughnessy, and this is Invest like the Best. This show is an open-ended exploration of markets, ideas, methods, stories, and of strategies that will help you better invest both your time and your money. You can learn more and stay up to date at investorfieldguide.com. Patrick O'Shaughnessy is the CEO of O'Shaughnessy Asset Management. All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of O'Shaunacy Asset Management. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions.

Clients of O'Shaughnessy asset management may maintain positions and the securities discussed in this podcast. My guest this week is Brian Christian, the author of two of my favorite recent books, Algorithms to Live By and the Most Human Human. Our conversation covers the present and future of how humans interact with and use computers. Brian's thoughts on the nature of intelligence and what it means to be human continue to make me think about what work and life will be like in the future. I hope you enjoy our conversation. I'd love to begin with some framing from you around just what your general interest is that unites the first two books that you've written and published and the third that you're working on now. How would you sum up your set of interests?

I think I have this very on-paper, eclectic academic background where in college I studied computer science and philosophy and then I went to graduate school for creative writing. And I remember at the time raising a lot of. of eyebrows at family reunions and so forth, people saying, so you're studying computer science, this very precise exacting engineering discipline and philosophy, like this abstract, vague field. How do these two things connect? Increasingly people don't ask me that question. And I think we're perhaps just coincidentally living through the great synthesis of computer science and philosophy. That as the sort of computational metaphor for mind becomes increasingly literal, we've developed these neural networks, for example, starting in the 1940s, through this analogy to the way that the nervous

system worked and are now discovering in the last 10 years that this is the best mechanism that we know of for actually implementing AI, which maybe shouldn't surprise us that we're rediscovering what evolution found after millions of years of trial and error. Computation gives us a way of asking questions about what does it mean to think, what does it mean to have a mind, what does it mean to make rational decisions. And I found, as I was studying these two things in undergraduate, the questions that I was interested in asking in philosophy about the mind were in some ways better answered through the tools and the vocabulary of computer science than they were in philosophy itself. So that's the sort of collision of those two fields has been kind of the overriding interest of my whole professional career.

What do you think the biggest open questions are today in? AI and computer science? I think it's clear that we're on the road to building AGI. And the open question that I think is on the minds of many of the people in the technical research community is how many significant breakthroughs away are we? I was just at an AI conference a few weekends ago and we're talking about this. And I said, I think we're zero to two major breakthroughs away from AGI. And that got a general consensus of, hmm, and nodding. And I think that's in some ways the Holy Grail. That's the biggest question hanging over the field. So I think that is going to be one of the great things that we'll learn in the next couple decades.

We're going to talk about the most human human in some detail in your own kind of journey, which was so interesting to define sort of what it is that makes us human, that that line, that goal keeps moving, the goalpost keeps moving. But can you define AGI for people listening? What sort of the history of that idea is? What is like the equivalent Turing test? Is it still the Turing test that would lead us to believe we've achieved this? and why might the answer be zero? Right. Okay, so AGI is artificial general intelligence,

which is a fairly recent term, but the underlying idea has been around for a long time. It basically means a system that is human-level competent or beyond at not just a specific thing that humans do, but basically everything that humans do. So we've long had systems that could outperform humans at arithmetic since the 40s, and then things like chess since the 1990s.

and we're seeing these particular domain-specific milestones kind of fall into the rearview mirror, more recently go, for example. But there's still this question of the kind of flexible general intelligence where you can ask someone to write you a poem, you can ask them to add some figures, you can ask them to ruminate on their childhood. What do we need in order to build a system capable of competence with that degree of flexibility? So that's the question of AGI. And I think the Turing test, I mean, Alan Turing was, of course, one of the great founding fathers of computer science and was amazingly prescient in thinking about some of these philosophical questions decades before other people were thinking about it. Already by 1950, he writes this famous paper called computing machinery and intelligence.

And the opening line is, can machines think? Of course, you have to remember this was back in the time when the computer, as we know it, was this like room-sized thing processing. hunch cards, but he was already asking these philosophical questions. He was kind of seeing the long view ahead. And he answered the question, not as a philosopher, but as an engineer. He said, I don't know, I don't purport to know what thinking actually is, but let's just imagine a practical test that we can run. We will sequester a random person in a room down the hall, and we'll be sending text messages back and forth with that person. And we'll also have this computer that we're sending text messages back and forth with the computer. And the computer claims,

that it is the random human being hidden in a room and that the other person is the computer. Of course, the human is claiming the same thing. Turing says that, in his view, by the year 2000, he predicted, we'd reach a point where after five minutes of interaction, the program could fool a panel of judges 30% of the time, and that as a result, we would come to speak of machines as being intelligent without expecting to be contradicted. He wasn't making a metaphysical point. He was making a cultural point.

And this is one of the famous predictions of computer science that didn't come true. So starting in the early 1990s, a series of annual Turing Test competitions were inaugurated, famously by this quirky New Jersey millionaire who had made his fortune selling plastic, portable, roll-up, lighted disco dance floors and decided to bankroll this Turington competition and name it after himself. So it's called the Lobner Prize. Starting in the 1990s, they actually started doing these competitions. They'd convene people. They'd have people sending these texts back and forth for five minutes. And the Turing test threshold of 30% was not crossed by the 2000s. But I got very interested in this in the year 2008 because the top computer program at that

year's competition managed to fool 25% of the judges. So it was just a single vote short of hitting this famous threshold. You were seeing newspaper headlines at the time, like humanity dodges a bullet. There was this thinking that perhaps the next year would be the one that machines finally breached that. And so I got involved in my first book by being one of the human participants in the 2009 competition. So I was the guy sequestered in a room down the hall with five minutes on the clock to persuade a panel of judges that I was the human being, not the mechanical imposter. So we're talking about it now. So let's just go right there. I mean, I thought this story was so fascinating. So a lot of people will be familiar with the binary turning test. And I want to come back to that binary line for AGI, past which things get very interesting if we do cross it. But first, I bet that most people won't know this idea of the most human human, that there's not just a binary prize yes or no with the percentage of judges fooled. There's another thing, which was the subject of your book. So define what that is and tell us the story of your participation in this event. There's this very bizarre aspect to the competition, which you're highlighting, which is that, After each conversation, the judge will not only make a binary decision about whether they thought they were talking to the human or the computer, but they will indicate a degree of confidence and basically say on a scale of zero to 100, how sure are you in making that determination?

And so every year there is a computer program that gets the highest confidence score out of all of the other computer programs. And you win this title called the Most Human Computer and you get a bronze medallion and a few thousand dollars. But there is also, of course, every year a real person who gets the highest confidence score out of all of the real people. So they were better able to persuade the judges that they were a human being than the other human beings. And this person wins the so-called Most Human Human Award. And so when I learned about this, I was riveted. And indeed, I found myself kind of approaching my role in the competition from the perspective that I was not only competing against the machines, but I was also, in fact, competing against the other real people.

And it opens up these bizarre and, to me, fascinating questions of, what do you actually do from a tactics perspective if you have five minutes to act as human as possible? In a competitive scenario, what does that actually look like? And so for me, that's a case of this philosophical question where the rubber is hitting the road. What are the distinctive qualities of the human mind? what is inimitable by software, at least at the current time. And that all has to operationalize and an actual strategy you're going to bring to what seems like this kind of high pressure small talk scenario.

What changed the most? So when you first encountered this problem, you're trying to figure out how to be the most human human and then sort of the journey ends at some point. Maybe it's ongoing still. You're still thinking about it. But what were the tactics that changed the most? What were some surprising findings about things that were effective at trying to be the most human human?

So I would say there was at least one big change between the early 90s and when I participated in 2009, and then at least one major change since. So one of the biggest differences between the early 90s and the late 2000s was that in the early 90s, there was this idea that you had to make the conversation topic specific. The original Loebner Prize competitions kind of handicapped the human players by forcing every conversation to a predefined topic as a way. to make it just a little bit easier for the programmers. And this led to very strange judging decisions of if your ostensible topic is hockey,

can you talk about the Cold War implications of the famous 1980s U.S. versus Soviet Union Olympic hockey game or are we now into politics? So it was, of course, very difficult to police the boundaries of what was or wasn't on topic. There was sort of a classic signature that a program might demonstrate pretty impressive competence within a prescribed kind of channel of speech.

As long as you kind of followed the predictable path through that subject matter, you were likely to encounter things for which the programers had prepared. But if you tried to push against that envelope, the only possible thing that would happen to you would either be you'd get just nonsense or you'd get this very noticeable attempt

to direct you back to the kind of main threat of conversation. That started to change in the 2000s because of the Internet. Suddenly it became possible to approach the writing of a chatbot program, not as if you were writing a choose-your-own-adventure book where you had to manually key in all of these different conversational branches, but to think about it more like something like a wiki where you can kind of scrape text from the web and assemble it into this kind of giant compendium.

So I would say the software program in the 2000s that really exemplified this is what's called, Cleverbot, where it just hung out on the web talking to people. And the ingenious trick that the programmer used was that everything you said to Cleverbot was going into the database of things that humans say. So if you show up and you're like, howdy, how's it going? It tags that in its database as this is a conversational opener that humans tend to use. Yeah. It's almost like supervised learning in some kind of interesting way. Yeah. And every time it would then say something to you, you'd react and it would tag that in the database as this is a human reply to this sort of thing.

And you would essentially combat these bots in a totally different way. If you were trying to explore the breadth of knowledge, you would be amazed at what they could competently respond to. It's basically anything that someone on the internet talked about over an 11-year span is in there. So if you say, what's the capital of France? It would say Paris. If you said, what's 2 plus 2?

It would say 4. there's an example that I give in the book, which is I asked it, what is the capital of Romania? And it said, Budapest? I don't know. Which, and the real answer is Bucharest. And so that's the kind of graceful degradation that you expect from a person, but not a computer. And if you gave it popular culture, things have you said, a line of Bohemian Rhapsody, it would give you the next line. You'd say Thunderbolt and Lightning. It would say very, very friendly. And this was just kind of dazzling at the time, just the breadth. But the tell in this case was that you got the sense that you were not talking to the same person over time, because indeed you weren't. It was kind of this mosaic of the entire

internet. And so sometimes you would see it spell the word flavor with the British you, sometimes with the American spelling. Sometimes it would claim to be female, sometimes male, and so forth. And so the maxim here is that you get the sense not that you aren't talking to a person, but that you aren't talking to a person. And this was something when I was taking the Turing test that I was hyper-conscious of trying to come across, not only to give good answers to the questions that I was posed, but to give answers that fit together

and presented a picture of a single coherent individual where the things I knew or didn't know made sense as part of this single life history. Now I think we are starting to move into what I would think of as kind of a recognizable third era of text systems, where you have, particularly with the rise of deep learning and neural networks, you can feed huge amounts of text into a system in a much more nuanced and sophisticated way

than simply cataloging it as a database. So in this case, you can have a text system that is perfectly capable of saying things that it had never explicitly seen before, just by using this neural network model that had been trained on this corpus of text. And if you train it on a reasonably coherent corpus, then it's going to appear reasonably coherent. In the significant milestone on my mind, when I'm thinking about this, is a system that just a couple months ago was published by OpenAI called the GPT2.

And famously, they did not publish their code because they were worried about misuse. So this is a significant milestone for the publication norms of the AI field as we start to inch closer to something like AGI. There are now these ethical questions around doing science where the Open A.I. team felt the potential for bad actors to get a hold of this system, which can generate infinite amounts of very plausibly human-seeming text on any subject and further destabilizing online. discourse and so forth. And I think that risk is totally real. Before you answer the question about why potentially the number might be zero breakthroughs through two, I'm just curious if you think we need an updated test from the Turing test, or if there already is one out there in the world, where we'd be able to say, okay, this is AGI, this clearly isn't, some binary threshold. And part of this question is, is there such a test, like conceptually, theoretically, because it seems to me, as we said

earlier, like the goalposts are always moving, like things that we would have said our AI 20 years ago happen and like, oh, no, no, that's not AI. We get that now. It's still clearly not a human. So talk about what that threshold looks like and whether you think we can actually ever get to it. The Turing test as a threshold is very controversial. And many people have argued over the last few decades that it needs replacing with one of any other various proposals. There's one alternative test that's called the Winnegrad schema. Not as catchy. It's not quite as catchy. But these things often have the form of brain teasers. So it'll be like, what is bigger the moon or a pizza. Things that are obvious, but if you've spent your entire life living on the

internet, never experiencing anything firsthand, it might be trickier than it would seem. There are questions that involve, like, resolution of ambiguous pronouns. So if I say, I opened the oven that had the pizza in it, and I put it on the table, what did I put on the table? The oven or the pizza? There's all sorts of questions like this, or like the police arrested the protesters because they were being violent. Do I mean the police were being violent? Do I mean the protests were being violent? There's a lot of these interesting questions

that come up even in machine translation around ambiguous grammar that turn out to be surprisingly deep questions that require world knowledge, actual experience of the world, or at least that's the idea. What I found eerie about the result that OpenAI published with this system, GPT2,

was that with a single, generic model that they trained, they were able to get world-class state-of-the-art results in something like 18 out of 20 of the leading computational linguistics tests. So things like I describe, things like you read a short story, and then it asks you which of four sentences is a better summary of that story, things that almost look like an SAT type of a question. There are 20 different benchmarks out there in the linguistics community, and a single model was able to get record-breaking performance on 18 out of those 20 benchmarks. So that is the kind of thing that gives me some evidence that we're inching towards something

like AGI. And I think I've stood up for the Turing test as a valuable benchmark. I know many people within computer science and beyond are dubious about whether it has ever made sense as a benchmark or still makes sense as a benchmark. But I think Turing saw pretty clearly that language is a medium for, accessing all different kinds of intelligence. You can ask someone verbally to do a math problem. You can ask someone verbally to produce some kind of art and that they can produce verbal art. That it doesn't necessarily capture the entirety of the human experience. There are certain things

arguably that can only be expressed through dance or whatever. But it sure seems to capture a lot. And so I think that from a practical perspective, I think the Turing test has been passed when you are using Twitter or Reddit. It really isn't clear whether the speech that you're reading was written by the person that the account is claiming to be, or was it written by that person's publicist, or was it written by a bot who has hijacked that person's account, or what's happening? I think we're kind of past that point now. But the broader idea of the Turing test, that language is this nearly universal channel for tapping into all of these different types of intelligence. I think that's as true now as it was then. And I think that will continue

to be kind of one of the main ways that we will demonstrate something like AGI in the future. Let's plan an axiom and say that we achieve it, that kind of how we think about AGI is achieved, and now we have this tool. Let's talk about what the very interesting positives and very interesting potential negatives of this technology could be. So you already mentioned, Open AI is guarding this for fear of nefarious use. So maybe we'll start with the negative because it's visceral. So what are the specific concerns? I know you're writing about some of these ideas now. I always love Isaac Asimov's rules. So what is sort of the state of the art of thinking, whether it's from people like Nick Bostrom, like yourself, about things we should really be

concerned about as a species when it comes to AGI. And then we'll also talk about the great potential benefits. The way that the great worry has been framed by people like Nick Bostrom, people like Eliezer Yudkowski, is through this idea that you have this all-powerful maximum that will maximize whatever objective function you want to give it. However you're keeping score, it will find some way to maximize that score. And the field of AI has a long and colorful history of totally demented ways of finding some loophole that maximizes the score while nonetheless doing nothing like the behavior that you originally intended.

There are examples where OpenAI published an example where they were trying to train a program to play this boat racing game. game. And the objective of the boat racing game is to win the race, but it's very hard to encode what it means to be winning the race. You have to somehow get the program to recognize, I'm on a lap, what is a lap, where are the other people? Because it's circular, they might seem like they're ahead of you, but you're actually about to lap them. So there's a lot of nuance that goes into actually defining that, what it means to win. So instead, they use the proxy of score. So they said, well, just try to maximize the points that you win in this game. And their program found this little

weird cul-de-sac where I could just spin around doing donuts forever, collecting these power-ups while everyone just passed by. And you tune in hours later, and it's just doing these donuts, collecting these power-up points infinitely. So there are a lot of cautionary tales of things like that. I mean, earlier in the 90s, there was a group that was working on sort of evolutionary, artificial life, where they would create these kind of rudimentary organisms that would do various things, kind of simple geometric environments. And one of them was, they wanted to evolve an organism for fast land travel. So they would reward it for how quickly it could reach some predefined goal point that was like 20 meters ahead horizontally. And the

organism that developed was like this giant column that just tipped over. And so it hadn't learned anything about land travel, but it had learned a very quick way of getting 20 meters in front of you, which is just be 20 meters high and fall on your face. And there's just this catalog of these sorts of things where a programmer says, okay, that's not what I meant. You did what I asked, but not what I meant. And so there's a fear among people that as these programs get increasingly competent, the stakes for making that kind of mistake will go up to a potentially catastrophic level. So there's this famous thought experiment of the paperclip maximizer, which the humble paperclip factory develops this super powerful maximizer, and they say, we'd like you to boost our paperclip output this quarter.

and one thing leads to another, and now every atom in the invisible universe has been turned into a paperclip, and there's no sentient life anywhere. It's all just paper clips. And Eleazar Yudkowski has this famous quote, The AI does not hate you, nor does it love you. You're simply made out of atoms that it can use for something else. And I think this sort of concern of how are we going to get the objective function right went from being perceived as sort of a fringe idea to increasingly becoming a standard part of the way that machine learning engineers are being trained. Like a kill switch, basically. I mean, that's one idea, and there are a number of ideas. So the idea that there should be some sort of off switch,

there's a lot of technical literature that's being written starting around kind of 2015 through 2017 that were a flurry of papers on if the AI has an off switch, but it has a way of preventing you from hitting it. What do you do to incentivize this robot to allow you? you to switch it off. And one of the critical ideas there is that the AI needs to be uncertain about what it is that you want, such that if you try to turn it off, it can interpret that as evidence that it has the wrong idea about what you wanted it to do. Otherwise, if it's totally confident that it knows what you want it to do, then any attempt of yours to switch it off,

it will be like, no, no, no, no, I'm helping you. Let me help you. So this idea that the objective function that you give the system needs to encode some degree of uncertainty is increasingly becoming, I think, pretty widely accepted, certainly in the technical AI safety community, which is a subset of AI more generally. But people are, I think, motivated by thought experiments like this paperclip maximizer to rethink some of the fundamental premises of the field. The field has this premise that you have this objective function, which you maximize as aggressively as you can. And it's up to the human engineers to be wise about crafting that objective function just so that they've eliminated all of these possible loopholes. Increasingly, the field is moving towards rethinking some

of these fundamental assumptions and saying, what does it mean to build a system that doesn't have an objective function as such, but observes the way that I behave and tries to make inferences about the things that I'm trying to accomplish based on what it sees me doing and tries to be helpful, but maintains some uncertainty over exactly what I want or what I appear to be doing. And as I said, that uncertainty turns out to be very critical. So there's some promising work that's now happening on these things. So you've mentioned a number of terms, which I didn't expect to go as far down this rabbit hole as we have, but I'm glad we did. You mentioned awareness, experience, some interesting terms, and we do a lot of work with optimization. And there's objective functions. There's also constraints.

And you've talked about some of these constraints already, Asimov famously, that was maybe the strength of his system was that it cannot harm, you can't harm humans in whatever you do. So you can't maximize paperclips if it's going to kill off humanity. So my question is the degree to which self-awareness and experience that the machine may have or not have factor into this and putting your philosopher hat on, if it does have self-awareness, let's say that's a threshold. a machine becomes aware of itself, might that then self-regulate such that it perceives some objective good and bad spectrum in the world? And if it's told to maximize paperclips but becomes self-aware, it stops because it realizes this is in violation of just like objective good in the universe or something. I realize this is an insane sounding question. But self-awareness maybe is the true threshold for EGI, whether or not that holds credence. Yeah, there's a lot of

different threads that intersect in what you're saying. So I mean, there's one. One thing is this question of what's called moral realism, which is, are moral truth's objective facts that exist in the world the way that physical and mathematical truths exist? Some philosophers are moral realists, some are not. Sam Harris would be a moral realist, right, with a moral landscape? He's on that end of the spectrum. There's peaks of the, yeah, okay. Right.

The argument that we can use science to uncover facts about ethical behavior. If that's true, then I've heard people make the argument. I'm not necessarily sympathetic to it myself, but I've heard people make the argument that let's just let AGI do its thing because it will be smarter than us, and so it will inevitably discover these moral truths that we don't know about and we should just let it rip. I don't find that persuasive for a number of reasons,

but that's a position that people take. The self-awareness question is interesting to me. So there's two threads that I would kind of identify within that. So one is the question of consciousness, and I don't even know where to begin on that. It may well be the case that we, by the end of this century, have systems that are without any argument as cognitively flexible as we are, and then some. But we have no idea if anyone's home, if there's any light on, so to speak. It may well be that we have to live in a very weird limbo of not knowing whether these systems are due any sort of moral respect.

So in ethics, this is known as being a moral patient. So we may have to live with a lot of kind of cognitive dissonance around the question of, do we owe these systems anything and do we need to treat them well? I think that's a very interesting question. And I, in some ways, I would be just as surprised if we got the answer to that question as if we didn't. It would be very weird to live in a world in which these systems were everywhere and we didn't know if they were conscious or not. They maybe claimed to be, but we had no way of knowing.

It would also be very weird if science just told us consciousness is. this. This process or this pattern is consciousness. We've figured it out. That would be very strange also. So one of two very strange things will happen and we'll have to find out which of those it is. I would encourage people interested in this. We'll stop here on consciousness, but if people are interested to read Hofstadter and read Dan Dennett, and there's some fascinating stuff on this question that you can spend a whole lifetime trying to figure this out and maybe get nowhere, but you'll certainly be interested along the way. Yeah, for me, I don't know where to even begin to get traction on that argument, but it's fascinating. And the other side of what you're talking about with self-awareness

is this idea of how does a system setting consciousness aside take its own existence and its own behavior into account when it thinks about how it interacts with the environment. And that has been another one of these long-held assumptions within the field of machine learning that people are just now starting to poke at. So typically the framework that's used in machine learning treats the learning agent itself as kind of outside of the environment. The environment can't do things to it. It can't be changed in dramatic ways by the things that it encounters. But that's not true of real life. You can do things that kill you. You can do things that change what you believe or what you value. And when you're in a situation, it's important to understand the role that you yourself play in that situation.

you're not just confronting it as if from the outside. And so there's been a research agenda. I know people at the Machine Intelligence Research Institute have an entire vein of research now into what they call embedded agency, which is what does it mean for a system to reason about itself as part of the environment in which it's behaving?

And that's the kind of thing which will become increasingly relevant as AI systems become increasingly powerful. They will take up more space in the environment that they're operating, and they will need to pay more attention to themselves in that way. And so this is, I think, very nascent. There's not a lot that we know at this point. I would say the folks at Miri have been working on this

for a handful of years at this point, and I think they're starting to build some track record of work in this area, but it's really early. And so, I mean, this is part of what's so exciting for me about the point that we find ourselves at in the history of AI is we've made enough progress in doing things the classical way that we're starting to see the weaknesses

in the classical paradigm. And we're saying, okay, does it need to have an objective function? Does it need to be treated as exterior to its own environment? What does it mean to reimagine what a system could be like if we relax some of those? What would be the kind of skeptics take on all this? So as I mentioned, I do a lot of work with the fanciest of these new algorithms. And very often what we find is that the stupid, simple,

old linear regression does almost just as well as the coolest deep neural net or whatever. And that I love the idea of AGI, both positive and negative, thinking about it that's just like an objective function maximizer. It's sort of this magic box. You feed it an objective function. And it just gets you as much of that as possible using all the available resources like on the internet or in the world. There's a really interesting thought experiment. Our experience is that magic boxes don't exist today. That it's incredibly hard to build a system that gets you the thing you want. It often takes a lot of smart people a very long time. So what would be like the most skeptical viewpoint on AGI where someone says either it's impossible, it'll never happen, or it's

way further away than we think it is today? This sort of comes back to this question of are we zero to two or more significant breakthroughs away? There are many obvious criticisms that can be made of the current state of AI in terms of the things that it can't do. Humans are really good at operating in situations where the number of actions you could possibly take is almost infinite. So one of the great achievements of the last 20 years was going from a system deep blue that was superhuman at chess to a system AlphaGo that was superhuman at Go. One of the biggest differences between those two games, not the only difference, but one of the major differences is what's called the branching factor.

So in chess, you have at any one point in time about 30 moves available to you, and then your opponent has about 30 replies. in Go, you have on average 200 or more possible moves, and then your opponent has 200 or more possible replies. And so the exponential just blows up way faster. And we've now got methods that are impressively capable of dealing with that kind of complexity. But you compare that to something like just existing as a human being to having a conversation. How do you even cap the number of possible things that could be said next? I mean, there's a nearly unlimited. amount of actions that you have to choose from.

So I think there's a reason to feel skeptical about that, that we don't really have great methods for dealing with infinite action spaces. Some of the classical algorithms that are useful in game playing scenarios, like there's one called Q learning, which basically says, try every possible action in every possible situation nearly infinite number of times and just build up a memory of which, actions tend to lead to high expected rewards. Well, that algorithm makes absolutely no sense if you're living a normal life where most critical junctions only come around once,

or you have a nearly infinite number of things that you could do in any situation. So the idea of just try everything an infinite number of times and see what happens, that just doesn't make sense as a strategy. So that's kind of a non-starter. So I think that's part of the skeptics argument is like these sorts of things are really cute, in the context of Atari or even go, but they quickly disintegrate when faced with the real world. I think there's some truth to that. Humans are really good at what's called hierarchical planning.

So when we think about what we want to do, we think about it in this very abstract level, not at the motor command level. So it's like when I woke up today, I wasn't planning like the motor actions that I needed to achieve in order to come here and talk to you. And even now, I'm not thinking about the motor actions that I'm taking and forming words and so forth,

that we plan in this hierarchical way. Okay, first I need to make coffee, then I need to get the subway, then I need to, you know, whatever. We don't really have good systems for doing this kind of hierarchical planning. We have systems that are pretty good for the equivalent of muscle memory. So anything that you can do sort of instinctively we're pretty good at. And AI had an earlier history of working on these sorts of more deductive planning-based systems. A lot of that work has kind of fallen by the wayside in favor of the deep learning stuff, which has its own strengths and weaknesses.

I think a lot of people are saying there needs to be some grand synthesis of these two research programs that have until now basically existed in two different silos. The skeptic, I think, would say there's a hell of a lot of work to be done at reconciling those two things and we're only at the very beginning of doing that. I think whatever's the opposite of the skeptic, the booster, their response would be, with a big enough neural net and enough time and enough compute, the net will just figure out how to do hierarchical planning

if that is important. And maybe it's not that important, you just think it is. That's the gauntlet that's been thrown down, is how much can you do with a big net, a lot of examples, and a lot of electricity? is it just a matter of pumping more electricity into enough GPUs to train the net? I think that's a pretty sharp dividing line between research communities right now and AI. There are some that think, yeah, it's just a matter of enough compute.

Other groups saying, you're just going to get the wrong answer faster, but we need to totally rethink the actual structure of how we are building these. So a very practical question. With everything that you know about all of this that we've talked about, everything you've researched, what advice would you give to people building careers. So we're in a political cycle now where things like basic income are being discussed. There's all sorts of fascinating books, Britain and Rise of the Machines or Robots or I can't remember the name of that book, but fascinating, thinking about how all of this is going to affect work in the

future. So really, I guess what I'm asking is, in your view, what are like the most defensible areas of human activity, whether that's some sort of creativity or asking great questions, coming up with the objective functions that you then feed the machines? Like, what, would you recommend people focus on as they think about either early or late in their career, sort of adding value? There are sort of two ways that I can approach this question. So my second book, which we haven't really touched on at all, we're going to get to, yeah. It's called algorithms to live by, and it looks at things like career decisions from an explicitly algorithmic perspective. So we can get into this more later if you're interested, but there's

this paradigm called the explore exploit trade-off, which is how much of your energy do you spend gathering information, how much do you spend just sort of committing based on the information? Can we go there now? Because that was my favorite chapter in the book. I like prescribe it to everybody I ever meet. I think it's the most interesting of the many interesting algorithms in that book. So could you explain that one in some detail? Yeah. So there's a number of decisions that we face throughout life that take the form of a tension or a balance between trying new things and committing to the things that seem to be the best. Whether that is, where to go out to eat? Do we go to our favorite restaurant? Do we try a new restaurant? Who do we

spend time with? Do we reach out to a new acquaintance we'd like to get to know better? Do we spend time with our close family or our best friend? And I think the same thing is true in investing. The same thing is true in managing your time and your career. So I think the structure of this problem, kind of this iterated decision that you get to make over and over again about do you continue to put energy into the things that have seemed promising, or do you spend your energy trying new things is applicable across a wide range of areas in human life. You can even think of, for example, a clinical trial as having that same structure. And indeed, the FDA has been increasingly interested in looking over the disciplinary fence at the computer scientists and saying maybe those

algorithms that you're using to optimize ads could also be used to optimize human lives. So the way that a computer scientist approaches this question is through something that's called the multi-armed bandit problem. And it's a sort of a weird name. It comes from the slang for a slot machine as a one-armed bandit. So in the multi-armed bandit problem, you walk into a casino that has all these different slot machines. Some of them pay out with a higher probability than others, but you don't know which are which. And so quite simply, what strategy do you employ to try to make as much money in the casino as you can? Well, it's going to necessarily involve some amount of exploration, trying out different machines to see which ones appear to pay out more than others,

and exploitation, which to a computer scientist doesn't have the negative connotation that it has in regular English. Exploitation, meaning just leveraging the information you've gained so far to crank away on those machines that do seem to be the best. Now intuitively, I think most of us would recognize that you need to do some amount of both, but it's not totally obvious what that balance should look like in practice. And indeed, for much of the 20th, century, this was considered not only an unsolved problem, but an unsolvable problem, and sort of career suicide to think about it. During World War II, the British mathematicians joked about dropping the multi-arm bandit problem over Germany as like the ultimate intellectual sabotage, just waste

the brain power, like nerdsnipe, all of the German mathematicians essentially. And in many ways to the field's own surprise, there came a series of breakthroughs on the multi-arm bandit problem through the second half of the 20th century. And now we have a pretty good idea of what exact solutions look like given a number of constraints, but also what sort of more general flexible algorithms look like. So the critical, I would say, insight into thinking about this problem is that your strategy should depend in some ways entirely on how long you plan to be in the casino. So if you feel that you have a long time ahead of you, then it's worth. it to invest in exploration because if you do find something great, it has a long horizon

to pay out into the future. On the other hand, if you feel that you are about to leave, then the return that you would get on making a great new discovery is going to be much smaller because you have fewer opportunities to sort of crank away on that handle once you find it. So for a number of reasons, we should naturally segue. or transition from being more exploratory at the beginning of a process to more exploitative at the end. And I think that's an intuition that makes sense, but the math bears that out very concretely. And it's been interesting to see this idea that emerges in computer science in the late 50s through the 70s

is now getting picked up by psychologists and cognitive scientists who are interested in human decision-making. So, for example, Alison Gopnik at UC Berkeley, who studies infant cognition, has been thinking about the explore exploit trade-off as a framework for how the infant mind works. If you think about how children behave, we have all these stereotypes about children are just kind of random. They're generally incompetent at things. There's a huge literature that shows that they have what's called a novelty bias. They're just relentlessly interested in the next thing, in the next thing, and the next thing, and the next thing. And rather than viewing that as just kind of low willpower or attentional control, you can view it as the

optimal strategy. And if you've just burst through the doors of life's casino and you have 80 years ahead of you or whatever, it really does make a lot of sense to just run around wildly pulling handles at random. And the same is true for being in the later years of one's life, that we have a lot of stereotypes about older people, that they're very set in their ways, very habitual, resistant to change. There's a psychology literature that shows that older adults maintain fewer social connections than younger people. And it's tempting to view that kind of pessimistically as, oh, I guess it must be kind of sad, getting older or lonely or something. In fact, if you build an argument from the mathematics, you can see that older adults are simply in the exploit phase of their life. And they are, again,

doing the optimal thing, given where they are in that interval of time. And so you have psychologists like Stanford's Laura Carstinson appealing to the explore-exploit tradeoff to make this argument that no, older adults know exactly what they're doing, and they're very rationally choosing a strategy that makes sense given where they are. They have a lifetime exploration behind them. They know what they really like. They know the people and the connections that matter to them. And they have a finite amount of time left to reap the fruits of some new connection or a new discovery. So they're very deliberately enacting this strategy, and the math should predict that on average older adults are happier than young people despite our preconceptions. And her research bears out that that appears to be the case.

So if you translate this kind of fascinating tradeoff to career and life advice, you've done some of it already. Obviously, the bias would be when you're young, try a lot of stuff. And maybe when you're older, think more about using the best solutions that you found versus trying to find new ones. I'm fascinated how this maps on to like the life cycles of businesses, for example. In the business context, explore might be innovation, and exploit might be run the same playbook to earn high returns on capital or something, and something you know works. And it seems like you kind of always want to be handing off to a next batch of exploration or innovation while thoughtfully maintaining something that you know works. In a business context, if you want to survive for a very long time.

Yeah. There's a couple of things here that I think are interesting in a business context. One is that implicitly the casino framing that I've described assumes that those probably probabilities are stable and fixed. And of course we know that the world is not stable and not fixed, that things change over time. And this is true in our personal lives as well. The restaurant that you love gets a new line cook, and the burger's not as good.

These things shift. And so this is known in the mathematical literature as the restless bandit problem. So how do you play this game when these probabilities are, let's say, on a random walk? So they're drifting. And this is a very interesting case where the theory is not yet consolidated, but humans seem to, in practice, have no problem.

Like if you put people in a lab and give them a restless bandit problem, they have no trouble making choices within that environment, but we don't yet know what the mathematics of the optimal solution looks like. So here's a case where the computer scientists and the mathematicians are asking the cognitive scientists, what are your models for how humans are actually approaching this because there may be some insight that we can use from the theory side? One of the implications of thinking in this way that I think is particularly relevant in a business setting is if the interval of time you perceive yourself to be on

determines the strategy that you should employ, then it should be the case that if you observe someone else's strategy, you can infer the interval that they're optimizing over. So we give the example in the book of Hollywood. So most people have noticed, it feels like we're living through this deluge of sequels. X-Men 12 and the Avengers 7 and whatever. So I spent a while digging through the data,

and it turns out that this is objectively true. There's a sea change in Hollywood that I want to say in 1980, two of the top 10 grossing films were sequels. By 1990, it was like six. By the year 2000, it was eight. And I think most recently it was like all 10 or something like this. And we can infer from that that Hollywood has taken, like, like a very hard turn towards an exploitative strategy, that they are milking their existing

franchises rather than investing money speculatively to try to develop new franchises that will last them into the next few decades. So from that, I think it's reasonable to infer that, for example, movie ticket sales are declining, which turns out to be the case, that Hollywood, perhaps correctly, perceives itself to be at the waning time of this kind of golden era of cinema going. And if that's true, then they really should invest all of their money into just squeezing everything they can out of the existing franchises. So that's an example that I think is applicable more broadly. So you can look at different industries, different corporations, and see, oh, they've really cut their R&D budget and they've given that money to marketing, let's say.

Well, that would be an indication that they feel that that area has matured or has plateaued or something. Yeah. Yeah, that's a fascinating research idea that we've sniffed around. It's very hard to draw firm conclusions, even though there's a lot of companies, sample sizes actually aren't all that big. But fascinating idea that you can observe trends. And we've certainly seen one, one exploit strategy in public markets might be paying dividends, buying back shares, consolidating the share count versus spending on CAPX or R&D. What's interesting is those firms actually outperform. They do much better than the one. that are spending what we would call it recklessly, which maybe is a bit counterintuitive. But I love this idea of the broad numbers of backwards and forwards applications of exploring versus exploiting.

You mentioned that that was sort of one of two ways to approach my original question of kind of advice for career. What would the second one be? Well, the second avenue is sort of totally different from this way of thinking, which is just what will the impacts of something like AI or AI be on the economy? And I'm reminded of McKinsey did a report. on which jobs they thought would be the most robust. The big picture thing that was interesting to me

is that it cuts across the traditional class lines. It's not a white collar versus blue collar thing. It's not an upper middle class versus lower middle class thing. It's very sector dependent. And I remember that what they identified as the most resilient or robust jobs. At the top end, it was Gardner, legislator, and psychotherapist. I thought that was very fascinating. And it's sort of this eclectic mixture of things.

I don't think of myself necessarily as a prognosticator about these sorts of things. But my way of thinking about it is that there is a lot of kind of human machinery around how capital moves, how laws get made, how licensing and permitting happens. That's still done as sort of a human negotiation level. I know a guy. I'll talk to Joe and will sort of. it out. I think humans will maintain oversight of these kind of flows of power and capital, even if the actual value is being created by software. So kind of, that's my somewhat off the top of my head thought, is position yourself closer to the flow of that value than the actual

creation of the value, which is maybe somewhat counterintuitive. I don't know, as far as the question of UBI. I don't have a great intuition for that. I am interested in Norway has this sovereign oil wealth fund. What would something like that look like? I don't know. In many ways, I think there is already a restlessness in the labor force that a lot of the careers that employed some of the most numbers of people are the most vulnerable. So people who drive cars or trucks, people who work in warehouses, doing sort of pick-and-pack things, a lot of those jobs are just one innovation away. And it's not clear to me. I mean, there's also going to be a political response, as well as just a pure sort of economic response. So I grew up in New Jersey where there's,

at least in my childhood, there was a really robust toll collector union. And they had machines where you could toss your change in a bin, and it would automatically sort your change and give you whatever you needed back from that. And there was an effective effort to unionize the toll collector so that you still had a human being in the booth, counting out your quarters and so forth. And that's an example where, it's not for lack of technology. We had a coin sorting machine, but there was kind of a political process that was directing the actual level of implementation. So I often encourage people to remember that. People will fight to use licensing. requirements and regulation to maintain those things despite the actual technological capability

having radically changed. So it's very hard for me to know which areas will look shockingly different than the world looks today and which things will be in some ways shockingly backwards for their time because we've had for political reasons to sort of hold the line. But as to what exactly that looks like, I'd say my guess is as good as anyone's. So to complete our arc of starting very theoretically and ending more practically. I'd love to close with one, two, maybe three final examples from your second book, Algorithms to Live By. It's certainly the one that I've recommended like crazy, and I think people would just all love it, even if it's one of those beautiful books where you should read the whole thing, but it's also feasible to skip ahead a chapter if one thing isn't resonating

or whatever. So we talked about explore exploit. Thinking back on it, are there a couple that you found most interesting and or useful? Those are sort of the lenses that I'd love you to apply to the many chapters and maybe close with two examples. One thing that comes to my mind is the idea of what's called optimal stopping. The multi-arm bandit problem and the explore exploit tradeoff presumes a framing that's highly iterative. You can pull the handles again and again and again. You can kind of fluidly go from one machine to another and back. There are many decisions in life where you are forced to make a single binding commitment.

That could be anything as banal as pulling into a parking space. it could be something like purchasing a house or signing a lease. It could be something like marrying your spouse. And there's kind of a separate mathematics of cases where you need to find the right moment in time to go all in and commit to an option and no longer gather any further information. And I think that mathematics is very kind of instructive both in a specific way but also as a broader set of principles. There's this very famous result called the 37% rule, which is that, let's say you're looking for an apartment and it's a really competitive marketplace. I'm from

San Francisco, so it's this kind of hellishly competitive situation where you have to just force the check into the landlord's hands. If you're in a situation like that where you encounter a series of options one by one, and at each point in time you must either immediately commit and then never know what else might have been out there, or decide to walk away and keep exploring your options but lose that opportunity forever. What do you do to try? try to end up with the best thing possible, even though you won't necessarily know at the time whether you've found the best option that might be out there. There's this beautifully elegant result that says that you should spend the first 37% of your search, or 1 over E, noncommittally

exploring your options. Don't bring your checkbook. Don't commit to anything no matter how good it seems. You're just purely setting a baseline. And then after that 37%, whether it's 37% of the time that you've given yourself to make the decision, or 37% of the way through the pool of options, be prepared to immediately commit to the very first thing you see that's better than what you saw in that first 37%. This is not just an intuitively satisfying balance between looking and leaping. This is the mathematically optimal result. And there are strategies like that that I think are wonderfully crisp in the recommendation that you give. But they, of course, rest on this bed of many different assumptions about exactly how the problem is structured and exactly what your goals are.

And specifically, it's better than anything you had seen in the prior 37%. Yeah. Got it. Okay. Yeah. This rule presumes that your entire goal is to maximize the chance that you get the very best thing in the entire pool. But it comes with a 37% chance, of course, that you have nothing at all because you've passed it by. Yeah.

And you'll never find something that's good. And now many people would have. would find that unacceptable. And so, of course, we can go down the rabbit hole of how do you modify this, and the solutions get less and less clean as you wiggle the assumptions around. But I think there's broadly just this intuition that one of the highest level takeaways for me from working on the book and just thinking in computational terms about decisions in my own life is some decisions are just hard, that the classical optimal stopping problem, it turns out due to a weird mathematical symmetry that if you follow the 37% rule, you will only succeed 37% of the

time. The other 63% of the time you'll fail. And that is the best possible strategy you could enact in that situation. So there's no better that you could possibly do than failing 63% of the time. In a weird way, that's some measure of consolation, because often in real life when we find ourselves not getting the outcome we wanted, we can rake ourselves over the coals or try to reconstruct our entire thought process and say, where did I go wrong? I think it's some comfort that computer science and mathematics more broadly can, in effect, certify that you were just up against a hard problem. And I think broadly that, for me, is some measure of comfort, that if you have the kind of the vocabulary to understand the type of problem that you're facing and you have

some intuitions about the general shape of what optimal solutions look like, then even when you don't get the outcome that you wanted, you can in some sense rest easy because you knew that you followed the appropriate procedure or the appropriate process for dealing with that situation. So that's an elegant place to end. And so I'll move to my final closing question. I ask everybody, which is for the kindest thing that anyone's ever done for you. The kindest thing that anyone has ever done for me. I want to offer some very concrete thing, but I would have to think pretty seriously about

that. What comes to my mind is just the idea of being loved unconditionally. by my parents. I think most of the difficult things that I've faced in my life in one form or other took the form of not being totally comfortable with who I was, whether it was in school, feeling self-conscious about being a nerd and not answering a question in class, even though I knew the answer because I sort of wanted to fit in or didn't want to seem like I was enthusiastic about school because it wasn't cool to act like that. Throughout my life, I think some of the most meaningful things to me have been ways that I've come to just accept who I am.

And so I think one of the kindest things that you can do for someone is to see them exactly as they are and accept them. And to be able to do that for oneself is tremendously powerful. Yeah, I haven't heard it put quite so eloquently as you just did, but unconditional love or some version thereof is a common answer and think for a good reason. Yeah. So thanks for that great answer. Thanks for a fascinating conversation. I love the ones that barely touch investing. And this has been a great example.

So I hope we can do a round two someday. Maybe after the next book. It's been a pleasure. Likewise. Thank you. Hey, everyone. Patrick here again. To find more episodes of Invest like the best, go to investorfieldguide.com

forward slash podcast. If you're a book lover, you can also sign up for my book club at investorfieldguide.com Forward slash book club. After you sign up, you'll receive a full investor curriculum right away and then three to four suggestions of new books every month. You can also follow me on Twitter at Patrick underscore Oshag, OSHAG. If you enjoy the show, please leave a quick review for us on iTunes, which will help more people discover Invest Like the Best.

Thanks so much for listening.