Into the Impossible With Brian Keating - P-hacking, Reproducibility & the Nobel Prize: Guido Imbens (#269)

Episode Date: October 30, 2022

Guido W. Imbens, along with David Card and Joshua Angrist, shared the 2021 Nobel Prize in Economics for “methodological contributions to the analysis of causal relationships”. In 2017 he received the Horace Mann medal at Brown University. An honor shared by your host Professor Brian Keating. He is The Applied Econometrics Professor of Economics at the Stanford Graduate School of Business since 2012, and has also taught at Harvard University, UCLA, and UC Berkeley. He holds an honorary degree from the University of St Gallen. He is also the Amman Mineral Faculty Fellow at the Stanford GSB.  Imbens specializes in econometrics, and in particular methods for drawing causal inferences from experimental and observational data. He has published extensively in the leading economics and statistics journals. Together with Donald Rubin he has published a book, "Causal Inference in Statistics, Social and Biomedical Sciences”. He is a fellow of the Econometric Society, the Royal Holland Society of Sciences and Humanities, the Royal Netherlands Academy of Sciences, the American Academy of Arts and Sciences, and the American Statistical Association. He holds an honorary doctorate from the University of St. Gallen. In this episode, Professor Imbens give his lecture on his Nobel Prize-winning thesis. See the video with the slides here: https://youtu.be/X632K3n8PPI Watch the video with slides here: https://youtu.be/q1cPyE9rAD4 Connect with me: 🏄‍♂️ Twitter: https://twitter.com/DrBrianKeating 📸 Instagram: https://instagram.com/DrBrianKeating  🔔 Subscribe: https://www.youtube.com/DrBrianKeating?sub_confirmation=1 📝 Join my mailing list; just click here http://briankeating.com/list ✍️ Detailed Blog posts here: https://briankeating.com/blog.php 🎙️ Listen on audio-only platforms: https://briankeating.com/podcast Subscribe to the Jordan Harbinger Show www.jordanharbinger.com/podcasts for amazing content from Apple’s best podcast of 2018! Can you do me a favor? Please leave a rating and review of my Podcast:  🎧 On Apple devices, click here, https://apple.co/39UaHlB scroll down to the ratings and leave a 5 star rating and review The INTO THE IMPOSSIBLE Podcast. 🎙️On Spotify it’s here: https://open.spotify.com/show/2G3PRMUhxGQkyQzLiiCqlf?si=8656119458df4555 🎧 On Audible it’s here : https://www.audible.com/pd/Into-the-Impossible-With-Brian-Keating-Podcast/B08K56PXJX?action_code=ASSGB149080119000H&share_location=pdp&shareTest=TestShar Other ways to rate here: https://briankeating.com/podcast -  Support the podcast on Patreon https://www.patreon.com/drbriankeating  or become a Member on YouTube- https://www.youtube.com/channel/UCmXH_moPhfkqCk6S3b9RWuw/join Learn more about your ad choices. Visit megaphone.fm/adchoices

There's other parts of the economics PhD program. They haven't really changed for a long time, but actually clearly should be changed. And one of the things is we should have more of an ethics thing. One thing that we are behind physics and some of the other disciplines, as far as I know, there's never been a paper in an economics journal that's been retracted. Hello, podcast listeners. Today you are in for a treat where I am bringing to you. another one of my very cherished interviews with Nobel laureates.

In today's episode is a first for me. It is an interview with a Nobel laureate who's not a physicist. That will not detract in any way, shape, or form from the delight you will take from listening to this conversation. With Stanford's Crito, Hymbenz, who is an economist of great renown, not only for winning the Nobel Prize, but for winning an award that I won two just this past year, where I met him at Brown University. You'll hear a little bit about that. And how Brown made a not insignificant impact on his life and career, how he was almost derailed from becoming a Nobel caliber economist because of the ultimately foolish lack of wisdom and

intelligence by investment banks on Wall Street, not to hire him when he was a newly minted master's student who spoke Dutch and that was exactly what the job of advertisement on Wall Street was looking for. They didn't even interview him. How stupid is that? You'll hear more than just knowledge from Professor InBenz. You're going to hear a tremendous amount of life advice, of wisdom, of perseverance. This is a long conversation, two hours long, so much fun to talk to such a generous, genial intellect, genius, I should say for alliterative purposes. And I really enjoyed it. It was such great fun to converse with, with him and really to share, you know, two of the most interesting hours I've spent in a long time. So I know you're going to enjoy it.

And I want you to sit back and relax. And before you do, to do a couple things on behalf of yours, truly. It's my birthday season, as you'll find out, Guido's birthday as well. We're celebrating. So I want to ask you for a present, which is to do a very, very small thing, which is to leave a review of the end to the Impossible podcast, perhaps a five. five-star rating and a review, which you can leave on Audible and on Apple Podcasts, but you can leave a number of stars and asterism on most of the podcast apps that you're familiar with.

So please do that. And I read each and every review on iTunes and Apple Podcast, and I'll read one that I got recently, which I really cherish because this is something I'm going for with this podcast. And this is from a person with the initials in name NMac 103. At the end of August this year, he or she said, Brian Keating equals science teacher. I love the way Dr. Keating explains things. I heard him speak on the James Webb Space Telescope and then on what's happening with the Star Beetlejuice.

He makes astronomy so much easier to understand. He really has a gift for teaching. Another one from somewhere in the United Kingdom, where you can also leave a review. Not seeing any claims over here in the UK. So this is a good show and it really enjoys it. from Xerxes. Oh, that's, that's impressive over there in the UK. So this is just a delight. It's a small little present you can do for me. You can also subscribe to my YouTube channel, Dr. Brian Keating on YouTube or search into The Impossible. You'll find it there. And I want to

just really thank everybody for all the goodwill that they've expressed during this birthday season. And maybe the last gift you can get is to subscribe to my newsletter, which may be a gift for you. You may win one of the 100 meteorites I'm going to send out to lucky new subscribers to my mailing list at Brian Keating.com slash list, Brian Keating.com slash list. And so now we're going to take a deep dive into the study of economics, which is really distilled into extremely easy to understand terminology by today's guest Huito in Benz. We do show some slides from him from his actual Nobel Prize lecture. So to see the slides, you should go to my YouTube channel, Dr. Brian Keating. And whenever you're dealing with economists, you have to watch out for their invisible hands.

And that is something that you will not need to worry about too much on this podcast because the supply of wisdom is only matched by the demand that you, my brilliant audience, has for knowledge and wisdom. So with that, I implore you to open your mind as well as your wallet as we take a deep dive with our first Nobel laureate and economic sciences. Dr. Professor Huito Inbens. Enjoy. Any sufficiently advanced technology is indistinguished. from magic. It is a great thrill to have on today's guest, who is Huido Inbenz. I've learned how to pronounce his name. He is one of our many, many Dutch guests who honor me by coming on.

And of course, you know, Guido, that the telescope was invented, not by Chaléo Chalé, as most people think, but actually by, of course, a Dutchman, Hans van Mavent. than a Lepersche. And so he invented that. And then another Hansman, Lansman, by the name of Hans van Lerich, I think you could pronounce the name better than me, he invented the microscope. So we will look into the telescope and look out in the universe, and we will look into your crystal ball and your microscope later on in this podcast.

But it's a great delight to have you on, especially since it's our birthday season. You and I are born the same week of September. So happy belated birthday to you and happy early birthday. Tomorrow is my birthday. So this is a great treat for me, Huita, to have you on the podcast. Thank you so much. I'll give you a quick introduction. Dutch American economist at Stanford University.

He won the most prestigious award in all of society, humanity, the world, which I have won too called the Horace Man Medal. at Brown University for outstanding graduate student performance. And then he went on to win another medal called the Nobel Memorial Prize in Economic Sciences with jointly with Josh Angrist for their methodological contributions for the analysis of causal relationships. And it sounds incredibly complicated. And of course, Huido, you are the first non-physics Nobel Prize winner of the 14 Nobel list I've had on. You are the first.

So first of all, thank you so much for going. Thank you. Thanks for having me. I'm always happy to see a fellow Brown University graduates. And it was a great pleasure to be there kind of when I won the Horace Mann Medal. So it's, it's good to see you. Thanks for having me. Yeah, it's great to be with you. And when I was introduced this past May for by the Dean Andrew Campbell of the Graduate School, he said, you know, it's very, we have 100% correlation between winners of the highest man medal and Nobel Prizes, although the record only lasts for one year. So as I explained, one thing my audience loves is when we dive deep into books.

And from your work on inference and causality, you'll know that there's an expression, which is never judge a book by its cover. But I claim what else do you have to go on when you don't have any other prior information? So I wanted to begin with hearing it from you, and maybe someday we'll get your co-author, Donald Rubin on, your book, which I've begun to read and really am enjoying. It's called causal inference for statistics, social and biomedical sciences, and introduction. And it has a very, very minimalist cover, I would say. But, Huita, could you please explain the origin of that book, the title and the cover, if any? thoughts that went into it. Yeah, sure.

So the origin was sort of very clear. So I was an assistant professor in the economics department at Harvard at the time. And so Don Rubin was in the statistics department, which is in the next building in the science center. But for whatever reasons, that was right next to the building that had the economics department. And so at that time, I've been working with, done some work with Josh Angrist, In fact, we started the work for which we shared the Nobel Prize coverage. We also shared with David Carr. And so it was sort of clear that it was a connection between the work we were doing

and the work Don Ruman had been doing in the statistics literature. So I actually, overlapped with Judge Angus only for one year. Then he went to University of Jerusalem for a couple of years. but as I decided to go kind of talk to Don Rubin and see kind of if we had enough in common kind of what we could share. Because that's always kind of a big challenge to start talking with people in other disciplines. It's kind of, you know, it's very hard.

You often have a different language. You're used to making particular assumptions that the other discipline may actually think are very controversial. and people have different ways of thinking about what the development problems are. So it's always kind of a big challenge. But at that time, I was very early in my career. I wasn't really very set in my way. So I was kind of quite happy to talk to Don and kind of try to explain how we were thinking about it

and understand what he was doing. And kind of remarkably, because Don was much more senior at that point. But he was actually very open-minded. He was kind of, you know, he was sort of a little hard. just get started. He was like, well, you know, these economists, I never understand what they're doing. It doesn't really make any sense to me, but I sort of, we persisted for a bit. And then, then, then he got very interesting in what, what Josh and I were doing. And actually, I remember once getting a call for him and he said, well, hey, I'm at the airport here in Rome. And so I had a couple of

hours. And I was looking at the paper you gave me. And, you know, I think kind of, it's a It's not really right, but there really is something interesting in here. And it's kind of, we should talk more about this. We should, no, no, we should talk and kind of figure out what is really there. And then so a little later, we actually started teaching a course together. Again, Harper was kind of a very tricky place to be junior faculty, but it they gave me an enormous amount of freedom. So I went to my chair in the economics department and said, hey, I want to teach this course

with someone in the statistics department and the chair said basically said, well, whatever, I don't really care. Go ahead and do that. And that was just, so we started teaching this course on causal inference. And a funny thing was we wrote this course description there. And then the registrar's office would put out this whole book with all course descriptions, which was not online in those days was just a big print book their spell checker replaced every instance of the word causal but casual so this was a causal casual inference and we're going to do casual this and casual that so the truth was very confused we had a huge turnout the first day we quickly left when they realized actually no it was still going to be you know it's still going to be math in there it wasn't

it wasn't it wasn't just all yeah it was uh yeah it was uh a casualty almost of the publisher. Wow. That is it. That actually, of course, has happened many times when National Public Radio announced the Nobel Prize last year. They said that Josh and I had won for our study of casual relationships. Yeah, so what do economists wear? They wear, you know, very light T-shirts when they do casual.

So who is the book for? Is it a textbook? I mean, it's a Cambridge. publication, right? Yeah. So that book really came out of that course we're teaching. We were trying to kind of combine these perspectives from statistics and economics. So it was intended for students in statistics and in social sciences, kind of all the areas

where people in the end are interested in estimating, in learning about causal effects. And because that's true in all of social science. We don't really care just about the correlations we see in. in the data, in how people behave, in how much people there earn. We want to know what policy makers could change, and for that to have some effect. We want to know in advance what these effects are. So we want to learn about causal effects. And so it's, on the graduate students, we've taught various versions of these courses,

kind of also to policy institutions. So it was intended for a broad audience to give people an idea how to think about causality and causal effects. And in that kind of instance, I mean, we'll get into a little bit more of your work. But I hear a lot in the economics literature about all these revolutions, you know, marginal revolution, the inflation revolution. you're credited with with something called

the credibility revolution can you and your colleague David Cart can you know first of all what are these revelations how many revolutions do we need in a field we have a quantum revolution

every hundred years in physics what is it such a rep what does it take to constitute a revolution as I said well so here this is not a term I call it actually Josh Angus

coined that term. But sort of in the 80s so economics kind of always has a couple of different components. There's kind of theoretical work where people do sort of apply mathematical modeling of economic behavior and then there's empirical work where people actually take models to the data and try to use them for giving policy advice

and helping their industry. decision-making. And so there are often, we were very explicitly interested in causal effects. And kind of over the years, and that goes back to the early 1900s in kind of Timberger was one of the founders there. It was one of the people who won the first Nobel Prize in economics, which, of course, was established much later than the other prizes in 69. And so they started kind of with building these, these,

economic models and try to still get credible causal effects. And over the years, these models became much more complex and much, much fancier. And so partly, economists got a little carried away by their cleverness and kind of tried to build these really complicated things. And so in the 1980s, people became a little disillusioned with that. And there's this famous paper that sort of Carr, the Angerist and I all cite in our Nobel lectures as a source of inspiration by a guy at UCLA at Lima at Lima that had the title, let's take the con out of econometrics. So actually, I had him give a guest lecture recently and they said, well, you know, now he wanted to change the title to who put the tricks into econometrics. But he was incredibly good writer.

He's actually someone you should have on your podcast. I'd love to have an introduction. He's a very witty writer. And in his paper at some point he writes, you know, the problem with economics is that nobody really believes the results anymore, or at least nobody believes anybody else's results. And he shows some of these empirical papers where people really make this this quite outrageous claims and in a way that kind of it suggests this somehow

think these are relevant for for policy makers and he looks the particular example he looks at this what is the deterrence effect of the of the death penalty and you know because that's a very controversial right in the political discourse but there's these people trying to write these papers and they claim, you know, for every execution, we save three lives from people who would have gotten murdered otherwise. And other people who write these papers well, for every execution that leads to five more murders. And they all claim that they estimate these effects with great precision. And it's sort of very clear, if you look at that, seriously, that these papers are really

meaningless. There's no way you can learn that from the data. There's assumptions people make are really just outrageously strong and it just doesn't make any sense that you could possibly learn that from from that type of data. And there was a lot of empirical work of that type where people were making these strong claims and it really just didn't hold up. And so what started that in the late 80s, and this is kind of people at Princeton, kind of where both David Carr, David Cart was on the student, and on the faculty there, Judge Angeris was a student there. There were a bunch of people who started trying to find ways of getting,

sort of being clear when you could actually get credible results of these data, satisfying Lemurs concern that not only they would believe, that people would really believe it themselves, but also that you could convince other people that these results were actually serious. and that policymakers should take these serious. And that really started what Angeris much later called the credibility revolution. We kind of tried to make the empirical work more credible and not rely on these very fancy models that may have looked complicated

and that could maybe fool some people, but really that were not really very believable. Yeah, when you begin the book, you talk very practically about issues, which almost have an ethical kind of counterpart. And I wondered, was there a motivation, you know, since the book, or maybe the book is so timely for what's called the reproducibility crisis in the biomedical sciences? So part of the title is, you know, includes biomedical sciences as one of the target audiences for causal inference for statistics.

So can you talk about that? Are we in a reproducibility crisis? And if so, is this book in your work, does that have any impact, say, on the ethics of the way that science is done? Or is it merely to, and importantly, to really provide a framework to assess how one is not thinking causally properly?

What is the influence on the reproducibility crisis? Ambition comes in all shapes and sizes. At First Citizens Bank, We roll with your goals because we're built for what you're building. Fit for your ambition for Citizens Bank. So economics also has a huge reproducibility crisis. And it's sort of at a lot of different levels, but including kind of the very mundane level that people publish papers

and they may even post that data. And then if you try to just even making all the assumptions the authors make, even using that code, it doesn't, the numbers don't even come out the way they are in the papers. And so I'm one of my roles is, I'm an editor of economic, I was one of the leading journals in economics. And there, we're now moving towards having the papers at least narrowly reproduced before we published them, where some of the economics journals started at a couple of years ago, but for a long time, there was no comment for a long time. People didn't even have to share the data.

Now, economics in general is moving more towards, okay, we do need to really make sure people share that data. We need to have much more transparency and reproducibility. And that's sort of partly, you know, we need to train the students better. When I was an undergraduate and even as a graduate student, when we wrote code for estimating our models that was not intended for anybody else to use just writing for ourselves. So I might put in comments in my code in Dutch or in things that were only comprehensible to me. But now, of course, that's no longer acceptable. You work in teams,

you need to be able to do it in a way that someone else can use it. And it needs to have a clear pipeline where the data come from, how they're used. And that, But that's kind of a very narrow definition of reproducibility. But then more generally, it's sort of clear we need to get much better at checking that results that people find in a particular context work in other settings. External validity, of course, is a huge issue. And that ties in very closely with causality. Experimental data often, of course, are great things. for internal validity.

But the external validity is not always so clear. And you see in social policy debates, people continue to refer to experiments that were done on early childhood interventions, kind of on early schooling, on Head Start-type programs. Where these experiments were done in the 60s in kind of very small towns in Michigan.

And, you know, it's very helpful to have those experiments. But it doesn't really tell us what would happen if we did similar type, headstar type programs now in very different locations. So clearly in social sciences, we need to worry a lot more about external validity than you guys have to do in physics. Right. Although it's hard for it. We can't easily reproduce, you know, a spaceship, you know, if the James Webb Space Telescope has some results. And you say, well, I won't believe it until it's reproduced. I mean, that could be kind of expensive, even by NASA standards.

And I wanted to, you know, pivot a little bit to your work as an educator. I mean, you're a renowned educator, phenomenal, clear-headed thinker. And, you know, one thing I've thought about with my students, we don't really have required classes on ethics. It's just sort of, you know, if you come upon it, maybe it's interesting, maybe it's not, you'll study ethics. And our colleagues in the medical school, in the law school, even in business schools, they at least have pay, I would say, I don't want to say lip service, but way more than the lip service that I personally pay. I'll just speak for myself.

What do you think is the role of, you know, instantiating a higher level of credibility via starting early at the ground level? Or do you think it's, you know, at students teaching them ethics, teaching them reproducibility, metrics, best practices, et cetera. As an educator, is that a worthy goal? If so, how do you guys do it in economic sciences? I think we are there, we're equally far behind as physics. And so I think that it is becoming an issue.

And it's something we need to deal with. And in some sense, just looking back over my whole career, there's a number of institutional aspects of the profession that haven't really changed in since I since I got into the profession. Actually, the job market for economists still works exactly the same. We people apply for jobs in November. We interview people at a meeting, general economics meeting in January, and then we fly people out for the seminars and then we offer them jobs.

And that has been exactly the same since sometime in the 70s. Only change now. The restaurants you take them to. It's only different on Alma Street. You take them to a different place on Alma Street. Yeah. The main change, well, one change, and that was already controversial. We moved to meetings from between Christmas and New Year to after New Year.

That took a lot of people buy in. internet. And the other change is now two years ago, we started doing the interviews online. And suddenly everybody realized actually, you know, it's not ideal, but it does mean it's much levels the playing field in a way that having these interviews be online, more people can attend the interviews. It's much cheaper for everybody. And so we are probably not going back effort to doing the interview part in person. And kind of the same way, there's other parts of the economics PhD program, they haven't really changed for a long time, but actually clearly should be changed.

And one of the things is we should have more of an ethics thing. One thing that we are behind physics and some of the other disciplines, as far as I know, there's never been a paper in economics. journal that's been retracted. That may be because we're just superior beings and we don't, their editors are all great and they, we never published wrong things. But sort of clear that that's not right. It really means we just haven't had a system where we were able to retract papers

if it turns out they're wrong or they're fraudulent. Other disciplines have had problems where people have published fraudulent papers. And it must be the case that we've done so as well. And we just haven't had a system where we do that. And now, I imagine only sort of at least explicitly saying when people submit a paper, be aware that we have the right to retract things after publication if we find that something was completely wrong. And I think that goes hand in hand with the fact that we actually need to educate the graduate students better in terms of the ethics of the profession.

There's other aspects of the profession where the whole publication process where we ask people to review things exactly what the ethics are is not made very clear. Actually, I know someone at San Diego, Paul Neuhaus, has a course kind of on professional issues, kind of about how to referee papers, how to do projects, how to read papers, kind of all about how to function, effectively in the profession. And I think we need to have more of that that kind of needs to be a part of the graduate program really at every PhD granting institution. It's not clear exactly how much these things are discipline-specific, whether we should just have a general program for all incoming graduate students, especially since people come in from very different cultures.

Actually, one thing that I always find fascinating. So here at Stanford, we have this, and you may have in San Diego as well, we have the honors code for students that if I give an exam, I can't actually be in the room where the students take the exam. They're supposed to monitor themselves. They're supposed to then tell me if someone else is cheating.

And once I was at a party, we were talking about that, and we asked people if they would actually do that, if they would inform on their fellow students, if they saw them cheating. And all the Americans said, you know, yes, they would do that, the majority. And all the non-Americans or all the Europeans said, no, no, no, they wouldn't do that. They wouldn't inform on other students. and they view that as kind of ratting out your fellow students. Because at some level, it's a much better system to have an honors code. So we should police ourselves rather than needing some outside authority.

But given that so many of our graduate students come from Europe or other countries, we need to instill that in students. And we need to teach them ethics much more effectively, given that we, we need to. don't do any of that than we do at the moment. Yeah. No, I think it's, I think it's, you know, time of your colleague, Carl Wyman, in physics at Stanford, also Nobel laureate. We've had him on the podcast.

And, you know, he's talked a lot about how, you know, education, at least in physics, is at the equivalent level, in his opinion to, you know, teaching bloodletting and leeches and so forth in physiology back in the, you know, so he's done great work and kind of flipping the classroom and other things. We also have Eric Mizzure scheduled to come on the podcast from Harvard and not too distant future. But I've always wondered, yeah, I mean, your point about cultural differences is incredibly significant. There's Rabbi Sachs who passed away a couple of years ago now. But he used to talk about the differences between guilt cultures and shame cultures.

And, you know, broadly maybe U.S. versus Europe and Asia could be divided along those axes, a kind of internal motivation, guilt versus not wanting to shame somebody in the European or Asian So I think it's fascinating. We're always saying, oh, we don't have time. And, you know, I wonder, you're so productive. And I do want to get to your lecture slides because they're so delightful. And I know my audience will love it. But what is your workflow?

I know your dedicated, you know, family man, father, your renowned educator, author, teacher. What do you, what's a day in the life of Huita, Inven? What does it look like? How do you accomplish so much? but so I don't really at some level of you myself sort of getting

a long done though so I do try to stay away from the guilt because that always I do not like that and I find it very I'm productive but you know part of it

I just I just love the work I love working with the students so you know what I do my day I spent time with my family and my kids and I do work and I don't really

do a huge number of other things I go biking here with some colleagues here living, you live in San Diego but so living here in Palo Alto biking is just amazing and stuff but I don't

I don't really enjoy watching TV very much so I don't really do a huge mind of that But even that, no, I don't know about you, but I do sort of have self-control problems for a while during the pandemic. I was using the, there's this app called Freedom that kind of, oh, yeah, that doesn't allow you to, to surf the, the internet for a while. The whole screen goes green. You're now free to do, do the stuff you're supposed to do.

Right. I use that for a while because I find myself kind of getting too, to destroy. And in the end, for me, what's always been kind of both productive and enjoyable, it's kind of having time to think and not be distracted, but it takes a fair amount of effort for me to get into that state where I can really think. And I need a couple of hours to really do that and be productive. And because that's over the older I get, the harder it is to find that time to do that. and not get distracted

because there's always a lot of other things to do. But I try to, that's kind of the part I'm most disciplined, that I try to find ways of making sure, even if I'm on the computer, to not move around to different screens and stuff and not have all these other windows open that if I'm stuck,

that I would start doing other things and really to try think, deeply into things. Don't ruin my co-author on the books that I have this expression that, you know, you need to be willing to think so hard that your head hurts. I really, really think deeply. And when I was a kid, I was, I was into playing chess and kind of... Yeah, I was just going to ask you about that.

Yeah. Yeah. I felt, that actually felt like very good exercise for what I do now. I don't really play much anymore because, partly it's not very relaxing, but the ability to kind of really shut out distractions and focus on one problem and kind of just try to make progress on that irrespective of how long it takes. I find it a lot of fun and it's even though it's kind of frustrating in the moment when it's, you know, making progress. That's sort of what I enjoy about, one of the things I really enjoy about the job.

All right. Yeah. Isaac Newton reportedly was asked when he was asked on a podcast in 1764. How do you do what you do? Similar question to what I just asked you. And he said, I do what I do by thinking without ceasing. And he would do things like lock himself in a dark room for three days at a time to study. I don't want to compare myself to you.

No, no, no. I'll do it. Yeah, the shutting out the distraction. And it's just, you know, in that sense, I kind of feel fortunate that when I grew up, there was this chess culture where sort of when I was 15, 16, we would kind of play with regular adult time control. So you would have these five, six hour games and you would do nothing else. There was, we know telephones, that we know.

talking to other people, you would just focus on a sequence of problems. And I still find it a very useful skill. Of course. I was super good at it. But I learned a lot from... Yeah, I mean, one of the quotes I have here is that you credited it, you know, with your passion for econometrics starting that. So I wonder, you know, speaking of...

Isaac Newton, I just brought up, this will be make sense in a minute. But when I wrote my, my second book is called Into the Impossible, which is interviews with Nobel laureates. And I was speaking with Barry Barrish, who won the 2017 Nobel Prize co-recipient for discovering gravitational waves with the LIGO experiment. And I'll ask, I asked him some questions I'll ask you at the end of this conversation in 20, 30 minutes. and that was, you know, what advice you give to your former self? And he said basically to have more confidence and eventually to get over having the imposter syndrome. And I said, what do you mean get over it in the future? Like he was speaking in the future tense.

And he said, no, when you win a Nobel Prize, you have, I got the imposter syndrome worse than any other time my life. Because when you win it, apparently, you know, I'll probably never find out. You could tell me if this is true. But at least for him, he had to sign this ledger that attested to the fact that he received his medal and his share of the prize money. And he is a curious guy. So he looked, well, who signed this, you know, 50 years ago? And he saw, you know, he saw Chandra Sechar and then he saw, of course, this guy, Albert Einstein. And he said, I'm not worthy of Albert Einstein.

I cannot be in the same book. I'm just a guy from Oklahoma. I don't belong in this book. And I said, Barry, guess what? Albert Einstein had the imposter syndrome when it came to Isaac Newton. He thought Isaac Newton did more than any other human being effectively since Jesus Christ, who was Isaac Newton's biggest hero. So I want to ask you, you know, on a personal level, when you achieve this great high,

and I'm not speaking about the Horace Mann Medal, which, of course, the only medal we'll ever share together. But does it impact you like on a human level? It's got to affect you. But does it, how does it affect you? Not in terms of, you know, my next paper has to be a really good one. But in terms of meaning and issues of, you know, how you spend your time and how you view yourself in the grand kind of scope of history, does it affect you? I mean, not everyone's the same as Barry or, you know, suffers from certain, you know,

feelings of inadequacy. but did it instead maybe inspire? Can it do that? How did it affect you on a personal level? Yeah. I certainly, I tried to push that away, but I certainly suffered from the imposter syndrome at various times, some of them very recently. So I went to Brown. So economics compared to physics, I think is kind of much more hierarchical, where there's a number of, because it's very cheap for economists to move around. So you end up having the top economists concentrate at a very small set of universities. And so going from Brown to my first job at Harvard was something unusual.

I'm not sure. I don't think they'd ever hired someone from Brown University before. And so at some point I remember one of the senior faculty saying, was in reference to someone else who'd been hired from a similarly ranked university as Brown and said, well, you know, we don't always, we don't very often hire from the provinces. He might even say providence. He might even say providence. Yeah, yeah, no, exactly. So that didn't really, really help much with the imposter syndrome. And so, but, you know, and growing up in Holland, Harvard was a place that, I heard of

that's probably the only American uniforms of at the time. So when I was there, I never really had the feeling I belonged. They didn't really have the expectation of getting tenure. I didn't get tenure there. And I didn't really bother me because that was never really on my horizon. But I had a great time there because I connected with some people, including angriest, including proven, including senior, colleague in the economics department, Gary Chamberlain. So it was a great time for me to learn a lot of

things. At the same time, I remember sort of going to the seminar in my area in economics, and there were four senior people there, and it was incredibly intimidating there. So partly, some of them weren't particularly supportive of the junior people. They were very senior. They were very senior. sort of they had won many prizes themselves. They saw themselves as kind of doing, having done very important work and doing very important relative to what Angeris and I were doing at the time. And then, you know, some of them sort of made it clear

that they felt much more highly of their own work than they thought of ours. It wasn't really quite clear about it they were right, but it was kind of a tough, finally, it was a tough environment. And I certainly tried to do things a little differently with my students now. I think that was not really a very supportive environment for junior people. And I think that was probably not great.

There were parts of it that were great. And there were some people who were incredibly supportive. I mentioned Gary Chamberlain and before kind of, but it was, that certainly, you know, it certainly didn't make me feel like there was sort of a natural place for me. That was, there was a place that I belonged. But I kind of over the years, and even at the time, I felt the work I was doing was good and I got a lot of satisfaction out of, out of doing the work. And I enjoyed doing that. So it didn't really bother me. I wasn't looking that much for outside validation and certainly didn't have any grand ambitions that

this would be work that would really meet with widespread acceptance. But I thought it was interesting and I thought it was good and I got a sufficient amount of positive feedback that other people thought it was interesting and important and novel. And so that helped me, they kind of continue to inspire me to keep doing it. Yeah, well, I think, you know, the, sometimes it said the opposite of the imposter syndrome is the Dunning Kruger effect where you have a little bit of knowledge. And then that makes you feel like you have this expansive perspective on the field. I always joke, I have the most knowledge of anybody about the Dunning Kruger effect.

But, you know, it's sometimes said in Jess that, you know, economists have predicted, you know, nine of the last five recessions. I can't resist asking you. I don't know how many podcasts you listen to, but nowadays, if I talk to an economist, I have to ask him or her about recession and blockchain. So I beg your indulgence, but first, what are your thoughts on recession?

Are we in the U.S., at least, in the West maybe, headed for one, or are you more optimistic that we can? Yeah, I didn't know. That's really, so a lot of the things that I won the Nobel Prize and one of my colleagues at Stanford with one before, I see, you know, people are going to ask a lot of questions about things you don't really know. And I remember kind of earlier when I was,

we had a colleague of mine in economics won the prize. And then at a dinner, he was talking about that he'd gone to Korea and he'd had dinner with the finance minister there. And we said, well, what did you talk about? I said, well, I talked about monetary policy, and stuff. He said, well, we said, well, what do you know about it?

You're an economic nutrition. And he said, no, you know, I can talk about those things. And so I try to avoid answering it. I don't really, I'm not really going to know much more than you do about the, the recession. The blockchain. Well, I've actually kind of, who's an economist, who's a much more general and very prominent economist, is for,

very knowledgeable about both blockchain and so know more about recessions too. But so I, that's not, yeah. I guess the only thing that I thought might appeal to you about blockchain is that the ability to have very rapid, very reproducible experimentation on the blockchain. So as a way of doing what you do so well, which is, you know, these kind of getank and, these very rapid and very reproducible. And I wonder if things like, if people look into things like, well, universal basic income is something that people talk about.

It seems like that could be something that would be very amenable to a blockchain, you know, deposit into, you know, I'll give you my crypto wallet later. But are they kind of metricians? Are they looking at, you know, possible applications for blockchain? Because most of them that I've talked to, I mean, Paul Krugman, I haven't talked to Paul Krugman, but they're very dismissive of it. I mean, your fellow laureates and so forth. But are there people in the field that are taking it seriously?

I've thought about it for science. but yeah so all i know really is from uh talking to my wife who knows much more about this and so i think there clearly are real possibilities and there's real things possible through the brock saying that they were not not possible before so it's clearly here to to to stay exactly what form is going to take i don't know and so it's not an area i've really thought very deeply about And so at some level, there was sort of a conscious decision. So that I, you know, nowadays in economics, people actually move around much more than they used to. But I've had a remarkable, at some level, remarkably narrow career.

In Holland, I started doing economics because that's actually an undergraduate major. You can choose. I chose that while I was in high school, not really knowing what that was going to be like. And I've stayed there ever since. The field has changed, but I've probably been good at it by doing it for a long time, longer than anybody else by this point. So that's great. Yeah, it's great to know one's strength in their character so well that you can be so focused.

And that's probably attributing to a lot of your success. I wonder if we could go over some of your slides from the lecture. So if you want to take over screen sharing, I think you should have permission. Let me first kind of just talk you briefly through how I got here. So I was born in the Netherlands, so I went to high school and there. At that point, actually, I wasn't really quite sure what I wanted to do. I was leaning towards physics or mathematics by my older brother, I was already studying that. So I decided I want to do something a little different.

And then my economics teacher gave me this book by Timberg and that got me interested in economics. And so then I ended up going to the university in Rotterdam where they had a special economics program that was actually set up by Timberg and one of the first Nobel laureates in economics. But actually that worked out incredibly well. I really enjoyed it. Then at some point, they had an exchange program with English university, and so not thinking

that through particularly well, I ended up spending two years at the University of Hull in the north of England. Then, again, by accident, there was a professor there who was about to take a job at Brown University in the U.S., and he asked me if I was interested in going there and doing a piece. At that point, I was planning to just go back to Rotterdam and finish my degree there, but going to the U.S. seemed interesting challenge. So I went there, so did my PhD at Brown University, greatly enjoyed it there. Then I was, again, thinking about actually going back to the Netherlands and taking a position there, but I got a job offer as an assistant professor at Harvard. And again, that seemed the opportunity too good to pass up at the time.

So I went there. Then after that, I moved to UCLA, then to Berkeley. Then actually back to Harvard. It was at the time my wife was at Stanford, and we're trying to figure out which place we'd work for both of us. So we went back to Harvard. But then 10 years ago, we moved back to the West Coast to Stanford so it's where we've been ever since.

Your summer starts now with Memorial Day deals at the Home Depot. It's time to fire up summer cookouts with the next grill, four-burner gas grill on special buy for only $199. And entertain all season with the Hampton Bay West Grove seven-piece outdoor dining set for only $49. This Memorial Day get low prices guaranteed at the Home Depot. While supplies last, price invalid May 14th or May 27th, U.S. only exclusions apply. see Home Depot.com slash price match for details.

So, kind of what is my research about? The general theme is it's about causality. It's kind of there. Obviously, we all know that correlation isn't in general, the same as causality. But the question is, how do you actually get causal effects? Because in social science as well as in biomedical sciences, typically what we're interested in is causal facts.

You want to know what would happen if we change something. What would be the causal effect of some policy or companies may want to change the way they do business, the way they approach customers or the way they present options to customers. We all want to know what would actually happen if we change something in the environment or changed some of these things. And so all of my research is about how to do that in a credible way. And sometimes we can do randomized experiments like we do in drug trials. But in many cases, we can't. We can't really do experiments to decide whether education is a good thing or a bad thing. We can't prevent people from going to college and force others to go to college just to learn what the effect is.

So we need to do clever things with the data to figure out what the causal effects are there. And that is kind of where most of my research is. is, and the well price was sort of partly for the work trying to figure out to exploit natural experiments, to exploit situations where there's some random variation that can help us learn about causal effects in the absence of formal randomization. And so, again, to stress the point that correlation is not causality, there's kind of many cases where we see simple correlations. The roost of crowing is very highly correlated with the sun rising.

Shark attacks are very highly correlated with ice cream sails, but it's clear that in both cases, there's no causal effect going from one to the other. Ice cream sales don't cause shark attacks. You may be correlated because in the summer people go swimming and are more likely to get attacked by sharks, and the same time they're more likely to eat it. ice cream. Here in these data, these are real data, you see that this correlation is actually negative.

That's because it's share architects in Australia and ice cream sales in Florida. So when it's summer here, people go swimming in Florida and, oh, sorry, go buy ice cream in Florida, but at the same time, there's few people swimming at that time in Australia, so there's few shark architects. But so given that the correlation is not the same as causality, what do we really mean by causality? And this is where I was very heavily influenced by some of the work Don Rubin had done in statistics. And he kind of viewed it as all about thinking about some manipulations, thinking about some action you can take. So here on the slide is this poem, part of a poem by Robert Frost. Two roads dispersed in a yellow wood.

And sorry, I could not travel both and be one traveler. Long I stood and looked down one as far as I could to where it bent in the undergrowth. And it sort of really illustrates well the problem. We want to think about cause of effects as there being some decision point where you could go left or right. and the causal effect is the outcome at one end of the road versus the outcome at the other end of the other road. But you can't really ever see both because you can only take one of the roads. So you want to think about a causal effect as not the effect in the abstract sense, but the effect of a particular manipulation of a particular action.

I can go to college or not go to college. I can take an aspirin or not take an aspirin. or not take an aspirin or you can send people some information or not send them information. This is the general notion of counterfactuality, right? Exactly. So you can take one of these actions

and see what happens there. You can take the aspirin and maybe your headache will go away, but you'll never know for sure where they would have gone away had you not taken the aspirin. Right. And so you can never directly observe

any of the causal effect. And that's kind of the big challenge. And so to deal with that, we want to think about what the manipulation is we're going to clearly need multiple units, multiple people, multiple objects so we can see some that are exposed to a treatment and some that are not exposed to the treatment. And we're going to think about why they actually got the treatment they got. Was it chance or was it choice? And that's kind of really an important thing here.

Because this really, actually, let me skip a couple of slides. That's really where the statistical traditions and the social science traditions diverged and where to some extent the work with Rubin and with Angerriss and part in the book with Rubin kind of brings back, brings together these traditions again. But in statistics, the study of course of facts was really in the prehistory. context of randomized experiments where we're just really very sure we get what we want. We take a set of individuals or some population. We split them into two groups.

We give some of them one treatment and some of them another treatment. And because we actually randomize, as Fisher kind of stressed, if we actually do the fiscal randomization, we know that even if these groups are different, the chances that they're very different is controlled. and we can put precision on how different these populations could be. And so we end up with very credible estimates, of course. That's, of course, why the Food and Drug Administration insists for new treatments that we actually do randomized experiments.

They don't want to just have the drug companies say, hey, some people took it and they're doing great, and some people didn't take it, and they're not doing it. Well, we want to make sure that we can control the error rate there will be confident that we get cause of facts. But in economics that didn't really work, sometimes it works. In effect, the Nobel Prize in 2019 went to a tree of economists. We started doing experiments in economics, and that's been very effective in some settings.

It's sort of clear that we can't do that in many cases. We can't force people to go to school and prohibit others from going to school just so we can learn with a high degree of credibility what the cause of fact of schooling is. I've often thought that, I'm sorry to interrupt, but one of them, when I was a graduate student at Brown, we had, I had a fellowship that required me to go and teach in public schools in Providence. And I remember teaching them back then. This is in the mid-90s, late, you know, early 90s. and I was teaching these students and I taught, I was teaching, you know,

how many planets there are, et cetera, et cetera. And I said, I just thought to myself, if I come back 30 years from now and I say there, I ask these adults, you know, who are children in the early 90s, I ask how many planets are there? You know, and if they say eight, you know, whatever I was teaching them back then,

it could have been nine actually. But anyway, they say eight, I'll have no idea if I had any educational impact. But if I teach them that there's, 427 planets, then at least if I come back 30 years later, I say, you know, how many planets are? And they say 437, at least I'll know I made an impact. And, you know, a lot of my colleagues looked at me in horror. But, yeah, these are the types of experiments that would be great to, you know, see if we can do

numerically or, you know, blockchain. I don't know. Do air drop money into people's block, you know, wallets. But it is fascinating. Yeah, the type of. Yeah. So in the end, kind of very early on, economists kind of decided in a lot of cases we couldn't really do experiments. Yeah.

Actually, the starting point wasn't even cases where you could envision doing the experiments, but where it's even hard to see what the experiment would be like. So Timberg and kind of in the 1920s was interested in estimating the demand for a particular product. So for Holland in those days, you know, if you've ever seen the, the Dutch paintings of Van Gogh, kind of from his Dutch period, kind of the potato eaters, potatoes were big in Holland in those days. And so he wanted to know what the effect was of the price on the demand for potato flour. And as economists, and this is actually kind of interesting where, you know, from a computer science perspective, when I talked to my friends there from a statistics perspective, they're like, wow, the effect could be positive or negative. Actually, economists are pretty sure that prices go up, the amount is going to go down.

But we don't know how much. Yes. And we don't want to simply, and that's really kind of a fundamental insight there from economics way back. We don't want to just look at quantities where prices are high and when prices are low and say, well, that difference tells us what the difference is in demand. Because prices are not set randomly. It's not like an experiment. Prices are set by sellers because they think that's what the market can bear.

That's what they can get away with. And so when they see that the demand is high, they go to charge more. And so disentangling how the prices are set, what the supply is and what the demand is, is kind of a key challenge here. And that's just from looking at prices and quantities, you can't really do that. And so part of what the economists have been doing since the 1920s is kind of trying in those settings to disentangle these different forces and estimate what the effect of prices

on demand or the effect of prices on suppliers. And so I'll come back to that kind of what you would do in this particular situation and kind of with these demand and supply settings. But what kind of at a general level they did was they tried to build a model for how people behave in these markets and then try to use that to disentangle these two forces, kind of supply and demand and kind of essentially what the solution is going to be, is to find something that affects supply. without affecting demand, something that's going to shift the supply function,

but not change the demand function. That's going to help you. That's what is called an instrument, and that's going to help you figure out what the demand is. Now, instead of one of the papers I did with Josh Angress and another co-author of ours, Katie Greedy, is we looked at it and sat in the Fult and Fish Market. This was data that actually Katie had collected for her thesis on the price and quantity of particular type of fish sold at the Fulton fish market, whiting.

And so to get to, she wanted to estimate the demand function. So the question was, what is there that changes supply, but doesn't change how much people are going to demand at a particular price? now there may be a strange question and so that's not an easy question to answer directly

but kind of from an economics perspective sort of very clear and she came up with a very nice solution there she looked at weather conditions at sea things like wave height so if there's a lot of waves

if it's very choppy it's just going to be hard for people to catch the fish more dangerous out there It's kind of, you know, if it's stormy, there's not going to bring as much fish onshore. Because if I'm interested in buying fish, if I'm a buyer there, I don't really care what the weather was at sea. Right. All I care about is what the price is.

Yeah, how it translates to price. And so that's something that doesn't change the demand function, but it's going to create variation in the price because it made it made it harder to catch the fish. So it's going to change supply. It affects supply side. And so that's kind of the way the economists kind of look for variation in some part of the model that doesn't affect other parts of the model. So we're very confident that when I interested in buying the fish, I don't really care about what the weather is at sea,

whether it was hard for the fisherman or easy for the fishermen. All I care about is what are they charging me per pound? And Tim Bergen is very interesting. He has a, he studied physics apparently. Yes. At Leiden with Paul Aaronfest. And he apparently met many famous physicists like Cameron Anas, who won the Nobel Prize for Super Connectivity. And a man by the name of Albert Einstein.

And he actually, he may be his son, but there's a famous radio astronomer. He's passed away now named Yop, J. AAP. How do you pronounce that? Huida. Yop. Yeah. So he's a very famous, and he actually wrote the canonical, very slim monograph in my field of astronomical polar imagery. So what a, what a family. And his brother won the Nobel Prize. I was going to say that that's amazing. He's a hero. He was a very remarkable guy. And so, you know, we talked a little earlier kind of about what things to do beyond academics. But he was

a big institution builder. He built some of the government advisory institutions. He started a degree program in econometrics in Holland. He did a lot of stuff for the United Nations. During the Hungarian uprising in the 50s, he actually took in refugee children from Hungary in his house. He was just a remarkable human being. Scholar and an intellect. Amazing. He was just a very, very remarkable person.

And it's sort of interesting kind of, you know, that his work inspired me when I was in high school, but it was only much later that I read his work, but it still resonates very much. It still has a very modern feel to it. I was fortunate enough that when I was a student in Rotterdam, I actually did go to a public lecture he gave there. He was still coming there occasionally.

So I do feel this intellectual kinship with him. So but later, the economists kind of started building these more and more complex models. And so here is a very interesting slide. So the picture on the right is a picture of this of an economic model, but a physical version of an economic model. So when I was an undergraduate in Rotterdam, they would actually bring this out kind of once a year. So it was a model where water flows, modeled the flow of money in the economy. So there kind of were all these basins in this machine that showed the GDP, and then there were levers that you could pull to change interest rates,

to have a monetary policy, to have a fiscal policy to change exchange rates. And it was a physical model that allowed you to simulate how the economy would respond to various government policies. And it kind of, you know, and it's a very complicated model here, and it could do many remarkable things, but it also symbolized kind of how complex these models became over the years. And at the same time, how they became less and less credible over the years. So when I was an undergraduate there, they would wheel this out and then they would demonstrate the model and it wasn't working anymore. Whatever you would do, the water would leak out of the machine and would end up on the floor. And GDP would very quickly be down to zero. So it symbolized kind of the state of economics kind of over the years in more than than one way.

And then kind of by the 80s, there was this paper by law. Lema that I mentioned before, let's take the con out of economics, where he really took issue with the fact that very little of the empirical work was credible, was believable, and that it was losing influence as a result, because whatever policy people were supporting, they could find some work that showed that I was a great idea, and someone else could find the opposite paper that would propose the opposite and provide support for that. And when Angeris and I came into the profession, there was a real sense that a lot of the empirical work was just not credible.

And what we were trying to do was find ways of improving that, find ways where people could actually do empirical work that was believable, not just. to them, but that was believable for others, that was useful for policy makers, so that that was robust to kind of a lot of the auxiliary assumptions people were putting in their models that made assumptions that you could actually talk about, that were transparent, and that you could, where you could build support for the results in incredible ways. And of course, that's not always possible. Kind of, you know, Lima looked at these analysis of the deterrence effect for the death penalty, and there at some level, you just cannot do it. There's no data

available in the U.S. that would really tell you one way or the other what the effects are there. And so it also kind of forces us to be a little bit more humble and acknowledge that sometimes we're not going to be able to give credible answers, but in some cases we do. So let me, I don't know how much time we still have. It's up to you. I'm happy to go as long as So let me go. This is kind of one of the key examples that was cited in the award. So let me go through some detail here.

So when Josh Anger was really this thesis, he looked at, he was interested in the effect of estimating the effect of serving in the military on earnings. And so this is kind of one of these cases where obviously we can't do a randomized experiment or no randomized experiment have been done. And you're worried that people who serve in the military are different

for people who don't serve in the military in lots of different ways. They may be more, less risk of us, they may have more skills, they may be more highly educated, they may be different in many, many ways. So how are we ever going to find comparisons that people would actually find believable? And so the... The first step was to go back to the Vietnam War when there was a military draft and kind of

realized that because the way the draft was implemented through a lottery, that we get two groups that are comparable, just like in a randomized experiment. The people were, we had low lottery numbers who got drafted, who were draft eligible, were ex ante exactly the same as the people who didn't get drafted. we had a high lottery number. So there, it's just like a randomized experiment. These groups are not identical, but a comparable ex ante because of the randomization. Problem was that not everybody who was drafted served in the military,

and not everybody was not drafted didn't serve in the military. In fact, if you kind of look at the numbers there, actually let me skip a couple of slides here. If you look at the number of the percentage of people who was drafted who actually served, it's about 31%, but 19% of the people who didn't get drafted, served in the military. So these things weren't perfectly correlated.

And so the question is kind of how can we make progress there? And the starting point then was to think about the people were not drafted and thinking of there being three types of people. Some people were always going to serve in the military, were going to volunteer. And for them getting a high lottery number or a low lottery number, it didn't really matter. They were happy to go serve in the military.

The exact value of the lottery number didn't matter. Then there was another group where who wasn't going to serve in the military irrespective of their lottery number. They might have had a medical condition. They might have had bones spurs. Whatever. Again, having a high lottery number or low lottery number doesn't really matter for them. But then there's this group in the middle. This is what we call the compliers. They kind of actually do what the draft was intended to do. intended to do. They get drafted, they serve, they don't get drafted, they didn't serve. They weren't actually going to volunteer, but they were willing to submit to the draft

if they did get drafted. And so if they were not drafted, they would end up in the civilian labor force, but if they did actually get drafted, then they would switch and they would serve in the military. And you can kind of see that really what you would like to do is kind of know who these compliers are. Because then for that group, you actually do have a randomized experiment. There, the randomization does exactly what it's supposed to do. It switches people from the complying, from the not serving to the serving population. But the problem is we can't tell, we can't tell directly who these people are. We can, we can partition the population kind of into these four groups, whether they were drafted or not and whether they actually served or not,

but you can't tell whether someone is a complier, because if you see that someone was not drafted and didn't serve, they could be, it could be degenerate, right. It's a degenerate, right. It's a degenerate. It could be that they would never serve or that they are a complier. And so the question is, how do we disentangle these groups? And that's kind of where, where the insight was that if we're willing to, so we're trying to disentangle these groups. But from the, from seeing draft status and veterans status, we can already,

first of all, we can estimate the fraction of these street types. And here we find that in fact about 20% were volunteers, about 70% would never have served in the military and then there's only this 12% in this compliance group who actually the one we're most interested in they would serve if they got drafted but not if they didn't get drafted but we still can't tell exactly who they are we can tell how many there are but we can't tell which individuals there there then the next step was to uh to assume

that for these guys who served who always served and who never served, the draft just really doesn't matter. It didn't affect their outcomes. Now, that's kind of still a tricky assumption. It may work very well if these people were exempt for medical reasons, but it may not work so well if they made choices to get out of military service, if they would move to Canada or they would do other things to stay. out of military service.

But if we are willing to make that assumption, then we can estimate what the effect is for, of military service, for this group in the middle, for these 12% of the complies. And for the data that Angres collected, he estimated that there was about a 20% decrease in earnings. So these people actually paid a huge tax later in life in terms of their earnings.

by serving in the military where their counterparts, we had a lottery number that meant they didn't have to get, they had to do military service, had earnings that were much higher, because they had additional time in the labor market, they had additional experience, and they ended up with better paying jobs. And so this is the kind of natural experiment

where the system, the structure, helped us by giving some credible randomization for some part of the problem, and then we combined it with some of the economic modeling to disentangle these effects in a credible way. Here, kind of just as a way, sort of how they may work in other settings. Similarly, when I grew up, we had military service in the Netherlands. Of course, there was a little different. Holland wasn't actually at war at the time. and in fact they didn't need as many recruits as they were getting.

So the military kind of had a problem. In fact, they had too many people who were serving in the military. And so they decided to deal with that by just exempting one year entirely. So there was one lucky birthday in 1959 where people didn't have to serve in the military. and the years before and afterwards, everybody in principle had to serve, and about 45% or so, on average, did serve in the military. There were still people who had medical exemptions, there were still people who had religious objections and other things,

but typically about 45% or so served in the military. But then there was this one year where nobody served. And so what you see then much later, here we looked at earnings in 1990, when these people were in the middle of their careers much after their military service. We see that the group in the middle, this 1959 birth cohort, actually made considerably more money than the people born in 58 or the people born in 1960. And you see that this is based on national data.

So it's a large sample, even for a small country as the Netherlands. You see that these people just make significantly more money. And so that seems very plausible that comes from not having to serve in the military. In fact, the numbers are not as high in the angriest case, but it's about 4%. These people make about 4%. And to kind of make that even more credible, if you do the same exercise for women who never had military service in the Netherlands. For the women, there was no such difference.

The 59 cohort was no different from the 58 or 60 cohort. And so that lends additional credibility to these estimates. And so that kind of satisfies the Limer concern that you want to make these things credible. you want to find you want to come up with estimates where you're you're really willing to put your reputation on the line that these actually are incredible causal effects i said let me go to one more example and so this goes back to the the timburg and the study so here's what this data actually look like from the food to fish market study And Katie collected these data first season.

That was actually just a remarkable thing what she did there because she would have to get up at two in the morning, every morning in Princeton to drive over to Manhattan to the filter fish market. You know, that was a tough place to collect data. There was concerns there still with organized crime involvement, and made the building where they kept the records burned down. And I've been there once just to see what I was like.

And it was not a very pleasant place to collect data. To do a thesis. Yeah, to collect thesis data. Every day, every transaction just follow the seller around. Oh, yeah. Or the transaction and kind of how much fish they sold, what price they sold. And there are kind of lots of interesting things that came out of that. But so here, just kind of looking.

at this cloud of data points. Sometimes prices are high, sometimes prices are low, sometimes quantities are high, sometimes quantities are low. Correlation is pretty much zero. There's nothing there. But then what she did was say, well, the first thing that the economists would do, they say, well, what do these points really mean? We want to think of every point there as the intersection of a supply and demand function.

Now, that doesn't really help very much on its own because both supply and demand clearly change every day in this market. So that doesn't help us kind of figure out what is what is what. But then what she did was collect data on what the weather was like at sea in the day before. when these fishermen were actually out there, the catching the fish. And so, she defined kind of three sets of days, or at least kind of here, we're using three sets of days, stormy, fine and intermediate.

And so stormy was when the waves were high, when the wind was strong there. And what you see is on those days, the prices are relatively high. and the quantities are relatively low. And we think of that as those days' supply was low. Demand wasn't really affected because nobody cares. The buyers don't really care.

They don't care whether it's store we. They see what the price is. And so then if you look on fair days, the price is relatively low. The quantity is relatively high. And so if you then put us two together, that helps you figure out what the demand function is. It helps you disentangle the supply and demand, kind of very much in the same way that Tim Bergen did with his potato flour data. Right.

You know, not in a big data setting. I think he had like eight observations, but it sort of helps you disentangle these things. The same way the draft lottery creates this random variation in whether people serve or not, at least for the compliers, here the weather creates this random variation in the supply function that allows you to learn about the demand function. And that's, you know, then the sort of more complicated stuff in the background where you want to deal with the fact that is actually also heterogeneity in the demand function and it may not be linear, but it's having this instrument, having this variation in the supply that helps you trace out

what the demand function is and that gives you more credible estimates of how demand would change if you change the prices. And just the vocabulary, the price elasticity is the derivative of quantity, log quantity versus low prices? No quantity versus lock. So it means anti-correlated high quantum. quantity more price. And the minus one is actually very important because if it was, you know, obviously it should

be negative. Yeah. Prices go up, demands should go down. But if it's between zero and minus one, it actually means that you would sell, you would, your revenue would go up if you increased prices. And so then sellers should just be increasing their prices because they would actually increase their revenue with it. And the fact that it would sell less, it wouldn't matter.

because the revenue would go up. So you would expect in these markets the elasticity to be, elasticity to be minus one or less. So that's, and again, the raw correlation between prices and quantities here is actually pretty much zero. So it really shows that this is making some sense. Now, the last example is from a paper, actually I did with Don Rubin and a student of ours, Bruce Sassadot, and that came directly out of this first course we were teaching on causality. We're interested in what the effect was on the universal basic income. Initially, actually, we were interested in what the effect was on child poverty. And there's sort of been a lot of work on that.

And essentially all the empirical work was just comparing kids in rich families with kids and poor families. Because, you know, you need to have kids who grow up in rich environments versus poor. And sort of clear, it's not really clear what that means because rich and poor families are different in lots of different ways. Partly their parents have made choices, their parents may have different, that may be different in many ways. it's not clear that any comparison of that is really going to give you directly the cause of facts you're interested in because the government's not going to take kids from poor families and put them in rich families. All they could ever hope to do is give these families more money. And so you want to analyze data that mimic that type of thought experiment. And so what came out of

discussion is that it would be very helpful to look at people who would play the lottery and who'd won the lottery. There, that actually mimics very closely, just getting a guaranteed stream of income. It's a particular nice thing, the nice thing of the lottery there that we went to in Massachusetts. But that's true in general. That's too often for these lotteries. They don't give you a lump sum, So you win half a million dollars. You don't get a check for $500,000. You get a check for $25,000 for the next 20 years. And so it really is like a universal basic income.

No, it's not quite the same. It's not coming from the government. And your neighbors may realize you won the lottery, and there may be some pressure on you to do things. But that may be after a couple of years, that may wear off. and it may be very much like you now have this extra income for a while. I can't resist asking you, if you had the opportunity to take your Nobel winnings,

lump sum, which I assume you did, versus paid out, which makes more sense, you know, for give us a year. I think for these respondents at some level it makes a lot of sense getting it in over the years. Because I think what you see, there's. often people don't quite know what to make of it very early on. And so they may waste a lot of it early on, but then after a while they figure out that, you know, this is not going to go on forever. I need to actually make a plan here. And what you see in the data as well as, you know, a lot of people don't do crazy things

with it. They don't stop working. They don't just waste at all. people actually save a reasonable amount. Now, because for the Nobel Prize, the thing is it typically goes to people who are fairly well off already, so it's not really clear that it changes their life

quite as dramatically as... Do you think there's a limit often, and you kind of alluded to this earlier, that you know, you really didn't do the work to win a Nobel Prize, you didn't... Is there a limit? Is there an amount of money? I mean, there are prizes worth more than the Nobel Prize.

You know, the breakthrough prize is worth three or four times. Is there a point you think that people would start getting influenced by the financial remunery? Or is it, you know, demand is completely inelastic? I don't, no, I mean, because our jobs, all of it, your job, our job, my job, we are incredibly fortunate. We get so much freedom to look at questions that we're interested in. And so you want your life to have some meaning. And so it's sort of, I guess part of the problem in economics is often that we can't get people to ever retire.

Because they like their jobs. They're incredible. You get to talk to really smart people all the time. You get them to listen to you. You get to hear their ideas. So it's not quite clear. You get to travel to interesting places. So it's not really clear what I would really want to change in that.

Oh, yeah. Fair enough. Fair enough. I'm more venal and driven by these issues than most. But anyway, sorry to interrupt. So I think the aspect of the Nobel Prize that has sort of changed things most for me is the fact that now it's even easier to reach out to say to some place, call someplace, say, hey, I'm going to be passing through.

I'd love to give a talk and everybody's immediately happy to accommodate that. And so it makes my life considerably. easier. It wasn't difficult to begin with, but it just gives me a huge amount of freedom to do exactly the type of research I want to do and go listen to the people I want to listen to or talk to the people I want to talk to. No, I imagine. I guess one serious question would be, you know, a lot of Nobel laureates and I'm only using them as a proxy for, you know, highly accomplished scientifically, you know, literate and capable people, and even restricting the pool of data to just experimental physicists who have won the Nobel Prize, eliminating all the theories, you look at them and

almost uniformly, I mean, none of them died completely poor, maybe, you know, like a pauper, but many died, you know, these are people that invented, you know, like Shockley, the transistor, you know, who was up at Stanford, right? there are people that invented the laser charlie towns at berkeley um they never really received any of the financial benefits of their inventions and obviously they all do the work you know they would do it if they got no money as you're saying it's the i call it the best job in the world and you can prove it's the best job in the world to be a university professor cledo because it's the only job that neil armstrong would accept after he walked on the surface of the moon he became a professor of engineering

So we get to do what Armstrong wanted to do. But in all seriousness, some of my colleagues have proposed things such as, you know, we, physicists collectively, not me, you know, invented the transistor, you know, the laser, the internet, you know, at least the, you know, some of the hypertext protocols at CERN. These are huge capital, you know, kind of providing revenue providing things. As Faraday once said, you know, when he was asked, what's the use of electricity? He said, I don't know, but someday you'll tax it to the king. But I wonder, Guido, is there any thought about, let's say we, you know, one of my colleagues thinks,

oh, you should tax every email, you know, a thousandth of a cent in order to pay for physics research, you know, just thinking, again, very parochially. Is there, you know, do people study this? This would be a very expensive experiment, perhaps. But in other words, how do we, you know, provide for the future of the field of physics by, we can't go after the fact that, you know, give us money because be invented the transistor. Is there anything we can do in the future to ensure, you know, the financial security, not personal security of myself or my phone?

Yeah. So I'm going to kind of make a little bit of the distinction that between the personal security and the security of the field. Because in terms of the personal security, you know, what I see in economics, now there's a lot more people going into the tech working for the tech companies. they kind of they have realized that kind of like computer scientists and statisticians, economists can actually contribute a lot there. And so some of the PhDs go into that, into those companies and they make a lot of money. They make, and some of my colleagues

have done that on a temporary basis. Some have gone permanently. And if they do that permanent, They often end up making four, five times as much as they make in academics. But in the end, very few choose to do that because the academic jobs just have incredible perks as well in terms of doing what you want to do. So I don't know. So in that sense, I don't, and here around in Silicon Valley, I kind of talk to some of the plenty of capitalists and entrepreneurs, and I think what they're doing is really interesting.

But it also takes skills that I don't necessarily have, and I think some of the things, even though I could well see that they would end up being very successful, I wouldn't find very interesting kind of to be doing. And so I would not want to, and said, I made my choice here, and I'm incredibly happy doing what I'm doing. That's that for the government, for society as a whole, we need to make sure that we keep the people going into these fields. And we need to make sure we fund them well enough.

And we need to kind of realize that exposed some of the work people have done, even though that may have looked ex-ante, like it wasn't going to be particularly useful, turned out to be incredibly lucrative for society. and we need to make sure that those inventions keep coming. So finding some better ways of financing these things without nickel and diming it's the way it gets done at the moment that clearly would be good. That makes sense, yeah. Sorry to interrupt. No, I think there,

it used to be that some of the companies had bigger research labs kind of the sort of both here of course with the Bell Labs in Holland the Phillips Electronics Company was for a long time the biggest company in Holland they had a big physics lab kind of with the very distinguished physicists were working And they as a company kind of took a lot of their social responsibility. They funded all their employees, kids kind of going to college and stuff. I got the funding from them when I was a undergraduate. So you would like to see some of the tech companies taking some of their responsibility as well

in terms of funding the PhD programs that they take, that they take a lot of. students from. It's one thing for the government to pay for their PhD programs if these students all go back into public education. But if they go to the private sector, then maybe the company should pay a little bigger share of the cost there. So I think we do need to think about how to change the funding there. But so in this case, kind of with these having the lottery gave us a very credible way of figuring out what the effect of the unearned income was. Now, again, it still turned out to be fairly complicated. If you just look at the winners and losers samples, these were actually quite different.

Because there was not because the lottery was not random. It was just there was people would buy different amounts of lottery tickets, people there was differential non-response because people didn't necessarily answer a survey. But in the end, we ended up with kind of both with credible the estimates and we ended up with the estimates that were credible. We could demonstrate that they were credible by looking at how much these differences were prior to winning and that these statistical methods kind of eliminated any difference between losers and winners prior to the time. they won the lottery. So here, this was, again, something that would really satisfy

Lima's concerns that these were estimates that you could take to the bank and that you could, they were credible. In a purely, you know, efficient market, wouldn't there be some influence, say, on the expected future value of money in that, you know, in a low interest rate environment, it might be perceived by, you know,

obviously a knowledgeable person, that it's better to take a lump sum. and, you know, avoid inflationary concerns, or is anything in macroeconomically? Yeah, in principle, that's absolutely right. And in fact, there's a market that kind of souls that, because if you win this guaranteed stream of income, you can actually go to the bank and ask for a lump sum, and they will discount it for you. In practice, and that's sort of behavioral economics saying what people get initially,

they sort of tend to stick with that rather than go through the trouble and changing that. So a lot of people just stuck with the 20 years payment system because that was what they were offered by the lottery and they didn't want to have to think through and worry about taking advantage off by the bank by not getting enough or getting or not trusting the bank there, they clearly did trust the lottery at that point. So in practice, this may not mimic perfectly universal basic income, but it's much better than just what, and that's kind of essentially what economists had done before, sort of comparing people with rich, with high earning spouses and people with low earning spouses.

It's sort of clear that the selection of spouses is not random, but. either, that that variation is not particularly credible. Yeah, that was fascinating. Thanks. You said this place was steps from the water. We just haven't found the steps yet. How much did we save? Enough.

Enough to get lost. Or you could book a stay with Hilton. Welcome to your ocean front room. Just steps from the water. The Hilton sale is on now. Book on Hilton.com or the Hilton app and save up to 20% to get the stay you expected. When you want savings, not surprises. It matters where you stay. Hilton for the stay. Thanks. Yeah, it's great to have such a thorough

and delightful wide-ranging conversation. I wonder, Guido, you've been so generous with your time and as you economists say, time is money. But I wonder if you wouldn't mind answering these four final questions that I call my fantastic four questions. Great. Okay. Well, they all involve some way legacy or, you know, advice to your former self. And most of my audience is young. A lot of males listen to my audience. So, yeah, you can kind of interpret that as you like. But a lot of them really do reply quite well to, you know, advice and how to overcome the challenges that you certainly have had to deal with. And I think the first thing I'll start with is, as I said, you know, it's right before my birthday, right after your birthday. And in the Hebrew tradition that I partake of, there's a notion that the righteous lived to be 120 years old. So that's the age Moses lived to in the stories of the Old Testament. And at that time, he gave an invocation kind of, that's what the last book of the Bible is about. Sorry to give a Bible lecture if you already know this.

But the final book called Deuteronomy in English or Devarian in Hebrew. It's basically one long will. He's giving like a will, not of monothea. military will and he wouldn't get into the promised land and not all of us get into the promise land. But he gave a ethical will and it was kind of advice and then from the sort of wisdom that he had acquired over his 120 years. I want to ask you when you reached the biblical age of 120, what sorts of wisdom would you want to give, not just to your biological children that you're so devoted to, but to your ideological children, which there are many around the world.

Yes, because I realize I'm now almost halfway. Midlife. It's time for you to get a Corvette, a red Corvette and have a midlife place. Yeah, so these are great questions because I'm very reluctant to answer them. So I think it's about the importance of listening. Because now I've done a lot of talking the last hour. but you're the guest. I'm much more comfortable.

I learn much more from listening and then kind of pondering what I heard and read. I was talking to a politician, Dutch politician the other day who was an academic prior to being a politician. And at some point he said something that struck me, that resonated with me. They said, well, a lot of the politicians he talked to,

he was admiring how confident they were in their opinions and that he had a hard time ever getting them to question their opinions and listening. It was having a hard time getting them to listen and kind of to actually think about alternatives and kind of think about really questioning whether they were. right in their opinions and what other what alternatives were and I I really get a lot of joy out of reading out of listening and then thinking yeah reflecting on what I I

heard or read and at some of I feel very lucky that I grew up in a in a world where that that may have been easier when I was there was not there were not quite as many distractions as as there are in the current environment. And so I think a lot of it's about the importance of listening and kind of getting exposed to different opinions. Yeah. I have to put the part of my audience that is doing PhDs, whether it's in graduate school or

that is in academics. Also, it's not easy. It's not, it's hard. You're all doing what we're all doing. It's very hard. And it kind of may,

you know, on your podcast, you have a lot of incredibly successful for people and that may make it look like it. It's all easy. But even for these people, was so successful. There's kind of a lot of times when things are very hard. When I was in graduate school at Brown in my second year, there was a time when I was very fed up with the, with graduate

school. And I started applying for non-academic jobs. And so I remember at some point I was applying for a position at a bank and they were, it was a bank in New York, and they were looking for someone who was fluent in Dutch and we had a master's in economics. I thought, you know, I just finished my master's part of the program. And I was fluent in Dutch. It was fluent in English. I thought, you know, this is, I'm going to get this job. And it was a lot of money.

They were paying a lot of money. It all seemed good. And they didn't even interview me. So I ended up being by force, basically staying in the graduate program. and because they worked out very well. But yeah, I was going to say, it's too bad. You could have had a good career. Yeah, there's a lot of times when these things, you know,

they don't go your way. You get papers rejected. It's, that's kind of pain. That was painful early on in on my career. It's still painful. The first four papers I go back after winning the Nobel Prize were all rejections. And so I'm not, you know, I don't want to completely. I've been very fortunate in the way of our papers have been handled over the years.

But it's very hard. For us, it's very hard for everybody. And people should not underestimate how challenging it is from a mental perspective in these jobs. And people should be aware that everybody else is going through this as well. That it's part of the imposter syndrome. But people who are people. who clearly belong exposed, have suffered from that greatly. And you mentioned a conversation with your Dutch politician friend.

It reminds me of a quote from the Nobel laureate Lev Landau, who won the year before you were born. And he used to say about cosmologists, my profession used to say cosmologists are often in error, but never in doubt. So that's kind of one of the blind spots we have to guard against. Okay, the second of my fantastic four questions is a quote from, revolves around the quote from Sir Arthur C. Clark that we opened the podcast with, which is that any sufficiently advanced technology is indistinguishable from magic. I want to ask you not necessarily in your career, but it could be, what kind of magical ideas have you encountered that you would feel worthy of, you know,

kind of having sort of a swagger about as a species from economics? or any idea, you know, for billions of years from now when aliens find the Earth, what kind of things should humanity in your field, your work even, or completely, you know, remote from your work, chess or something? What could be the most biggest, mass, magical accomplishment of human. Yeah, you know, that's, that was probably the trickiest from these, for our questions. I was reminded of the fact, to be moved into our house on Stanford campus about nine years ago, And so I had the brilliant idea of having the kids make a time capsule to put in the walls

because we kind of did a major renovation there. And so we were going to open it 30 years, so about 21 years from now. And so the kids kind of all wrote some stuff. My wife and I did. My parents did. And now kind of thinking back at least on what I wrote myself, I don't know what anybody else wrote, it's sort of pretty disappointing to see how little there was,

actually that I thought is still relevant now. I thought, you know, this is the time that we were driving kids to school all the time. And so I thought, you know, 10 years from now we're going to have to sell driving cars and it's going to make it so much easier to get the kids to school and it would solve these traffic problems. And, you know, a very easy prediction seemed like we would no longer have print newspapers because that's so clear that that makes no sense to kind of print these things out and bring them to people. So we still have a subscription to a print newspaper. And then you get the pandemic, and it's just nobody saw,

I mean, there was just never on our radar screen that could actually happen. And I remember kind of reading that in Japan, the schools were closing down. And it just seemed unimaginable that we would get to that point in the US, that the kids would not go to school. Yeah. Of course.

On the other hand, it shows the incredible resilience where we can actually, you know, we managed to make that work in a lot of places. And we've actually learned a huge amount, you know, and that it's not really fair. It worked very well in some places, but other parts of the populations suffered greatly as a result. but there was an enormous amount of resilience there. At the same time, when the internet came out,

it just seems such an incredible promise. And there have been a lot of disappointments. Yeah. Where it didn't really work as something to make democracy stronger and to weaken authoritarian regimes. It kind of really seems like it's pretty much the opposite. And it's... Reenforces it.

Yeah, it's just so hard to... to, but it seemed like this would allow everybody to consume information and sort themselves. And it turns out actually it doesn't really work that way. Somebody talked to the, this guy was editor of some news organization. He said, well, he thought us before when we had newspapers that were, where the business model was coming from advertising that effect the advertisers. had a great moderating effect on the content because, you know, if you're Procter and Gamble, you don't want to have crazy stuff in the newspapers that you, that you advertise in.

Right. And at some level, at the time, that seemed like a bad thing, we don't want Procter and Gamble to kind of censor what we can read. But maybe that it was sort of a good thing to have somewhere some moderation, where now anybody with crazy ideas can kind of find a platform to to do that. You don't need to. So I don't really know what the big idea is sort of, you know, personally, I did kind of find some of the ways of thinking about causality that I was exposed to early on in my career. I kind of find that just, that was eye opening for me.

And then it's some of the work. and kind of just the life kind of of of Timberg was very inspiring to me. But I'm not quite sure what I would want to put on a monolith. That's right. Well, we can move it to another quote, famous saying from Sir Arthur C. Clark, he used to say when an elderly, but distinguished scientists states that something is possible, they are almost certainly right. but when they say something is impossible, they are probably wrong. Now, I'm not calling you elderly.

But you are a scientist. You are an economic scientist and have a great deal of insight. I want to ask you, are there things that you've changed your mind upon or maybe a sea change in your field that wasn't really something you anticipated that, you know, in retrospect maybe could have been more obvious or, or. maybe even something you've completely changed your mind upon, not saying you were wrong about, but you've had a turn of events hard about. Yeah, so I'm going to say two things here. One is the, you know, so now I'm editor of one of the economics,

so I kind of read a lot of paper that I try to kind of think about whether, I've tried to keep a very open mind about things where people are suggesting things that are way out of the box. And so I had this paper a while ago where someone said, well, you know, all of economics sort of builds these models where we discount the future a little bit. We don't put as much value on the future as we put on today. And I think they're saying, well, that was all wrong. We should we should discount the future much more.

We shouldn't really care about the future. Now, you may say, well, actually, we should care much more about the future. Yeah, we're going to be there a long time. Yeah, we've made a mess. you know, we've made a mess on some things. We should value the future very highly. But so this guy said, no, no, no, no, you guys got it all wrong. You're putting too much value on the future.

We should, you know, pollute away climate change, no worries. And his argument was, and this gets more to your neck of the woods. She said, well, there's some chance we're going to figure out how to do time travel. And once we figure that out, we're just going to come back and clean up the present. And so the fact that we make a mess of it now, it's not a concern. we should just give a lot of money to the physicists and they figure out time travel and then we solve it all. Yes, I endorse that. Yes, I said, you know, this is not really for the journal I run.

You may need to send it somewhere else. And in fact, so I'm going to reject this paper. But it's really a win-win thing because either I'm right and it was the right decision. Or if I'm wrong, that's no problem either. then I'm just going to go back and come back in the future, expose, publish the paper anyway, and then I look like a genius. Sir Roger Penrose tells a story about how one day Stephen Hawking was around Cambridge and he put up a desk and on a desk it said,

welcome time travelers. And Penrose and everybody's, what are you doing? And then the next day he published in the student newspaper an advertisement for time travels to come and visit them yesterday. And so he was thinking he could disprove any attributes, any possibility of time travel. Okay, Hito, you've been so generous. Last question.

Now we're going to go backwards in time. Maybe a time travel is really applicable here. And this is the famous quote I use from Arthur Clark that says, the only way of distinguishing the limits of the possible is to venture a little way, them into the impossible. And that's the name of the podcast. I want to ask you what mysterious aspect of life maybe perplexed you as a 20 or 30 year old man

that you'd give advice to your former self about now to give you the courage to do as you've done to go into the impossible. Advice to your former self. Yeah. So looking back at my career, I felt early on I wasn't very good at picking good questions. And I think I also also wasn't really very good at talking to other people about that. I said, but where I was very lucky, it was kind of connecting with people like Angress and Rubin, who at the time, to me, felt like they were much more confident, like, you know,

this is a stupid question. Don't work on these things. But sort of trying to figure out when a question is, it's worth spending a lot of time on. Once I was kind of motivated to look at a particular question, then it was just like playing chess. It didn't really matter that at some level it was whether it was relevant or relevant or not. Once it engaged me, it was very easy for me to just get deep into that.

And I do feel that a couple of times in my career, I kind of ended up with a question where I would feel that there's something here I don't understand, but I feel deep down that there is a solution, that there's some insight, that there's some better understanding, and I want to get to that place. And a couple of times that did work out where at some point I really felt, okay, this here, now I know something that I really didn't know before and that, you know, I don't think in this field people know that. It doesn't mean it's necessarily an important thing,

but I really felt a couple of times and like, wow, now I've learned something that this is sort of what I'm doing this for. I sort of suggest to my former self to kind of be more confident there in kind of trusting your interest. Because it's very hard because you do make mistakes. You end up working on the wrong things. But again, in combination with the listening, kind of just be willing to spend time thinking hard about the problems

and trusting your intuition that there is some solution. Because developing that intuition has been a long process. But I feel like I suppose earlier, I was actually better at that than I realized at the time. Wow. Yeah, that's the message. I think that's so important for my audience to hear. As I said, many people will listen to this thousands and thousands. And it's been such a great treat to talk to you and for you to share so much of your time. And from the land where the telescope was invented, I was afraid that you might use another Dutch saying on me, which is cluff in the moulin, is that? Clough in the mola. I thought you might say that about me.

No, no, no, this is great. It was a very pleasant conversation. It was great to see you. Wonderful. It's so nice to talk to you. I hope we get to see each other again, my fellow man medalist, and I wish you great success and many, many checkmates to come. If you do get some time to play some chess, that would be lovely.

Huido, thank you so much for joining us today. Take care. Well, that's a wrap, everybody. on this phenomenal episode, epic episode, two hours long, first Nobel Prize winner, not in physics. We have many Nobel Prize winners coming up on the podcast. And I really just implore you to stick around. And if you want to pay me back economically, it won't cost you anything. It's free.

There's three things you can do. All of them are free. And so there should be infinite demand for these things for you guys to do these because the supply is so plentiful. And that's to, A, leave a, or number one, rather, leave a rating and review of the podcast on Apple, or on Spotify, Audible, et cetera. I'll read each and everyone. I may even read your name and review online if it's good. No, just kidding.

I would read it even if it's bad. I've done that. Thanks, Mom, for leaving me that one-star review. And the second thing of the free trifect is to join my mailing list, Briancateen.com slash list, where you may be entered into win one of 100 fresh. Now, these are actually aged meteorites. Four billion years old plus space schmuts from the outer solar system

from a previously failed planet from a supernova explosion, highly magnetically susceptible, and you will enjoy it if you're one of the lucky winners. Brian Keating.com slash list. And finally, join my YouTube channel, subscribe. And that is Dr. Brian Keating on YouTube where you can search for Into the Impossible. My name, Brian Keating, you'll find it.

And there you'll see the slides and so forth from my guest. And we always have cool, interesting animations, videos, and customizations, as well as the interviews. And 10-minute thesis projects that I produce as solo episodes about the coolest, hottest, coolest, hottest, hottest, or coolest topics in all the lands of STEM,

science, technology, engineering, and math. So with that, I bid you a do. I thank you for tuning in on this birthday edition, asking you to get me those three free birthday presents, and bidding you a do. Until we meet again, have a magical week. Yamava Resort and Casino at San Manuel is California's number one entertainment destination

for today's superstars. Catch the Jonas Brothers return to the Yamava Theater stage on April 30th, the powerful vocals of Demi Lovato on May 17th, and the signature Southern Country Rock of Eric Church on July 19th. Tickets on sale now at yamava Theater.com, only at Yamava Resort and Casino, celebrating its 40th anniversary.

You win? Must be 21 to enter.