Odd Lots - How to Forecast the Future

Episode Date: August 26, 2019

Every day, people are bombarded with predictions of what will happen in the future. In recent months, talk of 'inflection points' in the markets has heated up, and the possibility of the U.S. economic expansion, now the longest in history, coming to an end is being actively discussed. But how do we know if such predictions are good ones? And how can we learn to be better forecasters ourselves? On this week's episode of the Odd Lots podcast, we talk to Philip Tetlock, the Leonore Annenberg University Professor of Psychology and Management at the University of Pennsylvania, and the author of numerous books and papers on the topic of predictions. See omnystudio.com/listener for privacy information.

Thanks for listening to Oddlots. Follow the show on Amazon Music for more future episodes or just ask Alexa, play the podcast Oddlots on Amazon Music. Hello, Oddlots listeners, it's Tracy Alloway. As you may know, Oddlots is hosting its first ever live event on Thursday, September 19th in New York City. Join me and Joe Wisenthall as we host an evening of great conversation and live music. The Oddlots variety show will feature some new and old Oddlots guests, including the Economist, Stephanie. Stephanie Kelton. Sam Antar, the former Crazy Eddie CFO and convicted felon, will give us some of his best stories. Meanwhile, Lee Bukhite, once dubbed the philosopher king of sovereign debt lawyers, is coming out of retirement for us. Zoltan Pozar of Credit Suisse and formerly of the U.S. Treasury will be on stage with Brad Setzer, senior fellow at the Council for Foreign Relations, talking all things, bonds, and trade. We'll also have some markets-themed music, courtesy of Merle Hazard, the most important

country singer in economics and an early Oddlots guest, and even Joe has promised us a song or two. You can stick around after the show for some drinks and a chance to chat with us and our guests in person. So keep listening to Odd Lots to learn how to sign up to see us live on Thursday, September 19th. We hope to see you there. And welcome to another episode of the Oddlots podcast. I'm Tracy Allaway. And I'm Joe Wisenthall. So Joe, I feel, and I think we've discussed this before, but it feels like the world is sort of at an inflection point right now. You think, I mean, you think so? Yeah.

Isn't it always kind of an inflection point? Yeah, I guess that's true. But at least in markets, it feels like, you know, warnings that were late in the cycle, that we could get a recession at the end of this year or in 2020. Those definitely seem to be heating up. I think what I would say, and where maybe we would most likely agree, is not that the world is at an inflection point per se, but that what comes and goes is periods when suddenly people feel the turn is about to come. And we've had a series of these over the last 10 years, whether it was Q4 of last year, early 2016, with the economy going down, the euro crisis. From time to time, it's like there's this global collective anchor. anxiety that rises and whether it's any more real or not is up for debate. But I would absolutely agree right now you're getting a lot of like recession calls, bear market calls, start of the easing

cycle calls, things like that. A wave of global anxiety. That's a good way to put it. And this isn't really, as you mentioned, the first time that we've seen this. And I'm not just talking about markets. We've seen sort of inflection points happen in politics recently, right? So Brexit springs to mind, the election of Donald Trump as well. Yeah, and I think the fact that we have another election coming up in the U.S., again, it's one of these points where people want to call some sort of meaningful turn in something. There's no doubt we're hearing a lot of that these days. Right.

So all of this is a roundabout way of saying that we're getting a lot of forecasts and a lot of predictions, a lot of people trying to call or see it into the future. And so I thought it would probably be a good idea to do an episode on forecasting. I love this idea. I mean, one thing that I've always wanted to do and never done is, you know, obviously on TV, I talk to people all the time and they make forecasts like, oh, we're in this stock or interested in this sector or we're telling clients to do this. And I've never like gone back and it would just be too much work for me and like actually made a database of all their calls. But I've always thought that's like a fascinating project or that, you know, that because who knows?

You know, people just make these calls and how often do they ever get revisited to see if the person was actually right or useful in some way? Totally. And I have a feeling that our guest for this episode is going to have something to say on that particular topic. So without further ado, why don't I go ahead and bring him in? Our guest is Phil Tetlock. He is Annenberg University professor at the university. of Pennsylvania. He's also the author of numerous books and papers on forecasting. So, Phil, welcome to the show. I'm glad to be here. So, Phil, I guess my first question is, you know,

one of the thrusts of a lot of your work is that experts tend to get forecasting wrong. So is it weird? Does it feel weird that you're sort of the expert on why experts fail? I guess I've gotten accustomed to it because I've been doing it for about 35 years now. I started out just after I got tenure at Berkeley in 1984. I've been tracking the accuracy of experts' predictions pretty much continuously since then. So it does not exist. But the basic idea of keeping track of how accurate people are is a really good idea. And insofar as we all kept track asking of how well calibrated we are,

I think the research suggests we would actually get a little bit better at it. So you've actually done what I've said for years that I've wanted to do, which is create some database of forecasts and actually go back and look at who's right and who's wrong. How did you get the thought that that would be worth doing initially? And what is such a database look like? Because most forecasts are not particularly binary. people might say, oh, I, 50% chance or 75 or 90% chance. So usually things aren't just this will happen or this won't happen.

And often a forecast can be wrong, but the methodology turned out to be right or maybe someone predicted 95% chance. And that was still the right framework and the 5% odds did hit. So how do you even go about constructing a database of forecast and measuring what turned out to be right or wrong? It takes a long time. The key thing to note is you're 100% right that there are only two conditions under which you can definitively say a particular forecast was right or wrong, and that is that

the forecast it was rash enough to say 100% chance and it didn't happen or a 0% chance and it did happen, in which case you know conclusively that that specific forecast was wrong. But a forecaster says, you know, there's a 70% chance, like Nate Silver said, there's a 70% chance of Hillary Clinton winning the election in November 2016 was made silver wrong, or do we happen to inhabit a world that was 30% likely on November 1, 2016? So the solution to the problem is statistical. You need to keep track of lots of forecasts over time. If we collect hundreds of your forecast over the course of a year,

you say there's a 70% chance of things happening, those things happen about 70% of the time. When you say there's a 90% chance, those things happen about 90% of the time, and so forth. If there's a close correspondence between your subjective probability estimates and the objective frequency with which events occur, you can be said to be reasonably well calibrated. And that's a very desirable feature of forecasters. So when you ran your analysis, your statistical analysis of all these expert forecasts, and I'm assuming some non-expert forecasts as well, what did you find? You find that people are not very well calibrated for a start. There's a lot of overconfidence. Many experts would say things are 80 or 90% likely or even 100% likely.

They would occur 60 or 70% of the time. So there were big gaps between subject of probabilities and object of probabilities. There's a lot of overconfidence. People were not qualifying their forecast appropriately. And that is one of the better replicated findings in my field. which is the logic of judgment. People are overconfident. Not everybody's overconfident.

There are a few souls who are even systematically underconfident, don't have enough confidence in their judgment. But if you had to bet on what kind of mistake people are making in any given moment, overconfidence would be the better bet. You know, I'm thinking about something, one of our colleagues, I'll give a shout out to him, Lorken-Rosh-Kelly. He works with us here at Bloomberg, and he's very fond of strategists at banks and other experts who love to give a 40% forecast on things.

And like, oh, we see a 40% forecast of this person winning the election or a 40% forecast of this country leaving the EU in the next five years. And it's like the perfect number because it's, you know, it's still, it feels unlikely. But it's close enough to 50% that if it happens, you know, You know, you could still say, oh, I told you was significant, but if it doesn't happen, you could say it was unlikely. And I'm curious in your findings and in your research, you know, we think of forecasts or predictions is the point is to be accurate. But it feels like a lot of the reason people give forecasts is just to be interesting, just to have their voice heard, just to get their clients to pick up the phone. And I'm curious how that plays into your analysis of forecasts when the whole purpose is maybe not even to get it right. It's just to provoke a thought or to have your name out in the news.

Of observation, it's a very delicate dance that people play. On the one hand, you want to say things that sound interesting, so people don't roll their eyes and think that this is boring and a useless conversation. On the other hand, you don't want to say things that are so interesting that they could prove to be wrong later. So you're, so 40% seems to be sort of in the sweet spot zone. Or if you were to use language, you say, well, I think there's a distinct possibility that Putin's next move is going to be on Belarus or on Estonia. Distinct possibility is wonderful in exactly the same way.

The 40% is pretty good. I mean, if it happens, I say, hey, I told you, distinct possibility. And if it doesn't happen, I said, I merely said it was possible. So you're covering yourself very nicely. It is as though the art of punditry is the art of appearing to go out on a limb without actually going out on a limb. I mean, Joe referred to forecasts as non-binary, but I feel like instinctively a lot of people want to know whether something will or will not happen. And yet we have all these forecasting calls that are sort of, you know, 30% chance, 40% chance. Are probabilities a cop-out in that

case? Is it something that people hide under? Well, probabilities are not a cop-out if you're participating in forecasting tournaments in which we can systematically track how often you're 40% things happen. And if things you say 40% likely happen 40% of the time, you're pretty well calibrated. And it's not simply being used as a cop out. So, okay, so you've built this database and you've been tracking forecasters ability for years. And you mentioned that there are forecasting tournaments and we can really track this stuff and that we can track how well calibrated forecasters are not on any sort of individual prediction, but by whether over time their predictions of 70% likelihood events happen seven out of 10 times and so forth.

What are some of the interesting patterns you've discovered besides that people tend to be overconfident? What kind of people, what kind of approaches tend to distinguish the better forecasters from the worst ones? Because ultimately, I think that's sort of the point of your research. There are two classic biases that we have found over their minds, and the other is that people are too slow to change them. And it's the combination of those two things that... So I'm curious, beyond sort of individual characteristics that make people a good or bad forecaster, did you notice any discrepancy in the type of forecasts being made? Like, for instance, did political forecasting tend to be more or less accurate than something like economic

or financial forecasting? Oh, it really depends on the time frame and the, and the, and the types of questions you're asking. I think both economic and political forecasting can be pretty hazardous to your reputation. I think what we noticed more than anything was that the types of forecasters who tended to be better did tend to be a little bit more boring. They were more likely to say on the one hand, on the other hand, they were engaging in more explicit balancing and say, well, there's this causal force and there's that causal force and you have to balance them against each other. So the types of forecasting talk that make forecasters appear, you to the media tend to be, to also tend to make forecasters less accurate.

So a forecast is going to be more appealing to the media, it would seem, if they can come up with a compelling soundbite. And they say something like, well, you know, I think this Saudi regime is going to collapse within the next 12 to 24 months. That's a very dramatic forecast. It would have a lot of consequences for the Middle East and for world politics. A forecast who says, well, you know, people have been predicting the major regime change in Saudi Arabia off and on.

for the last 40 years, it hasn't happened yet. The base rate prediction is not very likely. There are some reasons for some concern, but you can steal your eyes, start to glaze over, listening to the more accurate forecasters tend to bore people. Yeah, so I'm just thinking, so at the time that we're recording this episode, just in the last day, and by the time you're hearing this, this would be old news, but in the last day Deutsche Bank, for example, announced a major restructuring of its bank. And I'm just thinking about how like the imperative for the news media is immediately to find people who will come on this morning and say something about whether this restructuring of the bank is likely to be enough.

Did it go far enough? Will it restore the bank to robust profitability and so forth? And it sounds like that imperative is almost exactly the opposite of what's likely to make a good forecaster. And anyone who already has their mind made up, it already has a strong view on the efficacy of the plan, at least going by your heuristic, that the people, that good forecasting is not correlated with quick judgment or quick decision making, which seems like we're kind of like highlighting, most likely highlighting some of the worst. people we could be highlighting. Well, it really, you have to make a decision about what kind of business you're in. If you're in the accuracy business, you're going to look for the kinds of forecasters we've been looking for in the work we've been doing with the intelligence community and elsewhere.

And these are going to be forecasters who are not very entertaining. If you're in the entertainment business, you're going to be looking for people who are entertaining. There's a separation in Bloomberg and probably most in other, you know, sophisticated media companies between analytics and the front end, right? Absolutely. The news doesn't stop on the weekends. Context changes constantly. And now Bloomberg is the place to stay on top of it all.

Hi, I'm David Gurra. Join us every Saturday and Sunday for the new Bloomberg this weekend. I'm Christina Rafini. We'll bring you the latest headlines, in-depth analysis, and big interviews. All the stories that hit home on your days off. And I'm Lisa Mateo. Watch and listen to Bloomberg this weekend for thoughtful, enlightening conversations about business, lifestyle, people, and culture. On Saturday mornings, we put the past weeks of events.

into contexts, examining what happened in the markets and the world. That on Sundays we speak with journalists, columnists, and key political figures to prepare you for the week ahead. Join us as soon as you wake up and bring us with you wherever your weekend plans take you. Watch us on Bloomberg Television. Listen on Bloomberg Radio, stream the show live on the Bloomberg business app, or listen to the podcast. That's Bloomberg this weekend. Saturdays and Sundays starting at 7 a.m. Eastern. Make us part of your weekend routine on Bloomberg Television, Radio.

and wherever you get your podcasts. So what about experience? Like, to what degree can, if you're an expert, presumably you have, you know, probably decades of experience and, you know, you've been studying a particular subject matter for a long time. You've noticed patterns or you can reach for historical analogies to describe a current situation and extrapolate from that. Does experience help offset the problems of overconfidence at all?

You do with the experience. People have different styles of thinking. And some people with experience become extremely skilled at creating very compelling, very articulate, justifications why they must be right. So experience can actually solidify dogmatism for people with that cognitive style. And for other people, experience mellows them, and they become more attuned to the limitations of their prior worldviews. And the implications may become better calibrated. But it's not a one trajectory.

People mature in different ways. Talk to us about how you train people to get better. So obviously, like, let's start with the assumption that there are some people that aren't just looking for media soundbites. Maybe to use your example, they're in the intelligence community, and they really want to make better forecasts about how things will happen in the future. be better at predicting, say, Vladimir Putin's next move. What is the, how do you start and how do you, what's the general approach to becoming better at that? Well, it is again not going to be all that exciting. And it bears a strong resemblance to what Danny Connman proposes in his bestselling book, thinking fast and slow. It is start with the base rates. And if you look at the base rates,

you'll see something quite interesting. And that is, people frequently claim. To go back to the beginning of our conversation, people frequently claim that they're at an inflection point in history. If you look at how many inflection points there have been, this is a very long list. The vast majority of claims about inflection points have been false positives. So you would naturally, I think, be wary of claims about inflection points. Another claim you'd be wary of is military coups or revolutions. They're relatively rare events. So someone who's making a dramatic claim, but it's going to be a regime change in a

country within a particular time frame, the likelihood of their being right is pretty low. Phil, Joe mentioned this in the intro, but what role do you think accountability plays in the sort of forecasting industry? Because it feels to me like given the volume of media that's out there right now, you know, either social media or traditional forms of media, it feels like you have a lot of people who will make, you know, say a hundred predictions and maybe one or two of those are right and then they get trumpeted for those right calls or, you know, they law themselves for those correct calls and people sort of forget about the other 98 or 99 calls that were wrong and no one bothers to go back and check on them because there's just so much forecasting and so

much information out there in the world. So how can we develop accountability for forecasting? My markets are and forecasting tournaments are excellent ways of allowing people to track their accuracy on judgment calls for which there aren't ready financial market equivalents. How do these – I've never – I'm not familiar with these. So how does they forecasting tournament work? You ask the probabilities on events that are specified by well-defined questions, such as whether Putin is going to be the president of Russia after 2024. for. And those probability, in that case, it would be a probability that would be yes or no. There are lots of ways of doing it, but they all boil down to the core idea, which is keeping score. Sticking with the intelligence community framework, how does the role of groupthink play into this and

avoiding group think? Because if we think back to what are considered a lot of the intelligence disasters over the last several years. The idea of some idea takes hold and no one feels comfortable yelling stop and suddenly everyone congeals on the same idea. Is that something that in your work you focus on, sort of like these cascades, where someone puts forth an idea and everyone feels compelled to fall in line or there's extreme pressure to voice a concern or voice skepticism on things? And are there ways or strategies that aspiring for?

forecasters can use to eliminate or reduce that bias? Group think is a big problem. In forecasting tournaments, we typically have people make judgments independently of each other. It doesn't mean that all team or group decisions are going to be bad ones, but it does mean that forecasters need to value accuracy above all. If your primary goal is pleasing your boss and you have an opinionated boss, it's going to be very hard for you to offer that boss an opinion with a probability judgment that points to a policy different from what the boss prefers.

So it's another version of the question, what business are you in? Are you in the entertainment business? Are you in the pleasing your boss business? Or you in the accuracy business? Accuracy business is often not the first business people are in. It's often not even the second business people are in. People are trying to have successful careers. They're trying to avoid embarrassment.

There's a lot of other things people are doing, aside from accuracy. So forecasting tournaments create a really weird social environment. They create a world which only one thing matters, and that is minimizing the gaps between your probability, judgments, and reality. And that's it. That's the sole objective. I'm so trying to understand. So as you point out, like, forecasting tournaments are very weird, because in the real world, that's not how predictions are made and people are aware of what other people are thinking and talking about. So when you consult and when you talk with people, How do you foster a culture?

And I guess this is really what matters to the end consumers of your research is how do you foster a culture where more people feel that they're in the accuracy business? Comes from the top. People are typically look up for their normative cues about what's appropriate. So if you have a boss who's open to being wrong, that helps a lot. So, Phil, I know the majority of your work has to do with statistical analysis. of probabilities of forecasts, but I'm curious, could you give us a sort of case study that you've come across of a forecast that has gone very, very wrong and that sort of brings together some of the themes or lessons that you've been talking about?

When I was doing this work, and everybody thought we were at a major inflection point, virtually everybody, and they were more or less right about it. I mean, most inflection point calls are wrong, but they were right that the Soviet Union, And at the time I was starting off on this work was at an inflection point. People didn't have any idea where the Soviet Union was going to go. The conservatives thought that the Soviet Union was incapable of reforming itself. The liberals thought that Reagan was driving the Soviet Union into neo-Stalinist retrenchment to become more aggressive.

Yet Gorbachev came along in March of 1985. He became the General Party Secretary, and he proceeded with Glasnos and Perestroika to liberalize the Soviet Union in ways that were really astonished. Now, after the fact, the conservative said, hey, we forced them to do it, but they didn't really expect the Soviet Union as capable of reforming itself beforehand. And the liberal said, well, we knew it all along because the Soviet economy was crumbling and the Soviets needed to do this. And Reagan had no role in it at all. The paradox was that nobody was really very close at all to predicting what would happen. But everybody after the fact had a confident explanation for what would happen. Is there anything that people could have done better prior to the events unfolding that could have made their forecast better? Or is it the kind of thing that it was so novel and, you know, it was kind of so unexpected that this has just be a really hard thing to forecast in any meaningful sense? It was a hard thing to forecast, but there were clues that Gorbachev was different. And even a conservative like Margaret Thatcher was signaling that based on a.

early meetings with Gorbachev. I think that the key factor here is, you know, how fast are you willing to change your mind in response to incoming evidence? So after Gorbachev became General Secretary in early 1985, there were lots of little bits of news that suggested this was going to be a different style of leadership, and maybe not just a different style of leadership, but different substance, different substantive policies would be pursued. And it would be, it was your willingness, I think,

to make small, rapid adjustments in response to the news. It wasn't that, and there was any one big item that absolutely turned the case, but there were lots of little bits of news that accreted over time. And I think that's one of the defining features of the best forecasters is that they're more granular. They make distinctions among more degrees of maybe than normal people do. An old joke in my field is people, there's three degrees of uncertainty. you know, yes, no, and maybe. The best forecasters are people who just know the difference between a 40-60 bet and a 60-40 bet,

or even a 55-45-45-bed, so it's somewhat analogous. There would be no doubt, for example, if I said, you know, good poker players could do that. And you'd say, well, sure, they must be able to do that in a repeated play game. We get rapid quantitative feedback. But you said, well, good geopolitical and economic forecasters also do that. And you'd say, well, can that really be possible? And the answer is yes, it can. So do you have a favorite forecaster?

You know, it's like asking who are my favorite children, right? I know. I wouldn't pick any particular person as a favorite forecaster. But I think there are lots of very admirable people out there. Well, in the beginning, you mentioned Nate Silver and his prediction, or not prediction, but his assessment, maybe a better way to put it, that Hillary had a 70% chance of winning the election. He's someone who has a very sort of clear understanding of probabilities, and he puts, you know, he has a difference between 85 and 70 and 50 and 20 and probably fits very well into your database. By and large, do his 70% forecasts prove to be right about 7 out of 10 times? I have not analyzed Nate's data, and he analyzes it himself, and he has reported how well-calmed.

calibrated, the forecast on the 538 site tend to be in both sports and political forecasts. And I think they're pretty good in the sense of being well calibrated. I don't know the data on resolution. They're scoring pretty well. There are two key facets of being a good forecaster when you're doing subjective probability scoring. One of them is what I mentioned earlier, calibration. So when you say 70% likely to 70% things happen 70% of the time,

The other is resolution. Now, so there's a sneaky and lazy way of being well calibrated. So if you're a weather forecaster in Seattle and it rains 60% of the time and you say, hey, I'm just going to say there's 60% likelihood of rain every day. And you know what? I'm going to be well calibrated because rain will happen 60% of the time. You would be well calibrated, but you'd be very uninteresting. So there's another property you need to ask a forecasters beyond calibration. And that is you need to ask them, are you good at assigning much higher probabilities to things that happen than to things that don't?

Are you good at being justifiably decisive? So you want forecasters who are two things. They're appropriately humble, which means well-calibrated, but they're also justifiably decisive. They say interesting, decisive things when they have a warrant. It's a combination of those two things that makes someone a so-called super forecaster in our work. But I think that the Nate Silver Group at 538 is doing the right things. And I think a number of other organizations are starting to do the right. So on that note, how should forecasters deal with tail risk events?

Because, of course, as you put it, you know, you could just sort of do an average of probabilities and you might look very smart and very well calibrated. But at some point, there is a chance that a big unexpected event is going to come out of nowhere. and sort of shift the entire regime of statistics in some way. How should forecasters deal with those kind of unforeseen risks? In terms of shades of gray, it's not that things are, there are some things are foreseeable and other things are unforeseeable, that there are black swans and then there are white swans. There are swans of varying degrees of grayness.

And the best forecasters, I think, recognize that there is a continuum and that tail risk is a problem. and you have to judge how important it is. You'll never miss a war or you'll never miss a disaster if you always predict disaster. But the cost you're paying in false positives is ridiculous. The question is how high a price are you willing to pay from making lots of false positive predictions about the Dow is going to fall below 2000 in the next six months, that sort of thing. And by futures contracts based on that belief, skin in the game, as it were.

Those are judgment calls, and you never escape making probability judgments, even though the probabilities may be extremely small. It's very difficult to say whether someone is well calibrated in distinguishing between events that are 1,000 likely and 1 in a trillion likely, right? But there's a huge difference between 1 in 10,000 and 1 in a trillion, right? So, Phil, you mentioned people predicting war or natural disasters. it does feel sometimes like the people who forecast negative events, you know, recession is coming, war is coming, Donald Trump is upending the global order, those sorts of things, that they seem to make more waves or more inroads than people who predict either a status quo or positive trends. Do you think people like to hear dire forecasts more

than extremely optimistic forecasts. But they seem to find them more interesting and they pay more attention to them. Can the best forecasters or the super forecasters, as you call them, can they always articulate their approach or do some people just have some sort of deeper intuitive sense? And I'm thinking about you use the poker analogy. And one of the things about poker is that there's different ways to play it. So some people are extremely mathematical in their forecast. they calculate everything. Others seem to much more clearly operate on feel and they just have a

good feel for whatever reason. That turns out to be a successful strategy for them too. Is there a range in the approaches that people use or some people can very methodically lay out their approach like a Nate Silver where they build all these models versus a more intuitive field-based approach that maybe can't be written down as well? Well, Malcolm Gladroll wrote a book. I don't know if you're a member it called Blink. And there are some people in my field who wrote a much less well-known book that was a rejoinder called Think. I think I remember that. So you've got a dualism here between people who endorse Blink and people who endorse Think. I lean toward the think end. I'm not precluding the possibility that when you're dealing with events that occur over and over again

and there's lots of repeated play and there's lots of opportunity to build up deep experience and automated cognitive processing, that some people can become intuitively very good at it. And they may even be doing rather complex calculations in their head very rapidly. So it's not that the intuition is ESP here. It could be that some of the best forecasters have simply overlearned the probability calculation heuristics to the point where they unfold virtually automatically. Just as like a master pianist, right? It doesn't have to think about every key and just, you know, a great time.

tennis player doesn't have to think about where his arm is. So, Phil, I feel like I have to ask, will you give us a forecast? Okay, I'll give you a forecast. Even though I'm not a forecaster, just for you, just for you. I will give you a forecast. And that is, I don't think the forecasting practices are going to change very fast. I think that the majority of people, because they don't want to offend people in power, They don't want to offend clients because they want to be entertaining in media performances or elsewhere.

And that accuracy will continue to be a very secondary goal. But that gradually, over the next 10, 20, 30 years of forecasting tournaments and prediction markets will become an increasingly common way for people to resolve certain categories of disagreements. Well, hopefully your appearance on this podcast will help marginally change the trajectory of the forecasting. over the next several decades. But really appreciate you coming out, a fascinating, fascinating discussion. Thanks so much, Phil. Okay. Take care. So, Joe, I'm just going to point out that Phil's forecast did not contain a probability. Oh, huh. Even though he did a pretty uncontroversial forecast, even he didn't put a precise number on it. But I really, I really like that conversation. I I think it's just a great topic because we see this all the time and not just in the fact that people come on and give forecasts and never are really held accountable for them.

But just in what is the purpose of forecast? And how many times we all encounter people who give forecasts but whose job is clearly not accuracy. And I'm thinking a lot of there, I think there are a lot of asset managers like this who use stories as a way of gathering clients. And maybe they tell a bearish story or a conspirator. story about the Fed or whatever, but that story really has nothing to do with how they then go on and invest. Right. Certainly in the investment industry, there are plenty of people whose positions just don't add up to the world viewpoint that they tend to express, which, you know, again, I would think of a lot of the really bearish people out there who for the past eight years

have been saying, you know, move into cash, buy gold, the end is coming. And yet, clearly they are in the business of putting cash to work in some. way or another. So, you know, it seems a little bit out of sync. You know, we didn't, you know, we only talked a little bit, and I'm sure if we read his work and studied it, there'd be more depth. But in terms of becoming a better forecaster, this idea of just sort of looking at the default, and his point is, well, coups are pretty rare and wars are pretty rare. And regime change is pretty rare. And sort of start. Starting there, I mean, another thing in stock markets is like bare markets are really rare and stock market crashes are really rare.

And so this sort of idea of like, as you put it in the beginning and as we were talking about, we're at a moment in which lots of people are talking about inflection points. And it feels like things might turn. But just starting from this assumption that we're probably not an inflection point and that most of these turns that we think are, oh, they're bound to happen now, probably won't happen now. It's like it's an interesting starting point. I mean, obviously that's not enough because sometimes the turns do happen. Right. But starting from that assumption, it's like that you're probably the right move is to fade the expectations of a turn. See, it feels like an interesting toehold to get into it to then move from there.

Right. But see, I think this is where human psychology and preferences come in because no one is going to remember or reward you for continuously calling the status quo. correctly. But they will probably remember you if you did call the big regime shift or the big bear market. And that's why so many people remember quite a few names that predicted the 2008 financial crisis. But no one remembers as clearly, you know, people who called the gigantic rally that we had after 2009. The only, you know, the only one, and like an exception to someone like Warren Buffett who just buys stocks and doesn't do anything fancy and has done very well. So there's a, no, seriously, like there's a few people like that. I feel like Warren Buffett is always the

exception. Yeah, that's true. Like you can't just point to Buffett, but that's true. All right. Well, on that note, this has been another episode of the Odd Lots podcast. I'm Tracy Allaway. You can follow me on Twitter at Tracy Allaway. And I'm Joe Wisenthall. You can follow me on Twitter at the stalwart. You should follow our guest on Twitter, Philip Tetlock. He's at P. Tetlock. And you should follow our producer on Twitter, Laura Carlson. She's at Laura M. Carlson, as well as the Bloomberg head of podcasts, Francesca Levy, at Francesca Today. And be sure to check out all of Bloomberg's podcasts on Twitter under the handle at podcasts. Thanks for listening. I'm Francie Lacqua, an award-winning journalist. And I've got a new podcast. Leaders with

Francine Lacroix from Bloomberg podcasts. I've interviewed everyone from heads of state to fashion icons about the news of the moment. But I've always been curious who are these people as leaders. I don't think there's one right way to be a leader. Make decisions. A poor decision is always better than no decision. Listen to new episodes every other Monday. Follow leaders with Francine Lacois wherever you get your podcasts. What separates good leaders from transformational?

ones. I'm Jessica Chen, and in season two of Leading By Example, we'll sit down with executives like Grace Chen of Bertie Gray to find out. It's important to understand where you spike, but also really acknowledge where you don't and find people who can fill those gaps. Listen to Leading By Example, executives making an impact on the IHeart Radio app, Apple Podcast, or wherever you get your podcasts.