Odd Lots - Bridgewater's Greg Jensen on AI, Inflation and What Markets Are Getting Wrong

Episode Date: July 3, 2023

Every industry is trying to figure out just how AI or Large Language Models can be used to do business. But Bridgewater Associates, the world's largest hedge fund, has already been at it for a long time. For years, it has explored AI and adjacent technologies in order to analyze data, test theories, develop novel investment strategies and help its employees make better decisions. But how does it actually use the tech in practice? And what's next going forward? On this episode, we speak with co-CIO Greg Jensen about both the possibilities and limitations of these advances. We also discuss markets and macro, and why he believes that investors are still too optimistic about the Federal Reserve's ability to get inflation back to target. See omnystudio.com/listener for privacy information.

Thanks for listening to Oddlots. Follow the show on Amazon Music for more future episodes or just ask Alexa, play the podcast, OddLots on Amazon Music. Hello and welcome to another episode of the Odd Lots podcast. I'm Tracy Alloway. And I'm Joe Wisenthall. Joe, I think it's fair to say. There is a lot of excitement about investing in AI. There is also a lot of excitement about using AI to invest. Yes. I mean, it's what I think. there's like a new like chat ETF I saw an ad for and there's like oh we're getting you know I think I saw another like project who is like we're going to have chat GPT pick the stocks for us and I you know I get it it's kind of exciting and maybe there's some new way of like these super advanced digital

brains that can beat the market etc but like I don't totally get it well I also feel like there's a tendency nowadays for people to talk about artificial intelligence in a sort of abstract manner. You hear people bring up AI almost as a synonym for just software at this point. I think you pointed out recently that the Kroger CEO mentioned AI like eight times on the earnings call. So a supermarket chain, right? Yeah. And you know, it's like machine learning, tech, algebra, algorithms, it's all existed for a long time quantitative investing. But it feels like because of the excitement around a few specific consumer-facing products that have been unveiled over the last six months. And the way they've captured people's attention, people like, you know, suddenly there's a lot of interest in

like, how are companies using this tech to do something? Yeah. Well, I'm glad you mentioned that because today we really do have the perfect guest. This is someone we've actually spoken to about AI before last year. In fact, someone who is at a firm that has a lot of experience, using machine learning and AI of different types, and we're going to get into the differences between all those technologies. I'm very pleased to say we are going to be speaking once again with Greg Jensen, the co-chief investment officer at Bridgewater Associates. So Greg, thank you so much for coming back on all thoughts. Yeah, it's great to be here. The exciting topic. Yeah, so I actually revisited our conversation from last year. I think it was in May of 2022. And you said

two things that stuck out in retrospect. So number one, you said that markets had further to fall, which turned out to be correct. And two, you brought up artificial intelligence as a major point of interest for Bridgewater. And this was all before chat GPT really became a thing. And everyone started talking about AI at every single conference and earnings call and so on. So I guess just to begin with, maybe you could lay the scene and going back to Joe's point in the intro, we are used to hearing these terms. So Bridgewater does machine learning and systematic strategies and quantitative trading strategies and AI and things like that. What's the difference between all of these things and how do they relate to each other at a firm like Bridgewater? Yeah, great

question. So I think to answer that, let me take a step back for a second and give you a little bit of my background because it all kind of comes together in a way to connect these different pieces. So, you know, even as a kid or whatever, I was certainly interested in kind of translating and predicting things using some mix of my thinking and technology. So I can think back to in the late 80s using stratomatic baseball cards or you know what they are, but programming them into computers to try to calculate the way to create the best baseball line up and use that in in fantasy baseball type situations and similar things with poker and whatever and trying to learn how to kind of use technology to combine with human intuition

to get at what was different ways to create edges. And then in college when I heard about Bridgewater, Bridgewater was a tiny place at the time, but the basic idea that there was a place where we were trying to understand the world, trying to predict what was next, but doing that. that by taking human intuition and translating that into algorithms to predict what was next, kind of mixed two things that I loved. I love to try to understand the world. And I love the idea of having to discipline to write down what you believed in stress test,

what you believed and utilize that, right? So if you go back, and this is now in the 90s, kind of where artificial intelligence was at the time, most of the focus was still on expert systems, was still on the notion that you could take human intuition. you could translate that into algorithms. And if you did enough of that, if you kept kind of representing things in symbolic algorithms, that you could build enough human knowledge to get kind of a superpowered human.

And Bridgewater was a rare example of where that worked, where given the focus of trying to predict what was next in markets, given the incredible investment that we made, indoor creating the technology to take human intuition and translate that into algorithms and stress test that. It's an incredibly successful expert system, essentially, that was built over the years. I'd say probably the most profitable expert system out there. And that's really what Bridgewater has been about, which is building this great technology

to help us take human intuition out of the brain, get it into technology where it's both then readable by, let's say, investment experts, but also runs on a technology basis. And that's kind of where algorithms, let's say, the mix. of algorithms and human intuition. It was really important. If you go through the history of our competitors, they're littered by people that tried to do something more statistical, meaning that they would take the data, run regressions,

and then after regressions, let's say basic machine learning techniques, to predict the future. And the problem that always had is that there wasn't enough data. The truth is that market data isn't like the data in the physical world, in the sense that, A, you only have one run through human history, You don't have very many cycles, even cycles that debt cycles could take 70 years to play out. Economic cycles tend to play it around seven years. There's just not enough data to represent the world.

And secondly, that the game changes as participants learned. So the existence of algorithms as an example changed the nature of markets such that the history that preceded it was less and less relevant to the world you're living in. So those are big problems with, let's say, a more pure statistical technique to market. So you had to get to a world where statistical techniques or machine learning could substitute for human intuition. And that's really where kind of the exciting leaps are now, that you're getting closer. It's not totally there, but you're much closer than you've ever been, where large language models actually allow a path to something that at least mimics. intuition, if not is human intuition, and that you can then combine that with other techniques, and suddenly you have a much more powerful set of tools that can deal, at least take a big leap

forward on dealing with the problem of very small datasets and the fact that the world changes as people learn in a way that up until the big breakthroughs in large language models, I think we're much further away. So that's a huge change in the limits of ways that statistical machine learning could affect something with small amounts of data, something where the future varies from the past, all of those problems. We're closer to having at least ways to take on more and more of what humans have done at Bridgewater and what humans generally do in investment management firms. And that's a huge leap forward that's going on now.

I have one very short quick question. I realize just now that not long after we talked to last, year last spring, like a month later, you won your first World Series of Poker bracelet. So congratulations on that. I always say that because you mentioned poker. Did you play the World Series this year? I'm heading out, actually, after this. Because I know there are some, okay.

Congrats. Congrats. Yeah, good luck. And good luck. Yeah, and it kind of connects to this because I don't get to play very much poker, but I really studied what machines were learning about poker. So much has been learned in the last five years, ten years. And one of the, you know, basically trying to translate that into intuitions that I could use,

you know, that basically can't actually replicate a computer play spoke very complex way, but you can pull the concepts out, right? And this actually mirrors to what part of what we're doing at Bridgewater, which is that as you get to computer generated theories, that if you can pull the concepts out of these complex, algorithms, you know, you can make more of an assessment, a human assessment of whether they make sense and what the problems might be. And that's really a big deal. So there's actually a link between what I'm doing in poker imperfectly for sure and many of the concepts that we're trying

to apply at Bridgewater. And like you said, just we had talked kind of before the LLMs had really hit the public scene. But I mean, just to give you a little bit of background for me, If you go back to 2012, first off, we brought Dave Ferrucci, who had run the Watson project at IBM that had beat Jeopardy into Bridgewater. And that was a time when I was trying to experiment with, okay, what can we do with more machine learning techniques? And Dave was trying to take what he had done to win at Jeopardy, but actually put in more of a reasoning engine because while what happened on Jeopardy was impressive, it was pure data. It had no idea why it was doing what it was doing. and therefore really a lot of the path with Watson or whatever was going to be very hard to move forward with because at its end it was just statistical and it didn't really have any

reasoning capability. So Dave came to Bridgewater and later partnered with Bridgewater to roll out a company elemental cognition that's focused on using large language models, et cetera, but overlaying a reasoning engine that essentially helps with things like hallucin hallucinations that large language models have and focus on what is human reasoning and how does it work and how does that limit views that are unlikely to be true. So that's one thing. And then in 2016 or 17, I was introduced to Open AI and actually as they transitioned from a charity to a company, I was one in that first round and met a lot of the people and looked hard at their vision

using scale and technical scale to build general intelligence and build reasoning. So I both worked with Dave Ferruci and sort of understood many of the people at OpenAI at the time and moving forward with those things. And then I was literally the first check for Anthropic, another large language model, kind of people that had been at OpenAI. And so I've been passionate about this, trying to take different paths to how will we build a reasoning engine, to overlay on statistical things and a couple different approaches that were being applied at the time and obviously different, they panned out to a different degree, but many things are coming together

now to say, okay, you can actually, in a way, at a pace and a speed humans can never do, you could replicate human reasoning. And that's a huge deal. And if you can really break through that, you can start to apply it in so many ways in our industry, I believe, and obviously way beyond our industry. You can get the news whenever you want it with Bloomberg News Now. I'm Amy Morris. And I'm Karen Moscow here to tell you about our new on-demand news report delivered right to your podcast feed.

Bloomberg News Now is a short five-minute audio report on the day's top stories. Episodes are published throughout the day with the latest information and data to keep you informed. Yes, there are other products like this from a variety of news organizations. But they usually rerun their radio newscasts throughout the day. That's not what we do. We create customized episodes that can only be heard on Bloomberg NewsNow. And we don't wait an hour to publish breaking news. When news breaks, we'll have an episode up in your podcast feed within minutes.

So you're always getting the latest stories and developments. Get the reporting and the context from Bloomberg's 3,000 journalists and analysts we're all over the world. Listen to the latest from Bloomberg News Now on Apple, Spotify or anywhere you listen. You talked about earlier generations trying to embed human knowledge. And I'm wondering, you know, if an analogy is like, I remember when Deep Blue came out. And they had all the grandmasters sort of work with IBM to like come up with this great

computer program that was basically as good or eventually better than Gary Kasparov. But then the next generation of chess computers didn't even have the grandmasters playing. It just learned the game from ground up and crushed those. crush those previous generation. Is that sort of what we're talking about here with the transition from earlier engines to the new sort of LLM focus, which is like the sort of reasoning comes out of the computer rather than having to be taught directly by the experts? Yeah, I think something like that is happening, right?

You got that in chess because once you had the ability, you had enough data and enough compute, you were able to do enough sampling. that the pure, that you got to the point where the pure data process with, you know, good human intuition on how to build that data process, but, but a data process was able to beat that those rules-based things. Now, chess, unlike markets, is, you know, a little bit more static in the sense that while people, while there are adversaries and the adversaries, they'll try to learn your weaknesses, it's more static in the rules of the game are steady and those types of things so that that sampling could work. Although it's interesting. I love the like, because it

is an analogy to some of the problems that pop up and will pop up. If you take Alpha Go, right, on the Go game, Go got also after chess, obviously, but Google was able to create this game that was beating the pros and radically beating the pros, killing everybody and getting better and better and better, although, you know, I don't know how up to date you are, but then there was this loophole in it where that's that another person who was a mediocre Go player and a, but a. computer scientists who thought there might be a hole in this super AI, used a little program to find the hole. And what it illustrated was the AI had no idea how to play the game. Because what a

six-year-old wouldn't, the mistake the AI was prone to was a mistake a six-year-old playing Joe would never make, where if you made a large enough encircling, if you now go works, but if you encircle the other guy's pieces, right, you eliminate them all. And something that never work in a human game is make a really big circle. And because it never came up in human games and because when they perturbed human games and started playing computer against computer, they basically started with a seed of human games, they never perturbed it enough to try this out, to try a massive circle. And a human would never let the massive circle have it. It's so easy to defend against. But actually the best Go algorithm in the world allowed it to happen. Right. And now a

a mediocre go player with a little bit of AI found a way to beat this incredible Go game. Again, because the Go algorithm at that time had this tremendous amount of data, but the things that weren't in this data, it wasn't aware of. And it wasn't in any deep sense understanding the principles of the game. So that's the type of data problem you can have, even with a massive amount of data, played millions and millions of games. But to play every possible go board, there's more possible go boards than there are atoms in the universe. So it was never going to calculate every possibility. And it never got to reasoning, right?

And therefore, that was a weakness, right? And on the other hand, had you mixed that, blend that, even with a basic reasonary, that a language model could come up with understanding the rules of go and being able to talk about it, there's an element of knowing those things that humans already know. That's possible with a blend of, let's say, a statistical technique like AlphaGo was using and a reasoner to prevent these types of mistakes. I like that story because it makes me think I have a chance against the super smart supercomputer.

Okay, that's kind of comforting. But I definitely want to ask you more about weaknesses in AI and large language models. But maybe before we do, you know, just sort of setting the groundwork once again, but when we see headlines like Bridgewater Restructures, will put more focus on AI. What does that mean exactly? What does it mean for a firm, an investment firm like Bridgewater, to build up resources in AI? And then secondly, could you walk us through a concrete example of how AI would be deployed in a particular trading strategy? I feel like the more concrete we can get with this, the more helpful it'll be. Yeah, great. So I think, As we restructured, one of the things that as we made the transition at Bridgewater,

you know, from Ray having the key ownership to ownership at a board level and that transition, we have done something we hadn't done in the past, which is essentially retain earnings in a very significant way, which allows us to invest in things that, you know, aren't going to be profitable right away, but are the big long-term bets that we're making. And certainly recognizing that there's a way to reinvent a lot of, lot of what we do using AI machine learning techniques to improve what we're doing to understand the world, accelerate that. And specifically, what we've done on the AIML side is we've set up this venture.

Essentially, there's 17 of us with me leading it. I'm still very much involved in core Bridgewater, but the 16 others are 100% dedicated to kind of reinventing Bridgewater in a way with machine learning. we're going to have a fund specifically run by machine learning techniques, which will take me into Tracy, what kind of strategies you can do. That's what we're working on right now in that lab and pressing the edges of what AI is capable of. Now, AI, like machine learning is capable of. Right now there are big problems, right?

A, you take large language models, and they have two types of problems. One thing is the basic problem is they are trained on what structure of language. So they usually return something that looks like good structural language. They don't always return accurate answers. So that's a problem. It hallucinates. It makes things up because it's more focused on the structure of what word or what concept it would come next than whether it's accurate in what concept comes next. Can I just say when I hear AI hallucinations, it becomes so science fiction for me.

It's very like robots dream of electric sheep kind of. It's just so surreal. Yeah, well, I mean, in this case, you can imagine what's happening, right? Because it's just what it's what it's trained on, right? So if you're just, if basically the basic concept is give it any stream of words and it'll predict based on having read everything that's ever been read, what comes next, right? And that if it's a little bit wrong in what comes next, it can misfire and give you something that sounds like something that could come next, but actually's wrong, you know?

And it's just what it's trained on, right? It's trained to predict the next word, slight errors in that create those types of issues. Now, the algorithm is pretty remarkable, particularly like we, as I said, I've been tracking in AI as an investor for a long time and looking at their technology for a long time. And, you know, up until there's GPT 1, 2, 3, and many versions of between, and then at GPT3, it started to have some use, GPT 1 and 2 were, you know, barely coherent. GPT3 was, you know, somewhat usable for certain tasks. Three and a half, which is what chat GPT is, you know, got to a certain level, like on Bridgewater's internal

tests, you suddenly got to the point where it was able to answer our investment associate tests at the level of a first year, IA, right around with chat GPT 3.5 and Anthropics, most recent quad. And then GPT4 was able to do significantly better. And these are, you're or at least what we thought were conceptual tests, significantly better than our average first-year investment associate that went through training. And similarly, it's able to take the LSAT and do well, et cetera. So it can be basically pretty smart. It is pretty smart on a wide variety of things with errors, but pretty smart on a wide variety of whether it's the MCAT or the L-SAT or Bridgewater's internal tests or whatever, a whole wide variety of things. This is a big deal that it can achieve all of those

kind of academic things. And yet it's still 80th percentile kind of thing on a lot of those things, which is remarkable to be 80th percentile on many, many different things. But at the same time, it's 80th percentile for a reason. There are flaws, meaning it's not a hundred percentile. And so that leads to like, you need to find a way to work through those flaws, right? And that's really where, you know, so if somebody's going to use large language models to pick stocks, I think that's hopeless.

That is a hopeless path. But if you use large language models to create some theories, which it can theorize about things, and you use other techniques to judge those theories, and you iterate between them to create sort of an artificial reasoner where language models are good at certainly generating theories, any theories that already exist in human knowledge and putting those things connect together, they're bad at determining whether they're true, but there are other ways to pair it with statistical models and other types of AI to combine those together.

And that's really what we're focused on, which is combining large language models that are bad at precision with statistical models that are good at being precise about the past, but terrible about the future. And combining those together, you start to build an ecosystem that can achieve, I believe can achieve the types of things that Bridgewater analysts combined with our stress testing

process and compounding understanding process at Bridgewater can do, but it can do it at so much more scale because all of a sudden, if you have an 80th percentile investment associate, technologically, you have millions of them at once. And if you have the ability to control their hallucinations and their errors by having a rigorous statistical backdrop, you could do a tremendous amount at a rapid rate. And that's really what we're doing in our lab and proving out that that process can work. I see. So is the idea that AI could possibly generate theses or ideas that can then be rigorously, you know,

statistically fact-checked by either the humans or, you know, existing algorithms and data sets? Is that the idea? Yeah. And then, yes, but the idea goes further. But yes, that's to start. Language models can do that. Statistical AI can then take theories and generate whether, like, those have at least been true in the past and what the flaws with them are and refine them, offer suggestions on how to do them differently, which then you could dialogue with. So then the other strength of language model has that humans are weaker at is now take a complex statistical model and talk about what it's doing.

and there's ways to train language models to do that, that then allow sort of a judgment to say, okay, now let's think about what's happening here and reason over what's happening. So you use the way we've modeled this kind of out is language models can come up with potential theories. Now, there's a limit to that, it's not the most creative thing in the world,

although it's theory at scale for sure. And then there's, and again, that's language models with good, you know, you got to tune your language models in a certain way, so it's not straight out of the box. But then you can use statistical things to control that. Then you can use language models again to take what's coming out of that statistical engine and talk about it with a human or other machine learning agents and we kind of report back on what you're finding and what that is and the types of theories that are out there that might

run contrary to what you believe, which can lead to more tests and other things. So that's the loop that I'm very excited about. And as I said, up until the thing that statistical AI was limited because it was focused on the data of markets, where language models, the good thing is it has a much better sense of something that a statistical model wouldn't really have. A statistical model of markets doesn't get the concept of greed. Language models pretty much understand the concept of greed. They've read everything that's ever been written about greed and fear and whatever. So now they can start to think about statistical results in the context of the human condition that generates those results. Big deal and really a radical difference.

Let me ask you one very simple question, and it might be one that speaks to an anxiety of listeners. If already GPT can perform at maybe the type of level that a high quality first year or second year associate or analyst at Bridgewater can do, does that mean fewer hires in the future? humans being hired at Bridgewater or does it mean the same number or more humans doing even more? Like, is it a replacement? Like, what does it mean for like the type of person that would have been 10 years ago, first year employee at Bridgewater? What I think people should expect at Bridgewater and just generally, but in a hurry is things are changing quick. That really requires people to be capable of playing whatever role is necessary. necessary in order to do that, right?

Like, if you go back at the clock at Bridgewater when I started, or just before that, right, we were, you know, we were using egg time, like we had rules on how to trade, but we were using egg timers and humans to like do these things. And over time, computers could do more and more of that. We kind of got to this point where it was, I'd say, kind of humans settled into the role of intuition and idea generation. And we use computers for memory and for constantly running those rules. rules, accuracy, et cetera.

That was a transition. It got to 50-50 technology and people. And now this is another leap, right? And it's definitely true that it's going to change the roles that investment associates play. Now exactly how, and you still need the foreseeable future, you're going to want people around that working on those things. There's edges that these techniques I'm describing certainly won't do

well for an extended period of time. And there's how to build the ecosystem of these machine learning agents, et cetera. And so what I've found is certainly the people in the lab, you want people who are curious about these new technologies. You want to utilize them. And that's, that's going to be really part of the future of work, I think. I think it's going to be very hard in any knowledge industry to not utilize these. And we're seeing this huge breakthrough in coding, right? That that is so democratizing in a sense that that you don't you really need to know what you want to code more than you need to know coding you know and that's a big breakthrough so a bunch of people that weren't as well trained or as capable in C++ or in Python or whatever can suddenly get

what they want so much faster so all of a sudden the skill sets are changing and they're changing in ways that I think are as surprised to many because it's actually a lot of the knowledge work a lot of the things where content creating and whatever that I think people thought would be later in computer replacement that are happening faster. So the main thing is, I'd say right now there's so much in flux that having flexible, the more you need flexible generalists who can have an eye towards this, an eye towards the goal and be able to utilize whatever tools are necessary to get there, that's really where I think, you know, you're seeing a fair amount of change quickly.

So you mentioned earlier that just the existence of machine learning can impact both the current environment and the future. So I think you said the future data points aren't going to look like the past data points simply because machine learning exists. Does that sort of reflexivity between machine learning slash AI and markets become more of an issue as AI and machine learning becomes more and more popular and more entrenched? Yeah, I think it's a big deal, right? And I think it's both something that's going to cause actions and something I'm super excited about.

Obviously, I'm excited about the power of this that I think there's ways to utilize it really well. And it'll also, there will be a lot of mistakes. Like you're saying, there will be funds that will, you know, use GPD to pick stocks and not really deeply understanding what's happening and why or what the weaknesses of that might be. there are already plenty of times where statistical, pure statistical, because there's not enough data, you're not building with those fundamental issues in mind.

Not that it was directly in markets, but in the housing market, what Zillow did is a great example, right? Zillow goes out and uses an AI technique that wasn't fit for purpose for what it's worth, but they use an AI technique to predict housing prices and then go into the market to start buying houses that they think are undervalued, right? And they have a couple problems. One is while they had a ton of housing data, it was over a relatively short period of time. So even though they had what looked like tons of data points because they have the price of every house and everywhere or whatever, there's still a macro cycle that affects everything that was underestimated in what they did.

And secondly, they underestimated what it would be like in theory versus in practice when it's actually an adversarial market. Every time they won an auction, there was something about that particular lot that the other people bidding on that lot knew that. that they didn't. And so it ended up, obviously, being a huge problem for Zillow. And they kind of had a big impact on the real estate market and then a big failure. And that's the kind of thing you're going to see over and over again. If because the basic problem that the data that you're looking at isn't necessarily the data you'll face in real world, you're not facing the adversarial problem when you're looking at that data the way they were, you're not a statistical technique that's very good its seasonality and trend following might not be very good at understanding macro cycles and so on.

So that was another case where, you know, Zillow is a case and I think we'll see it over and over again where the recognition that it's not as simple as taking machine learning out of the pack and applying it to this problem, even when there's a ton of data, right? Some of the places where there is a lot more machine learning going on, very short-term trading arguably is better for machine learning because there's a lot of data and you can learn faster over that data and there's there's some merit to that and and in terms of tangible places this is now years ago but where we started applying some of these techniques were in things like monitoring our transaction costs and looking for patterns and shorter term data because there's a lot

more data but on the other hand the data often it's like having the data of your heart rate for your whole life you can feel like wow this is a um yeah i've got every heartbeat for you know you know 49 years that seems like a lot of data, but it's totally irrelevant when you've art attack. So that even when there's lots of data, it can be misleading. And that those are the types of issues that will lead to these techniques having huge problems, which means it's not out of the box. AI is going to solve all these problems. You really, and this comes back to, you have to understand the tools, what they're good

at, what they're bad at, and put them together in a way that use what they're good at and protects them from what they're bad at. Now, nothing, no process we're coming up with, we'll do that perfectly. But the more and more you could do that, I think the more and more you could become, let's say, better than humans at that because humans have many of those fallibilities or versions of those fallibilities that these processes will have. And that's like, that'll be the question of how far we can, how far we can take that and how much human judgment is better than those things, which is stuff, you know,

we'll be experimenting with as we go along. I'm June Grasso, inviting you to join me for the Bloomberg Law podcast. Every weekday, we help you make sense of the legal stories that shape the nation and the world. Listen for complete analysis of the biggest court cases, the latest actions from Congress and regulators, and the legal moves driving the markets, from corporate law to constitutional law, and from state courts to the Supreme Court. At Bloomberg Law, we go beyond the day's headlines. We speak with top attorneys, judge.

scholars and policy experts to break down what the rulings really mean. We do this every weekday, then bring you the best conversations in our daily podcast. Search for Bloomberg Law on YouTube, Apple, Spotify, or anywhere else you listen. On the East Coast, listen as you start your day. And on the West Coast, catch up in the evening. That's the Bloomberg Law Podcast with me, June Grosso. Subscribe today wherever you get your podcasts. So, you know, one thing that, you know, your founder, Ray Dalio years ago, like, sort of, he wrote down a set of rules.

You've talked about this before. He wrote down a set of rules about how he understood the sort of the machine of the markets to work. And one of the issues with AI, and I think you're sort of been getting at this, is that, like, AI legibility and the understanding of, like, okay, you put it in, you pose a query to a large language model. it creates some output. You don't really know like what it did to get there. And so that's, you know, that's sort of different than dealing with a human analyst. You could say, well, did you think about that? Did you think about that? Can you talk a little bit more about like the sort of, I don't know if that's a weakness or how do you sort of get around that fact that like it's still difficult to query an AI model and say like, how did you arrive at X or Y conclusion?

Yeah, and I think that's really important. And that we, but also something that's more and more breakable. Because even with humans, one of the things, like one of the places where I think there are a lot of areas where Bridgewater has a strength, right? Bridgewater has a strength. And we never went from a statistical model. So we built data based on what we needed for reasoning. And as a result, we have a better, longer, cleaner database than I think anybody has. We've been thinking through this problem that you're referring, which is how do you actually get out what somebody means?

You'd be surprised how hard it is to truly get from a human. humans don't actually know why their synapses do what they do. They actually, like when you ask somebody to describe something, you get some partial version of what they're thinking if you took like an intuitive trader and you start peeling back all the reasons. That's very hard. We've been doing that for a long time and having expertise in doing it. And I would say that humans don't even know what they're doing often.

But there are ways to, you know, like you're saying, query and force questions and what about this and what about that that will help pull out human intuition. And what you find with machine learning algorithms, if you get good at this, and this is, you know, going back to 2016, 2017, has been critical to my work is there's a way that you can query machine learning algorithms like you query, like it's different, but the concept's the same as how you query humans to get at why they really believe what they believe. And as I was saying, I think there's actually elements of large language models interpreting what statistics. statistical AI is doing, that allows that process to accelerate.

And I think it's very critical. You really want to know because that's the way you find the flaws. If you go back to my Go example and you say, you can think about, if you can query a model and think about what it's done and what it hasn't done, then you can figure out what data is missing, right? And you need to set up adversarial techniques in order to keep querying an algorithm for what it's doing. And again, I think that's still an area of research, but a process that's moving along

quickly to basically get to the point where the standard is, even though a machine learning technique might be doing something very different than a human is, that it can still explain itself. And it might not perfectly explain itself just like humans don't perfectly explain themselves, but to a very high degree of confidence across a wide range of outcomes that you have a sense of what's going on is possible. And that's part of the design of what we're putting in, which is, well, how do you query? How do you give it more information, remove information, et cetera, see how it changes its mind to determine roughly what's going on? You know, you mentioned the data sets there.

And I guess it's a cliche nowadays to say, well, a model is only as good as the data that it's trained on. But it's a cliche because it's true. Do you use your own internal data for the large language models? Or where are you actually pulling in data from? And then secondly, like what type of data have you found so far is most useful? for these types of projects. Well, I think the things that are most interesting to us, A, we're trying to learn things that we don't already know,

so we're being careful about what kind of Bridgewater knowledge we put in here because it's not that helpful if we reinvent Bridgewater. Someone helpful, but it's not as helpful as, let's say, reinventing everything that we don't know about, that other people have thought about, et cetera. And so, point one, in the lab right now, at least we're focused on not making this two Bridgewater center, on purpose because it's in that way,

learn things that we don't already know. And if you just fed a bridgewater information, which we may well do, that could be a productivity enhancing thing, but you'll quickly, you know, produce something very similar to Bridgewater where what's been amazing so far

is we're producing good results by Bridgewater standards, but different, very, very different conclusions and different thoughts than what we have internally. So I think that's point one choice. Now on raw data and cleaning data and how you put together data, now we are benefiting from Bridgewater scale on that.

That's been a big, that's a big deal that over the years, again, precisely because we took human intuition and said, what data do we need to replicate that intuition? We have a unique database where if everybody else is pulling from, data stream, Bloomberg, et cetera, we put together the data we needed to feed our intuitions. Oftentimes that data didn't exist. We had to figure out the way to create it. And also we're big believers that you need to stress tests across a very long period of time. So we have much longer data histories.

Now, those things are certainly valuable in a context of small data, any quantity of data, any like the understanding the data and being able to therefore, for a given theory, find appropriate, unoptimized data, those are big deals. And that we are using. And, you know, that does allow us to move forward more. And on the large language models, you know, there's still a lot of work to be done. but you certainly can train through reinforcement learning to make sure that they're not making mistakes that you know about. And so there's ways to do that.

Now, we've been trying to avoid that for the reasons I was describing before. I avoid doing too much of that of injecting our own knowledge and use external sources to do that. But that's still part of the tool set that will be available, that yes, you can train it more directly on things you already believe to be true if you want to do that. And that certainly will lead to answers that replicate your thinking more quickly. So just on this point, one thing I wanted to get your opinion on is how good is AI at predicting big turning points or structural breaks in market regimes? Because I don't know about you, Joe, but one of the first things I did with chat GPT was I asked it to write, you know, a financial news article about inflation just to see

whether, whether our jobs were, um, were in danger. And you could tell that it was trained on not quite current data. It was talking about how inflation has been stubbornly low for many years and the Fed is trying to get it to the 2% target. But how good is AI at predicting those regime changes? Because if you're running, you know, a macro fund, I imagine that's one of the important. things that you need to do is try to figure out when something is fundamentally changing in the market. Yeah, and I'd say terrible if you use it in the sense that you're using it, right? Like that it's a little bit like saying, well, how good are people at that? Well, people are pretty darn bad at that, right? That doesn't mean that there isn't a way or some people who could do such

a thing, right? So AI, like, it's hard to just think about AI as a thing or think of like, okay, well, if I'm just going to use chat GPT for that, you're exactly right. chat GPT as it comes out of the box is only trained over to a certain history. And it doesn't care, like unless you know how to make it care. It doesn't care that it's, you know, it's just answering a question about inflation based on everything it's ever read about inflation. Time isn't even that important unless you make time be very important to it. And predicting. And so you have to know how to use the tools to generate the type of outcome that you're describing. So do I think like AI out of the box will do that? No, absolutely not. It'll be.

awful at that. Are there ways to take what's embedded in AI to come up with a way to do that? I was embedded in language models. And if you combine that with statistical tools, yeah, there's a path there. But it's not going to be as simple as open up chat GPD and ask it that question. There's more involved. But if you basically, it is helpful to have an analyst that's read everything that was ever produced, even if they stopped reading in 2022, in 2021, in 2021, I should say. There's a way to use that, but you have to use it correctly and not misuse it in order to try to generate that answer. All right. So I can't just ask a large language model when will inflation get back to the Fed's target. But I'm speaking, I'm not speaking to a large

language model. I'm speaking to CIO Bridgewater. And, you know, I do, I am curious, you know, I do want to talk a little, we do want to talk a little macro. And I, you know, before we sort of like, I'm not going to directly ask you when inflation will be back at the Fed's target. But what strikes me about the last year, and since the last time we talked, that's really blowing my mind, is that rate hikes have been a lot faster than people expected. Inflation is hotter than people expected. The unemployment rate is lower than people expected. What is it that people misunderstood a year ago about the economic machine such that the Fed has hiked rates much faster than people expected? And yet it's been surprisingly ineffective at cooling things down. And to this day, there seems to be a surprising amount of economic momentum with Fed funds at like 5.5%.

Yeah, it's a great question. I have a bunch of thoughts on it. You know, certainly I can't speak for all people, but I can speak for myself. I've been wrong about a bunch of those things. So just to talk about what I certainly, and let's say we at Bridgewater, didn't nail. Like you're saying, I thought the degree of and certainly are. everything that we had understood in our statistical models or whatever, and we knew that we could easily be wrong,

but that the degree of tightening was fast and high relative to history and that any tightening like this in the past had led to significant downturns, although the lead lag is somewhat variable and still possible, that's right. But I think a lot of things that happened different than I expected was, A, usually when, let's say, as they were last year, stocks were falling and short rates were rising, that formula in history always led to the personal savings rate rising. People seeing higher interest rates available to them, asset prices falling, housing, slowing down, etc. Usually people save more money, which meant there was less revenue for companies,

which meant there were layoffs, which meant savings rates rose more when the employment market weakened. And, you know, a recession was caused through that mechanism. And what's happened in this period is that I think, and now I could be wrong, that general, let's say, impact of the higher interest rate and wealth effect impact was offset by the fact that wealth had been changed so radically in the 2020, 21 period by fiscal policy and that we have fiscal policy as extreme as the war. And the ripple of the length to which that disrupted, let's say, those other relationships was interesting. the degree of it was interesting. I think there's ways we should have, you know, looking back now.

I think there are reasons that we should have, I should have known that. And some people were pointing to that. But that created much less of a reaction in household balance, in household savings rates, as you normally did. You came out of the recession with better balance sheets than ever. People were willing to dis-save. So even as rates climbed and actually debt growth collapsed as it normally would, but what simultaneously collapsed outside of debt is, or let's say, increase was the willingness

to spend down the cash that households had built up. And that cash doesn't just disappear when one person spends it. It goes on to others balance sheets, whether it's corporate balance sheets, other household balance sheets. And so that what's been happening, it appears, is that money's been spinning around in a way that made the rate hike have much less impact than I believe it would have had pre-COVID if you had anything like that, that rate hike. On top of that, within the U.S. economy in particular, corporates had extended their duration so the impact is taking longer, the effect on corporates, although I think it's happening, but it is taking longer. And so there are a few other things. And then obviously the benefit of when nominal, what did

happen is rate rise is created a decline in nominal demand, but that's mostly shown up in inflation. So nominal demands fall in pretty much as much as I've expected. It's been more inflation falling than real growth falling, which again, I think there's reasons that that's the case. But before there was this massive demand shock from the what the Fed, what the central banks and the treasury had done to get everybody's balance sheets up. And supply was struggling to keep up with this massive demand shock. And now demands falling, but supply is still catching up to that old level. So on net, real growth has come out stronger.

Now, I could see all that in the rearview mirror. I didn't by any predict that that would be the way it would play out. But I think that's why you've had this stubborn strength in the economy. And that's created a certain amount of stability. Now, equities have rallied significantly since then. There's some of the negative wealth effects have eased. At the same time, though, a lot of that excess cash that was on balance sheets have been distributed. So there's a mix of pressures here that looking forward, you know, we do think inflation is still coming down a bit, although on net we've entered what we think is a more inflationary environment such that 2% inflation probably more likely to be more of a bottom than a cap.

And we do think fiscal policy as the way to deal with the recessions is probably politically the more likely outcome than let's say moving back in the next recession to more QE. and fiscal policy is a lot more inflationary and effective in a sense of stimulating growth quickly as we've seen. So I think you're going to see a world where we are still adjusting to a higher inflation world that's de-globalizing, although everything we're talking about on the productivity front, maybe machine learning changes that we'll see. But largely X a major productivity miracle, I think de-globalization, the move towards fiscal policy has changed the long-term inflation. path in a way that markets haven't fully adjusted to. Because markets right now believe the Fed is totally credible that inflation is going

to return to target basically with very little problems. When we measure the pressures, we don't think so. We think it's going to be much more challenging to get inflation where markets expect it. The impact on earnings is going to be a lot more negative than the markets are currently expecting, and it's going to take longer and be harder. So big differences between what we're seeing and expecting and what the markets are currently pricing. So I think last year you were talking about the possibility of a recession in 2023.

Is that off the table now? So you're still positioned, it sounds like, for a level of higher inflation, but it sounds like maybe you're a bit more optimistic on the growth front. Yeah, we've been wrong on growth. So I'd say, look, we think it's going to be a struggle where in a sense. state of disequilibrium in the sense that relative to a given level of growth, we think the level of inflation to the Fed target, that they're going to have a difficulty achieving growth and inflation at the levels they want and are going to have to give on something. In the short run,

I think that's leading to, you know, higher rates, the expectation that the massive easing is coming is unlikely. The Fed's going to continue to have to be tighter, longer than the market's expected. So that's bad for, you know, let's say bonds and long-dated short rates. It's also probably bad for equities. And at the same time, we think growth will be struggling. It's nominal growth slowing. I think nominal growth is going to continue to slow. And as nominal growth slows, while you're more in stickier inflation, things like wage growth and some of the service areas, more sticky inflation, you get more of a challenge as nominal growth falls for it to just flow through to inflation. So my view is you end up with growth disappointing a bit and inflation

disappointing on the high side a bit, ending up probably bad for bonds and a little bit bad for equities and generally weak growth. And if that weak growth starts to translate into rising savings rate, you could easily end up into a recession and one that's going to be difficult to deal with. But yeah, I'd say we've teamed, I've tamed and we've tamed Bridgewater some degree our view on growth while still negative, not as extreme as it appeared. And it's a more gradual process that's unfolding. And then on the inflation front, while we've had, we expected a quick decline inflation as novel GDP fell, we do think we're in the range where you're in the much more stubborn

part of inflation. It's more harder to continue to get those inflation falls going forward. So just to be clear, though, you do think there is a gap between either what the market sees in terms of how much more work the Fed is going to have to do or what the Fed thinks, how much more work the Fed is going to have to do and what basically you think the Fed is going to have to do if it actually is serious about getting inflation back to something resembling its target? Yeah, I think so. I mean, I'd say the Fed seems a little bit more realistic than the markets do on what it's

going to take. But right, that we think that's right, that when you look at what the markets are saying, that it's super optimistic. It could come true. You do need, essentially, to get an equity rally from here. You have to have lower rates fairly quickly into a world where earnings are. are pretty good. That's kind of the discounted line to get above that. You need even more than that. And I think that line's super optimistic relative to what we're, you know, what we measure.

And again, our, I'm using the words, but I'm describing a process that's based on studying, you know, hundreds of years of economic history and how these linkages work and building all of that into a systematic process. But just spitting out kind of the output of that is that it doesn't appear that it'll that the Fed will be able to achieve that and that we're in this disequilibrium where you still have more inflation relative to growth and you don't have an easy way to close that gap. So we'll see. We've been wrong about that in terms of at least what the market outcomes have been for the last six months or so after having been incredibly right for an extended period of time. And that's part of it. We get a lot of things

wrong and that's normal. But I think when you break down why we got it wrong and the ways in which that, you know, we've learned from that and the ways in which our processes have taken in the new information still leads to this view that the markets are overly optimistic about how easy that's going to be. All right. Well, Greg, we appreciate you coming on and outlining your thought process both around the markets and AI and how you're actually deploying this new technology. So really appreciate it. Thanks for coming back on the show. My pleasure. Good to talk to you. Good luck in Vegas. Yeah. Bring home another bracelet.

We'll try. Thanks, Greg. That was great. So Joe, I feel like I have a slightly better conception of exactly how this kind of technology can be used for investing. So the idea of maybe you have the AI models come up with feces or ideas that could then be rigorously fact-checked because all the AIs are hallucinating and things like that. That makes some sense. Yes, absolutely. And I think, you know, you asked the question that's like, can AI do our jobs?

And I don't think the answer is yes. And I think it's like, can the AI replace the stock picker? It doesn't sound like the AI is yes. But like, can the AI augment the way someone's thinking, test, come up with theories that then can be rapidly tested, have that sort of go back and forth and sort of do some of the work that currently sort of like junior analysts do in terms of like theory, testing ideas and stuff like that. But you could see how it could be a force multiplier at a large fund.

Yeah. But I mean, to that sort of turning point question, that also seems to be maybe the big weakness here is that if you have an algorithm or a model that's been trained on years and years of prior data. So rates going lower and lower and inflation staying below 2% seems very difficult to project what might change. Which to Greg's point, humans aren't very good at that either, but you would hope, like, right, like that's what we want to just be able to ask Chad GPT or whatever, you know, I'm using that as like a stand-in for this technology. Yeah, or maybe you ask AI, like, what would you need to see in order to start taking the prospect of regime change seriously.

Yeah, I like, I mean, he talked about this idea of like the sort of like adversarial way of thinking about it, which I think is like really important. And he pointed out the sort of like disaster of the how the home eye buyers. Yes, the Zillow analogy was really interesting. And then they got adversely selected because it's like, well, if Zillow is in the market, we know they're going to overpay. And so everyone suddenly dumps all the homes on Zillow. and it was not anticipating its own role in the market in response to your question,

which I think is like a really interesting dimension to all of this. Yeah, that sort of reflexivity between the models and the markets, I think we're probably going to be hearing a lot more about in the future. On that note, shall we leave it there? Let's leave it there. All right. This has been another episode of the Odd Thoughts podcast. I'm Tracy Alloway.

You can follow me on Twitter at Tracy Alloway. And I'm Joe Wisenthall. You can follow me on Twitter at the stalwart. follow our producers on Twitter, Carmen Rodriguez at Carmen Armin and Dashel Bennett at Dashbot. Follow all of the Bloomberg podcasts under the handle at podcasts. And for more OddLod's content, go to Bloomberg.com slash Oddlots, where we have transcripts, a blog, and a newsletter. And for even more, if you want to chat with fellow listeners about all these topics, there's even an AI channel in there. Check out our Discord 24-7.

People talk about all these things. Discord.g.g. slash odd lots. And if you enjoy odd lots, if you like these conversations, please leave us a review, a positive review, please, on your favorite podcast platform. We'd really appreciate it. Thanks for listening. A lot of short daily news podcasts focus on just one story. But right now, you probably need more. On Up First from NPR, we bring you three of the world's top headlines every day in under 15 minutes. because no one's story can capture all that's happening in this big, crazy world of ours on any given morning. Listen now to the Up First podcast from NPR.