Invest Like the Best with Patrick O'Shaughnessy - Eric Sorensen - How Quant Evolves - [Invest Like the Best, EP.139]

Episode Date: July 23, 2019

My guest this week is Eric Sorensen, the CEO of Panagora asset management, which manages more than $46B for clients across a variety of strategies. Eric began his career serving in the Air Force as both a pilot and instructor in high-performance jet aircraft. He then accumulated 40 years of quantitative research and investment experience, with a Ph.D. along the way. Please enjoy our conversation on the changing landscape of quantitative investment strategies. For more episodes go to InvestorFieldGuide.com/podcast. Sign up for the book club, where you’ll get a full investor curriculum and then 3-4 suggestions every month at InvestorFieldGuide.com/bookclub. Follow Patrick on Twitter at @patrick_oshag   Show Notes 1:15 - (First Question) – His background in the Air Force             1:23 – Boyd: The Fighter Pilot Who Changed the Art of War 3:18 – Training people on high-performance machines 4:47 – Traits that made for better pilots 5:51 – The evolution of quantitative equity research and its stages 7:56 – How his research led to becoming a practitioner 9:10 - The early feature sets in his research 10:44 – Tradeoffs in the spectrum of interpretability 12:08 – Early days of his practitioner career 13:24 – Risk Premia and the 5 C’s 14:28 – Quantitative Equity Portfolio Management: Modern Techniques and Applications 17:13 – Applying the 5 C’s to value investing 18:38 – Knowing when a strategy/signal is broken 21:24 – What does this strategy plan mean for his firm today 24:56 – Mixing expert systems and portfolio construction 30:07 – Natural language processing 32:00 – The cultivating the power and creativity to ask good questions 35:13 – The concept of a research graveyard 37:45 – State of risk premia today 40:04 – Active equity process 46:37 – Frontiers of research that he’s excited about 48:53 – Safe havens for non-quantitative investors 52:16– Advice for young quants 54:36 – Quants on the buy-side that he admires 55:41 – Kindest thing anyone has done for him   Learn More For more episodes go to InvestorFieldGuide.com/podcast.  Sign up for the book club, where you’ll get a full investor curriculum and then 3-4 suggestions every month at InvestorFieldGuide.com/bookclub Follow Patrick on Twitter at @patrick_oshag

Hello and welcome, everyone. I'm Patrick O'Shaughnessy, and this is Invest like the Best. This show is an open-ended exploration of markets, ideas, methods, stories, and of strategies that will help you better invest both your time and your money. You can learn more and stay up to date at investorfield guide.com. Patrick O'Shaughnessy is the CEO of O'Shaughnessy Asset Management. All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of O'Shaunsi Asset Management. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions. Clients of O'Shaughnessy asset management may maintain positions

and the securities discussed in this podcast. My guest this week is Eric Sorensen, the CEO of Panagora Asset Management, which manages more than $46 billion for clients across a variety of strategies. Eric began his career serving in the Air Force as both a pilot and instructor in high-performance jet aircraft. He then accumulated 40 years of quantitative research and investment experience with a PhD along the way. Please enjoy our conversation on the changing landscape of quantitative investment strategies. I thought we'd start all the way back. You have a unique background and that you spent some time in the Air Force. And one of my all-time favorite books is a book about John Boyd and sort of the Uda Loop and all the interesting things that he learned. And I know you were an instructor

in the Air Force for a while. And I would love to use that, just a brief story of your time there, and sort of the foundational lessons that created the springboard for the rest of your career from the Air Force. I appreciate that. I was in the Air Force in part because of the environment. There was something called of Vietnam War, and I was an undergrad in college at the University of Oregon, and I joined ROTC. And my father had been a military officer in the Army in the Second World War, and we had a kind of a proclivity for serving. So I went to pilot training, and that's a one-year program and became then soon after an instructor pilot in a high-performance jet airplane. So tell me about what kind of planes you flew and maybe the early days of the instructor. I'm especially interested in that stage.

Well, thank you. I flew an airplane that used to train, advanced training for people that are getting their pilot's degree in the United Air Force of their wings, and to see the third plane that they would have been in. And it's the last one before they get commissioned as a pilot. It's called an F5 or T-38. It's a fantastic airplane. Northra built it, still in use. The one we flew carried 4,000 pounds of fuel.

and when it didn't have fuel, it weighed 8,000 pounds. It's just a rocket, no lift, just a rocket. Of course, supersonic and high performance. And think about an aircraft that is somewhat similar to the business where you and I are in today. It's a machine with limits, rules, great performance, but you can overextend it. But it's also operated by human.

So there's nothing like being in a high-performance aircraft and pulling 5Gs with your right hand on the control stick and your left hand on the throttles and being forced down on that seat and just that feeling. And I don't think the machine could do that by itself. So I've often thought about what we do now. It's a combination of intelligence, human intelligence, decision-making, thought process, instinct, emotion, and using whatever the latest tools are to do that.

And I really enjoyed that. It's an amazing analogy, and maybe it's one we'll kind of return to through the conversation. I'm curious for new pilots that were operating this sort of high-performance machinery for the first time, how you taught them to transition. And there's an obvious analogy here, which is fundamental analysts that haven't used quantitative systems and maybe they should. So what were the key steps in getting someone used to working with such a high performance machine? That's a very good question. First of all, these students who I call them students in quotes, they were like one year younger than me.

I had been one a year earlier. And then I was an instructor for about five years. They would come into our flight where I was an instructor pilot. and they had just finished training in another jet, which was subsonic, and a very different aerodynamic characteristic. So this is a new experience. This thing is going 250 or 300 knots by the time to get the gear up and leaving the – I mean, it's fast.

They've got afterburners and so on. So the first couple of flights, they're a little nervous. But for an instructor and for their program, they have to go solo relatively soon in that aircraft, like after maybe six or seven sorties or missions. And as an instructor, you have to make a judgment. Is this person, in those days there were just men, now they're women, can he do this? And so, in a sense, I learned early on in my career how to discern that and then to take the risk to empower people. Because you get out of the plane, out of the back seat as the instructor and you watch your student go and you keep your fingers crossed.

That doesn't crash and burn. Were there common traits of whether they're temperamental traits or otherwise that made for very good pilots relative to say more mediocre ones. The mediocre and very good, let me talk to that. There is a huge disperion even after people graduate in what you consider skill and ability. And upon graduation, some were recommended to go to be aircraft commanders or instructors and others had to serve some time as a co-pilot. So there was a, even though they all were above a certain minimum, that's part of it. The other part of it is, I don't think there's anything you can point to. It's not athleticism. It's not necessarily intelligence. It might be being able to do well under pressure. There's a series of

check flights you have during a program like that. And thereafter, you have them even as a fully rated pilot. You get inspected by the inspector general. You're put in there, no notice. And someone's grading you. And when you're going 400 knots and you're doing different maneuvers, you've got to be able to do that. But I don't think it relates to any other area of physiological or otherwise. We had big people, small people, large. We had good athletes, bad athletes. We had smart people. We had kind of average intelligence. You never know. Fascinating. Yeah, it really is interesting. One of the things I've been most excited to talk to you about is that you've basically seen the major stages of quantitative equity research evolve. And obviously, this has become, it was a niche thing for a long time.

It's now probably a dominant trend in markets and equity markets and all markets. So I'd love for you to describe kind of in your view what the major stages of that evolution have been, sort of draw a history. And then obviously we'll spend most of our conversation talking about today and the future. Okay. When I got to the University of Oregon, there was just the beginning of a explosion in financial economics, financial research. Probably the prior 20 years, most of the really good, they were economists, not necessarily a quote-unquote finance professors. And there were two or three areas, four areas actually, two in particular that really began to shape the literature and the way people conducted themselves with respect to doing dissertations and so on. One was the concept that

you ought to earn a rate of return in markets that's consistent with the risk you take. And formally, it's a capital asset pricing model. And I like to think of that as asset pricing equilibrium. Jack Traynor, who did not get a Nobel Prize, who I knew well, he's now no longer with us, was a pioneer of that. And Bill Sharp, Jan Massine, and John Littner got Nobel Prizes for that. And basically, that was an area that was new,

but it was descriptive. It was like, here's the way the world ought to look. Then there was another area of literature coming out of Chicago and mainly Gene Pham and others, which is also descriptive. not asset pricing equilibrium, but asset pricing competition. It's hard to win. Because if information is readily available, entry costs are relatively low and you can make money.

Maybe you want to try this. And I remember one author way back, and I've used it a lot, said, if there's a lot of competition and there are some cost to entry, the game may be worth winning in terms of if you get the right result, but the game may not be worth playing because of the expected value of the outcome is negative. And that's really interesting. So then there was a body of literature on proving or disproving efficient markets, those two things. So talk about your insertion in this stream. The research that you were doing, obviously we're both active managers.

So we disagree with some of the things that these Nobel laureates have put out there. So talk about how your academic research started to dovetail into a practitioner world. And there were two other areas of research. And the fourth one I'll mention is where I spent most of my time, call them areas of innovation. One was the whole area of, well, empirical studies. So maybe you can do it. anomalies. So the academics had to explain it away as this couldn't happen really small cap or earnings surprise or whatever. And that behavioral explanations because people don't think right.

They sell it the wrong time by the runtime. And then there was in my area, I did a lot of research on pricing. So I call it basically econometrics of asset pricing. Just straightforward looking at, hmm, can we get better at actually knowing we're a bond on a trade or a stock on a trade? So I did a fair amount of research, quite a bit of published academic research for a variety of purposes, where I had to use pricing models. And these were based upon multiple regression. And the variable in interest would be a price of an asset. And then the characteristics that would determine whether that price is high or low. And that was an area.

And that really led to a lot of the empirical work that now has found its way into investing, I think. Can you talk about that early feature sets? The outcome variable is price. I'm always fascinated by how people decide what features to include, what they do to normalize and scrub those features. So in the early days of the research, what sorts of things were you looking at? Well, my dissertation was on the question of municipalities at issue bonds. And the municipal bond market is a fairly large market. And there are investment bankers involved almost in every case in distributing those bonds, syndicates. And there's two ways it happens. One is there's a lead underwriter who negotiates with the issuer, whether it be a government or county.

and in other cases, the jurisdiction requires competitive bidding like you would on either contract to build a building. So there was a negotiated way of selling these bonds and a competitive bid. So the question would be, which one is better for the issuer to minimize the borrowing costs? So I had to develop a model, and others had done some of this too, and I was in a group that we did this, where the dependent variable was the yield or the price of the bond or something like that, and the independent things were factors, beta factors, multiple regressions. obviously the credit rating, obviously some other information about the municipality, the slope of the yield curve, the structure of the issue, and they're all basically tranches with different maturities priced

together. So you build these models. Number one, they had to be things that made sense. And today, even the business we're in at the firm I'm in, Banagora, we have to have intuitive causality when we investigate things. Doesn't mean we don't use fancy tools, but it isn't just price versus whatever we think can correlate with it. Can you talk a little bit more about that trade-off? So that's a really interesting question in today's market, which is certainly some firms would hold out the fact that they don't need to be able to interpret their models as a relative competitive advantage.

So talk about the trade-offs between that kind of spectrum of interpretability. They're probably a trade-off. Perhaps both work. I would think the one that doesn't have an understanding inside it or a, if you will, use a machine learning terminology, isn't supervised. It's unsupervised. just finding correlation structures embedded and then repeating, repeating, repeating, with algorithms.

And there the objective is to minimize the error. What if it isn't working? And it is trained on historical data. You really have a quandary as to why. Secondly, what do you grab? What data do you go for? How do you decide actually what to look at? So we have our equity team and our multi-asset teams are all, if you will, coexisting

date. The researchers are also portfolio management and vice versa. Some are older, some are younger, some have different talents. They watch these markets. They watch these performance results of our strategies, and they're all the time looking for new inputs, but they have to kind of make sense. Otherwise, you could just go after anything.

And there's a budget. If you're buying, quote-unquote, big data, I'd call it smart data. We build a lot of our own. It's expensive. So you have to really make a, if you will, a business judgment. I don't know how you would manage that without belief. So I go back to the question.

It's just a different belief. So now talk about the early days of your practitioner career. or so you start taking sort of this skill set and applying it to actual markets and actual trading. I was doing these research projects that was pricing either not just municipal bonds. I went to corporate bonds and yield curve modeling as well as things like preferred stock

and convertibles with different features. And I got involved with some investment banking people that said, well, what does it cost us if we put a sinking fund feature around a bond? Well, you had to have a model to say, what would the bond sell for with or without? So to me, this was like in my sleep. So I found out.

You could marry, and I love creative. I'll come back to what I think the motives are of why we're in this business. I think there are two, why we run a firm, why you run your firm. And the motive there was to get to the intersection where you get to research and discover things, and someone likes it. And it's useful. It's practical. So after time passed, I became department chairman at the University of Arizona Finance,

published quite a lot. I took a one-year leave of absence and went and worked with a small firm. And then after that, I never went back. I found Solomon Brothers or they found me in 1986. six, and they needed to build a quantitative equity group. They had none to speak of, like what they'd already done in a major way with their bond analytic group, not the proprietary stuff, but Marty Leibowitz's stuff on bond portfolio analytics. So I went there and started that business. So many listeners will be familiar with sort of the simple evolution of what we call risk

premia or what I might call like the core primary factors, things you've already mentioned some, value, momentum, things like this. So obviously that tends to be one thing that many quants use, certainly not the only thing, and it's become increasingly common. In the materials you send me, you've got this great thing called the five Cs. And so I'd love to hear what the five Cs are and use that as a means to talk about kind of today's research. That's a great question. Just as I observe the life cycle of strategies in competitive markets with a lot of motivation for people to participate when something's working and then seeing it ultimately not work for a period of time or even just ceased to exist. It strikes me that, first of all, you have this set of people

or minds that are creative and they like to discover and they think this might really work. And if it's truly creative and it works, that's great. And you're either adding value or you're controlling risk better and so on. But it's a very competitive space and people begin to copy it. We wrote a book at Panagora about 10 years ago, myself and two colleagues on sort of a follow-on to Granold and Kwan. How do you do quantitative management? We have a lot of stuff in there which we'd formally published in some pieces and some journals, which really disclosed what we're doing with what we consider to be sort of state of the art at that time. Well, you kind of do that, and people find out and copy it.

So you have a situation where there's a creative idea, it gets copied, money flows in, it gets a little congested or crowded, and ultimately there's convergence on a limited supply of potential assets you can buy or sell. And then the Federal Reserve or there's some external catalyst says, wait, things aren't the way we thought they were, and people begin to sell. And then there's ultimately capitulation because it becomes a liquid. The first one I saw, I observed up close and personal it was Solomon Brothers in 1987. And the day of the market was down 22 percent, which day was in October.

It's always October. And it was associated with a very good strategy, started by two academics and one businessperson from the West Coast. Tane Leland, Mark Rubenstein, who I know, both of them and another guy, I'm Larry O'Brien, L.O.R. And they had this thing called portfolio insurance. Think about our business. You got to have a great name. It's not death insurance. It's life insurance. You know, there's a label when I was at Pottenham in a brief period where they had on a product which was like, sounded too good to be true. But anyway, the strategy was to use the futures market

in their case to advise pension plans to hedge their equities, S&P. And there was other people doing it actually using options, which was very interesting. But anyway, that strategy became fairly large relative to the, and one day the market was down and these portfolio insurance programs kicked in. The futures gaped down and with index arbitrage and the cash markets gap down and so on. And by noon, the market's down 10%. By two in the afternoon, the New York Stock Exchange

and others are calling our head trader inequities and Goldman's and Morgan Stanley say, don't back away from the market. And it was down 22%. So there was ultimate capitulation. One of the interesting things there, you could have hedged in down drafts using options. And one firm in New York did.

I believe the name of the firm was, B-E-A, and I met the principal at some point in time, I thought a very bright guy. And he was doing this for pension plans, or his firm was, by selling out-of-the-money puts in the S&P, which were reasonably inexpensive. I don't know what the structure of implied volatility pricing was, maybe it was flat. Maybe they were even cheaper than an at-the-boney-putt, probably. Ever since that day, there's been a skew in that pricing structure. The out-de-money puts have never been cheap again, because people realized that was probably

the best way to preserve wealth because it's contractual. It was an insurance premium. That premium one not after it. Anyway, I saw that and then other episodes of emerging market, debt crises and long-term capital. And even to some extent, the 19, 2007, 8, 9 equity thing. There was a lot of money. And you mentioned earlier crowded. So I just think, and those things will probably unfortunately repeat. So I want to take these five Cs and apply them to something like value investing, for example. So you've got the kind of Andre Schleifer limits to arbitrage idea that for some of these strategies, maybe they go through this five C cycles many times. but their long run expected excess return is still positive. And so I'm curious to get your take on what differentiates, let's say, a signal or a factor that will have, despite knowledge of it, like value, will have persistent outperformance versus something like long-term capital or some other strategy, which is fully arbitraged away through the 5C cycle.

So what's the difference between those two? Well, the value proposition, I think, is a real one in a long-term sense. I do believe that things become mispriced. and if it's a viable investment vehicle, like say the stock market or the bond market or subsets of that, there is mean reversion. So if I were 30 or 40 years ago starting a money management firm, I'd be a value investor, equity, and I'd make my clients allow me to keep their money for five years. And I would always have higher wealth.

There's a thing called sarcastic dominance. Can you accumulate more wealth for your client over a five-year period? The trouble with the signal sometimes that are short-lived, I mean, there are periods of time where they're not working. We've been in that in the last couple of years in equities in quant equity. And maybe it's crowding. It may be other considerations, other influences. You mentioned things that ultimately fail and don't work. That does happen too.

But I think you have to have a philosophy that this strategy will work over time. And you can tweak it. And the way to think about this question is how to know when something is broken. So, one, I'm curious whether or not at Panagora you combine signals maybe like value that you think in some way will work forever with those that maybe you expect won't work forever. mentioned smart data. There's plenty of examples of information arbitrages that go away. So how do you think about identifying when a strategy or signal is broken and should be removed from the process? The categories of these elements, the drivers, I think, are good. They're broad. There are sentiment, is one, inequities, valuation, quality. You can call it ESG. The metrics you use, though, have radically

changed. I'll come back to that a little bit. The other part is portfolio construction. So there's lots of strategies, macro risk premium, risk parity strategy really is heavily dependent on how you construct, how you weight, risk weight. That's a pretty good proposition. My colleague Eddie Chan will tell you every major asset class has the same sharp ratio over any 20-year period. But during some periods, some of them really have great sharp ratios and other down. But if you know that and your risk weight, commodities, equities, global markets,

yield curves, bond markets in various countries, it really has a nice distribution. It doesn't have the fat tail on the left side. So that's a construction. I think that's fairly fundamental, but you have to stick with it. They can't just sort of discard it if, in fact, another asset allocation scheme, like fully, heavily oriented toward equities in a bull market looks better, quote, unquote. On the other side, in the equity side, you have to use better data. See, I did the first, I believe, the first quantitative equity ranking system

that was put in production on behalf of pension clients for S&P 500. We did it from Solomon Brothers. was the same stuff I'd been doing for the prior 10 years. It's an academic except applied equities. And every stock had a valuation. And the valuation was based upon projected growth and normalized earnings, but a factor model, which would adjust the way you actually, if you will, give a uptick or down tick to the multiple.

But I put it in the discounting mechanism. It was like a long-term DDM. But the factor model was in adjustments to the discount rate based on risk and volatility and stuff like that. And the reason it worked really well, because every stock, It was slow moving, so a stock that had a great value wouldn't change overnight, and the quintile spreads were reasonable. It'd buy the top one or two deciles, very simple portfolio construction.

And it worked because, well, not many people were doing it that way. And it was simple. The way it worked was, this was called the E-Model, and the way Solomon Brothers model. And the way we worked is a pension plan who had internal index funds. Several of them were clients of our firm. They would carve out some of their money and the internal counterpart to me, the person running the index fund would implement what we told them.

And so they tilt the portfolio toward this model. And then it turned out it did better than their S&P 500 index more or less over time. So the early days looked that way. Yeah. Obviously today, I'm sure, looks a lot different. Does. So there's two components to this that I want to dive in some detail on. The first is the data.

And the second is the models used to find patterns in the data. So simplistically, you have linear regression on top of book value. And in more complex terms, you've got deep neural nets and random forests on top of all sorts of interesting smart data. So talk about maybe kind of focusing on today and looking forward what that looks like now. So I think the concepts are similar. Be systematic. Be good at portfolio construction, et cetera. Like you said, I've already said several times.

You need to evolve. So talk about what that means for your firm today. I agree, Patrick. There are the two aspects to trying to win today. One is the data you're accessing how you implement it or you get it, how creative you can be with it. And there's a lot of data today. That's one.

We'll come back to that. And the other is, what are the tools you use to actually then extract value from that data. And they are the advanced, if you will, statistical techniques and other types of things that are needed today nonlinear, hierarchical, and maybe machine learning in some sense, or real sense. But the data, I give this talk about, we call this smart beta and big data. So I call it the smart data and big beta. Beta is too big. And the data have to be smart. It's a $50 billion smart. I said the group report a few years ago, I said the vendors of what now is called big data, The average of the revenues is enormous.

A lot of those providers really don't have well-defined business models. They just happen to be nichey. They know some data. So the same thing can happen there if you're buying from a vendor and other people have access to it. So we're aware of that. The other part of it is you can buy some or you can actually then create by different sources. So the creativity there comes with an investment team that really knows where the data sources are. First you're going to know what drives stocks.

One of our groups have, I think, a pretty good model for biotech. There's nothing to do with financials. We've had this for 10 years. It's a part of one of our strategies. It's now a standalone biotech pharma portfolio. Biotech have to do with drug trials. That's a coin toss in some sense. Well, it's not really because we'd call in the Goldman Sachs, the Morgan Stanley analysts,

the biotech analysts years ago and say, what do you look at? I don't look at prices. There's no earnings. We look at the FDA approval process, and we look at what they're working on and we visit them. So we actually, and a big career I use always expert systems, this team build an expert system to mimic what that analyst knows about those handful of stocks except we're not as deep, but we know it more broadly. So that data set we've perfected over time. And there are other examples too. And even as having said that, you have to continue to be head of the game.

So there, and I don't know actually how we're running that today, but early on I know it was compounds and what cohort group. operating the same kind of discovery process of drug trials. What's the likely success with kind of a Markov chain analysis, a probabilistic model, like these biotech firms really low odds, these above-average odds, and so you buy the above-average ones. So that's a data set that was unique. I call it smart data, and I say this about it. It has to be intuitive.

It makes lots of sense because the analysts from the cell side are doing it. B, you have to be able to gather it and collect it. And if you can do it prioritarily, that's really a big deal. You're not buying it from a well-known vendor. And see, I believe most of our work, we believe, and maybe others are different than this, we have to have a reasonable horizon for the payoff. So price tomorrow is very rarely a dependent variable in things we do. Yep, non-stationary.

Yeah, yeah. It's just all over the place. So that's a great example to talk about kind of mixing expert systems, let's say. So if you've got a biotech system, which is hugely different from something relying on financial statements or something in another sector. And so you've got, let's call them competing models or signals in terms of relative attractiveness of the asset, the underlying asset. How do you blend those two things thoughtfully in portfolio construction? So if you're in a portfolio, I certainly understand if it's a standalone biotech fund, you're comparing versus

other biotech companies. But if you're comparing a biotech stock to a consumer staple stock, and those two stocks are vying for weight in the same portfolio, how do you think about that integration? Well, we, fairly long time ago, we're using a technique for looking at the signals how they co-vary and effectively try and optimize information ratio. And through time, as you look at the inputs, you can actually get a weighting scheme that comes right out of that. So that waiting scheme will give you the optimal IR of whatever factors you're putting in, given the history of those factors.

So as they change, that waiting scheme will change over time. So one of them is just the alpha, if you will, the weight you put on a signal within the composite ranking. So if you're optimizing for IR, something we always struggle with a little bit, which is often the best way to do that is to reduce tracking here, not to increase the numerator. So how do you guard against that?

Well, the other part of it is the risk, which you call risk. And so when I was doing these early models for enhancing the S&P became, and then the real players came in, Wells Fargo, which became BGI, State Street. They came and saw me, they saw my counterpart of Goldman on the cell site. How do you do this? And they made businesses out of it. For us, it was just a niche thing, and it was fun. And we made some money.

We got paid, by the way, by transactions. So we weren't fiduciary, and it was a lot of fun. They perfected a lot of this and the rest of this kind of history there, but it evolved. So that's a good example of what you call smart data. If you're willing to share, I'd love to hear other examples just to continue to contextualize this for people thinking about this idea. But then also how you think about applying the very buzzword-driven new statistical techniques, which are just that. They're beefed up evolutions of very old techniques. We were talking about my friend Jeremiah alone before we started recording. And he always urges me to think, if you hear something really

fancy, just think stacked regressions, right? I heard that. Sort of regressions all the way down. But how do you think about using those liberally, talk about a powerful machine like what we started the conversation with responsibly and then marrying that with the smart data sets? The classic quantitative or fundamental portfolio, the manager or team or process ranks the securities. So you're going to buy the higher rank securities. The question is, can you use advanced sort of techniques to identify value and see relationships and get better inputs, right? And the buzzword today's machine learning or artificial intelligence is a term that I don't like to use with financial markets, but it's used. We have so many AI experts today in our

field. I'm going to digress for a moment here the way I think about that. And I think some of my colleagues would share this. First of all, you go back to the definition and you know the definition by Alan Turin in 1950, if in fact the machine in those days the computer, which is mechanical and somewhat artificial maybe in some sense, if it can produce. results that are indistinguishable from a human, then it's artificial intelligence, maybe even better than the human. Of course, that's been shown to be the case with a chess plane computer versus a human, go, driverless cars. You think about all those applications. The two things I think come to mind. One is regardless of how many possibilities there are in a chess game or in a car driving down the

road, they're finite. And two, they're rules-based and deterministic. The rules. So you can think about an expert system that can actually do that, maybe even facial recognition. You get into financial markets, A, both of those things break down. You have Brownian motion. You have stochastic processes. So almost an infinite number of outcomes of a security price. And is it rules-based? Not day-to-day. So that's an issue. But machine learning is a tool that actually can get inside and maybe come closer to getting a solution than if you were doing it just with your own head and some simple data. And I think some of the applications that are really interesting there, you mentioned stacked regressions. We started using decision trees back when I was at Solomon, my colleague then Joe

Mesrich, who's still a cell side quat. I think the best cell side quat on the planet. I mean, we worked together since 87. There are others I worked with too. Probably the only person I read with regularity. He's amazing. He's a good friend to mine. Anyway, we started using classification trees because they're nonlinear and hierarchical and they're rules-based. And the test statistics on R-squared is like the probability you've got the right answer. And we use regression trees, but there was too much noise in that. So we classified with quintiles or deciles. Well, today the work he's doing in others, they use multiple trees, random forests, or they'll do multiple trees through time so the structure can actually learn and can adapt to changes in what causes

return. With boosting, yeah. Yeah. So I think that's a pretty good application. As long as the engineer or the designer knows the potential limits and pitfalls. The trees have to be proven back. You can't just let them fully populate. Back to your 5G analogy. Because they overfit. And then you know you get spurious results and you can't count on it necessarily. The other's natural language processing, which I can come back to. So let's talk about that a little bit. So why and how is that useful today? We have some colleagues that have actually done quite a bit in that. And we have some applications and it's actually in our models. One application is to use the reading of filings, corporate filings. And there's a study done by a couple of professors at Rice

were mentioned Renaissance, AQR. I think we were number two. They got information from the SEC as who's hitting that website, Edgar, to get all that information. We come second with number of hits, but we come first with a dispersion of all the things we're looking at, even to footnotes. So you read the literature. You know that when a CEO's talking, and if they talk too much, repetitive, I mean, there are red flags and stuff. So you can, you can, can actually use natural language processing to associate certain terminologies and phrases in English to get inference whether this is actually a good thing or a bad thing for the company. Analysts do that, they do it by talking, but you can do it with natural language processing.

And it's the same techniques that Google, which in the public domain, invented in .vec 2, 3,4, whatever it is. The other is an example that a colleague of mine, Michael Chen and his colleagues worked on, which actually reads Chinese characters to try to get an insight and a sentiment in the A-share market. And that's fascinating. As an aside, when I was a Solomon Brothers, because we had a quantitative group, which grew quite large that I organized, but we had a pretty good fundamental research team, which worked with the Salesforce institutional investors and also with the investment banking people. And at one point, one of the things you needed to do as an analyst that was

understood is you need to detect deception by the CEO, CFO, and so on. So we actually had psychologists come in and give lessons in how to interview and pick up things that might be red flags or green. So NLP is doing that, like all quant, it's doing it maybe not as deep. It's doing it a systematically and broadly. That's breadth and scale. It works. Yeah. So one of the things we talk a lot about with our team, with other managers as well, is that in this world, asking good questions, and you've kind of put this a couple different ways, has really become the creativity, I guess, is your word for it, is become the powerful thing, the differentiating thing, because answering questions is getting easier and easier with more and more data, more and more techniques, et cetera. So as you

think about your research agenda, how do you cultivate that creativity? Is it more of a bottom-up type system where it's just about hiring creative people and letting them be? Is there a top-down component to that? How is the research agenda structured? A lot of it is who you hire and how you develop a business model, or if you will, a recruitment model and a leadership model and people that come in that are creative that are maybe potentially asking the right questions. A very good friend of mine who was a contemporary academic, came and visited me and visited with our firm for a while after I'd been at Solomon Brothers. He said, Eric, since you've been here, you've become a much better academic because I had, I knew what question is. You know, it was real. So I think, A,

yeah, trying to identify creative people, but B, giving them some rope to a fault. I've always done this, like with example of being an instructor pilot, I believe my best approach to managing creative activities and research and money management and a business that has to make money for clients is yet still to allow a lot of freedom within reason. And I think we have two CIOs and I've worked with these people for 20 years and they've hired people. And our equity group have now interns come in. Several of the group of the last three or four years have become full-time employees. They become then PMs. You can't over control. Nor can you come up with a list of like principles. You just have to enjoy it. I said earlier about the motivations. The two things. Yeah.

Yeah, I think about this and I think about discovery. It's a good word. That's why I became an academic. I had no idea that I'd ever be doing this today. You want to figure out stuff for? Or getting paid by people to manage your money or in Wall Street getting paid by people to trade their portfolios as a firm. But discovery's fun.

One of my analysts, young guy, smart guy does our NLP stuff. You can't stop him talking. He comes in my office yesterday. We're talking about. He says, I've got to come back and show you the entire agenda for all this stuff. I would say, I don't know what he's talking about. I know enough about what he's talking about. If I don't, he makes sure I get it. But that, you got to understand. Now, he's not going to be the type of person that's going to sell,

manage accounts, go out and pester, pester, pester, institutions finally get into a semi-final, final, get business, and then unload it, right? But you need that. So it's a fun thing to watch. And we have an environment where that is the way people do it. The other D is dollars. I mentioned earlier converting copy. People come into the business who think you can make money, which you can. If that's your sole motivation, that ultimately will not help you run an organization. So the right balance between having an economic setup that makes sense, competitive, and also allowing some freedom of discovery. Discovery in dollars is a good framework for thinking about this. I mean, there are institutes that sponsor medical research that bring in young doctors, physicians who are just doing research on incurable diseases today, Alzheimer's or some cancer.

Here's money. Here's your team. See you later. We'll see you later. And they're doing it because they just love to discover stuff. One of the things I also like to ask about is this notion of a research graveyard. So everyone's got things that they were excited, hypotheses that they wanted to test, that they went through the rigamarole of getting the data and testing and just didn't work. It failed.

And it gets its tombstone in the graveyard. So I'm curious if you have any sort of favorite tombstones, things that you've tried or your team has tried, that you thought were good hypotheses, but just didn't work. Oh, so many of them. Yep. I'm trying to think back and stir the ashes in my own experience. For example, I think it's the case that this work we did recently, my colleagues did on A-Share Market. He did his analysis of reading those 50,000 Chinese characters, and they're not sentences. And they did an association between the language used in the chat rooms. There's three or four of these blogs. It's a retail-driven market in China. And sentiment and positive negative sentiment. And because China, the country itself, golf is kind of heavily regulated and there's some censorship of what you can say. Two Chinese characters are two Chinese expressions. One is garbage or bad. And the other spicy chicken.

So you can say this restaurant sells spicy chicken or this restaurant is garbage. You sound very similar. And you read these blogs and you can associate that. So the slang is if you don't like it, it's spicy chicken. Well, you can do unsupervised machine learning to actually come up with a whole set those things. I don't believe we make any money on that, but I do believe this. We learned a lot. My colleagues did. And maybe ultimately that particular application will be useful. Is it in the graveyard? No. But there are other things, I suppose. I mean, volatility. We did a lot of work at Mezrech as well on forecasting volatility. It was a hot topic. And we hired Rob Engel, who became a Nobel Prize, a laureate, Bob Engel from University of San Diego, UCSD, and spent a year with us.

And we had a lot of fun with this. I kind of knew that we're not going to make any money on it, but it was really engaging. So we use arch and arch models, and we wrote some stuff about that and went out on the road. Because Mesrish, I'll mention him again. Well, I got Solomon in 87, and he was in another group, technology area somewhere and had a PhD in Mass Psych, and he was a time series statistician.

Everybody was doing cross-sectional, cross-sectional, even crudely at that. I couldn't find anybody who knew what Arima was or any time series, classic econometric time series. He did, asking to join our group and the rest of history. And I think, so we were. doing this volatility modeling. I don't know if you make money on that. I don't know. Markets are pretty efficient, applied vols compared with statistically predictive volume. What about the state

today of what classically is called risk premium? So if you look at Panagora's sort of lineup, if you will, of strategies, there are two separate categories, both systematic and quant, one with risk premium, one called active. So I'd love to understand the difference between those two, starting with what your interpretation of this idea of risk premium strategies looks like and sort of any comments on the state of those strategies today after what you said. It's been, in most cases, a tough couple of years for kind of classic quant models. It has been on the equity side, no question.

With respect to risk premium, we probably think of it more as asset classes. It could be currencies. It could be bond markets around various maturity structures and bond markets, certainly equity markets. And risk premium means if I'm going to basically expose myself to a volatile asset, I should get a return. And you can measure the sharp ratio of that return.

And I said earlier, sometimes they mean revert. But Eddie Chan, who actually was the first one to actually write down in a paper in 2006, he's one of our CIOs, runs a great product area, risk parity. And Ridgewater had a product at the time, phenomenal name because that's a pilot. All-weather. All-weather, man. We were always talking about the all-weather aircraft that was coming. Maybe it was the F-15 could fly in any condition.

By the way, you don't fly in thunderstorms. So there's no such a thing as an all-weather airplane. Or hailstorms. You just don't do it. Some good analogy there. It wouldn't like. But the formal way of doing that is to really. risk weight the assets. And so we have a product in that area. I think that has real staying power.

What happens is sometimes the proposition is that they all have similar sharp ratios over time, but they go through different cycles. Really, the correlations between the asset classes, if they're lower negative, it's a winning strategy. You can accumulate more wealth for your client with a risk-weighted approach over a reasonable horizon than, say, traditional 60-40. But it doesn't always work because work meaning the following. It's underperforming traditional. What do you mean underperforming? Well, last quarter, it was 3% behind the, say, 60, 40, which is 90%, 95% equity risk. Why? Because equities went straight up. So I tell this, Tim, I said, you're doing your job when it, quote, unquote, underperforms.

Because over three-year period, it'll deliver a better answer. Yeah. If you're not disappointed, you're not diversified, right? No, I know. And so we do that in long-only space. We also do it in, if you will, hedged or, if you will, beta or market neutral space. Same concept, capturing premium, but offsetting it by shorting something else. So then active equity is a very different idea.

So talk through from what I can clean on the materials that I saw. It sounds as though maybe we can use the biotech or some other examples. There are sometimes even single stock specific models that are being built. So it's not long short, decile spreads and value or something. It's more granular than that. So describe that process, that active equity process. We have two approaches that we've used. And actually, there are two different portfolio offerings.

One of them does, I never did like it, but we always talked about it. as every stock has its own model. See, there's so much idiosyncratic risk in a single stock, I don't think you can model it with valuation, but in terms of actually weighting factors, it's hard. But we built a process back, my colleagues, and I was part of it 20 years ago, and we have clients in this strategy today.

We call dynamic equity, which actually, rather than one size fits all, the weightings on a particular stock in the set of factors, whatever they are, value, momentum, growth, with different inputs over time, quality. The weights are different for different securities. no two securities with exactly the same weight. But the way the weights are determined by commonality within a larger group of securities.

So take all the gross stocks, the high growers, or take all the small stocks, or all the high beta stocks, and model with your inputs, what is the optimal waiting scheme for each of those clumps? Well, no stock is in one clump. In Euclidean space, it's sitting out there somewhere. So you map it into these four or five different risk, if you will, partitions, and then use a waiting scheme, which is drawing from whether the stock actually results. sides. That was pretty good. And even today, that's still better than one size fits all. But other people have actually done that, I've copied this to some degree. So that's a one size fits all kind of away from that. The other strategy gets very deep into the mechanics within a sector or when a set of common securities. What are the fundamental drivers? Like the biotech. Is there another example that you think

is interesting? Yeah, and I use an old example too. Financials. They really had a hard time. Quants did with their financial sex of their portfolio in 2006, 7, and 8. It was a problem. And my colleague, George Masali, who is the other CIO, the counterpart to Eddie Chan, who I mentioned earlier, is really a pioneer in what's today called big data. I mean, really, it's a real fundamentally oriented team, but they identify, and they were trying to figure out ways to really sort between good and bad banks. I mean, financials, the corporate, say commercial banks.

And one of the data sources they thought about, well, let's value the, asset portfolio. Let's figure out what's the quality of the assets, all of the loans in the portfolio. Categorically, there are all kinds of loans. And so they figured because we know the world, well, there's a government entity that actually cares about that, the federal deposit insurance corporation, because they guarantee to some degree the solvency on behalf of the depositors. Well, they know a lot about the quality of those loans. They could have their data. And it turns out they had at that time. I'm sure they had better data now, but they had data on the delinquency rates of a loan portfolio of commercial banks that were FDIC insured,

meaning what percent of the loans, one month overdue, two months, three months over it. So you can actually, if you knew that, you can give a haircut to the value of the assets and therefore value the bank. By the way, you can also do it with insurance companies, you can also have real estate and REITs. We do it all over now. But here's a great evolution of a factor. So George and Group, they call the FDIC, can we have that data that here if I found out about it?

Sure. and I like to describe it this way, it probably wasn't quite literally, but about a week later, a huge box of paper arrived at our office. All this stuff typed up, written down, or print it out. We can't use that. We can't. The labor going through, can you get it to us electronically? They couldn't, but they ultimately were able to. So going into 2007 and eight, I believe our financials performance as part of our long-only stock selector product actually held up a lot better than most of them.

Now, that same data today has been armed away. because I'm sure that one or two clever providers of what's called big data found out about it, went into the FDIC, said, you're giving that to Panagora? We'll buy it. Yeah. Okay. And then they'll resell it. That's a stylistic version of what I think is a good example. Let's take those two models again. How often are those models being updated or changing? Well, a factor may last four or five years and a half life maybe two years and you have to monitor it how well it's working. We have a model that our equity team use, a model in quotes, an actual expected value model.

model or a business model of whether we're going to go after a particular data set and then use it in production, big data, smart data, whatever. Part of the answer is how much does it cost? The other part of the answer is, if you look at the existing model, will it add value at the margin? So you can get a set of data that's a, if you will, a version of it, a pilot version. You can test it with your process and see. You can get a pretty good answer.

Will it add three basis points or four? Do we think we have to guess a little bit on the bound of that? that. And then turn around and say, what's going to cost us? Some of this big data is very expensive because the hedge fund world is smart people. They get in there in a hurry and it bids the price up. And so then we have to make a choice. Are we going to buy it when it's not economically worthwhile? We do buy some. Or are we going to try an alternative source? Or are we just not going to do it? And by the way, again, this team, we have a medical doctor on there. We have other people from different parts, a historical professional. The MD didn't practice medicine. He's been with us 10 years.

he built our biotech and pharma model. He loves this stuff. He just didn't want to be a doctor. But he kind of know he's in the field. We have a way of actually sorting through what will work and what won't work. And what will work is if a vendor says this is going to cost you a $200,000 a year and you can find, let's say you want to know, there's a building right over here. You want to know the occupants. You think you have some big data or some smart data that can figure out the quality, the same power of the occupants. So the real estate holding the REIT, whoever owns it, you can rank whether that's safe, risky or not. Underwrite the quality.

Right. You can buy that data. It's very expensive. Or you can go down the street here and look at a person who's basically got a washing windows or a person that's actually in the plumbing business or fixture business. And there's 40 tenants in that building. And that guy's got access or that person has access to information on who the tenants are so they know who to go after.

The trade organizations have some of this. It's free. So that's not always the case. It's a fascinating kind of application of some of the porter forces to business where the business in this case is. supplier power or whatever it is, the inputs into the quant process are complicated. And like you said, the discovery is what's so fun about this, is relating sort of raw data sets to outcomes that we care about for our investors. As you look forward today, what are the frontiers of research that

excite you most sitting on top of so many different strategies, new techniques, new data sets? Is there any one or two things that you're really interested in today? Or is it just sort of the same perpetual discovery that is driving you? On the multi-asset side, whether it's broad markets, equity markets or sectors, I think there's a lot of room for better time series work. Again, I mentioned Mesrich. A lot of people are focused in on cross-sectional comparisons and don't have a real passion for like over long cycles. Can we forecast better? Or can we be on the correct side of it? That's a whole area that I think's got a future. In equities per se, you look at what's happened. We have quant equity. It's under some pressure. It's like fundamental.

Basically, there's not a lot of flow or the flow slowed down. It'll pick up again. It's really got to be. For the ranking system, I like to think the ranking system of ranking stocks, I often compare it to the ancient Olympics. It's 1776 BC. The Greeks, they had a stadium, and they started a contest. Who could run around this 200-meter stadium, the fastest? They had a contest. And then over time, over the next 50 years, they proliferated the event, not with number of people participating. Yeah, people could participate. But with how many events that person had to participate in became the pentathlon. There was running, javelin throwing, discus, jumping, and wrestling.

Now, think about all those things. Think about a stock, five characteristics. Is it going to be good at the top on everyone? No, the winner of the pentathlon on a weighted average basis won. You wait these events. I think Jim Thorpe in 1912, they still have the pentathlon in the 1912 Olympics, was still viewed by many as the greatest athlete of all time. He could do them all.

He could have won single events. That's the traditional quant equity. We want to rank, and we get in there and do that. So that whole world is about data, and it's about the tools to actually forecast. We mentioned them. The machine learning or the AI type things as a new ways, which are basically extensions of linear regression or these data sets you and I've talked a little bit about. And you just have to hope and think that you can actually be ahead of the field in that.

I'm curious if you think there are any safe havens for non-quantitative investors. So like Mark Grandreason always says software's eating the world, kind of think about quant the same way. Like if quant can eat something in investing, it's going to. You can do whatever an analyst does more efficiently if it's systematic and structured. Do you think there are safe havens for the traditional, more. old school discretionary analyst or portfolio manager. We go to see clients. There's a group I'm thinking of.

They're usually more sophisticated clients. They like us. They like our competitors. They have multiple of us. First question. Head guy in the room, person in the room. What are you doing that's different? Be good, but be different.

Yeah. I think concentrated portfolios, forget this alpha beta separation. That's going away. Smart beta, as we call it, you can't have a monolithic cap weighted MSCI that everybody has to attract to. It's going to change. It is changing. And I'll come back to another metaphor about the Olympics of how I think that's working this way out,

modern Olympics, as opposed to the pentathlon. And therefore, those that actually can pick stocks and have concentrated portfolios, I think there's a whole world for them. There's a key, though, you can't hamstring them. You can't make them have a beta one. In fact, you ought to say, I like your team, I like your processes. And by the way, that's a large part of how you get hired, not just your track records. I believe you believe what you're doing.

I like your passion. I like your fees, but that's not the main thing. We think this will work. And by the way, if it doesn't, we'll fire you. By the way, we as money managers exist so we can be terminated. It really is. It really is an Asian principal problem. Social work is not a bad place to be as long as you can survive.

So concentrated portfolios, and one of the keys is, man, can I hold cash sometime? Yeah, you can do that. See, in 1971, 72, when I got one back to your address school, I started looking at this. Every master manager was alternative. The history of this company, probably, with your name on it, was alternative. Wasn't viewed as, okay, here's a mandate, look like the S&P 500, or look like the nifty 50. So we're kind of moving back to that, but with better tools. I hope we're moving back to that.

Where the goal is wealth creation for the ultimate owner or sharp ratio periodically, but it's ultimate wealth creation. So does quant equity or does fundamental survive? I think they both survive. What's your modern Olympic analogy? Well, I think about what's happened in Smart Beta as a metaphor. If you look today, let's say you're going to field an Olympic team today. We're well beyond just the pentathlon as a single activity.

In fact, people look at that and say, that's interesting, but they really focus in on the specialization, the sprinters as a group, the hammer throwers as a group, the high hurdlers. And so what an entity that's going to put together an Olympic team today will bring clumps of specialists that have one thing they do well. You don't make the 165-pound sprinter throw a javelin or the 300-pound shot-put player sprint. So what's SmartBeta?

SmartBeta is sort of a decoupling of alpha and beta, but it's actually getting the portfolio exposure to these factors, to these events, with clusters of, if you will, specialists. So we have a defensive equity strategy. We can run a low-wall one. We can run a dividend yield one, which we do. We can run a multi-factor one. And this portfolio construction technique, which trades off diversification benefits of the stocks

and the stock's specialty. So it's a little bit of simplicity. But I think that that stuff is, it's all over. It's not done correctly every place. there are lots of concentrations of risk in the way it's been done historically. You may end up with a whole bunch utility stocks for lowball. But there are better ways to construct the portfolio. And I think there's some room for growth there too.

You mentioned earlier the example of the guy who won't stop talking. He's so excited about research in your office. I'm curious if there's any advice you have for young, let's call them quants, interested in getting into quant finance as practitioners. Any advice that you would give that batch of people today in terms of things to focus on to differentiate themselves, tools to avail themselves of, et cetera. Again, my same team, and including that gentleman you just referred to, we look at machine learning in a certain way, and it can be applied all over. A different engineering problem or driverless cars problem, and there are, I don't know if it's unique to my colleagues, but it's a nice way to think about it.

There's a Venn diagram with three circles. There's an intersection called machine learning. One of those circles is domain, knowledge of the domain. And another circle of the analytics, the algorithms, the computational expertise or acumen, and there's data. So if someone wants to come into this business or any business they want to go in, they got to understand all of those three are important. So I have an acquaintance who runs a big money management firm and asks me, you know, you've got really good quant.

We go to the universities and hire people with advanced degrees and statistics. They don't all work out. Well, they've got that one circle. They know the techniques. Maybe they don't understand the industry, really. And certainly they don't know the quality of the data yet. They have to learn that. So I would say understanding that there's data,

that there are analytics, which you're learning in school, a lot of it. And the data may not be good data you're using because it's commercially available and universities can afford it. And there is, do you really know what a trader does, what makes prices move day to day? You understand dealers. You understand the byside clients. You understand stickiness. You understand whatever cycles. So you're not going to learn that overnight, but just be aware there are those other components. So get exposure. Again, I mentioned my colleague George Masali, who tells his story. He was an undergrad and then he went to MIT ultimately, but he had to decide whether there's going to be a fundamental analyst or a quant.

I think he's a good quant because he spent a lot of time thinking about one fundamentalists to do. So you think about domain knowledge and also data. I love that domain piece as a recommendation for young aspiring analysts. It is so incredibly important because it's like a search cost producer for the manager that's going to hire you. It's just such a hard part of the process that if you can bring that to the table, that's a huge relative advantage, competitive advantage for a person.

When in doubt, just hire a jet pilot and you're done. Good to go. Last couple questions. So the first is, you mentioned Joe Mesrich as an example of kind of what I'm after here. Is it okay to mention people in these? Oh, yeah, absolutely. Encouraged. We want this to be a tool for people to go do their own further research. So my question is specifically on the by side, if there are other quants that you especially admire who you think do really good work, who are worth reading their work, following, et cetera. Probably I don't even know who they are. I'm not necessarily sure it's people that are actually doing it for money. I think there are people that are doing it in other forums, maybe research groups. We have a hard time competing now. I mean, finding talent because they go to Google

or they can go to Amazon. And in Boston or other towns like Montreal is a huge work of AI. There's machine learning. And what used to be an outlet for those, that talent pool has now got more possibilities. I could name some people, but they're my competitors and I don't want to. Good answer. Usually I could trick people with that one. You didn't fall for it. No. And actually, I'm probably not in touch as much as if you ask that question to the CIOs I've mentioned earlier, I think they would have answers. Yeah, yeah. Because they're hanging out more with that group than I am at this point. So my closing question for everybody is for the kindest thing that anyone's ever done for you. Really? Oh, man. Love me unconditionally. That's a big term. That's a religious thing. Yeah. Love me unconditionally beyond any, no doubts. Wonderful. I'm good to go. I'm now free from guilt. That's fantastic. A unique answer. I've done a 135 of these haven't gotten something quite that direct and pure. That's the truth.

So thank you. I can tell you more about that. Great. Well, thank you so much for being here. Well, Patrick was great. Thanks a lot. Appreciate your time. Yeah.

Thank you. Hey, everyone. Patrick here again. To find more episodes of Invest like the best, go to investorfieldguide.com forward slash podcast. If you're a book lover, you can also sign up for my book club at investor fieldguide.com Forward slash book club. After you sign up, we'll receive a full investor curriculum right away. and then three to four suggestions of new books every month.

You can also follow me on Twitter at Patrick underscore Oshag, OSHAG. If you enjoy the show, please leave a quick review for us on iTunes, which will help more people discover Invest Like the Best. Thanks so much for listening.