Invest Like the Best with Patrick O'Shaughnessy - Leigh Drogen - Sink or Swim--How to Combine Quant and Traditional Asset Management Techniques - Invest Like the Best, EP.48]

Episode Date: August 1, 2017

Several weeks ago my conversation with Leigh Drogen on quant investing proved timely and popular--because everyone in asset management is facing the rise of big data, and the use of data science in investing strategies. Because of the rise of quants, many are asking themselves how to survive and thrive in a changing industry. In short, how can traditional managers compete with quants? This second conversation with Leigh was set up to answer many of the questions posed in the first one. If quants are taking over, what should other investors do about it? Leigh proposes a method by which old school asset managers can restructure their thinking and their process to compete with and even beat purely quantitative competitors. The method involves pulling the best from both worlds and combining them into a hybrid structure. But it will be impossible without a wholesale change in mindset, which is where we begin. Please enjoy round two with Leigh Drogen.   Links Referenced Revenge of the Humans Part II: A New Blueprint For Discretionary Management   Show Notes 2:14 – (First question) –  What role will ego and mindset play for traditional hedge funds looking to transition into quantitative investing strategies 4:21 – Describes the traditional process that hedge funds use to make investment decisions and how the internal politics can hamper it 6:08 – What value has portfolio managers played at hedge funds traditionally as the quarterback of a fund 9:57 – A look at what Leigh has seen as he sits with teams 12:20 – A look at places that have tried to simply add quant to their firm’s strategies without “tearing it down to the studs” and properly integrating them into the process 15:00 – Leigh is asked to define the basics of a good investment firm’s strategies 16:57 – Strategies for writing down core beliefs, whether it’s for yourself or your firm 17:49 – Exploring the second step, finding a differentiating view and how to succeed with it. 21:43 – The importance of force ranking and structuring the unstructured 26:14 – Building factor models 29:42 – How the portfolio manager position should have less room for subjectivity than at the analyst level 33:44 – Is anyone integrating this kind of high level data at the portfolio manager level into the decision making the way Leigh describes 35:07 – What blind spots are created by systematizing their processes 36:18 – Why much of this applies more to shorter and structured periods 38:23 – Shifting to portfolio constructions and what Leigh would do to create the right mix 43:39 – Shifting to management structures in these firms starting with the role of the CIO 45:24 – Looking at the different quant roles that exist in a firm and what they should be responsible for; data engineers, data analysts, pure quants, and quantitative engineer 48:20 – If you are an undergrad or grad student right now interested in asset management, what are the roles you should be thinking about targeting 49:25 – Why communication skills are still so important, no matter what role you are in 50:25 – With all of the tools and skills that Leigh has at his disposal at Estimize, why not institute an active strategy 52:01 – What has Leigh observed in the dispersion of skill in the Estimized data set 53:47 – What is the relationship between specialization and accuracy among funds 55:29 – The pros and cons of the generalist 56:56 – A look at Leigh’s background into War Theory and what lessons that he still draws on today 1:00:19 – How the field of study around war and battle relates to the investing world   Learn More For more episodes go to InvestorFieldGuide.com/podcast.  For complete shownotes, go to InvestorFieldGuide.com/leigh.  Sign up for the book club, where you’ll get a full investor curriculum and then 3-4 suggestions every month at InvestorFieldGuide.com/bookclub Follow Patrick on twitter at @patrick_oshag

This podcast is sponsored by CFA Institute, the Global Association of Investment Professionals whose mission is to lead the investment profession by promoting the highest standards of ethics, education, and professional excellence for the ultimate benefit of society. CFA Institute serves a global community of investment professionals working to build an investment industry where investors' interests come first, financial markets function at their best, and economies grow. The chartered financial analyst credential is the most respected and recognized investment management designation in the world.

The views expressed in this podcast do not necessarily represent the views of CFA Institute. Hello and welcome, everyone. I'm Patrick O'Shaughnessy and this is Invest Like the Best. This show is an open-ended exploration of markets, ideas, methods, stories, and of strategies that will help you better invest both your time and your money. You can learn more and stay up to date at investorfieldguide.com. Patrick O'Shaunisee is a principal and portfolio manager at O'Shaunacy Asset Management. All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of O'Shaughnessy asset management.

This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions. Clients of O'Shaunlessy asset management may maintain positions in the securities discussed in this podcast. Several weeks ago, my conversation with Lee Drogan on quant investing proved timely and popular because everyone in asset management is facing the rise of big data and the use of data science in investing strategies. Because of the rise of quants, many are asking themselves how to survive and thrive in this changing industry. In short, how can traditional managers compete with quants? This second conversation with Lee was set up to answer many of the questions posed in the first one. If quants are taking over, what should other investors do about it? Lee proposes a method by which old school asset managers can restructure their thinking and their process to compete with and even beat purely quantitative competitors.

The method involves pulling the best from both worlds and combining them into a hybrid structure. But it will be impossible without a wholesale change in mindset, which is where we begin. Please enjoy this great second round with Lee Drogan. So Lee, round one was us describing sort of the state of quantitative investing and the process, broadly speaking. We're going to get even more specific today, which is to describe how a more traditional discretionary asset management firm, specifically hedge fund, can successfully make this transition from discretionary to at least part quant. And we'll begin with something that is pervasive across this process that

will describe step by step, which is this notion of mindset and ego. So I'll let you start there by describing what role ego will or will not play in this transition for those that are successful and why this needs to be, first and foremost, a shift in mindset versus, say, tactics and strategies. Yes. So I started, it's funny that you bring up tactics and strategies. I started war theory in school. And often, we like to say that war is politics by other means. And if you don't have a political goal, it doesn't matter what tactics and strategies you use to fight the war or to fight the battle, you're going to lose in the end. And as we've seen, you know, Americans in Afghanistan, stand like we've used all the right tactics and strategy, but we have no political, you know,

outcome that we actually are attempting to affect. And in the kind of long, only and long short hedge fund equities world, it's very much similar today. So you're seeing all of these funds wake up and attempt to use tactics and strategies, but really have no kind of, the political aspect has not really been thought through yet. And I think they're doing it all backwards. And the political aspect is mostly the ego associated with PMs having control over the investment process and final decision making. And things flowing up to the PM instead of the PM basically just being like the offensive coordinator on a football team. That is the shift that needs to take place. And look, these guys make a lot of money.

And even more, like the power associated with it, I think is an important thing for them. And in order to make this shift, they're going to have to give some of that up. Let's level set by talking about where it is now. So describe the, to the extent there is a process, describe the traditional process between researchers, analysts, sort of the hierarchy that exists now, how decisions get made, and then we'll go from there into what it maybe should look like. Every fund is going to be a little bit different and you're going to get differences between big, long, only funds and, you know, smaller, long, short funds.

but generally what you've got is these analysts who will be given a sector industry and they'll be told go cover 40, 50 stocks. And you'll have a PM, maybe at, like, let's say our hypothetical firm, there's 15 analysts and there's, you know, three or four PMs. The analysts will then come to the PM with an idea and say, I think this thing's worth X, a year or two from now or whatever. and here's the big 10-page research report. Why?

Please go read it, and then they'll have some kind of discussion, and maybe the PM will put it into action, and maybe they won't, and there's this whole kind of politics process of getting your ideas implemented. And the issue is that a good analyst isn't necessarily a good politician within a firm, but these two things have been, unfortunately, combined,

and so you get really good analysts with really good different. ideas that may be very accurate, but they just can't sell it. And then the PMS don't use it. And then it's very hard to judge the accuracy of the analysts and how good they are relative to how the PMS use it. And the whole system does not align itself towards understanding how good are we, when are we good, when are we bad? When should we listen to that person? When should we not listen to that person? A good analogy that you've used already is this notion of PM as offensive coordinator versus PM as quarterback, so kind of calling versus making the plays.

And that's going to be an important part of the discussion, which is how have PMs traditionally added value and how should they in the future, assuming they migrate to this more quantitative or man plus machine type model. So talk a little bit about that in the past, in the more we'll call it the traditional model, where there's information flowing up to a PM. Again, we're speaking in generics and there's obviously an exception to everything we're going to say. But talk about that process in terms of how they've added value, quote unquote, historically, whether it's diving deeper into the names or just having a nose for, you know, more top-down type stuff. Talk about kind of the flavor of the quarterback version of portfolio managers. Yeah, I mean, the quarterback version is they're basically deciding on everything from position sizing to market timing to risk management, and they're basically relying on the analyst to just

give them kind of, you know, the fundamental view and thesis. That puts a lot of different variables in the PM's hands and they're juggling a zillion different things and how could you possibly be good at all of those things? I think, you know, while we're going to talk about, you know, maybe some of the beginning processes of idea generation and stock selection, the one that I think is just even more egregious is the portfolio optimization part of it. Computers do this much better than humans and God knows a human should not be deciding how to weight their portfolio. And just like the very stark example is you could be generating alpha but have negative returns because you're exposed to betas and the PM be like well I you know like I've done everything right like why

why is my portfolio not performing well well it's it's you know because you didn't you should have optimized to not be exposed to certain betas or if you wanted to be exposed to them like the machine should be doing that for you and so they're they're literally just juggling too much they're having a macro view of the whole thing from the top down. And then they're literally overriding the best ideas, on average, from the individual analysts, because inherently, an analyst's idea that is very differentiated should be controversial. And we know that in a study of venture investing, there's some really good funds like first round capital that have looked at the variables associated with making a good investment. And in many ways, when you look at like deep value

investing in this same process, it's exactly the same. Union Square Ventures has also done a study internally of this kind of stuff where they find that when they go into the partner meeting with an idea, what they want to see is a barbell of, I think that's an awesome idea and you're freaking crazy. Like, why would we ever do that? They find that their worst investments are the ones where everybody is like, oh yeah, that makes total sense. You do not want to be making those investments. And the problem in our world, long short and long only investing is that ideas that are like, oh yeah, that makes total sense are usually the ones that get put into the portfolio, the ones that are really controversial, which is where you really should be investing,

you know, they get thrown out by the PM. Yeah, this is, this is something that I have to remind myself all the time, which is that consensus and alpha do not mix. Exactly. Yeah. If you are in consensus, you are not generating alpha. Like, by definition. My favorite part of the conversation with Jerry Newman, who's a venture capital guy here in New York, was that he would log on to some, I think it might have even been Union Square Ventures website and like look at the portfolio and just think, this is crazy. Like, what are they doing? And you look at the funds IRA and it's like 65% or something outrageous like that. Fred Wilson will say he knows he's made a great investment when other people call him up and say, why the hell did you do that? So talk a little bit about your

recent experience where you sit in a very neat seat, which is you're not necessarily a competitor to, you're certainly not a competitor to your clients and the kind of hedge fund landscape, but you get an intimate kind of look into the structure of the firms with an eye towards helping them. So tell a couple stories about recent experiences where you're sitting with teams and sort of what you've seen and what they've been asking for. The last couple of years, as we've scaled out, distributing the estimated earnings estimate data to these long short and long only funds, inherently we sit at a really interesting part of their investment process of them judging their own estimates against

ours, of them looking at ours as certain signals to how stocks might perform. And I'll go into these funds and sit down and maybe it'll be like the CIO and a PM and a couple of analysts and a technology guy and maybe another person or two. And I don't go in there to like rip their whole thing apart. It's not my job. I go in there to show them how they can use our stuff to make better decisions. And then inevitably they end up asking, you know, other questions around that. But recently I've been in some of these funds and one of them, I sat down with a group similar to that. It was a, you know, $20 billion fund. And we got 15 minutes into the conversation. And one of them just turns to me and it's like, could you please actually rip our whole process down to the studs?

And I'm like, this is not what I'm here for. And it becomes obvious when you ask all these kind of questions as good, you know, I act as the salesperson in these conversations. Good salesperson should ask a lot of questions to understand how would you use it, you know, what's your process? But when you start asking all these questions about what's your process so that I can tell you where our stuff fits into it, a lot of them kind of just like wince and go, well, you know, like the analysts bring the, you know, the ideas and like then we discuss it. And I'm like, well, do you have a meeting that this is discussed? And they're like, no, not really. It happens ad hoc, and you can just see their faces. I don't have to say it. They realize we have to get,

we have to get, we have to get this together. And then they inevitably say, well, how would you do it? And I'm like, that's not why I'm here, guys. I'm glad to try and find you somebody to like go consult for you on this. But that's also why I wrote that, you know, the follow up piece to the first one so that I didn't have to go into every one of these funds and kind of try and teach them how I would, how I would do it or how I see other firms who are successful at certain parts of it doing it. Yeah, so I hear all the time, especially when presenting a purely quantitative strategy, often one of the first questions is, is there some sort of fundamental overlay? It's always that term overlay.

So the mindset is often that this is a two-tiered process, whatever direction that might be in. So you run a quant screen and narrow your universe and then you pick from within that, or you do the work first and then a quant screen sits on top. And I think really the key insight here is how difficult it is to integrate those two mindsets together. So I'd love for you to talk again about that mindset. You mentioned that term tear down to the studs. And I think that's really important, right? If you're going to emerge with a process that successfully integrates pure quantitative methods, data science, etc., that it can't be something that you bolt on.

So talk about people that have bolted on and maybe why that's gone wrong and then we'll get actually into the step-by-step process. Yeah. So I think a lot of these funds that have woken up first, the easiest thing to do, was to say let's go hire five data scientists and an engineer and then let's go buy a whole bunch of unique data and then let's go throw it at the PM at whatever part in the process we feel whatever and this simply doesn't work because yeah these are two different worlds two different ways of thinking the PM won't pay attention to it there are some firms that have tried to do this on a larger scale and get the PMs to actually pay for it like through P&L as if the internal team was

almost like a sell-side shop, and that hasn't worked, obviously. But the reason they're doing this is because you don't have to rock the boat politically then. These guys sit over here, those other guys sit over here, and neither the two shall meet until some work product gets handed over. And this is obviously not working. And the scary part of it is we're starting to see these firms, a couple of them retrench because a couple years down the road after trying this, everybody looks around and is like, is anybody using it? And then everybody looks at this cost center over here and says, why are we doing this? And it might actually, like, they've taken a step forward,

but they might actually take two steps back at this point because, you know, they did it wrong. And I think that's going to retard the industry a little bit, the fact that they tried to take a shortcut here because the buzzwords were big data and data science and all these things instead of like, let's actually look at the process by which we make investments in our culture, you know?

Like, what do we value in terms of, terms of how we should act. Part of the nice thing about that idea, the visual of bringing it down to the studs is the studs themselves. So let's go there, which is what the studs, I think, represent being kind of core beliefs about the universe and the universe itself. Like, where are we going to hunt? We can't hunt everywhere, probably.

Maybe some firms do that are massive. But talk about defining the studs themselves and why that is sort of the first step in either this building or rebuilding process. Yeah, one of these firms that I was in talking to, they literally couldn't eludicate the universe of names and the type of investing strategy that they employed. They were like, oh, yeah, we kind of do everything. It's like when an analyst comes up with a good idea. So it's a couple of things. What is the market cap that you're going after?

Like, are you a small and midcap investor? Are you a large cap investor? What industries and sectors do you believe that you have really good analysts in where you can analyze things and have an edge? do you believe in growth and momentum or do you believe in value or some other characteristic that you have a serious belief that you want to align your portfolio towards? I personally was taught how to be a momentum investor, right? You want to buy high and sell higher. You want to use relative strength in your favor, and then you want to get the hell out of there

before margins collapse at the top and then the multiple collapses. You want to ride the multiple up instead of trying to catch a multiple at its bottom. That was my philosophy. God knows there are a lot of different philosophies that work perfectly. I think the problem is a lot of these firms don't even know what their core philosophical beliefs are. And you can't build the rest of the process without that because the rest of the process is really, it's aligned to those things. If you believe in momentum and growth and relative strength and stuff like that, you're going to look at all sorts of different factors and variables and different

universe of names than if you believe in value and all those other things. I've actually been going through this exercise myself, not just in investing, but just more broadly speaking, trying to come up with a set of core beliefs or guiding principles. And it's incredibly hard to do. Like to actually put pen on paper, which I think is the right way to do this, and codify core beliefs, is really freaking hard. Are there any suggestions that you have for going through that process itself? Like maybe you've done it. It's like writing a constitution. The firm needs to sit down and write a constitution. And this is, we talk a lot, historically about this idea of like style drift and how managers across the board perform poorly

when they drift from their stated style. Every firm should have a stated style. And we know that when you have a specific strategy and you try and take trades outside of that strategy, those trades are going to be your biggest like losers, mostly because you don't know how to manage risk under those scenarios. And when you don't know how to manage risk, those are going to be the long tail of your bad returns. So assuming that people can sit down, identify a universe where have strengths, identify some core idea of how markets work and, or at least how they can operate in a way that has some sort of edge. So then step two is, okay, ride that edge. So develop sort of a differentiated view versus the market back to that consensus idea. So talk about that

second step and what's important there. So this is where I think firms need to get much more systematized in how they run the process of having analysts come up with ideas. Historically, there's some software like Code Red and Tamale, and some of these have been bought by like a faxed or an advent. And it's the system where the analysts will basically go in and put notes about things. And some funds like Citadel, even 0.72, have kind of these internal estimate capture systems. Nobody really uses it because they're terrible pieces of software. And they know it too. Like, this is not a secret at all.

And so they can't really get the whole firm to actually give them structured views on expectations. And so my belief is that they want to and they know it's what they should be doing and they want to try to do this. And the way that I'd like to see firm set this up is basically analysts have to give a centralized database a forward-looking view on EPS revenue, EBITDA, and then like whatever key performance indicators the company is really kind of bound to. So Netflix would be subscriber growth numbers and Starbucks would be same sort of sales, whatever it is. Give a forward year of estimates quarterly or two forward years or whatever your time frame aligns to. And time frame is another one of those like core beliefs.

Many firms say, oh yeah, we're long-term investors. And then of course, like, you know, they're in and out of the stock in a quarter. And they don't give their idea the time. And I think the reason why this happens is because if, if you don't have all the structured variables that add up to why you believe in a long-term thesis, you will cut and run much earlier before letting it actually play out because it's not actually put down on paper there. So the analysts on a regular interval probably three times a quarter have to update their forward-looking estimates. And then what they should be doing is, and this is my core kind of view on what fundamental investing is,

you have some fundamental factor, let's say EPS, you have a forward one-year EPS estimate. Let's say for Apple, it's, I don't know what it's, four bucks or five bucks, whatever. Apple trades it 15 times earnings now. You believe that if you're right, it'll trade it 18 times earnings a year from now. You multiply that multiple by your forward-looking estimate. That's your price target. And so three times a quarter, the PMs are going to get an updated view on all of your stocks. Let's say you cover 30 or 40 names as an analyst.

The PMs are going to know what is your forward one-year price target? What is the markets forward one-year price target, which is basically the consensus E times some terminal multiple that it's trading at now? And they're going to see where are the biggest deltas between your ideas and the market's consensus. And then once we have all of that data from all of our analysts, we can do a lot of different things with that. But the point is that instead of the analysts coming to the PM whenever they feel it's

necessary, it's incumbent upon the PM to then run the process of taking all of the analyst structured ideas and dealing with them. And that will remove, hopefully, a lot of the politics involved in when the analyst actually shares an idea, because sometimes they get pissed off at the PM, and they're like, well, I don't want to share my best ideas with you. I'm just going to, like, save them in this like Excel sheet over here on Google Docs so that when I go to another firm, I can like say, well, these were all my real ideas, but my PM screwed it all up. You don't want that. And so, in the... in this way, you would have a centralized kind of database of these views.

One of the key ideas here, which is just hard. It's a mindset thing, again, is this notion of structuring the unstructured, that a lot of this is gray area stuff and long reports, and it's not objective data points. You've talked a lot in the past about the power of just force-ranking stuff. It doesn't need to be, the magnitude may not matter, but at least get something on paper that you can track to be able to look back and evaluate performance,

evaluate effectiveness, maybe change your own methods. think that's really key. So let's assume that there's some way, and maybe this is a big assumption, which is that there's some efficient way of structuring the unstructured that's not just like an Excel spreadsheet. And maybe that, you know, someone out there listening could build a very lucrative software company. Yeah, absolutely. If they do this for hedge funds. It's something like, you know, you take something like Tamale or Code Red, and it would have been nice to have them literally put in fields where you could say, okay, my field is corporate governance. And, And each time I go in there to like update my whatever, I have to rate corporate governance for this

company on a one to ten scale. Well, if we collect that for several years, we can go back and say, was corporate governance or my expectation of what corporate governance was, it was that in any way correlated to volatility or earnings volatility or misses and beats or stock performance. And if we collect that on a set of maybe 10 different variables for each stock that the analysts, you know, these kind of unstructured concepts, what's the, what's the, what's the probability that the company gets bought out this quarter, you know, stuff like that. You just have so much more data to deal with. And that's the stuff you hand over to the quant. That's where the

quant starts to really like get involved. But you have to have this stuff structured. You can't just have, and I've seen funds try and take all of the unstructured text that their analysts were putting into Tamale and Kodrad and stuff like this and have and have their quants go through the semantic analysis of it. And it's like you are trying so hard. hard to do the wrong thing. You're putting so much effort into into the wrong thing. And it's really, yeah, it's just really funny. Just a point here, because it's something I've thought a lot about. And we're talking about systematizing things and that, you know, systems in general are good in the sense. But a key point I think about systems is that simple systems are better than

complex ones. It's the beauty and the elegance of a, of a force ranking one through X versus like price targets, let's say. There's something much better about keeping things. simple because if you try to drop, and there's a million examples of this, if you try to drop a complex system in like a design, top down design complex system, it will fail. And we've seen this as as pure quants a lot. It's why often equiluating trumps everything else. It's why force ranking works so well. And it also makes it more accessible as you're trying to make this transition from kind of shoot from the hip stock picking to a more systematic process. It's more of a baby step, right? You're not asked to input a thousand things or have false precision is a big problem when you take a traditional

mindset and try to make it quantified, I think false precision is incredibly dangerous. So I would highly recommend people think in terms of system, yes, but simple systems at the beginning trump complex ones. And then once you figure out what works within a simple system, then you can push a little harder and maybe make it more complex, but start with really, really simple notions. Financial people love complex things because they believe. if it's more complex, it must be better. And it's just simply not true. It's also a marketing thing. Oh, we're doing all these really complex things. Give us money because we're the only ones who can do it or you know, something like that. And there's only a handful of funds in the world where that is actually true.

Your rentex of the world. The rest of us should not attempt to go through that. It's, yeah. So to take stock, so we've got core beliefs, guiding principles. We've got this notion of structuring the unstructured by taking whatever it is, and at least entering it into a very simple database. So now we've got data. It's subjectively derived data, but that's okay. It's not even that it's okay. That is the core of generating alpha. Is the subjective nature of your forward-looking expectations being differentiated from the rest of the world? So ideally the process through which those subjective inputs are derived also itself is somewhat systematic or process-oriented. Should be. Hopefully, Yeah, you're looking at the same data each quarter.

You're doing channel checks. You're talking to customers, whatever, yeah. So now we've got data. So now we really start to get into our wheelhouse, which is actually building factor models. So talk about this crucial step. And this is really where, you know, instead of hiring a quant team to come sit in their own office and occasionally serve up, you know, a research report, this is where those skill sets become really interesting.

So talk about building factor model. Yeah. So this is where the unique data comes in. this is where the quants get hired. Quants don't enter the process really at all up until this point. There are data engineers that enter the process before this point and data analysts, and they help the fundamental analysts get a hold of any data that they may feel is necessary to make those forward-looking expectations.

But on the other side, there should be a set of quants that basically work with the PMs to say, okay, what is it about this industry or this sector that, we believe all of your names share. And so we know that in technology, which is a growth industry, that history surprise of, you know, analyst estimates is a factor that is correlated to future momentum and outperformance in stocks. And so what the quants will do is they'll talk to the PM and they'll talk to the data analyst and they will take a data set and they will create a factor model, which is basically a Z score. You know, we rank all the stocks in our our sector. So let's say there's 300 of them. Just to find what the Z score is. Z score is basically the

relative ranking of, you know, let's say one stock to the next based on some factor. So if we're talking about history surprise, if there's 10 stocks and two of them always miss and two of them always beat, the two that beat are going to get a 90 or 100 Z score and the two that always miss are going to get a negative 90 and negative 100 Z score. And then it's a normal distribution. So, or it's not a normal distribution, but it's a normal. It's a normal. It's a normalized distribution. It's a number of standard deviations away from the mean that any company is. So if a company is, I don't know, has an EPS growth rate of 50% and the average is 10 and the standard deviation is 20, it's 50 minus 10 to bite up a 20.

The point being, we're trying to normalize a range of, you know, of scores. So the quant is going to build a factor model that they can put into a system. And they're going to build not just one factor model, but maybe 10 of them. And the idea is that when the PM goes to look at all of their analyst expectations, kind of sorted by the largest kind of outlier expectations, they're also going to be able to see what do the factor model scores say? And the idea here is that while the analyst's, you know, long-term year or two-year-out expectation is going to be whatever it is, the factor models might give better insight.

into timing, position sizing, risk management. Should we actually invest in this name right now? Or does the post-earnings drift model from Estimize say we should hold off for five days? And that's where the PM really plays offensive coordinator and looks at all the different information, looks at the satellite photos, you know, talks to his quarterback on the ground, and they're going to be coordinating and this is where they add alpha. PM shouldn't be trying to add alpha in the what stock should we pick realm. they should be adding alpha in the how do I run a better process to incorporate all this different

data and make everything work together well. It seems like there's actually, you tell me, at the PM level, less room for subjectivity than at the analyst level. Should be, yeah. So talk about how that might marginalize that very position, right? It sounds more and more of that sounds like it could be replicated by a simple, you know, piece of software. See, I think that the PM still needs to exist and they need to.

to exist specifically for the purpose of software is not good at seeing outlier things. And so let's say you're just dealing with 10 names that the analyst ranks for their forward-looking expectations. And you've got your two that are really big positive delta outliers. Those are the analysts' two best long ideas. The PM is going to look at that and they're going to say, hey, like, I have experience in this name. I really wonder why the analyst thinks that the earnings are going to be so much different than the

market does. And they're going to go talk to the analysts and they're going to say, have you thought about these three things? And maybe the analyst hasn't. And they're almost playing coach at that point. And it doesn't mean that they should overrule the analyst. It doesn't, it doesn't mean they should throw it out. But maybe the analyst goes back and, you know, kind of has another conversation with himself or looks at more data. It's a process that the PM has to drive. The other part of it is that this is inherently an interpersonal kind of process. So I think that the core skill set of the PM has to go from being a stock picker to basically being a coach. They have to have a much better set of interpersonal skills.

And frankly, I have to say that women probably should rule this part of the process. They're simply better at this. They have less ego. and they have less irrational behavior when it comes to, especially, you know, in professional environments, working with people in a very egotistically driven, you know, environment. It's about data and they have to follow it. Now, the interesting thing about having the PM move into that role is that I actually think that you can now measure the alpha that both sets of people are generating.

Let's say you have your ranking of the stocks by Delta, both ups and you. side and downside that the analyst is giving you and they're giving it to you on a regular basis, you could theoretically say, I'm going to create a virtual portfolio equally weighted of that analyst's top five long picks and top five short picks. And I can judge the alpha that the PM is creating by how much they deviated from those picks. And did they do better than the virtual portfolio or worse when the actual returns come out? If they did worse, fire the PM, right? Like, just go with the analyst model. And if the analysts are bad, fire the analyst, right? They're giving you bad inputs for the PM to deal with. Right now, you have no clue who's adding value in this process. The other cool thing there is

that there's two layers of that, right? So assuming there's multiple analysts covering different lists of stocks, you can do it at the individual analyst level. So you can add value there as a PM, but then you can also be a risk allocator to which analysts. So you might be just equating or taking their exact picks, but if you give one twice as much money as the other, you know, that could be a key role. So this is the other kind of quantitative aspect to the PM's role, which is if we have all of the analyst picks and forward-looking expectations, the fundamental view, the multiple view, all those other kind of unstructured variables that we've worked with, we can actually say, this analyst has been really good at these types of stocks and these types of scenarios, and we can put

really easy confidence intervals next to their picks on that dashboard so that when the PM looks at the fact that like this is the biggest upside outlier pick but like the confidence interval is really low for them to be accurate they can go back to the analyst and be like look I know that this is one of your names like I need a lot more from you here to get confident in this because statistically you're not very good on this and like I might not put it into the portfolio to this point we've described so far is anyone actually doing this like is anyone this is like the platonic version of a want and fundamental integration. Is there anyone on the planet that you're aware of that's actually doing this sort of

to a T? Not the whole thing. There are some firms doing pieces of it. They're collecting different pieces of information. But I think, and this is really where I think somebody needs to build a piece of software, which we'll talk about in a second, to actually run the central nature of the process. So people have given software to the analysts to fill out expectations, a laestimized in a structured way, Code Red and others in the unstructured way.

People have built stuff like Omega Point, which will be the totally end of the process where it structures the portfolio once you've picked the stocks, like what should be weighted and underweight and overweighted and overweighted. But nobody's really built a piece of software either internally or externally from these funds that actually says, you're going to review this on a Monday meeting every week. Here's the names sorted high to low. here's all the factor models, red light, green light, or the factor scores. Here's the confidence intervals of the analysts.

How often have they been accurate when they have these views? I think you need a centralized piece of software to run that, or else it'll be two ad hoc and the PM. You let everybody go off in different directions. Something needs to be the center of that. I'll play devil's advocate a little bit here, and everyone will know that this obviously is not my view, but I'll ask the question anyway, which is, what are the, let's assume Affirm does this, what blind spots does it create? So what, what negative, nothing's perfect. So what are the

negatives of, let's say, this perfect piece of software that totally systematizes the whole process? What are firm's going to lose out on by doing this? One time ideas, kind of, you know, your special sits. This is not a special sit system. My own, you know, kind of personal view on special sits is, you're basically gambling in a sense. Unless you are, unless you don't have to take that many bets. Soros is in a sense a special sit person fund whatever, right? Like he, his back hurts. Yeah, like he, yeah, like when he feels, you know, that the temperature drops and his back

hurts, right? He comes back every two or three years, has one great idea and like makes a ton of money. You're not Soros. Like I'm not Soros and, you know, people out there aren't going to be Soros. And we're also not Buffett. Like, we don't have enough capital to use the capital as the advantage. I want to pause for a minute here, especially with the reference to Buffett, because it's important to point out a key caveat to all this, which is this, we're kind of talking about one competitive,

I don't know if corner, corner sounds too isolating, but one competitive part of the market. And timeline, you mentioned duration or investment horizon or whatever earlier, is a really key idea here because a lot of this applies much more to, I would say, shorter and more structured periods. We're talking weeks to two years. That's right. Three years. So if you are an investor or an allocator or anyone interested where the differentiated

view back to that idea is much more long term. And I would be very curious to hear what factors you think are relevant there. I've always wondered this. I've heard people say, you know, corporate culture could be something. And that's where hopefully you would be able to develop some structured variables and factor models that would give you an idea of, yeah, what are those things? And they're going to be different from the weeks to months to a year or two ones, yeah. The idea is still apply, but maybe just less so. So something like culture is going to change a lot less frequently than an EPS estimate. Or a view on an industry as a whole, right? Right, right, exactly. Like I like I robot here, not because I really love the company, but just because it's a story stock in an industry that is going nuts. And that, That wouldn't be a like three month pick for me. It would be like a five year pick. One of the things, the exercises that I went through literally just for fun,

so we're purely quantitative. But I love reading 10Ks, right, especially from companies that are very consistently coming through models. And it's fun to go through and literally just pick a stupid simple system, like one through five. Does this seem like it's a good buy right now relative to the price? And that exercise can be really, really interesting.

And even if you're a pure quant, sort of informed future research or ideas that we talked to the first time about the importance of creativity here in designing the hypotheses first and foremost. So structuring like any part of the process can lead to interesting outcomes. Now I would like to talk about portfolio construction specifically. And you mentioned it a few times, which is optimization, exposure to risk factors. These are things that are better handled by software, but may still also require, you know, PM's involvement. So talk about sort of what your platonic portfolio construction slash optimization slash risk management system would look like. Yeah. So we're just at the beginning of this.

Axioma has a, you know, a decent product. There are a couple other big shops that have, you know, OK products. There's a smaller startup called Omega Point that has, I think, a good one. The idea here is that you can generate alpha, but I use the example of, Tesla and Microsoft. So if you invested in Microsoft over the last six months, you created a positive return, but almost all of that positive return is beta. There really was no alpha in Microsoft. If you invested in Tesla over the last six months, it was almost all alpha because Tesla is not correlated really in any way to the broader market or momentum or any of those other kind of like main factors. Just to be sure everyone understands, can you just describe like the, basically the math

behind what you just said. So how do you decide why is Microsoft just beta? Why is Tesla more alpha? Yeah, it would be like what is the return of the stock outside of the return of like a portfolio of momentum names or yeah, in the case of like Tesla, was it all momentum? No, it wasn't or was it all value. Obviously it was value. But Microsoft was basically like there was some value in there. There was also some like general market, you know, beta like to the S&P exposure. It was it was stuff like where if you wanted Microsoft, you could have just gone invested in like a smart beta ETF and gotten the same thing. Yeah, this is a really interesting development in this whole world where for a long time,

Alpha was knowing about value momentum. Now maybe it's choosing or being patient in them. So there's the patience component too. But what's interesting is that the appetite now is a lot for, certainly in the head, really specifically in the hedge fund space, pure idiosyncratic risk, which sounds bad, but really means something truly uncorrelated to what have become very cheap sources of market exposure. I don't have a complete disbelief in leveraging beta. I think there are times that the PM should have a bit of a macro view on where the market is.

I remember, you know, my mentor, David Geller, he certainly had an overarching, like he would look at new highs, new lows. He would look at breadth as indicators of, you know, directionality of the indecis. And we would take exposure on the long and short side up and down, you know, relative to that on a larger macro scale for the book. I think this is where PMs should focus more on the unstructured nature of decision making. Is that like how exposed do we want to be at different times? Again, this is the alpha that they're adding. It's not the pure ideas. It's the like construction.

most of the construction should be done by the machine at this point, these systems. And what the PM can do is look at the system and say, okay, the system has completely beta neutralized all of my position. So my book is like beta neutral. I'm just going to, you know, generate alpha theoretically based on all the like, you know, historical betas. But I really think, you know, momentum's gotten hammered here over the last couple of months. And I think that we're going to have a really good earnings season and, you know, some of these other factors. and I think momentum is going to make a comeback, and they can go into the system and say,

I want to ratchet up the exposure to momentum here. And I think that's a good thing that they can do this. And look, they're going to get measured. And every year, we're going to look at the amount of alpha they created by toggling things up and down. And they're either going to get fired or they're not based on how good they are. But the machine should do most of the portfolio management. The other part that the PM can do is sit there and say,

okay, my analysts have this really big outlier long, pick and all the factor model scores like line up and they're bright green, I want to size up on this thing right now. And they can take an outlier large position and swing for the fences. And this is where I really think the machine and man kind of convergence allows the discretionary firms to potentially put up better returns than the quons. How do you, though, account for what, so we talked about how to evaluate a PM, right, which is to say, okay, versus as an equal weighted allocation to the analyst long and short picks, what value did they add or subtract, how do you count for the fact that these kind of big fat swings represent typically a

very small sample size? Like, how do you draw in, maybe it's, maybe there's just no way, right? Yeah, there's, I mean, it's, look, that's, that's supposedly the value that the PM is supposed to add, or else, why don't we just have a quant, like, why don't we just be quants and have the analysts be the human input to the quant model, right? Like, it's, it's one or the other. And the PM, can't get away with it. Like at this point, you either hold the PM's feet to the fire and measure like that or, well, I just don't have a PM, just have a quant run it. One of the things we haven't talked about is the role of a CIO. In a lot of the stories you've told me offline, a lot of these meetings you've had that have led to this piece that you wrote and this very conversation, the CIO's in

the room. So this would be a good switching point into us discussing organizational structure and sort of strengths and weaknesses, the roles that there should be, what, let's say hiring managers should look for in those roles, what allocators should look for when doing due diligence and so on. We'll start with the CIO. So what do you view as the role of a really good CIO as it relates to a team of PMS? Yeah, I mean, I think the CIO at this point has to be the one with the strongest backbone because they're going to have to stand up to the PMS. And in many ways, you know, the PMS are your firm right now. I really think the analyst should be your firm and not the PMs. The CIO is going to have to expend political capital and piss some people off.

Now, my view is that PMs are mostly mercenaries anyway, and so I think that in the future, PMs are going to want to work at the firms where the CIOs have a good grasp on this process, because you might have 15, 20 analysts at a firm, or if you're at one of these really big long-only funds like, you know, Fidelity or Alliance Bernstein or, you know, whatever it is, you're going to have a lot more analysts than that and a lot more PMs. And if I was a PM, I would want to go to a firm that has a CIO that is willing to have an iron fist grip on this process and actually willing to put, you know, these processes in place so that I can get measured better. And that I believe that it's fair what I'm getting in terms of comp relative to the other PMs. And I think that has to, that has to

happen at this point. So let's go down the stack then. So obviously we've already talked about CIO portfolio manager analyst. That's pretty clear. Let's talk about the quant roles that exist, both on the data and the research side. I'll just let you outline kind of who they are, what they do, who do you hire first? Because this is the nitty gritty of we are into tactics a little bit now, but that's okay. So talk about the various quant roles, quant and data roles. Yeah. So I think you need to start with the data engineers because they're actually the first ones that are going to be flowing data into the analyst's kind of forward-looking decision-making process. And so these are classic kind of engineers that understand how to deal with data.

They're not all that expensive. Then you need this data analyst. And this is actually, I think, the hardest position to find because it's such a interdisciplinary set of skills. You have basically an understanding of being an analyst, first off, what drives stocks and the models that the analysts are building and all that. But you also have to have some technical skills, be able to talk. quant with the quants and have some technical skills you know regarding being able to manipulate data in helping the analysts out these people are few and far between but they're you know there's more and more of them these days you think about

Matt Ober who went to third point who is technically the the head of quantitative something something over there or the head of data science but but really even though he runs the whole group and there's somebody who needs to run this group he really acts as that data analyst for all the different, you know, PMs and analysts over there. And there's a reason why Third Point is up so much this year. And some of those things have to do, and I know this directly, like have to do with Matt and what he's done over there.

So there's the data analyst. Then you have your pure quant. And these are the guys that are literally taking the data from the data analysts and doing research, like pure research into factor models. And helping the analyst with like, yes, like this data set that I got from this credit card company is correlated to EPS and revenue when the company reports, you should use it. And then you have like the most expensive one, which is your quantitative engineers. And these are the guys that need to have a quant background, but also a deep engineering

background. And they're going to be actually creating the factor models. They may also work with external companies, you know, like ours to integrate other factor models from third parties. But they're really expensive right now because they could go work basically anywhere. So there just aren't enough of these people and let this serve maybe as some career or job advice. One of the interesting things is when you find someone that can span or kind straddle two different worlds and translate between the two, that's an incredibly valuable skill set. My interns this summer from Wharton, like that I'm telling and they have these quantitative skills and then a couple of them also have these like really good interpersonal skills

and I'm like, yes, go study both and become the center of that. Let's go that direction a little bit more. So if you were an undergrad listening or a grad student listening and you're interested in asset management in general, and obviously this would depend on a person's inherent skill sets, but are there, is there one or more avenue that you think has the biggest like supply demand imbalance in the kind of long-only and hedge fund world right now? Right now, there's just too many econ and polyside people. And then they're trying to hire all these pure data science people that know nothing about finance. And the problem is, like, you really need somebody who knows both. So we have an intern this summer from Chicago, University of Chicago, and he's studying, you know, econ and compsci.

And I'm desperately trying to get him to also add some polysyc courses. Because if you can't write and you can't communicate an idea, you won't, you won't be able to be the center of that kind of thing. And he doesn't want to be pure quant. And he also doesn't want to be a pure analyst. Like he would be perfect for this role. But you also have to have that skill set. It's amazing how I was thinking about this the other day that in any given hierarchy,

the higher up you go, the more communication matters. You know, you mentioned backbone and things like that matter too. But it really is amazing. Like it's rare to see someone, even at the top of a really wonky, say, quantitative stack. that isn't a pretty good communicator. Look at Ross Garron at Cubist. I mean, he's one of the best people in the industry, and there's a reason his fund does so well.

It's, you know, he's a great quant first, but he's also just generally a great manager and communicator in person. And you're just, you're not going to find, especially on the discretionary side, firm's work, it's not going to be a pure quant at the top, I guarantee it. Like, it's going to be somebody with an interdisciplinary skill set. It's also encouraging that that's a skill set that you really can't, like almost anyone can improve a lot.

in that skill set. You know, you can go read books on writing, on communication that really do actually work. It makes me think of, back to Estimize again. And this will just be kind of a fun tangent, which is, so you've got the ingredients for all this, which is the subjective inputs, the differentiated inputs, the ability to track which subjective inputs are good at which names by what degree, and a software mindset. So it just begs the question, why not turn this into an active strategy? Yeah.

I would love to someday go back to running money. I'm jonesing for it. But I view our next step in the evolution of what we do at Estimized as adding the multiple, which we're about to do, being able to aggregate up to full price target. Of course, we already have the big statistical engine that measures all these things and spits out confidence scores and intervals and stuff like that. We can already build factor models, which we do. But I want to give these firms the centralized dashboard now. And that's kind of our next big, probably two to three years of this company, is trying to be the center of that process for these firms. And it's obviously a big addressable market, which we're excited about.

But more, I would really just intellectually, I want to see if I can help solve this problem. Because if you can help solve this problem, you're going to get two things. Like some of these firms, and I really honestly think it's only 20 to 30 percent of them are going to survive the shift. but you're going to get a bunch of these people from the old firms, creating new ones, and starting fresh with a process. And I think those are the people who are really going to win. And I'm excited to be friends and colleagues in some sense with those people who attempt to do that and be a big part of their success at the end of the day. Can you share it all of some ideas around what you've observed in the dispersion of skill within the estimated data?

set. So I'm always curious to hear like, okay, so of course it's intuitive that, you know, one person's going to be better than the next and better than the next. But like how tightly, how narrow is the most skill group? Like it is the best two or three guys orders of magnitude better than, you know, the next five guys than the next 10 guys. Talk a little bit about like the distribution of skill that you've observed as your data sets gotten bigger. We were really interested in this from the beginning, most of it just because my co-founder and I are data nerds to begin with, and this was, in a sense, an academic experiment. We were not surprised to find that it is a normal distribution of skill. Although what happens, and this is one of the reasons why the data set is so good, is

the people on the left side of the distribution who suck tend to show much more attrition at a rapid rate than the people obviously on the right side, which you really want to keep around. theoretically this is what you should want at your firm as well, right? You should measure and then get rid of the bottom 20 or 30 percent like every single whatever interval. In terms of how much better, I don't really have like the pure statistics around that, but I can say we were a little bit surprised at the persistence of who was better and who was worse. Analysts who come in are persistently better than the ones who are kind of more random right in the middle. Yeah, that was going to be my next question. Like to what degree can you parse out luck versus skill? Really well. More than we thought

we could. We thought that this was a more random environment than it is. And what about specializations? So do you find more accuracy among people who only enter, say, estimates for one sector versus more generalists? We skipped over this concept of generalists, which I know you have a strong opinion. So I'm curious first, before we go to your opinion, I'm more curious about the relationship between specialization and accuracy. Yeah, you want. So we know specifically that our confidence scores and thus the select consensus model that weights each estimate in the consensus, those models are dependent on your accuracy within

that sector and have nothing to do with accuracy outside of that sector because we found zero correlation between accuracy in one sector and another. I'll bet that if we looked very, very closely at the data, that there would be correlation between like tech and consumer or like energy materials. but you don't need to go that deep in order to literally just say that analyst has this score and this Z score in this sector and you're going to overweight them, but in that sector, you know, don't pay attention to it. So yes, specialization is incredibly important. The other one was number of estimates was highly correlated. So we find that analysts who made more than 10 estimates, but less than 50 per quarter, companies.

Yeah, companies were more accurate than people who made less than 10 or more than 50. 50. Very interesting. Yeah, it seems to be relatively intuitive in the sense that like if you're doing more than 10, you're probably like serious. You're being thoughtful. You're being thoughtful. You're seriously involved.

If you're estimating like one or two, you know, we don't have that much confidence because maybe you're just coming to do Apple or whatever. If you're doing more than 50, you're probably a generalist in like spreading yourself a little bit too thin and you don't really have as much conviction in your ideas and you're not paying as much attention. Talk about the pros and cons of the generalist idea. It's a term that that is always asked, right? in every meeting and every due diligence questionnaire is, are you looking at the world broadly,

or are you very narrowly focused on material stocks or whatever it is? So obviously there are pros and cons. Where do you kind of fall out in terms of generalists versus specialized analyst? Yeah, I think the analysts need to be super specialized. There are so many different variables associated with each individual sector and industry. The only reason to be a generalist is if you're a PM, in which case you should be, But then, even then, hopefully your fund has a focus on a specific sector industry and you're not just like doing everything across the board. Or if you're at one of these big, long only shops, you're not dealing with every sector.

Like, you're in charge of a book that looks at these things. And it's the CIO's job to like allocate amongst different sector books or whatever. So I think the PM, you know, should understand markets in general and all these things. The analysts should come out of industry. And I've seen, you know, the best analysts that I know in tech, some of these guys, if they're in enterprise tech, they worked in Silicon Valley. Like, they know all the players. They have friends at these companies.

You need differentiated information or a differentiated understanding of this stuff in order to do this. If you work, you know, if you trade energy names, like, you better have come out of working at one of these companies or at least been a consultant to them in the past. I don't think there's any other way to do it at this point. One thing that you've brought up both times that we haven't dug it on, I remember in college my favorite, literally my favorite class. and they technically called it a philosophy class was just war theory.

I would love to hear a little bit about your background there. What drew you to it? And I would love to hear some of the kind of key things that you remember in terms of frameworks and lessons maybe that you still draw on. I mean, we talked a little bit about one of them, which is all war is politics by other means. And what that basically means is that when you get involved in anything, you know, know what the final outcome you want is.

You know, don't just say, like, we're going to go fight a war against terrorism. That's a war on a tactic. That's not a thing. Like, you're not, like, you're not eludicating the core of what you want. Always know what you want out of something. I mean, obviously, you know, some of this stuff, you know, there's a big part of what we learn, which is Klausvitz. And there's a big part of what we learn, which is like Sun Tzu. You know, those are kind of two sides of the same coin.

but the Sun Zoo stuff is more like, know you've won before you've gotten into something. Like when you go to make a bet, put all the odds in your favor and know that like you're making a sound bet there. Don't go to war and like guess at whether you're going to win. The thing that really, you know, drew me to the whole space was, I was also studying behavioral economics at the time on the other side,

so polysci and econ and two kind of different specializations. And in behavioral economics, you learn that markets are, incredibly irrational because humans are irrational. And we learned a very specific side of that irrationality and then how you can take advantage of it. On the war theory side, you learn how people behave in that realm. And I love stories about people and generals and decisions that they make. I named my two cats Hannibal and Alexander after Hannibal of Carthage and Alexander the Great. And it's not like, I'm really not a blood and gore kind of person at all.

It's more the strategy associated with, they were innovators. These guys were innovators in understanding the tools that they had and the outcome that they wanted and how to get from one to the other. Can you describe maybe a story or something specific for what their innovation was? Oh, man. I mean, Hannibal of Carthage, you know, there's a famous, and I'm going to forget the name of the battle, but there's a famous battle where he basically outflanks the other army and comes up with basically one of the core tactical moves of kind of ground-based hand-to-hand, you know, man-on-man combat, and it gets copied by generals for another, you know, thousand years.

Is this battle of Knai? Yeah, that rings a bell. I'm like forgetting my exact history here. and he wins an improbable battle with this strategy. And it's everything from just the how he comes up with it and how he implements it. Those kind of things are really, you know, interesting to me. Because in many ways they dovetail with how we come up with investment strategies, what we believe in, what everybody else believes in at the time, you know, those kind of things. Yeah, I was just going to say that the parallels are eerie, right,

in terms of hierarchy and structure, you know, generals, The CIA, analysts and grunts. There's all sorts of parallels. And my study of military history is much more limited than yours, but it does seem like the most interesting episodes are always about the development of new weapons or strategic technologies. I'll call it strategic technologies. And, you know, Blitzkrieg's always been my favorite, which is this kind of discombobulating way of approaching battle that's just incredibly philosophically interesting. when you don't expect it. Then everyone gets used to it and it's an arms race like anything else.

It seems like a great field of study. It's a good inspiration for me to go back. There's another one for today specifically, I think, is really important parallel, which is in the military, unfortunately, and we've suffered from this in the past as well in the U.S., we tend to elevate generals and colonels through the ranks because they've won a great battle or they had a big outlier event. Performance chasing. It is directly performance chasing, right?

And so what will happen is the general will make a big outlier decision and he shouldn't have based on all of the odds and the facts. But instead of the like three quarters of the time when he would have lost that bet, he wins the bet and we end up elevating him in the ranks because of that because well, you can't not like promote the dude who just won this great, you know, this great victory. And there are some people and we know like, just by statistics, there will be some people that flip heads, you know, five times in a row where it was complete luck and they did the wrong thing but got the good outcome and they end up

becoming like a three-star general and now they're making a bad decision the next time. This happens all the time. And this happens all the time in finance as well. And this is why I think measurement versus outcome is so important because you might flip heads five times in a row, but like, you know, the next couple of times it will be completely random and you'll be like, why is this person not performing? Yeah, I kind of want to wrap up with that idea, which is that if you can get your mindset wrapped around this notion of simple systems of measurement and simple systems of measuring outcomes and then just constantly wonder and be curious about what matters and what's predictive, first of all, you know, there's a couple key things here. First of all,

you're going to be surprised often by what matters and what doesn't. But second, you kind of need to work hard at the beginning to get yourself in the mindset of this. Like I use this app every day that it's called Way of Life. It's just like a bunch of checklist stuff. And the last thing on the checklist is like basically well-being, yes or no. Like do I feel a high sense of well-being, yes or no? And then you can go. So I've got almost two years of data now on this thing and you can go.

What's correlated to my well-being? And see which of these variables is most correlated with my binary yes or no well-being. And it turned out that it was sleep, like something's incredibly simple. And I tracked all this other stuff, you know, did I do this? Did I do that? All this complicated stuff. And stupid, dumb, like, did I get a good night's sleep was the most correlated to well-being? I don't, yeah, I'm not surprised.

But had I been designing, you know, some life change ahead of time, sleep would not have been the thing that I focused really hard on getting better sleep. But now that's what I'm thinking about because I did one of these simple, simple, simple. systems. And it just seems like, you know, your message and we'll make sure, I think this might be released simultaneously with this new piece of yours. The piece itself covers some of what we've talked about, but really gets into some deep detail about what people, what asset managers, portfolio managers, et cetera, should be doing. So we'll post that and I encourage everyone to check it out. Thanks, man. Absolutely. Hey, everyone. Patrick here again. To find more episodes of Invest like the Best, go to

investor field guide.com forward slash podcast. If you're a book lover, you can also sign up for my book club at investorfield guide.com forward slash book club. After you sign up, you'll receive a full investor curriculum right away and then three to four suggestions of new books every month. You can also follow me on Twitter at Patrick underscore Oshag. If you enjoy the show, please leave a quick review for us on iTunes, which will help more people discover Invest Like the Best.

Thanks so much for listening. Thank you.