Annie Duke - Ideas Lab Series - October 12th, 2022

[00:00:00] Annie

What you have to remember is that life is just way too short to be spending your time on things that aren’t worthwhile, and you need to quit those things to go do things that are. Overall that’s really what our long term goal is. It’s not to grit out some horrible thing that’s making us unhappy for no reason other than we think it builds character. It’s to get us to somewhere that resembles a fulfilling and happy life. Look, if the thing you’re doing is hard, but ultimately it’s going to get you closer to that goal of a fulfilling and happy life, you should be able to stick it out. But if it’s not, you have to switch as quickly as possible because you don’t have that much time.

[00:00:47] Introduction

Imagine spending an hour with the world’s greatest traders. Imagine learning from their experiences, their successes, and their failures. Imagine no more welcome to Top Traders Unplugged, the place where you can learn from the best hedge fund managers in the world so you can take your manager due diligence or investment career to the next level.

Before we begin today’s conversation, remember to keep two things in mind. All the discussion we’ll have about investment performance is about the past and past performance does not guarantee or even infer anything about future performance. Also understand that there’s a significant risk of financial loss with all investment strategies, and you need to request and understand the specific risks from the investment manager about their product before you make investment decisions.

Here’s your host, veteran hedge fund manager, Niels Kaastrup-Larsen.

[00:01:41] Niels

For me, the best part of my podcasting journey has been the opportunity to speak to a huge range of extraordinary people from all around the world. In this series, I have invited one of them, namely Kevin Coldiron to host a series of in-depth conversations to help uncover and explain new ideas to make you a better investor.

In this series, Kevin will be speaking to authors of new books and research papers to better understand the global economy and the dynamics that shape it so that we can all successfully navigate the challenges within it. With that, please welcome Kevin Coldiron. 

[00:02:21] Kevin

Thanks, Niels, and welcome everyone. If you’re a professional investor, if you manage your own money, if you’re just trying to follow the markets, you’re grappling with the problem of making decisions under uncertainty. Our guest today, Annie Duke, makes her living helping people get good at this. She got exceptionally good at this herself over the course of a twenty-year career as a professional poker player. She has career winnings of over $4 million. She won the first ever World Series of Poker Tournament of Champions. She’s the only woman to have won that competition and the NBC National Poker Heads Up Championship. When she retired in 2012, from poker, she became a decision strategist, consulting and writing about learning under uncertainty.

She’s written a lot of books. The most recent of those, Thinking of Bets, became a national bestseller. We’re recording this on October 3rd, the day before the release of her latest book called Quit: The Power of Knowing When to Walk Away. The messages and themes from that book are what we’re going to talk about today.

Annie Duke, welcome to the Ideas Lab. 

[00:03:40] Annie

Thank you for having me. I’m excited to talk to you. 

[00:03:45] Kevin

Yeah, and I really just wanted to say how much we appreciate having you on the show, especially today, because I know you’ve got the book coming out. 

[00:03:53] Annie

I have to say, yes. So it’s a little crazy. And you know, Kevin, the thing is, I think people imagine (who don’t write books, and maybe this is for other people who aren’t me) that it’s just really cool. You’ve done the book, you’ve finished it, you turned it in, you’ve done all the editing. Now it’s just, ooh, excitement because the book is going to be released into the world. At least for me, it’s actually very stressful because the way that I’d like to put it is that you think about different identities that you might have, right? 

One identity you can have is “I’m writing a book.” Kevin, if you were talking to somebody at a cocktail party and they were like, “I’m writing a book,” you’d be like, “they’re cool.” That’s a cool thing. You’d be like, “Oh, I wish I had written a book.” You admire them and, whatnot. It’s an interesting thing because when you’re writing a book, it’s just great. But when you’re releasing a book, when the book is actually getting released into the world, it’s now like running down the street naked.

Okay, before I was writing a book and you thought it was really cool and I gave you the elevator pitch, which is what sold the book, but you haven’t actually read anything that’s in the book. There’s nothing to critique. There’s nothing for you to reject. There’s nothing for you to, whatever, give a bad review about. When you release the book then you’re just exposed. There’s no “take backsies,” right? Anyway, yeah, I’ve been feeling it for sure, definitely, like this weekend, today, tomorrow, I’m (a little bit) holding my breath until tomorrow. I imagine I’ll be holding my breath for about the first week.

[00:05:49] Kevin

Just waiting for the reactions, even though you know that you’ve put so much work and a lot of the lessons really from a multi decade career into this, but you’re still… 

[00:06:02] Annie

Yeah, exactly. It doesn’t matter how much you’re talking yourself into it, ” But I did a good job, and my editor likes it, and I’ve seen some reviews and they’ve been good.” It’s still this really big moment of exposure. In an ideal world, you wouldn’t care. In reality, I write these books for myself because I want to learn about the topics. It’s just that a part of that is it does end up that other people see what’s been grinding around in your brain. 

It’s a moment. It’s a moment. Obviously, this is not my first rodeo. I’ve decided that I can withstand this moment just fine. But the day before and the day of, it’s always a moment. So you caught me during that moment, Kevin.

[00:065:56] Kevin

Like I said, we appreciate you talking to us during what I know is a stressful time. I think that’s a nice way for you to start off because, like you said, this isn’t your first rodeo, yet you still have these uncertainties, and I think we all can relate to that.

Let’s get to the book. There’s a lot of really great stories that you’ve put together (great narratives) to illustrate the points that you want to get across. But I thought maybe, before we get into those stories, if we could just start with what are the main messages and points you really hope that people take away from this?

[00:07:38] Annie

Let’s just start simple. Let me ask you this, if I called you a quitter would you take it as a compliment? 

[00:07:49] Kevin

Unlikely. 

[00:07:51] Annie

Yeah, so, really, if I think about what the broad message is, it’s why? There are clear circumstances in which quitting has to be correct. I can come up with some of them: you’re playing a football game and you get a concussion shouldn’t you quit? You’re climbing a mountain and there’s a snowstorm, shouldn’t you quit? You have an investment that’s very quickly going to zero, shouldn’t you quit somewhere along the way?

And I’m not saying that you should quit just because the investment’s going down. That’s obviously a question of, “Do I believe this is going to win in the future or not?” But if you don’t believe it’s going to win anymore, you shouldn’t be doing it. I think that, as a society, when we think about that opposition of grit versus quit, grit is a virtue.

So much so that when we think about the way that we parent our children, we tell them not to quit things because we want to build character. It’s synonymous with character. Whereas, quitters are losers, they’re failures. I think that’s just like a huge shame. That’s the main message that I’m trying to get across in the book is that quitting is a good thing when applied at the right time.

Grit is not always good. It’s really good for sticking to things that are worthwhile, but that are also hard. The problem is that grit is also really good at getting you to stick to things that aren’t worthwhile just because you think that quitting is bad and sticking to things is good. If we don’t rehabilitate this idea of quitting, we’re going to be in really big trouble. It’s a really important skill to get good at and develop in order to be a great decision maker. 

[00:09:35] Kevin

It’s funny, I’ve been really looking forward to speaking with you, and I’ve been telling people about, “Oh, I’m interviewing Annie Duke, she’s a former champion poker player, she consults with companies on decision making.” And people are like, “Oh that’s super cool. When’s the episode coming out and what’s she going to talk about?” And I said she’s going to talk about how we should quit more often. They’re like, “Oh, she’s wrong, but send me the link anyway.”

Early on in the book you talk about one of the reasons why quitting is seen as advice or we need to rehabilitate the idea of quitting is that we don’t see the quitters. They’re invisible. You illustrate that with the story about an incident of a climbing accident on Everest that ended up becoming quite famous.

I was wondering if you could recount that for us [that story]. I think not only does it show how people who quit at the right time ended up becoming a footnote (sort of invisible), but also that the powerful forces that drive even the most experienced people to keep going when they shouldn’t – to quit too late.

[00:10:55] Annie

Yeah. I think I say that if you’re going to write a book about grit you should probably include climbing Mount Everest, because that takes a lot of grit. I think it’s only fair that if you’re going to write a book about quitting, you should also start on Mount Everest which is what I do with this book.

This story is about Mount Everest, the stars, the heroes of this story are three climbers, Dr. Stewart Hutchison, Dr. Lou Kasischke and John Taske. They’re part of a climbing expedition. They’ve gotten very popular, right? So this is in the nineties when everybody’s climbing Everest. This expedition is led by a very experienced expedition leader. There’s three Sherpas and then there’s eight climbers. These three climbers become friends. They’re part of this same expedition and they’re heading up the mountain. So on Summit Day, They’re starting out from camp four. One of the things that they do on Everest is, when you’re climbing, they give you a turnaround time.

A turnaround time, simply put, is just a time that no matter where you are on the mountain, you’re supposed to turn around (even if you haven’t reached your final destination). It’s meant to really protect you from the very bad decision making you can make when you’re in the middle of the climb.

When you’re in the middle of the climb you might be suffering from all sorts of things like hypoxia, for example (which is just lack of oxygen), which is going to affect your climb. Also, when you get in the shadow of the summit, we’ve all heard of Summit Fever (this desire to just get there no matter what, because it’s just right there no matter what the conditions are), that’s actually really dangerous. 

You don’t want to be continuing. First of all, you don’t want to continue if there’s bad conditions, like a snowstorm or whatever. Also, you don’t want to continue past the turnaround time because if you do, you’re going to end up descending the mountain in the darkness. And not only in the darkness, but when you’re tired. 

When you’re going up and you’re trying to reach your destination, you have lots of adrenaline, you have lots of oxygen. Then, on the way down, that’s when you start to really suffer the adverse effects of fatigue and lack of oxygen. And you might be in the darkness, and dark is really bad when you’re climbing on Everest. 

The turnaround time, on summit day, is 1:00 PM. What that means is I don’t care where you are on the mountain, I don’t care whether you’ve reached the summit or not, if it’s 1:00 PM you have to turn around and come back.

Hutchison, Taske, and Kasischke are climbing up the mountain, and this happens to be particularly slow. They’re caught behind a clump of people who are moving really slowly, because the mountain’s really crowded. Remember this is when it started to get very popular to climb Everest. 

Their expedition leader comes up behind them and Hutchison says to the expedition leader, “How long is it going to be till the summit?” Just recognizing that they were moving slowly. The expedition leader says, “Three hours.” So, it will take three hours to get to the summit from where they’re standing. The expedition leader then scrambles ahead and goes in front of them. 

Hutchison holds Taske and Kasischke back and says, “I think we have a problem because it’s almost 11:30 in the morning. If it’s going to be three hours till we get to the summit, we’re not going to get there till 2:30 PM. The turnaround time for summit day, we have been told, is 1:00 PM. I don’t see any world in which we get to the summit before 1:00 PM. Even if we go really fast, maybe we get there at 2:00PM. So, I think we should turn around now.” 

Taske agrees right away. Kasischke was a little bit tougher just because he had the goal to climb all seven of the tallest mountains in the world. It’s called the Seven Summits, and Everest was his last one. But they did convince him, and he turned around and they went back to Camp Four, and that’s the end of the story. 

Kevin, I assume it’s immediately obvious to you, why? You don’t know who these people are. It’s not a particularly [compelling story]. Where’s the tension? Where’s the plot line? They were climbing. They followed the rules. They turned around at 11:30 in the morning because they realized they weren’t going to make it to the summit by 1:00, and then they lived. These people are not the heroes of any story, at least not ones that we read. The thing is that they should be the heroes of the story because they did follow the rules, and they turned around when they ought to.

The interesting thing about them, as much as when I tell this story, people are like, “Never heard of them.” It turns out that most people actually have heard of them. They just don’t remember them. The reason is that these people were part of the expedition that was chronicled in Into Thin Air, Jon Krakauer’s book. They were part of the documentary that was made called Everest. The movie that was made that starred Jake Gyllenhaal (I think), called Everest. 

Their expedition leader was Rob Hall, who people might remember, died on the mountain. He got to the summit at 2:00 PM. He was the one who told them it was going to be three hours to the summit.He got there at 2:00 PM He waited there for two hours for another client, Doug Hanson until 4:00 PM And then both of them perished. They never got down off the summit, which was the tragedy. 

When we think about that, the protagonist of the story, it’s Rob Hall and Doug Hanson are the protagonists of the story. We don’t really remember these three people. But, Krakauer actually talks about them. He says quite a bit about them and he calls them “the best decision makers on the mountain that day.” But they’re invisible to us. We don’t remember them. The people who stick to it are the heroes of our stories, not the ones who just turn around. We don’t know who they are. 

[00:16:50] Kevin

Yeah, it struck me that there’s a lot in there. The first thing that popped in my mind, just because I’ve got an investment background, is that the turnaround time is a stop-loss order.

[00:17:04] Annie

Exactly right. I put those stop-loss orders and turnaround times, I put them under the category of kill criteria. Kill criteria, really simply put, are generally either a signal or a state, that you could be in, at a particular point in time, that would tell you that you ought to stop – that you think about well in advance.

A stop-loss order is not something where you just do it on the fly. It’s like you open the position and then you put in a stop-loss at that time (in advance) which is what (obviously) a turnaround time is as well. Yeah, they’re totally related to each other. They’re in the same category of strategies for being better at quitting.

[00:17:45] Kevin

Yeah, you talk about how, when you’re setting the stop limit or you’re setting the kill criteria, you need to draw on history. You need to reference people who’ve done it before. Obviously, the Everest turnaround time is a great example of that. Presumably that’s been built up over time, based on years of experience.

I wanted to just ask you about that. I was thinking about that. That makes sense, right? A lot of people in the investment world do the same thing. They’ll set stops based on the past volatility of a particular asset, things like that. But do you think there’s some value in (how do I want to put it), learning the lesson the hard way? 

[00:18:38] Annie

No, I don’t. The reason why I don’t think so is because even if you put in a stop-loss, you’re going to learn the lesson the hard way because you’re going to cancel the thing which, I bet you, you’ve done in your life.

You’re laughing, but I know you convince yourself, right? You’re like, “Oh, I thought that then, but now I know something different,” and then you cancel it. Yeah, the thing is, look, here’s the question that we have in terms of learning the lesson the hard way. Let’s say that we do have a bad outcome that comes from quitting too late. Question number one is, are we going to learn the lesson? When that happens to us, and then we go into the future and we’re in an experience where something’s free falling or there’s some new information that’s come in that tells us that our original thesis wasn’t even right, are we going to pay attention to those signals?

The answer (and I think that this is the Rob Hall lesson) tells us really clearly is no. If Rob Hall, who’s one of the most experienced alpinists in the whole world, didn’t pay attention to his own turnaround time, then I don’t think that we should trust ourselves to actually pay attention to these things.

Actually, there’s another climbing story that I think shows this really well. There was this researcher, Jeffrey Rubin, who did a whole bunch of research on what’s called escalation of commitment or getting entrapped. Here’s a very simple example: you’re waiting in a line at a grocery store and your line starts to move really slowly and you see there’s another line that’s moving a lot faster. You don’t switch, because, “This is my line and I don’t want to have wasted the time that I’ve spent in the line, even though it would be correct to switch.” 

He did a whole bunch of work and he was (along with Barry Staw, from the motivational side of things) one of the people who had the most expertise in these problems of escalating commitment to losing causes – not reacting to the signals the world is giving you that, “Hey, I ought to be quitting.” He’s an expert in this. 

He’s doing like the hundred highest peaks in New England, climbing (I think) his last one. A very heavy fog rolls in. He’s climbing with a graduate student of his. The graduate student says, “Oh, it seems really bad. The fog is really awful and heavy. I don’t think we should continue.” Jeffrey insists they keep on going. The graduate student turns around and Jeffrey Rubin dies on the mountain. His body was found two days later. 

This is an expert in these problems. What are you going to learn the hard way? Because the fact is that we have the intuition that when the world starts turning against us; when those signals start coming in that the thing that we started is no longer worth continuing, that we will quit, but we don’t. We keep sticking to it. Why? 

For all sorts of reasons, not the least of which is we’re not quitters. Your friend said, “Oh, she’s wrong about wanting people to quit more because grit is good and we stick to things and whatever,” and we don’t like to admit we’re wrong. For that reason alone, I don’t think you’re supposed to learn the hard way, but you’re going to learn the hard way anyway. 

Rob Hall’s the one who set the turnaround time. He didn’t follow it. What we need to realize about these things, like stop-losses, or turnaround times, or kill criteria, in general, is that what they’re doing is increasing the probability that you’ll quit when you should. That’s what they’re doing. They’re not giving you a guarantee that you will. 

We know that these three climbers would not have lived without a turnaround time. They followed it, but not everybody followed the turnaround time on the mountain that day. Three more people did than otherwise would have. That’s the other thing that you need to realize, trust me, you’re going to run headlong into that. Hopefully, the next time you do, you’re going to be more likely to actually follow the turnaround time that you’ve set for yourself which, in this case, might be a stop-loss. That’s the thing that we have to remember.

No, hard knocks aren’t going to get you there. You have to have other things in place because you’re not going to pay attention no matter how much you’ve experienced it in the past. 

[00:22:51] Kevin

That makes sense. I want to ask you what we can do in addition to just having the stop-loss, or the turnaround time, or the kill criteria? 

But before we get to that I wanted to ask another question which I think relates to maybe why we’re not good at quitting on time. You reference a lot (in the book) of academic studies that relate to finance and investment. One of them talks about how we tend to take gains too early and we let losers go on too long. I think a lot of the listeners will be familiar with that. 

You framed it in a way that I hadn’t thought about before. It really made me think. You talked about, in poker, that players who quit while they’re ahead end up minimizing the hours when they’re playing well; and at the same time (I think you said) if you soldier on when you’re doing poorly, that maximizes the time when you’re playing badly. Our tendency is to play more when we’re not playing well. That made me think, does that mean we think we’re worse at stuff than we really are? Is that a general tendency?

[00:24:11] Annie

No, no, we think we’re better at things than we are, in general.

[00:24:19] Kevin

Even though that bias is making us do stuff longer than we should, and having results worse than we should. We still think we’re better. 

[00:24:30] Annie

No, we think that we’re better than we are… This is a different set of biases that’s causing this to happen. First of all I do want to make the point (just in the sense) about what I think of having some rule of thumb that sticking to it is just always good. I also don’t want you to come away from this with the rule of thumb that quitting things is, also, always good. What matters is, is the thing you’re sticking to worthwhile or not? This is the point about maximizing the number of hours that you’re playing well and minimizing the number of hours you’re playing poorly. That’s what’s determining it. What’s my expected value right now? 

If I’m in a situation where I think my expected value is low or negative, I shouldn’t stay in it. If I’m in a situation where I think that it is high, it shouldn’t matter much to me whether I’ve been winning or losing. As long as my expected value is still there, I should continue to play.

That’s that issue of worthwhile versus not worthwhile, which is hard. Which is one of the reasons why we have things like stop-losses, because we should recognize that when we’re losing we may be a very poor judge of whether we’re losing because of bad luck or we’re losing because of something that we’re doing.

I don’t want people to say, “Oh I shouldn’t be gritty, ever. I should just quit a lot.” What I want people to do is say, “What matters is the context and we need to become calibrated at this decision.” Sometimes sticking it out is the right choice, but not always. It’s not like grit should be synonymous with character. Sticking to things because they’re hard isn’t a good thing if the hard thing isn’t worthwhile.

But likewise, don’t quit too much. Here’s where we can see a calibration problem. Of all the aphorisms, almost all of which are just like, stick to things and you’ll succeed; try, try again, blah, blah, blah. Winners never quit, quitters never win. There’s one aphorism, which is to quit while you’re ahead, which is pro quitting.

That is terrible advice because it feeds into a bias that we have, which is exactly that, to quit while we’re ahead. Again, independent of the expected value. Alex Imas has done some great work on this with retail traders, where what he’s shown is that when you look at what’s happening in terms of stop-loss orders, retail traders tend to cancel those. They cancel them to keep the position on, to hold onto it. 

On the flip side they’ll also cancel, take-gain orders, but they cancel them to sell early. In both cases the order is getting canceled, but in one case it’s to sell it early when you’re in the gain. In other words, they’re trying to quit while they’re ahead, before they’ve stopped-out at the top – they’ve topped out. On the other side they’re sticking when they’re behind. 

Now, the question is, so what’s going on here? What’s happening? It has to do with this issue that we have that has to do with mental accounting and also what’s on your ledger (both things). We have a mental account and what’s on our ledger, those tend to line up, but not perfectly. It’s whether we feel that we’re in the losses or in the gains, and the desire to take us from a state of winning to having won, and our aversion to taking us from a state of losing to having lost. 

Let’s think about this. If I buy a stock at $50 and it’s trading at $40, as long as I haven’t sold it, I’ve got a $10 loss on paper. As long as I haven’t sold it, I still have the gamble on. Vol can go my way, whatever. I can get back to $50 as long as I keep the bet going. It’s when I sell it that it goes from a loss on paper to what’s called the sure loss.

This is something Daniel Kahneman, the Nobel Laureate has talked about, which is Sure Loss Aversion. We are averse to that moment of turning something from on paper to realized. We don’t like it. There’s the canceling the stop-loss. Again, independent of whether you think that the bet is still positive expected value.

In other words, what should determine whether you sell or hold at $40 is merely would you buy it today? That should be the only thing that matters – less history, would you buy it today? But that is not what matters. What matters to us is, I’ve got to hold this because I need to get my money back. 

I actually saw a conversation between two people on Twitter where somebody had recommended to somebody that they buy, I think it was Bitcoin at $50,000ish and it was trading at $26,000 or something, and the person had sold it and they were like, “How could you do that? Now you can’t get your money back.”

That’s a good example. But now it’s done. Now on the reverse side, we really like to take gains on paper and turn them into realized gains. We buy it at $50, it’s trading at $55. Our take-gain is $60 and we’re canceling it to sell it at $55, because then we don’t want to keep the bet on, because what if it goes back to $50? Then we’ll be sad. We want to turn those gains on paper into realized gains. 

Notice that this problem with turning things from on paper to sure gains, or sure losses, distorts both sides of the quitting question. It causes us to quit too early when we’re ahead, which is why “quit while you’re ahead” is such bad advice. It causes us to stick too long when we’re behind because we don’t want to take the sure loss, we’re averse to that. 

[00:29:57] Kevin

We had a client, when I was running my firm, and quite a big well known client. They had a rule where they would look at you and say, “Okay, based on your strategy, we expect a loss of X%, would be like a two standard deviation down move.”

They had a rule that said, if you hit that; if over (let’s say) a year we had lost what they considered to be two standard deviation. We had a really bad year. Then that triggered a decision. The decision was either to fire us, or give us more money. The thought process was, if you still believe that the manager’s got skill, then this is the ideal time to give them money just after a bad patch, hopefully betting on mean reversion. If you don’t, then you should get out. 

And then there was the reverse, right? If you had a really great year, then they would take money away from you, which was frustrating. I don’t know what you think of that kind of rule, but to me that seemed to illustrate a way of dealing with some of the biases that you talk about.

[00:31:17] Annie

Yeah, again, this goes to the idea of kill criteria. You’re just setting rules in advance and pre-recommitting to those rules. I think that, in general, that’s just a really good way to deal with these quitting and sticking decisions, because once we’re in it, once we’re on a path, we’re going to be really bad deciders.

This is something that Kahneman talks about which is being in it; in the decision. The way that I like to think about being in it is I want to eat healthy, but there’s an open box of chocolate sitting in front of me. Oh, I’ll just have just one; like, the whole box, thank you. That’s the problem once we’re on a path. 

So, what we want to do is set out benchmarks. As an example here, because you don’t want to go the other way either of do we still trust the thesis? These managers are trading a certain strategy. Do we think they’re smart? Do we think the strategy still holds? When they told us why they were doing it and they were giving us a forecast of what the fundamentals would look like, and so on, and so forth, did the world move against that in a way that would make us think that they didn’t know what they were talking about?

As long as those things aren’t true, then I’ll stick. But if they are true, then I’ll quit. Even if they were winning, I’ll quit. We have to think about it from both sides of the equation. Here’s an example. If you got into, let’s say that you invested in people who were trading crypto. What they told you was, this is why you should invest in us. Because we think that crypto is, first of all, going to be uncorrelated with inflation. So, it’ll be a good hedge against inflation. Also, it will be independent of what happens in the market itself. In a market crash type of situation we think that crypto is going to be robust in the face of that.

Let’s say that they told you that was their thesis. They’re smart people. I don’t doubt that they’re very smart people who have this thesis. Now they have a down year this year. Are you supposed to invest in them again? The question is how do you determine what smart means here?

It’s to say was their thesis true? We have evidence about that. We now know how it behaves in a market chaos situation. We know how it behaves when inflation is going up. They may have other reasons that they gave you, but if those were the reasons they gave you; that’s why they were investing in it, that seems like a good time that you should say, it doesn’t matter whether you’re up or down, I think that I should be out of this on both sides of the equation. 

Frank Brosens from Tectonic Capital actually told me something once that was, I thought, the greatest story ever. He was trading some kind of swap in Japan. He would know better exactly what it was, but it was some swap, it was some credit swap or something in Japan. Anyway, they had a thesis about how it was supposed to behave under certain conditions. They ended up winning gigantically – a crazy amount. They immediately sold their position, not because they were winning but because they realized that they had no freaking idea what they were doing. Their model didn’t predict what had actually happened. That they could ever win that much. They just got out of it. 

So, kill criteria works on both sides of the equation. It helps you to try to figure out when you are supposed to stick and when you are supposed to quit. It has to do with what is my model of the world? What am I expecting to see? When I see these signals in the future, what is that telling me to do? 

[00:35:07] Kevin

Okay, so kill criteria are important. Then you’ve also talked about the fact that you can have kill criteria and not follow them, even if you’re an expert. 

[00:35:19] Annie

Yes, or like me, sometimes I would blow through my stop-losses at the poker table, trust me.

[00:35:25] Kevin

Are there other things we can layer on to kill criteria? You talk about, sometimes splitting the decision – the position initiation maybe with the person coming in and evaluating whether you should continue the position because they don’t identify with it? They haven’t got personal capital invested. You talk about the notion of a quitting coach, literally a mentor that talks to you (maybe not on a position by position basis), but just, “Hey Kevin, I’m not sure the podcast thing is working out for you. Maybe you should do something else.”

Can you talk about both those things, the splitting decision and the role of a quitting coach? 

[00:36:11] Annie

Yeah, splitting the decision, divide and conquer, and quitting coach really go into the same category. As I said, we’re very bad at making these decisions when we’re in it. Not just because we’ve already started to accrue losses or gains, which then distorts our ability to think about what we should do going forward, but also because of some cost issues, such as we’ve put time, and effort, and money into things that we feel like we don’t want to waste. If I quit now, I have wasted all of that time. Endowment – we are the owners of those ideas – that can become very difficult for us to let go of things. How our identity gets wrapped up in things, status quo. There’s a lot of cognitive debris that’s making it very hard for us to walk away.

The thing is that, that’s our cognitive debris. My cognitive debris isn’t yours. We have similar types of cognitive debris but applied to only the things that we started, that I myself started. I’m sure, Kevin, that you’ve had this situation where you see someone who’s stubbornly sticking to something, and you can see very clearly, from the outside looking in, that they probably ought to walk away from it.

That probably happens to you every day. I don’t know. It stands to reason that if you’re looking at other people and saying, “Oh they’re totally stubbornly sticking to stuff here,” probably other people are looking at you and thinking the same thing, because none of us are that special.

Okay, so then why don’t you get them to help you? Kevin, if you can see me so clearly (and you may not be right, but you have a different perspective than I do), and maybe I’m stubbornly sticking to something and I think that I see something the world doesn’t see. You just think that I’m an idiot (you wouldn’t use those words), and that I’m delusional. But, I’ve convinced myself that I see something that everybody else doesn’t see. It would be good if I talked to you, if I was able to actually get your opinion. This is where quitting coaches come to play. Now, the key with the quitting coach is that  you’re going to be loath to tell me that I’m being an idiot.

I have to give you permission to do so. I have to very explicitly say, I need help with this. I’m struggling with this decision. I don’t know whether I should stick or walk away. I don’t want you to tell me what I want to hear when I protest and say, I know I can turn it around or, but no, but this, but this, but this. You have to actually give me an honest assessment. I’m not going to be mad at you. I need you to have my long-term interest at heart as opposed to just cheerleading my own delusions. Which is where we are naturally. That’s what you can do in your personal situation.

I have talked to investors where they have a big enough team that we’re able to take a divide and conquer strategy. In other words, to say, when somebody starts something, you immediately start the debris accumulating. So as soon as you cross the starting line, you start to accumulate some costs. You’re going to get in the losses or in the gains. You are going to have ownership. You’re going to be endowed to that path that you’ve chosen. Your identity is often going to start to get wrapped up in that as well. It’s going to become your status quo. If we know that’s the case, why don’t we get somebody else to make the exit decisions?

Where possible, one of the things when I’m talking to ICs is, “Hey, can you make sure that the people who are choosing whether to make the investment are different from the people who are choosing whether to exit it?” When possible.

[00:39:48] Kevin

Do you get pushback on that? What’s the reaction when you say that?

Because there’s this kind of cult of the (sorry to interrupt), there’s a cult of this portfolio manager. I know investment teams are typically structured where ultimately there’s one person who has responsibility for both. Have you had clients take that onboard and actually change it?

[00:40:22] Annie

Yeah, so it depends. Institutional investors are definitely open to it because they have a committee. I’m just saying just divide the committee up. If you have enough people, have two committees, like a start committee and a stop committee. That’s something that people are pretty open to.

The PMs it’s mixed, but most PMs are willing to do it. What I recommend that they do is one of two things. Either pair with another PM (kind of buddy system it). If you have two PMs and now they’re going through these decisions themselves. Part of that can be setting kill criteria together, as an example, which is really helpful. Then you have accountability and you’re each helping each other. I think that can be really helpful in that coaching. That’s not divide and conquer. I’m not saying if you’re paired with me, Kevin can’t make the decision for me whether to exit, but you can help me to think through it, and you can hold me accountable to my kill criteria and that’s certainly going to help me as a PM.

PMs are definitely super open to that. The other thing that they’re really open to, if they have a big enough team, is just the simple thing of if a quant recommended a particular strategy for you to trade, then don’t let them make the decision about whether to quit it. Have somebody else on your team do that.

The PM then is still the decision maker, but the recommenders become different. That also has been something that people have been perfectly willing to accept. Well, not perfectly, but mostly willing to accept.

[00:41:38] Kevin

Yeah. I thought about this a lot when reading your book because the strategy at the firm that I ran was systematic. We used models and a lot of the logic was to deal with the issues that you bring up. You can set a stop-loss rule based on history. You can continually scan the universe and look for better opportunities. You basically stripped the emotion out of the buy and sell decision.

I’m curious, do you find yourself, when you’re in investment firms, do you recommend models, systematic approaches? Let’s start with that.

[00:42:19] Annie

I don’t pretend to know what’s right for a particular person. I will tell you that the places that I work with, whether it’s venture (which is the slowest, we’re investing at seed), all the way through options traders (which is a very fast cycle), is that the more you can systematize the decision, the better off you’re going to be.

What that means is that you have to be going through a very similar process in terms of the start decision. Either there has to be a rubric, you have to decide what you care about, you have to make particular forecasts, you have to really think about what are the facts and information that are going into the brief? What are the subjective judgements that we need to make? Again, be very careful about writing down the thesis, make forecasts, and make sure that all of that is incredibly systematic. So that you’re putting order to it and you’re trying to take as much of the noise out of the decision as possible.

That, obviously, also allows you to be much better at closing feedback loops, which is something that I think is really important. If included in it, is some sort of forecast of the future, as I make enough of those forecasts, I can start to see what the quality of my forecasting capability is.

Since these decisions are just forecasts, that becomes incredibly important. The other thing is that when an investment goes up or down, instead of trying to recreate history, because I’ve systematized it I can actually go look and say, “How did this fit in? Did we miss something? Is there information that’s not in my model that I might want to include going forward that I learned afterwards, so on and so forth?”

I think it creates faster learning and refinement of that. I just think that’s incredibly important. Then what goes along with that is that you can start to think much more clearly about what are going to be the post starting decisions? That’s the thing that we need to realize is that we put all this time into the decision to start and not a lot of time into the decision to stop.

When you look at even expert traders, expert investors, on the starting decisions they’re doing much better in the market. Alex Imas found, at least for institutional investors, they were about 120 bips better than the benchmark. On the sell side decisions, they were about 70 bips worse.

[00:44:31] Kevin

It’s crazy when you think about it, because the buy decision should be so much harder. You could literally pick from anything, so the search space is… 

[00:44:45] Annie

I know, and then the sell decisions are just from what you’re holding. I know, right?

[00:44:48] Kevin

Just what you own. 

[00:44:49] Annie

It’s nuts!

[00:44:50] Kevin

I was just going to say that, would you recommend just selling a slice of the portfolio that you own and just saying, “Hey, I’m smart. I know I can identify good opportunities, so periodically I’m just going to free up some capital and reinvest it back into what I know is my strength.”

[00:45:16] Annie

No, it would be better than the strategy people are using. Because what Imas was comparing these decisions to was random. What if you just randomly freed up capital from your portfolio in order to trade something new? 

With the expert investors, what they were doing was looking at the tails, the extreme winners and extreme losers. That’s not surprising. They weren’t looking at the whole portfolio. It was, again, going into a heuristic approach as opposed to actually thinking about expected value. That being said, I think the real problem is this, that when you buy something, you track it. It’s constant feedback. You’re seeing it tick up and down, and so that’s allowing you to get this constant feedback about what the quality of that decision to actually put that position on was.

When you sell it, it’s gone. I don’t think that people actually even think about comparing their quitting decisions (which are just selling decisions), they’re selling decisions to any benchmark. We know what the benchmark is when we buy – it’s beta. We can just look at if we just index the market, compared to what we actually bought, would we have done better?

Okay, everybody knows what that benchmark is, and we’re always comparing to beta on the things that we own. On the things that we sell, as you said, you have to create your own benchmark, which would be, I’m randomly freeing the capital up from my portfolio, and then tracking that in some way.

We just don’t even think that way because it’s out of sight, out of mind. It’s off the books at that point. I don’t think that we get any feedback on this. This is the reason why, when I’m working with institutional investors, when I’m working with PMs, I’m saying, you really have to get good at this skill because I don’t think anyone really actually understands, very well, how much value you’re giving up by not thinking about these decisions.

When we go to this more systematic approach, what we have to realize is that we have to systematize the risk-off decisions as well. What happens is that when we’re thinking about putting risk-on, we go through this whole process. We have models, it’s very systematic, so on, and so forth.

We can do things like stop-losses, but what we really want to do is say, “Given what my predictions of the future are (which is all that an investment is)… Let me take a simple example, implied is that interest rates are going to be within a certain band. Let’s say that’s a clear forecast that you’re making that causes you to decide to put a position on. Interest rates will be in a certain band. What has to go along with that is that you literally write down, if interest rates go above or below that band, this will be my reaction. 

Now, I know it seems like a distinction without a difference, because if it’s already implied in the thesis, then when interest rates move out of that band, wouldn’t you react to that? No, you wouldn’t, because you’re going to keep climbing up the mountain like Jeffrey Rubin did. This is what you have to remember. Think about, if this is the reason that I’m putting it on, what does that mean about, say, the fundamentals? What does that mean about the fundamentals? Which isn’t the only thing you would pay attention to. That’s just an example. 

If those fundamentals are moving in a particular way, that goes against what I was predicting that caused me to put this position on in the first place, I ought not to convince myself that now it’s really cheap. Which is what we end up doing. Notice that also gets you out of the problem of these managers just didn’t do well, so I’m going to fire them. Which is not a particularly good reason to fire a manager, not in the short-run. We know that people lose a lot of money to that because there’s regression to the mean in both directions.

This allows you to actually start saying, “No, what are the actual signals that tell me something about my expectancy here,” and to start doing something about it. 

[00:49:08] Kevin

I want to be respectful of your time, and we only have a few more minutes left. I want to maybe just talk a little bit. It relates to a lot of things you’ve said, but just about goal setting.

I think we’re told that having goals is good. We’re told that pre-recommitting to those goals, telling people what you’re trying to do makes it more likely. 

[00:49:36] Annie

That’s good too.

[00:49:37] Kevin

You spent quite a lot of time in the book talking about maybe the dark side of goals, or the downside of goals, and I think that’s important for everyone to hear about.

Could you just give us your perspective on that? 

[00:49:54] Annie

This goes in the same category. In no way, shape, or form am I saying that grit isn’t good. Grit is necessary for success. If you are going to be successful, you better have grit because whatever it is that you’re doing, that’s going to be the road to success, you’re going to come up against obstacles. It’s going to be really hard, and you have to find a way to be able to stick to it even though it’s hard, if it’s worthwhile. The thing that isn’t good is to say, “Therefore grit is always good.” That’s the thing that I’m trying to get people away from. 

I think that Angela Duckworth’s work is incredible. I think that people should go read Grit. I think it’s an amazing book, and I think she agrees with me. I don’t think that she thinks that this sort of blanket application of grit is good, period – is healthy for anybody. I think it’s a misinterpretation of what she’s saying. We can take the same type of thing into goals. Goals are good, but not everything about goals is good, because nothing is that categorical. 

[00:50:59] Kevin

You talk about this unbelievable example of a woman who is running, I think the London Marathon, and she breaks her leg. 

[00:51:11] Annie

On mile eight.

[00:51:12] Kevin

On mile eight and finishes it. You’re thinking, “Wow, that’s a crazy story.”Then you’re like, yeah, but there’s a lot of them. There’s a lot of people who break their legs and run marathons. 

[00:51:25] Annie

Yeah. Siobhan O’Keeffe… This is the thing about goals, is that goals create a target for you. They create a finish line and, as you said, they get you to be more gritty about it. They get you to stick to it and head toward the finish line. The problem is again, that some finish lines aren’t worth continuing on toward anymore. The problem is once you set a finish line, when we think about what it means to be in the losses, if I have a finish line and I’m not there yet, I’m in the losses cognitively.

If I break my leg on mile eight of a marathon, I am 18.2 miles short. It doesn’t matter that I ran eight miles. Cognitively I’m in the losses. If I’m 300 feet short of the summit of Everest, I am in the losses in comparison to that finish line. I’m 300 feet short of my goal, therefore I’m in the losses.

This is something Richard Thaler talks about, is that goals are graded, pass/fail. This is a really big problem with them. It doesn’t matter that I climbed 29,000 feet. I didn’t get to the summit, so I failed. It doesn’t matter that I ran 20 miles. If I didn’t run 26.2 miles, I failed. Siobhan O’Keeffe, who’s the woman that you’re talking about, was running in the 2018 marathon. Her fibula bone snaps on mile eight. Everybody tells her to stop the race, obviously. She’s got some good quitting coaches, i.e. doctors who are like, “Hey lady, stop running.” This is what you lose sight of, and this is what we lose sight of even in the investing world, is that by continuing to run the race she’s risking all the other races that she might ever run again in her life.

By continuing to stay in an investment that’s no longer worth it for you, that’s money and capital that you cannot free up. There’s a cost to that capital. You cannot free it up to go put it in another opportunity that would do better. We get so focused on wanting to get our money back, or having some goal, or not wanting to quit in the losses in the stock that we have, that we don’t move it over into something better.

This is what’s happening to Siobhan O’Keeffe. She’s short of the finish line. She’s in the losses. She does not want to close that mental account in the losses so, she won’t quit the race. Then three other people in the London Marathon did it as well. If you search other marathons, you’ll see that there are people finishing marathons with broken bones left and right because there’s a finish line and those goals are graded as pass/fail. 

Let’s be honest, Kevin, when you hear this story about Siobhan O’Keeffe, I know it sounds a little nuts, but isn’t there a part of you who’s like, “Man, I wish I had that kind of grit?” 

[00:54:01] Kevin

Totally. 

[00:54:04] Annie

Totally right. 

[00:54:05] Kevin

I thought, what mental… What’s the word?

[00:54:10] Annie

Fortitude, I don’t know, toughness. 

[00:54:18] Kevin

The pain. I think like, I’m a tennis player. 

[00:54:21] Annie

I love tennis too. I’m a total tennis player. 

[00:54:24] Kevin

Yeah, I’ll get hurt and keep playing. 

[00:54:27] Annie

I played district championships with a broken toe this year. 

[00:54:33] Kevin

I wanted to ask you about this kind of myopia. Do you think that’s something that we get a little bit better at as we get older? I checked Wikipedia and you and I are roughly the same age. Actually, we were both born on September 13th, making us Virgos.

[00:54:55] Annie

There you go.

[00:54:56] Kevin

Is it easier now (I’ll ask it a personal question), as you get older? The finish line maybe is a little bit closer and okay, it’s more clear that if I’m doing something now, I’m not doing something else. ‘A’, do you think that’s right? And ‘B’, how would you try to convey that, in a more fundamental way, to someone who’s further from the finish line, like a younger person?

[00:55:25] Annie

Okay, yeah, I think there’s two ways in which we probably get better at this as we’re older. Let me just say (not based on scientific research), this is my opinion. I just want to say that. 

I think the first thing is that a lot of these issues have to do with, again, what your own finish line is. That time is a limited resource, and we all have this limited resource. I know it feels really bad when you quit something because you feel like you failed. What you have to remember is that life is just way too short to be spending your time on things that aren’t worthwhile. You need to quit those things to go do things that are. Overall, that’s really what our long-term goal is. It’s not to grit out some horrible thing that’s making us unhappy for no reason other than we think it builds character. It’s to get us to somewhere that resembles a fulfilling and happy life. 

If the thing you’re doing is hard but, ultimately, it’s going to get you closer to that goal of a fulfilling and happy life, you should be able to stick it out. But if it’s not, you have to switch as quickly as possible because you don’t have that much time.

I do think that as you get older you become a little bit more aware of that. I imagine that’s helpful. The other thing that I think is really helpful is that I think as you get older, and more experienced at things, you’re better at seeing finish lines that are worth it and finish lines that aren’t. You can see over the obstacles to say, “Is this something that, actually, when I get there, is going to make me happy? I know this is really hard now, but when I’m finished, how am I going to feel?” I think having experienced these things over and over again, it’s easier to see. 

I’ve started books that I haven’t finished because I’ve realized that even when I’m done, I don’t think this is going to be a good enough book. I’m not going to be that proud of it. I don’t know if I’m saying something important enough. I’m able to work through that. Whereas something with, ironically, a book called Quit, it’s hard to write a book. There are a lot of really dark moments in it where you feel like you can’t find your way through the material, or out of it, or figure out how to say something right, or you’re butchering it, or whatever that stuff is. But, I could see to the end of it in a way that, maybe if I weren’t so experienced, I wouldn’t be able to. I think it creates a little bit better taste. That’s one of the reasons why I say when you get a quitting coach, find someone who’s experienced, who’s seen it before. They’re going to be able to see what that more long-term outcome is supposed to be for you in terms of whether it’s worthwhile to keep going at it.

Don’t just find a random person on the street. Find someone who’s been there, done that, seen that, over, and over again. It’s part of the value of mentorship. I think that’s where it can become really helpful. One of the things that people used to say in poker, when they were coaching me, and then I then started adopting for myself was that “poker is one long game.”

It was meant to get you to realize that it doesn’t matter if you win this particular hand. You’re going to play thousands of hands over your lifetime, like tens of thousands of hands over your lifetime. What matters is, again, what happens in the long run, because it’s one long game. I think that the more experience that you have, the more that you’re capable of getting yourself to see the world that way. I hope. Again, that’s not science driven. I will say again, one of the things that the science tells you is that on average, across all people, we just stick. We’re really bad. We do get better as we get older. 

Actually, wait a minute, I take it back. There is some evidence that this is true.

[00:59:15] Kevin

That we get better as we get older?

[00:59:16] Annie

That we get better as we get more experience. I’m going to give you some evidence. I just realized it’s actually in my book in a footnote, or in an endnote. Okay, there was this really cool study that was done by Colin Camerer, along with a variety of collaborators including Richard Thaler (Nobel Laureate), and it was on behavior of New York City cab drivers in the 1980s.

This is a super fun study. They collected thousands and thousands of trip sheets. What they wanted to understand, the way the cab drivers work is they rent the cab for 12 hours. They don’t normally own the medallion, the license for the cab.

This was before Uber. I had to go to you like “Kevin, you own the cab,” and I’d be like, “Okay, I’m going to rent it from you for 12 hours, in 12 hour shifts.” Now what that means is that the drivers are choosing when they drive and don’t drive within the 12 hours that they have the cab rented. In other words, they’re not driving for the whole 12 hours. Lucky for us, because I wouldn’t want to be in the cab with the person on the 12th hour, but whatever. 

They just wanted to understand, are they optimizing their time? In other words (to your point that you said before about poker players), they want to maximize the time they’re playing well and minimize the time they’re playing poorly. Cab drivers want to maximize the time they’re driving when there’s lots of fares around, and minimize the time that they’re driving when there’s no fares around. They wanted to know, are they following that strategy? No, they’re not. 

What they discovered was that it was pretty bad. They were actually minimizing the time that they were driving when fares were good. In other words, when they saw lots and lots of fares coming in quickly, the drivers were quitting really quickly. Then they were maximized in the time when the conditions were bad. In other words, when the fares were coming in very slowly, they saw that they were driving for a very long time.

This was so bad that compared to an optimal strategy, they would’ve made 15% more if they had followed the optimal strategy. Compared to random, which is your point about, would some investors be better just selling randomly from their portfolio? Probably they would’ve done 8% better if they had just been like, I’m going to drive five hours a day. They would’ve done 8% better or eight hours a day, they would’ve made 8% more money. 

What was going on here? We can go back to this goal problem, is that they had set an earnings goal. Let’s say it was $300 in a day, and when they hit it, they stopped because they passed. Remember, goals are pass/fail, and when they didn’t hit it, they just kept driving on forever trying to hit their earning goal. Also this is, quit while you’re ahead, keep going while you’re behind – all of these things go in here. What the researchers did find was that the more experienced drivers made that error less.

Now were they perfect? No, but they were better than the less experienced drivers. There is the evidence that experience can get you better at these decisions, but you’re still going to be very, very far from optimal. Which is why, no matter how old you are, no matter how much experience you have, you should be trying to get some rules in place like kill criteria, get quitting coaches, that kind of thing, that are going to help you.

But yes, the cab drivers got better with more experience, which I’ve now just remembered. Now I have some evidence for you that what I was saying was not made up. 

[01:02:43] Kevin

There’s a smidge of hope for people like me who are a little more seasoned. Hey, listen, I know one of the biases we have is it’s difficult to quit on time. I’m struggling with that now, but I really do want to be respectful of your time and wrap things up. I’m really glad you persevered, and did not quit, and finished this book. It’s actually one that there’s definitely obviously lessons for the investment community and for everyone in general.

Thanks so much for taking your time and being with us today, and we really wish you all the best. 

[01:03:28] Annie

Thank you so much. This was such a great conversation. 

[01:03:31] Kevin

Thanks. Okay, with that, I’m going to pass it back over to Neil’s. 

[01:03:36] Niels

Thank you so much, Annie and Kevin for a wonderful conversation on the topic of decision making and when to quit. 

Annie has a fun and easy to understand way of explaining what can be a complex topic. I really loved this conversation. Of course, she had been cheering loudly when she said, “The more you can systematize the decision, the better off you will be. This is, of course, trend following in a nutshell. But without knowing why this statement makes sense, no one is likely to change that decision making. I’m hopeful that all of you listening today took notes when she explained the reason this statement is so true. 

The fact that we usually don’t get rewarded for quitting is something we should all internalize and perhaps change in our own lives as we pass on our experiences to the next generation. For all of you who already love and believe in trend following, we now have new terms for a stop-loss, namely kill criteria, or turnaround time and stories to go along with them that we can use in our investor conversations. 

Make sure you go and follow Annie’s and Kevin’s work, and make sure you go and get a copy of their most recent books, because as you can tell from today’s conversation, some of these ideas and topics are not being discussed enough on mainstream media.  From Kevin and I, thanks so much for listening and we look forward to being back with you in the next episode, and in the meantime, take care of yourself and take care of each other. 

[01:05:08] Ending

Thanks for listening to Top Traders Unplugged. If you feel you learnt something of value from today’s episode, the best way to stay updated is to go on over to iTunes and subscribe to the show so that you’ll be sure to get all the new episodes as they’re released.

We have some amazing guests lined up for you, and to ensure our show continues to grow, please leave us an honest rating and review in iTunes. It only takes a minute, and it’s the best way to show us you love the podcast. We’ll see you next time on Top Traders Unplugged.