Annie Duke Mix 1.mp3

Annie Duke: [00:00:00] It's not that imagining failure causes failure, it's that imagining failure causes success, because if you imagine failure, you can see all the obstacles that might be lying in your path and then you can actually do something about them before you run into the obstacle.

Harpreet Sahota: [00:00:40] What's up, everybody? Welcome to the artists of Data Science podcast, the only self development podcast for Data scientists. You're going to learn from and be inspired by the people, ideas and conversations that'll encourage creativity and innovation in yourself so that you can do the same for others. I also host open office hours. You can register to attend by going to bitterly dot com forward, slash a d. S o h. I look forward to seeing you all there. Let's ride this beat out into another awesome episode. And don't forget to subscribe to the show and leave a five star review.

Harpreet Sahota: [00:01:32] Our guest today is a poker champion turned author, consultant and corporate speaker who's here to teach us how to get comfortable with uncertainty and make better decisions.

Harpreet Sahota: [00:01:43] She's earned bachelor's degrees in English and psychology from Columbia University and pursued a Ph.D. from the University of Pennsylvania, where she was awarded a National Science Foundation fellowship to study cognitive psychology. All that changed a month before her dissertation defense, when she shoved her research into the trunk of a car, told her professors she'd be gone for a while and headed west, where she eventually found her way to a poker table as a former professional poker player. She was once the leading money winner among women, having won a World Series of poker bracelet, placing first at the World Series of Poker Tournament of Champions and winning the 2010 National Head's Up Poker Championship before retiring from the game in 2012 as an author. She's written six books, including Thinking and Bets Making Smarter Decisions When You Don't Have All the Facts and How to Decide Simple Tools for making better choices. She's also the co-founder of the Alliance for Decision Education, a nonprofit whose mission is to improve lives by empowering students through decision skills education. So please help me in welcoming our guests today, the Duchess of Poker and best selling author, Annie Duke. And thank you so much for taking time out of your schedule to be here today.

Annie Duke: [00:03:00] Oh, my gosh, I can't believe you pulled the Duchess of Poker out. Let me tell you a story about that when I started playing poker. It was like  1994-ish when I started playing for real. And then like, right around 2002, 2003, it ends up on television and all of a sudden like poker players end up with like agents, which was very weird. If you knew what poker was like in the nineties, the fact that anybody would have an agent would be pretty hilarious. And I had an agent, like the first agent that I had ended up switching really thought the idea of having nicknames people would be a really good idea. And so he tried to push the Duchess of Poker. And so it lingers around. But it wasn't something that I actually use for myself because I was like, now I'm starting to feel like I'm good, I don't need the dictator. And so I think it's hilarious that you get that one out because it does that you sort of see it around the Internet a little bit. It's like this lingering thing from like two thousand three, that kind of sad stock.

Harpreet Sahota: [00:03:55] You could thank Wikipedia for that one. Yeah, exactly. That one from. 

Annie Duke: [00:04:00] I love that you brought it up though, I think it's so funny.

Harpreet Sahota: [00:04:02] Yeah, I used to watch poker after dark. A lot of 

Annie Duke: [00:04:06] Love that show! Mori Eskandani is the producer of that, is such a gem.

Harpreet Sahota: [00:04:09] Yeah. And I've never seen you on that show when I was coming up. And yeah, they use that term, I was like, that's such a cool, cool name. Duchess of poker. But before we get into the awesome work you've done, I was wondering if you could talk to us a little bit about where you grew up and what was it like there?

Annie Duke: [00:04:26] Sure, yeah. Oh, that's interesting. So, yeah, I grew up in the capital of New Hampshire called Concord, not to be confused with any of the Concords and the US. We pronounce a little differently in New Hampshire. I grew up in there, but it's a little I had kind of a weird existence because of what my dad did. So my dad had grown up in West Philly, actually went to West Philly high. His father never I think he I'm not sure if he finished sixth grade. If he did, that was the last grade that he did. And then my dad actually went to West Philly high and then ended up going to Haverford College and then made his way to Harvard. So it's like I think it's like a real American story of like the sixth grade education to my father ended up with a PhD like one generation.

Annie Duke: [00:05:10] So my dad got into Harvard Law. And after the first year, he decided to drop out because he realized that his passion was teaching. And he he really wanted to be an English teacher, much to my Jewish grandmother's dismay.

Annie Duke: [00:05:24] But so through that, he ended up getting a job offer after he got his master's in English at Harvard to teach at a school called St. Paul's School. And St. Paul's School is one of these like elite boarding schools. It's actually Episcopalian. And, you know, it's kind of like what you would imagine if you'll think of, like the Dead Poets Society or something. Right. Like it's what you would imagine in terms of an elite boarding school. Only five hundred kids go there. A lot of kids from like New York go there. So my dad got hired there. And gosh, I think it was like nineteen sixty one and he was like the token Jew, like he was diversity at the Episcopalian school. So I had this very strange existence because obviously my grandfather was not he wasn't really wealthy. So we, we were obviously living on a teacher's salary. There were a lot of perks but so we were not rich. But I grew up among incredibly wealthy people. So that juxtaposition was always kind of interesting for me. I was at an Episcopalian school and obviously Jewish. There were just all these kind of like schisms in my life, which I think is an. Actually, I think a pretty good code switcher because of that. So anyway, one of the huge perks that came with teaching at St. Paul's was that your children could go to that school for free and that school cost as much as a college cost. So this was this was a really big deal that we got to go there for free. So I went to St. Paul's has been my siblings and then went out to Columbia after that financial aid.

Annie Duke: [00:06:57] And that financial aid turned out to be a bit like an amazing turning point in my life, because as part of my financial aid package, I needed to do work, study and applied for a back to work study jobs. And the one that I ended up taking was with a professor whose name was Barbara Landow, who was a professor in the psychology department, and she had done her work at the University of Pennsylvania with Leland Henry Gleitzman. And I actually ended up being a research assistant for four years. And then she really encouraged me to go to Penn and pursue my own Ph.D. there and a super random it was like I was kind of trying to to choose among work, study jobs, and I just happened to click with her. I don't think that at that time I was thinking, like, I want to pursue an academic career. But she really encouraged me to do that. And I'm incredibly grateful for that, for that stroke of luck.

Harpreet Sahota: [00:07:44] And what kind of kid were you in high school and what did you think your future was going to look like when you're in high school?

Annie Duke: [00:07:52] So it's kind of interesting because well, I mean, I was a good student, but I really felt very keenly that I was a misfit at that school. I went to school with people whose last name was Rockefeller. My dad was like Richie letterer from West Philly. I think I would say that I would call myself disaffected. So I was kind of disaffected. I had this real drive to make sure that I was succeeding academically. But I think socially it was very tough for me, partly because I didn't quite fit in with the people who went there, and also partly because I think that I just felt those differences. So I was just like, I don't think I was particularly super friendly either. Right. I was sort of saying to myself, and then the other thing about sort of my high school was my childhood was that we did not have necessarily the most functional family because my mother was a pretty severe alcoholic all the way through when I was through high school. So I don't know. It's interesting because I don't think that I was really thinking about, like, what do I want to do with my life? I was thinking like, I need to succeed and get good grades right now so that I can go to the school that I really wanted to go to.

Annie Duke: [00:09:00] By the time I was in junior year, I knew I wanted to go to Columbia because I really wanted to be in New York during the summer between my junior and senior year in high school, I actually lived in New York and did temp work for a summer, really fell in love with the city. I wasn't doing an internship or something, but then in my senior year, I had the opportunity to do like an independent study. And so I spent the winter term of my senior year actually working with autistic kids at Bellevue Hospital, kids who were so severely autistic that they were hospitalized at that time. And so I started feeling like I wanted to do something in psychology, kind of at that point. But I don't know that I ever had like a super broad plan, because I think at that time I was really focused on just getting out of the situation.

Annie Duke: [00:09:44] When I was in, I wasn't particularly happy at the high school that I went to, despite the fact it was an amazing education for sure.

Annie Duke: [00:09:51] But I wasn't particularly happy at that high school and my family life wasn't like super happy either. And so it was like, let me just get to the next stage and then I think I'll think about it then.

Harpreet Sahota: [00:09:59] Thank you very much for sharing that. So from this set of experiences, very unique experiences you had, how did the inspiration or idea or did those experiences cause an inspiration or idea for your books?

Annie Duke: [00:10:11] Oh, that's really interesting. I mean, obviously, the main thing that's kind of going on in my books is this conversation between my graduate work and cognitive psychology and poker for sure.

Annie Duke: [00:10:21] I think there are a lot of reasons, though, that I was attracted to poker because we did the happiest times that I had as a child were actually playing cards with my family. And my father is an incredibly lovely man, by the way. And my mother, actually, she had many good qualities. She had a disease. She's very funny. She was very smart, super smart. So she had a disease. So what are you going to do? So I think that I have been playing cards a lot when I was young. And so I obviously sort of had some experience with that as I was going into when I went into poker. But I do think that there's this kind of theme of uncertainty and the limited ability that we have to predict the future and that if you can get a little bit better at that kind of anticipation, you're going to be better off. If anybody who's grown up with somebody where their parent in a situation where their parent is really alcoholic, one or both of them, what they know is there's a much higher level of unpredictability that occurs in those situations where in particular with my mom, you didn't quite know what kind of mood was she going to be in from one day to the next.

Annie Duke: [00:11:25] The other thing was that the mapping between kind of like your days and what her reaction to the. We're I think we're much more capricious, so I could clean the kitchen and get yelled at or I could clean the kitchen and get a lot of praise, right. So there was a lot of noise and that so I think that you just kind of get super used to like that kind of uncertainty. And you're really focused on like, how do you navigate that in a way that I think was certainly helpful for me later in life. But also I think that that undercurrent informs everything that I write about, like, you know, the fact that the future is much less predictable than you think. It is certainly something people are finding out during coronavirus that maybe things aren't as predictable as I thought they were. But I think that that's something that I was kind of growing up with in that sense.

Harpreet Sahota: [00:12:07] So speaking about cognitive psychology and poker, kind of a big question here. So I apologize in advance. What are bets, what are decisions and what's the relationship between them?

Annie Duke: [00:12:19] Oh, well, OK, so decisions are bets. So there that's. Easy.

Annie Duke: [00:12:25] What do I mean by that? Betting is basically saying I have some set of limited resources that I can invest in to and I'm choosing among options where how that option turns out is not deterministic.

Annie Duke: [00:12:40] It's probabilistic. So you can think about a flip of a coin. If I flip the coin, heads or tails very occasionally on land on the side will ignore that. So let's just make heads or tails and so I can think about when do I want to invest my money into a coin flip, even though I can't control whether it lands, heads or tails, I can think about whether it's an appropriate investment.

Annie Duke: [00:13:00] So that would depend on what the pricing situation was.

Annie Duke: [00:13:03] So if for every dollar I won, I lost the dollar, that would be a pure gamble, meaning I should break even in the long run because it will land heads or I'll be 50 percent of calling heads or tails. If I were offering you a dollar and twenty five cents for every dollar that you lost, I would be losing. That would be a pretty bad option for me to invest it. And if you were offering me a dollar and twenty five cents to my dollar, obviously I should do that all day because I will make twelve and a half cents on every flip. So but we don't need to just think about it as money because you're also investing things like your time, which is going to give you the best return on your time. We make choices about different types of return, return in health, for example, return and happiness. So it's not just returning money, but it's essentially saying I can't choose every single option at once because I don't obviously I can't do that because I'm a finite human being. And so what options am I going to invest whatever my resources are in which in a probabilistic sense are are going to get me the best results over time? That's what it is.

Annie Duke: [00:14:04] That's also happens to be what a decision is. And then we can take it a little deeper and say what that means is that that's a decisions both because they're the same thing. The decisions are really at their core forecasts of the future. So, like, if you're trying to decide between the chicken and the fish, what is that decision? It's a decision that for whatever the returns are that you're trying to get from your meal, it could be that choosing the healthiest thing would be choosing the tastiest thing, whatever, that when you choose one option over the other, it's on a probabilistic it's going to be more likely to return you the thing that you would like. So if I'm choosing between chocolate cake and a vegetable stew and my choice is health, presumably I'm going to choose the vegetable stew because I think that that's going to return me more health than, say, the chocolate cake. I might decide when I choose a chocolate cake if my value is I want something that's the tastiest. That may be the thing that I choose. And then often the choices are closer than that. It's like chicken breast versus fish where the health is going to be similar and you're thinking about enjoyment.

Annie Duke: [00:15:09] And whichever one you choose is just a prediction of which one's more likely to produce a happier version of you in the future.

Harpreet Sahota: [00:15:18] What's up, artists? I would love to hear from you. Feel free to send me an email to the artists of Data Science Gmail dot com. Let me know what you love about the show. Let me know what you don't love about the show and let me know what you would like to see in the future. I absolutely would love to hear from you. I've also got open office hours that I will be hosting and you can register like going to bitly.com/adsoh I look forward to hearing from you all. Look forward to seeing you in the office hours. Let's get back to the episode.

Harpreet Sahota: [00:16:06] So speaking about options and outcomes, I think maybe more than sometimes we tend to look at an outcome and we work our way backwards. If it's a particularly bad outcome and we say to ourselves, hmm, I think I made a bad decision. So why is that really not a productive way of thinking?

Harpreet Sahota: [00:16:25] Yeah. So we can just go back to this relationship that I just said. Between the decision you make and the outcome that you got, right, so we're living in a probabilistic world, meaning that there are very few decisions that you can make that are guaranteed to have one single outcome. So even if you make a decision, we're ninety nine percent of the time you observe a great outcome and one percent of the time it works out disastrously. One percent of the time it's going to work out, really. But that's the whole point. We don't have control. So if you take, for example, a situation like Carroll in the Super Bowl in 2015, he chose a pass play. That play was going to be intercepted somewhere between one and two percent of the time. And it got intercepted because by definition, one to two percent doesn't mean zero. It means one to two percent of the time. And we don't have any control over when we're going to observe that that difference. So we're kind of pulling apart the relationship. We're just correlating decision quality and outcome quality in the short run. Obviously, in the long run it plays out, but not in the short run on any individual decision.

Annie Duke: [00:17:36] But as decision makers, we act like those things are correlated, essentially one. So if we know what the outcome of a decision was, it was intercepted.

Annie Duke: [00:17:45] What we think is that that tells us something really significant about the outcome, about the quality of the decision that actually undergirds it. And the interesting thing is that what's really happening there is that we're ignoring this uncertainty that we've been talking about, because if you take a game like chess, for example, that doesn't really have a strong influence of luck and certainly not a strong influence of information, because you can see all the pieces. If you and I play a game of chess and I lose, we actually do know something about my decision quality. Why? Because the pieces don't move around at random. They only move by acts of skill. So the only way I can lose is if the way that I was applying the skill elements of the game was worse than the way that you were applying the skill elements of the game. Which is why we can actually work backwards in a game like chess. But in a game like poker, we cannot do that because if you and I play poker for about the same time that a chess game would take, say, an hour, we know that I can lose because I'm Pete Carroll and two percent occurred. Right. Like I could only lose if exactly. The Queen of clubs hit on the last card. And lo and behold, the queen of clubs hit on the last card.

Annie Duke: [00:18:54] Now, with any of these things, if you get enough iterations, obviously, right. If we played poker for fifteen hundred hours and I lost to you, we could say something about my decision capabilities compared to yours, but certainly not in the short run. And yet we see example after example where this is actually how the mind works, that we get this real cognitive illusion. That outcome really does tell you something about the decision quality. That's certainly true in the case of Pete Carroll, because five years later, we're still talking about it. But another recent example that actually from the 2016 election is really interesting. So we know that Hillary Clinton lost it and we know that she lost three states in particular, which were Pennsylvania, Wisconsin and Michigan. So knowing that the general consensus in the country is that she bungled her campaign and she made a huge mistake. So the question that we want to have is, will if that resulted, right. Are we under the same cognitive illusion that we are in the Pete Carroll case where she calls a play that's only going to fail in the particular way that it did between one or two percent of the time? And it fails in that way. And we all think he's an idiot. Right. This is the same thing happening in the analysis of Clinton. And the Clinton example has an advantage over the Pete Carroll example, which is because the Pete Carroll example is happening in real time.

Annie Duke: [00:20:14] There's no way to go back and look at the quality of his decision, except in retrospect, because because it happened very quickly. But presidential campaigns actually happen over quite a long period of time. So I don't know. Can you think of anything? Where in real time, people are analyzing the decisions that are being made more than a national presidential election,

Harpreet Sahota: [00:20:38] Not off the top of my head, no

Annie Duke: [00:20:39] No, no. Right. So and we can see this now with Biden and Trump. There's lots of stuff being written on Biden's campaign strategy and Trump's campaign strategy. Lots of people talking about the polling. What is the polling mean? What are the odds of different things happening? It's just I mean, there's volumes written on it. So here's the thing with the Clinton case. What I would argue is that if she were making a really big mistake, spending proportionately many more resources in places like Florida and Arizona and North Carolina and New Hampshire and Virginia than she was in Michigan, Wisconsin and Pennsylvania, if it were true that that was a huge mistake, that somebody certainly would have written about it at the time. Do you agree with that? Yeah, right. Because you only have the information that you have at the time that you're making the decision. And at the time, places like Florida and New Hampshire, where polling is tossups in places like Wisconsin and Michigan in particular, were quite far ahead.

Annie Duke: [00:21:32] Pennsylvania, slightly less so. So when you're thinking is a campaign like where do I want to be spending my time, obviously you're trying to swing these close states right over over into your column. And at least as far as I can tell, because I Googled it, there isn't nobody's writing that I can you believe what an idiot. Clinton is because she's not spending time in these three states. Now, there are plenty of articles that say that, but the first one that appears is on November 9th, which is the day after the election. So this is how you can tell this is a case of results and it has an advantage over the Pete Carroll example, because we can we actually have a record of what people were thinking at the time. And we know that across those two states, she lost those two states by something like seventy seven thousand votes, somewhere around eighty thousand votes literally across the three states. So those could have gone either way. The chance she loses all three seems like pretty infinitesimally small. I mean, I'm sure they're correlated with each other to some degree. But I think one of the states she lost by 10000 votes, like how can we like what?

Annie Duke: [00:22:36] And there was a polling error for sure in those three states. But there wasn't a national polling error. There wasn't any polling error in Florida. There was no polling here in New Hampshire. Like all of these other places actually polled. Fine. Some places actually the polling error looks like it was in the other direction. So as I recall, Virginia, actually, she won by a much bigger margin than the polls had predicted. So sometimes it goes in the other direction. This one happened to go in Trump's direction. But here's the thing about a polling error that everybody seems to forget is that you don't know there's a poll here until the vote has been taken, so you can only know it after the fact. So I think this is a really good demonstration of resulting. And you can see what the sort of after effects of that are.

Annie Duke: [00:23:19] I mean, first of all, we're calling something a bad decision that probably wasn't that bad because that's the only thing we have to learn from it. But it also affects future decisions, right? You're hearing people talk about like, oh, you have to spend all your time in Michigan, Wisconsin and Pennsylvania, but that might not be the right strategy for this particular election. In fact, I would argue that the polling in those three states are probably going to be much more accurate. They were in 2016 because the pollsters have the scars of finding out that they had underrepresented white voters without a college degree. And obviously they're working to correct that. So you might expect actually to be more likely to have a pull in here somewhere else.

Annie Duke: [00:23:55] And this is a really important resource allocation question because campaigns have limited resources. So if you're taking the wrong lesson from 2016, you may actually be mis allocating your resources. So these things have really big effects.

Harpreet Sahota: [00:24:08] So seems kind of irrational to take that line of thinking where we say, OK, bad outcome means bad decision. So I think that kind of is part of the..

Annie Duke: [00:24:16]  Well, not if you're playing chess,

Harpreet Sahota: [00:24:19] But why is it that our brains are not built for rationality?

Annie Duke: [00:24:23] There's a broader question, but let me talk about the resulting problem. Just really quickly. So the thing is that decision quality is actually just really hard to derive. If you take the Pete Carroll example, I can assure you that if we do the math, the past play was actually an incredibly high quality decision.

Annie Duke: [00:24:38] But it's complicated. I have to sort of draw out the whole decision tree. And what that means that I have to ask you to kind of hold your horses on applying causality, any opinion about causality, and start to explore all the other branches of the tree. Right. Not just the branches where he passes and it's either caught for a touchdown or it's just incomplete. But also like the branches where he hands it off to Marshawn Lynch or it's fumbled or whatever. Likewise with Clinton, what are all the branches where she has the exact same strategy in those one or two or three of those states in her direction, in different combinations? That's a lot of permutations where maybe she wins Florida.

Annie Duke: [00:25:20] Right. There's a whole bunch of counterfactual worlds that could exist. And that's just hard. It's hard to do that when we know what the result is. Once we there's many possible futures, but there's only one past. And that one path ends up taking sort of all of our cognitive landscape like it fills our our cognitive landscape such that we sort of forget we sort of take this cognitive chainsaw to other all these other branches, all these other worlds that could have unfolded. And we lap those all off the tree. In our left is the trunk.

Annie Duke: [00:25:51] Now, once we know sort of what the path looks like, in other words, the thing that actually happened and Michael Mauboussin talks about this quite a bit, there's a part of the brain, what you call the interpreter, which immediately tries to apply causality to things.

Annie Duke: [00:26:04] And what the interpreter is not going to do is if that was unlucky, that was random. It's just kind of not the way our brains work.

Annie Duke: [00:26:10] Our brains really like to make a create a narrative that makes sense where one thing leads to another and kind of an orderly fashion. We really aren't comfortable with randomness. And when we think about some of the other kind of biases like illusion of control, which is just that we have more control over our future than we do, an explanation of randomness doesn't really fit in with that narrative. And I think that it just sort of make you feel like you don't have the world to sort of out of control. And interestingly enough, that's actually related to why people believe conspiracy theories, because the explanation of just crap happens like random. It happens and sometimes there's not like a global cabal that's trying to make it happen is I think kind of it's just not satisfactory to the way our brains work, which are really causal interpreters. That's what we are. We're pattern recognizers. There's actually a word called apophenia, which is seeing patterns where they don't exist, which obviously conspiracy theory, belief is in there.

Annie Duke: [00:27:12] And it's not correlated with intelligence because it is just kind of like some people are more so, some people are less so, but we're all sort of causal interpreters. And in fact, some of the most intelligent people are the most adherent to conspiracy theories, because I think that they can create they can give these arguments and narratives for patterns.

Annie Duke: [00:27:31] So that's I think that's specifically why it is like resulting occur. But in terms of the other biases, it's just you always have to trade off efficiency and accuracy. And, you know, I mean, I think the classic example is just we're sort of built for false positives, right? Where sensitivity for us is very high and we sacrifice specificity for that. When if you have some suspicion that lion is coming, we just assume the lion is coming and we run away. And that's been quite good for the survival of the human species. So and there's all sorts of other things that are like that.

Annie Duke: [00:28:04] So a couple of things I want to dig into there, first of all, to start with the beliefs, it seems like we tend to inject a whole bunch of our own beliefs into any decision making process. So how could we turn decisions inside out so that we can free ourselves from our beliefs?

Annie Duke: [00:28:19] Yeah. So I'm glad that you asked this, because I think this is one of the main differences between Thinking in Bets and How to Decide - between those two books. So Thinking in Bets is about a lot of it was kind of like an exploration of luck. Kind of what's happening when you're sort of injecting luck into the equation such that we don't really know what to say about why an outcome occurred. Because these things are kind of pulled apart and How to Decide I took as a place where I could really explore this kind of information or belief, knowledge, sort of how that is influencing things and kind of a bad way or maybe a good way if we could make it do that. Think about that, that sort of intervening at a different place in the decision cycle. So luck is intervening after you choose a particular option, like you flip the coin and well, it's in the air. I say heads, whether that lands, heads or not is a matter of luck. So luck is intervening between the decisions that you make, the option that you choose, and the particular outcome that you happen to observe.

Annie Duke: [00:29:17] That's where luck is intervening. Where the problem with are the information that we have is intervening. Just the fact that it's incomplete in comparison to our nations where that's intervening is actually before that. So you've got your beliefs and those are informing the options that you choose. They're informing what you think your goals are, your beliefs. And I don't mean beliefs like religious beliefs and beliefs, like all the knowledge that you have combined, the models that you have of the world, the perspectives that you have, the world, the way that you think about the world. And that's what I mean by beliefs. I just want to clarify that. But it's informed and kind of all the process that happens afterwards. What do I think my resources are? What are the goals that I'm trying to reach? What are my values? What are the different options that I can consider if I'm considering a particular option? What do I think the possible ways that are reasonable that that option could turn out or what? I think the probability of those things occurring, that's I'll be informed by your beliefs.

Annie Duke: [00:30:14] So you have your beliefs and then you have the option that you choose. And incomplete information is intervening at that stage. OK, so we can build an amazing decision process. I think I can map out for you. Here's how you make a great decision in terms of the process of choosing options and if the foundation that that process is sitting on is faulty.

Annie Duke: [00:30:38] I don't care how what I built you right. Because the data, the inputs into that process aren't going to be good. And that's essentially the situation that we're in pretty much for every decision that we make,

Annie Duke: [00:30:50] Because the foundation of that process is your beliefs and your foundation has two weaknesses.

Annie Duke: [00:30:59] Weakness number one is a lot of the stuff that you believe literally, right this moment in your life is accurate.

Annie Duke: [00:31:05] And of course, we know that that has to be true because we can think about both the arc of humanity, like humanity as a whole, used to believe that the sun revolved around the earth. And now we don't think that anymore. So there's something that used to be an actor that we've corrected. And we know that there are going to be things like that, that we're going to find out in another five years, 10 years, one hundred years. Gosh, when it comes to coronavirus, it seems like you have to revise your beliefs every week or day. I don't know. Information seems to be coming quite fast and then go about your own life, not just the arc of sort of human knowledge. I know for me there's lots of stuff that I believe very strongly when I was 20 that I look back at and I'm like, I was actually quite a bad thing to believe. So then it stands to reason that the beliefs that you have at this moment in time are not perfectly accurate. So that's the first problem with that foundation. And let's say that those are cracks in the foundation. The second problem that we have a society is a whole bunch of stuff we don't know.

Annie Duke: [00:32:01] So because we're human beings, we're not omniscient, we certainly don't know how the future is going to unfold. That's a bummer. But also, there's just like I don't know I don't know what you're thinking. I don't know what facts you have. I don't know, like all the knowable things there are about the universe. And this is actually a really big problem. Like this stuff I know could fit on like the head of a pin. And the stuff I don't know is like the size of the whole universe. I mean, that's just kind of the problem. And so we basically have this really sort of cracked and flimsy foundation that's informing the police that we have. If you think about, OK, so how can I kind of solve for that? And that's really where my focus is and how to decide.

Annie Duke: [00:32:34] It's not just obviously I show people how to construct the decision process, which is you need to have a sound decision process. But then most of the focus is on this issue.

Annie Duke: [00:32:43] How do you actually become more accurate in the things that you believe? Well, we know how we can correct both of those problems. If some of the things I believe are inaccurate. And I also don't know a lot of stuff, the way to solve for that is to go explore the universe of stuff that I don't know and to explore that in a really objective way where I'm kind of like maximizing my ability to run into information that is different than the things that I believe to be true, to run into new information. So corrective information, new information, and also to run into I really want to kind of maximally run into people who have different points of view than I do, because two people can be doing the exact same information and they can come up with very different answers about what it means because people are applying different mental models or mathematical models or algorithms or whatever. And so I'd really like to be able to collide with those as well, because that's where all of the antidote is to the problems with the foundation of the decisions that I make. So that's what I want to be doing. The problem is, is you previewed that we don't do that. We don't interact with that world in any kind of way that's actually random. It turns out that most of the way that we're sort of interacting with the world and most of the way that we're thinking about our own decisions is from the inside view. In other words, it's from the little speck of dust of the facts that we have, the knowledge that we have the police, that we have the models, the way that we've modeled the world.

Annie Duke: [00:34:08] And when we actually go and interact with the world of stuff that we that universe of stuff that we we maybe don't know, we do it in a specific way to shock, to specifically maximize the chances that we're shining light on something that already agrees with the belief that we have. We interact with new sources that agree with us.

Annie Duke: [00:34:28] We interact with people that agree with us. The way that we talk to other people actually reduces the chances that if they did agree with us, that we would know about it. So we're kind of trying to hide, avert our eyes from facts that disagree with us. We're not interacting with people who disagree with us. And if they do happen to disagree with us, we're kind of trying to make sure that we don't actually find out about it. And we can think about this is like an inside and outside problem, which is the world from our own perspective, from that little speck of dust. That is the stuff that we know would be the inside view. Right, is that we're interpreting things within the things that we believe to be true. And what that means, actually, is that we don't actually view information objectively, will interpret the information in order to fit our beliefs. And the antidote to that is the outside view, which is actually getting to this world of stuff that we don't know, things that are true of the world, independent of our own perspectives or the way that other people might be viewing the situation that we're in. And that's the thing that we're all that we just sort of naturally are trying to avoid. But if you really are thinking about how can I improve my decision making, what you have to do is say I need to stop avoiding that. I need to try to maximize how much I'm sort of getting to that outside view in order to discipline the view that I have myself.

Harpreet Sahota: [00:35:48] That's why being smarter makes it worse, because you're consciously going out there and you're looking for information. That's gonna support and just help fill in the cracks in those foundations of your beliefs.

Harpreet Sahota: [00:35:58] If your belief is that that motivated reasoning?

Annie Duke: [00:36:02] Yeah, so the only the only word that I would take out of that is the consciously.

Harpreet Sahota: [00:36:07] OK

Annie Duke: [00:36:08] So we're all everybody's doing this, right? So so what you mentioned is motivated reasoning, which is essentially you can think about like you have beliefs and models of the world that let's call it forms a trench and you're kind of down at the bottom of the trench. So you're trapped in it.

Annie Duke: [00:36:24] That's the inside of you. And ideally, what we'd like to do is climb out of the trench and go and look around at the world and say, should I change my trench? I try to climb. Is there something that tells me that this trench of men, that these very strong models that I have of the world, maybe I should adjust those, not do a 180 necessarily to like I'm totally wrong, but like, man, if you're two percent off and you can save that two percent, that's really going make a big difference to you and your outcomes. So we don't do that, though. So motivating reason is basically every sort of information is sort of floating above this trench. We're pulling the information down into the trench with us in order to interpret it in a way that supports that we have now. We can see that all the time in terms of the interpretations that people have in terms of politics. Two people can look at the exact same video of a presidential candidate. And depending on what your beliefs are, some people will say, look at that phrase, how brilliant they are. And other people will say, look, that that proves what an idiot they are. Or, you know, you get this all the time. It's like, what is your interpretation of whether somebody is being serious and what they say? Well, it kind of depends on whether you like the person you don't.

Annie Duke: [00:37:30] Right. So we hear this argument about Trump all the time where some people say, don't you know, he's just joking. Well, so people who like Trump will say he's just joking and people who don't like Trump will say this is a five alarm fire. We have to take this seriously. Right. And you can see that you're looking at literally the exact same information, but you're interpreting it to fit the model of what you believe to be true about the world. And this is happening not just when it comes to things like politics, but also just things like what route do you think you're supposed to take to work where when there's traffic and you're late, you'll be more likely to say, well, I just got unlucky there. Traffic, if you think that that was a good route to take. But if your spouse, who was arguing that you shouldn't take that route at all, runs into the same traffic, they'll say, see, that's proof that that was a bad route. Right? So that's like in a small place. But it's also true, just about like your sales strategies or the the business tactics that you're applying or whatever it is. The problem is the Data is not truth. How you model that. We have to model the data.

Annie Duke: [00:38:32] We're interacting with the data as human beings. And that's where motivated reasoning gets in the way, because the way that we're modeling the Data fits our beliefs. Now, this is this is not for the most part, sometimes it is, but for the most part, this is not a manipulation or anything conscious that's going on. We don't realize that it happens. The person that we're really trying to fool is ourselves mostly. And it's because that's what Dan Kahan calls identity protected cognition. So essentially, the way that we think about the world is actually kind of motivated to protect our identity and what is our identity. But the things we believe, I don't know that there is another thing. That's [inaudible]. It's like you believe certain things about the world and you have models of the world. And this is sort of who I am, what I believe to be true. And that's kind of your identity. And so each individual belief that you might have really kind of those are the threads that your identity is woven out of. And so you attack a belief. I have if I run into corrective information and I have to say that that belief is wrong, that's an attack on me. It's an attack on my identity. And we try to avoid this. Now, why is this worse for smart people?

Annie Duke: [00:39:40] Well, smart people are just better at spinning narratives. They're better at looking at a set of data and interpreting that data to fit the model that they already have. That's just why. It's just like this kind of narrative spinning that's kind of going on in our heads. And this reinterpretation, you know, it's just like smart people are just kind of better at doing that. And then I think the other issue for smart people is that by smart, I mean like people who are subject matter experts or you have something that sort of makes them stand out in terms of the thing that they're thinking about.

Annie Duke: [00:40:11] I think the other problem is that a subject matter expert is much more likely to take a glance at some data and sort of intuit what it means.

Annie Duke: [00:40:18] And because they are subject matter experts, I think they're much more less likely to question whatever the intuitive response that they are, that where they just sort of glance at the data table really quickly, whereas somebody who's maybe not so adept, for example, with Data is not going to have that same intuition about what the data means. And I think they're going to have to dig into it a little bit more because they can't rely as much on their intuition because they're they just don't have a lot of expertize in that area. So particularly, it's a particularly big problem for subject matter experts, actually.

Harpreet Sahota: [00:40:48] Thank you very much for getting into that. It was really, really interesting. There's another thing you talked about, which was probably the the thing that shook me to my core when I read Thinking in Bets. And you touch on it as well, in How to Decide. Just kind of about fielding the unfolding future, right? So I was wondering, you touched on a little bit before, but I was wondering if you could take us for a ride through the decision multiverse. What is this place and how can we navigate it?

Annie Duke: [00:41:15] Yeah, so the decision multiverse. So we touched on a little bit. It's basically this idea that for any option that you choose, there's many, many different ways that the future could unfold. So if we think about this like sort of as a retrospective problem, it's very difficult to understand what you're supposed to learn from any outcome that you observe unless you explore the multiverse. Right. Like all the other branches that the universe could have taken. Right. That could have unfolded. So there's a term for that which is called a counterfactual, which is essentially sort of states of the world that could have been but do not exist now. Right. So you could imagine a situation where let's say that you got married when you were 30. There is another branch that your life could have taken if you had married your high school sweetheart, for example. So that would be an example of a counterfactual that didn't actually exist. A famous counterfactual be like, what if Hitler had been born a girl, then what is the world look like? Right. So so we want to be thinking about when we're observing any outcome in our lives that the past is caught for a touchdown or we hate the college that we chose or we hired somebody and they quit within eight months like these, as we're sort of observing these outcomes that we're trying to learn from, it's really impossible to know what we're supposed to learn from the outcome unless we consider the other things that could have happened. So that's the multiverse, right?

[00:42:43] We have to sort of start exploring that multiverse in order to properly situate the outcome that happened in its appropriate context, which is the context of the other things that could have occurred. So one of the ways to address the resulting problem or the hindsight bias problem is to start to do this, to really think, given that I am observing a particular outcome, what are the other ways that the world could have unfolded? And in particular, I think that it's really, really important to do this with your good outcomes. The reason I think that that's so important is that I don't think we actually do that. And when you don't naturally do it, it probably means that you want to. So let me explain. If you have a bad outcome, then it's kind of nice to go explore the multiverse because contained in that multiverse, what you get to find out is possibly you made a perfectly good decision and you got unlucky, right? It could be, yes. I got in a car accident, but it turned out I actually ran a green light, not a red one. So think about sort of like if you think about psychic pain when you have a bad outcome, you feel really bad. You got in an accident. So if you go explore the multiverse and you say, where does this outcome's situated in all the other outcomes that could have occurred, obviously, if I went to a green light, most of those universes don't contain my haven't gotten an accident. So now I can see, oh, Pete Carroll and I hit a one or two percenter and it turned out that my decision was pretty good. So it's like you're trapped in this room of being sad and now exploring the multiverse allows you a door out that you can sort of walk out and say, well, now I don't feel so bad about that bad outcome because because it turns out I made a pretty good decision.

Annie Duke: [00:44:23] But think about that from the reverse. What if you have a good outcome, like, oh, I feel pretty good about myself.

Annie Duke: [00:44:29] If our natural assumption, if it's if I made it, if I have a good outcome, it must be because I'm an awesome sales person or I knew that that college was going to be amazing and that's why I chose that college. Or look at this employee. They turned out to be a superstar. I'm such a great interviewer and I can sift through CVS really well and whatever you can think about, kind of I chose the chicken and was amazing. Look at how great and choosing things off, menus, whatever it is, you feel really good. So what what are you benefiting in the short run from exploring the multiverse? If all of a sudden you find out actually the reverse is true, the chances are it's going to get the outcome that I got only two percent. And it turns out, like most of the ways the world is going to unfold were pretty crappy. Now, all of a sudden, you don't want you don't want to find the luck anymore, then you're sort of turning a win into a loss.

Annie Duke: [00:45:21] You don't want to have a door out of that room where you want. Now, of course, that's true in the short run, but it can't possibly be true in the long run.

Annie Duke: [00:45:31] What a disaster it is. If you have a really lucky thing happen to you and it turns out that you just don't explore that. And so you think it was a great decision and now you repeat a decision that's actually quite a poor decision, that has a very high probability of not working out for you over and over and over again. And I can tell you a real example. It's not so much anymore. I think that everybody would be embarrassed to say it.

Annie Duke: [00:45:54] But I know when I was growing up, I had people who really legit claim to. That they drove better when they were drunk legit because they had gotten home safely, and this is the thing that we're trying to avoid, right, is that there's all sorts of examples where I can tell you every single human being, me included, has a billion examples of thinking that they drive better when they're drunk. It's just that it's not as clear. It's not so obviously a bad decision that we're able to do that motivated reasoning kind of interpreted in that way because we're not actually looking at the times that we win and at the multiverse. Right. To say what are the counterfactual worlds here? Like, what are the things that could have occurred? And was this actually just lucky or was there a good decision?

Annie Duke: [00:46:36] Or by the way, a lot of times the is going to be I made a pretty good decision and I got a pretty good outcome. But it turns out there might have been a better decision.

Annie Duke: [00:46:44] That's certainly true in poker. You can be playing poker and be making decisions that are winning. But it turns out that if you made a different decision, you would have won more. And I want to know all of that stuff because my goal is how do I become a better decision maker in the long run?

Annie Duke: [00:46:59] Not how do I bask in the glory of my win every second that I possibly can in the short run. And this is something that I really like to get into in this book.

Annie Duke: [00:47:10] And the reason why I dig into it in this book is because I think that I didn't actually convey this well enough. It's my fault in Thinking in Bets where when I was talking about this problem, I had so many people say to me, and by the way, I'm relieved for helping them with their psychic pain. I had so many people say to me, You really like that book changed my life because I was feeling really bad about X.

Annie Duke: [00:47:35] And then I really, really explored it and thought about it and realized that I made a really good decision. And that really relieved me of a lot of pain. And I'm very thankful for that. The thing that concerns me is that nobody ever came to me and said I had a really good result and I realized I thank you because I realized I didn't actually make a particularly good decision that resulted in that. And wow, you really saved me from making that mistake over and over again. And I think it was I just didn't emphasize like that side of the equation very well. So when we're thinking about the multiverse, we have to apply that kind of thought. We have to think about those different ways that things could have turned out symmetrically.

Harpreet Sahota: [00:48:10] When I read Thinking in Bets, one thing that it helped me was this concept with like that unfolding future. In 2018, when I read the book, I was like heavy the job search process. I was really bummed out at my prior job and I was just looking for a new job. And every interview I would go to I would just think like, oh yeah, I'm going to get the job. Like it's just bound to happen. Right? And it helped me to just kind of recalibrate my probability of getting this job right. I have to set a baseline like, OK, well, I'm probably one of a hundred people applying for this job. So right from the get go, when they submit their application, I probably have a one in 100 chance of getting this job. And I would update and iterate the probability of me landing that job as I go through the process. Just help me help me to see this unfolding future.

Annie Duke: [00:48:54] Ok, so let's dig into that example a little bit, because that is the perfect example of why the outside view is so incredibly helpful. So the inside view was, I'm going to go to this job interview and I'm amazing and I'm a great candidate. And so obviously I'm going to get this job. That's how we all want to feel. Right. It's like rarely do we say I'm going to go on a sales call and obviously I'm going to fail. So that's the inside of what you describe. Like what? And you can sort of describe as your hopes and dreams and like, you know, obviously you're not seeing all the other candidates that they're seeing and you think you're probably much better than average candidate for the job. And so you're rating your probability of getting the job at quite high.

Annie Duke: [00:49:37] All right. So how do we discipline that in order to get more equanimity around sort of how we're viewing the situation that we're in is to think about two things.

Annie Duke: [00:49:47] How would somebody else view the situation that I'm in, but also what's true of the world in general? And that's why I want to hone in on something that you said. You step back and you said, well, wait, I'm probably one of one hundred candidates that they're seeing. So you were like, oh, so assuming they're all equally qualified.

Annie Duke: [00:50:03] The base rate for me getting this job would be one percent and all a base rate is is what's true of the world in general in this for the situation that I'm considering.

Annie Duke: [00:50:13] So like examples of base rates would be like what's what's the probability for somebody like you applying for a job of you getting the job at any point in the process? So if you send in your resume, what's the probability that you'll get it if you send it?

Annie Duke: [00:50:26] So so you can also think about like if I opened a restaurant during Non Korona, the probability that that restaurant would still be operating just generally not up my restaurant.

Annie Duke: [00:50:38] I'm saying restaurants in general would be around 40 percent in a year. We could think about like what's the probability that a 50 year old adult in America dies of a heart attack? We can go look that number up. So that's totally the outside view.

Annie Duke: [00:50:52] So you went and you thought that you started there and he said, well, let me think about what the base rate is. Where am I supposed to start? The great then you went further and he said, but what we want to do is not assume because the world isn't one hundred percent random place, there are things that are particular to me that matter. But having anchored to the outside, which is the place I want to anchor to, which just kind of like gets me into a place where my optimism and my overconfidence and my illusion of control get some discipline to them, I can now start to think about what are the things that are true of me, though, so that I can start to move off, like how far can I get you sort of better than the base rate or sometimes worse.

Annie Duke: [00:51:29] Right. So if you were to think about the average 50 year old, an American, what their rate of heart disease is, I can think about things that are true of me and my overweight. Do I exercise? Do I eat McDonald's every day? If those things are true, I'm not an exercise or I'm overweight. And I that I should probably assume that my chances of a heart attack are much higher than the average. If I say I do yoga and run marathons and I never eat a car or any sugar and. Right.

Annie Duke: [00:51:59] So I'm like, I'm like the ideal BMI, then I should assume that probably I can come below what the average is and I could start to alter that. And that's what you did on top of that, which is OK if there's one in one hundred. Now, let me start to think really where do I compare compared to the other one hundred people. Right. And it starts to get you to actually create a marriage between the inside and outside of you that allows you to get a much more objective look at what the future likely holds for you. And the thing is that what's so wonderful about that is that think about it like if you're one percent and you say, well, I think I'm probably on average three times better than the average candidate. OK, great. So your three percent now. Right? So so that gets you into a place where you understand a couple of things. One is your expectations are or what they should be. But two, it tells you something really important to you to turn it into action. I get to apply to a lot of jobs if I think that what a lot of people do is they'll see a job. They think it's a fact. They really overestimate the probability they're going to get it. And so they'll apply just to that job or they'll only apply to five job because they're really massively overestimating the chances that somehow their resume is going to shine through and sort through in all their interviews are going to go well and whatever.

Annie Duke: [00:53:13] And now they have to actually keep doing this in sequence, which is just a huge time waster. Right. But if you get a really clear view through this marriage of the inside and outside view. You can actually figure out how many times am I supposed to apply to which then increases the probability you'll actually get a job in a shorter in a shorter time frame because you're applying for all those jobs. Now, in parallel, instead of waiting for unexpected somehow bad news and then having to put in a new set of resumes,

Harpreet Sahota: [00:53:42] Because there's so much outside of your control in this job search process, the funding for that position could get cut.

Harpreet Sahota: [00:53:47] Another interviewer could do better than you. The interviewer might not have liked the smell of your breath or cologne.

Annie Duke: [00:53:53] Yeah, they could have had a fight with their spouse that morning.

Harpreet Sahota: [00:53:56] Yeah,

Annie Duke: [00:53:57] They could be in a bad mood. It matters whether it's raining or sunny on the day that you come in to talk to them. Yeah, we noticed from like even judges, right? If they're a sports fan, the sentence you get is correlated with whether their team won or lost the day before. There's all sorts of outside influences that have nothing to do with your skill, you as an individual, what your case is.

Annie Duke: [00:54:22] They just influence the way that that interview is going to go that you literally don't have any control over. Those are matters of luck to you. And this is this is really important not just for job interviews, but like sentencing that the way the judges sentence people is influenced by all these outside factors. Is the weather good? Is it not? Did their sports team win? Did they not? So and so forth. These things are all kind of influencing these outcomes. And that's why we need to so get to the to the outside view and stop thinking that the inside of you is the only thing. And again, I just want to be clear, unless you're playing the lottery where the outside would be the only thing you care about, you want to actually marry the two. I'm not saying that the things that are special to you don't matter. Right. It's just that we tend to only think about the things that are special to us. And that gives us a very poor view of what the future is going to hold.

Harpreet Sahota: [00:55:11] And throughout this process, it is imperative that you kind of focus on optimizing all of those things that are actually directly within your control. Right. So whether it could be shitty, but you could still wear a nice suit, show up dry somehow when you get to the interview. You could still come up with good answers and you can still articulate yourself clearly. And it shifted my focus instead of focusing on the result, like, oh, my God, I didn't get this job, I started focusing more on, OK, what is in this sphere of things that I can control and let me make sure I executed every one of those to the best that I can and just forget if I get the job or not, like, don't even worry about that outcome.

Annie Duke: [00:55:46] I think that's actually such a great way to think about pretty much all decisions. Right. There's there's a whole bunch of stuff that isn't in your control. And, you know, depending on the situation, like there are ways. You can hedge against that, so, for example, like sending out more resumes than you think you need to, that's that's a way to hedge against the stuff that's not in your control, just like it. Get past the recruiter or whatever. It's a way to minimize the impact of some of the luck. But what we really want to focus on are the things that are in our control that can actually push us away from the base rate like we would like. Ultimately, if there if it is a game of skill, which obviously applying for jobs is, we would like to get as sort of the most distance between us and the base rate that we possibly can. And that's going to be to focusing on the stuff that you actually can control. One of the ways to figure out what you should focus on is to actually do what's called a pre-mortem, which is to say, let's imagine I don't get the job.

Annie Duke: [00:56:39] Let me think about why that might have happened. Right. So you could think like I made a really bad impression because I was unkempt. The interviewer was in a really bad mood and I could tell they didn't want to be there could be like my know I wasn't a good match for the job or whatever you can start to think about. What are those things that that might be true in the case that you failed? What are the reasons that you might fail and then you can start to address those in advance. So you can be thinking about if I walk in a room with an interviewer who is in a bad mood, what am I going to do about it? Because I have a choice about how I react to that. I may be able to put them in a good mood. For example, I could make it worse. How are the ways that I could do that and to not just sort of accept will these things sit on my horizon, but to imagine what are the things that I can actually change about the situation that I'm in? If someone's in a bad mood, let me say in advance about the ways that I can handle that so that you you don't get in a bad mood. You don't get discombobulated. You don't give a good interview because you weren't expecting someone to come in and be kind of pissy.

Annie Duke: [00:57:36] Right. If you say I didn't look the part, which I think is a horrible thing because I don't think people should have to look the part. I just want to say that. But let's just talk about the facts of life. Then you can think about, like, what can I do to still maintain my personality and the things that I want to be true of me, where I can also be sort of living up to the norms that are expected for this particular position. And that gets you to actually think about you might go research company and say and look at pictures of people and say, you know, is this a company that's very casual? Is this a company where everybody's in a suit and tie? Obviously, I would like to be, even if it's a super casual company, because it's a job interview, I want to be less casual than that. But let me try to figure out how I benchmarked to that. Right. So get you to start to think about these things. And by the way, you may come to the conclusion when you think in a space like that that I don't think people should have to dress a certain way to get a job or look a certain way to get a job.

Annie Duke: [00:58:32] And so, therefore, if I think that that's a huge factor, I don't want that job. And that's totally fine as well. But you've sort of explored that and found that out from having done this kind of premortem work that allows you to actually get ahead of things such that you can get more distance between yourself and the base rate.

Harpreet Sahota: [00:58:50] That's an interesting part about premortem so that it forces you to think about all the bad things that could possibly happen. And I think it becomes very uncomfortable to start thinking in advance about how everything can go wrong.

Harpreet Sahota: [00:59:01] Can you talk a little bit more about that negative visualization that you talk about. 

Annie Duke: [00:59:05] Sure. So obviously the power of positive thinking is like super popular. But I have a whole chapter in the book called The Power of Negative Thinking, and it's because of what we just talked about. Right. I don't usually have negative goals. Right. I don't think you should be sitting there going, I think I'm going to fail. So I'm not even going to shoot for the stars over here. Look, I think you should have positive goals. Of course, I think the destinations that you're setting for yourself should be amazing. The question is, how are you actually navigating, thinking about navigating to those destinations, which is just your decisions, your decisions are your navigation to those.

Annie Duke: [00:59:40] And that's why I think that the power of positive thinking really kind of goes wrong, because the power of positive thinking is really saying, like, you're imagining a goal and then you're imagining just sort of succeeding along the way and implied, obviously.

Harpreet Sahota: [00:59:54] And that is that if you imagine failing along the way, that you fail will increase the chances that you fail. Now, obviously, depending on what part of literature you read, that's more in some places that is more explicit. That causal relationship is more explicit than others. But I think there's no place that it's more more explicit than the secret, which was like in Oprah's Book Club pick, I think, which essentially I don't know. Are you familiar with the book The Secret?

Harpreet Sahota: [01:00:20] Yeah. Yeah, I've heard of that law of attraction.

Annie Duke: [01:00:23] Yeah. OK, so here's here's the idea of the secret. If you imagine positive things happening to you, and I'm not just talking about generally positive thinking, some specific positive things, the love of your life, proposing extra money coming your way, whatever, that if you imagine those positive things that your thought, your thought ways, like if we did an and have a magnetic quality to them, that will attract those things to you. Now, the flip side, they say, is that if you fail. If you, for example, find yourself in a really bad traffic in the morning, which is an example they actually give that you must have imagined failing and attracted that to you. Aside from the whole horrible sort of undercurrent of blame, the victim there, which I think is quite odd, obviously the causal relationship is stated explicitly and the mechanism is super wacky because your thoughts cannot attract the things that you think about to you.

Annie Duke: [01:01:25] That is really bizarre, but whatever. So that's kind of like I would say, like the power of positive thinking in its most extreme form would be the secret. But but this idea of this causal relationship, like think negative, you'll get a bad result, think positive, you'll get a good result that's implied and all of that literature.

[01:01:44] But I think about that is the difference between using a paper map and a paper map and Waze. With an all time paper map. I think about that is the power of positive thinking. Like, look, here's the place that I want to get to and I'm looking at this map and the roads are clear, awesome.

[01:01:58] Whereas what Waze is doing is really it's like an instantiation of the power of negative thinking. Where would you like to get to? That's the positive goal, by the way. I'm going to show you all the crap that can go wrong on your route, and I'm going to help you to navigate around it. So I think it's actually just quite the reverse. It's not that imagining failure causes failure. It's that imagining failure causes success, because if you imagine failure, you can see all the obstacles that might be lying in your path and then you can actually do something about them before you run into the obstacle, because we don't really take time to imagine those obstacles. Then we're going to be reacting to them that something's going to be in your way and then you're going to be trying like a chicken with its head cut off to fix it as opposed to having avoided it, either avoided it in the first place. That's one thing that you could do. You could just navigate around it and take a different route. Another thing that you can do is to imagine how you'll react to it in advance so that your less emotional when you are so that you already have a plan in place which allows you to be a little bit more nimble. And another thing that you can do is to essentially try to figure out what I can't avoid this obstacle, so how can I dampen the effects of running into the obstacle? So that would be like if ways happen to show you that there was traffic on every road that got you to where you wanted to go. You could just leave with more time. You could leave forty five minutes early. And that will actually mitigate the problem that you see, that there's kind of failure on all of those routes and it's an unavoidable obstacle. So I don't really know how you succeed otherwise.

Annie Duke: [01:03:32] And if you're not spending a lot of time imagining the ways that things might go wrong.

Harpreet Sahota: [01:03:36] I'm curious how much, if any, of your work is influenced by Stoic philosophy, if at all?

Annie Duke: [01:03:43] Oh, quite a lot.

Harpreet Sahota: [01:03:45] Because I mean the like it just shines through to me.

Annie Duke: [01:03:49] Yeah.

Harpreet Sahota: [01:03:50] Premeditation of adversity. And then there's this. Epictetus opens The Manual with there's things in your control and things not in your control.

Annie Duke: [01:03:57] Right.

Harpreet Sahota: [01:03:58] So that's interesting that that's actually had an impact.

Annie Duke: [01:04:01] Yeah, yeah. Yeah.

Harpreet Sahota: [01:04:02] I've got a hypothesis on the power of positive thinking stuff and I'm just a dude in the basement.

Annie Duke: [01:04:08] I'm so excited to hear it.

[01:04:09] I don't know anything about this, but I'm just wondering if so they say thoughts become things. You think about. Good stuff, good stuff will happen. How much of that is is like just the reticular activating system just kicking in and wanting to see these opportunities. And obviously you have to act on it. Does that make sense what I'm trying to get out there?

[01:04:29] Oh, yeah. So so that's interesting. So what I would say to that is that I think that Gabriele Oettingen's work on mental contrasting actually shows that the move to action actually comes from imagining the failure. So basically has to do with this: If you think about like the sort of hormonal release, when things feel pretty good, whether it's like prolactin or oxytocin are like all these hormones that we have that sort of feel pretty good about versus the stuff that's kind of going on in like the adrenal gland when where you have sort of fight or flight or like anxiety or things aren't going well, that essentially when we are experiencing pleasure or joy or happiness, we have a lot of hormones that are sort of like calming us down and when we're calm.

Annie Duke: [01:05:20] It's not like really a call to action, right. Like you don't need to run away from the lion when those hormones are kind of going. But when you've got the things happening more that are in the adrenal gland, that obviously that's what gets you to run away from the lion. So anyway, what she's found with the mental contrasting work is that when let's say, for example, that you have people who say, I have a they're college students and they have a crush on somebody.

Annie Duke: [01:05:48] And you have one group of those people just kind of like imagine like happily ever after with them, with their life would look like how amazing it would be. They imagine the success of this relationship with someone that they have a crush on from afar. And then you have other people who sort of think about their crush and they really go through like, well, what if I were to ask them out on a date and they said no, like they could say no or we could start dating and it could turn out to be is that relationship or whatever they imagined all these kind of ways that you might fail? That's called mental contrasting. And then you check in with them later. And it turns out that the people who actually did the negative thinking are 30 percent more likely to have actually asked the person out on the date. And I think it's because when you're doing this positive imagining, you're getting actually a lot of the benefits of the success itself, which is you're getting that kind of good feeling that doesn't actually spur you to action, whereas you recruiting kind of -  there's other sort of hormonal things going that are getting you to sort of run away from the line. And this is just an act that actually get you to ask the person out. So I think there's a physiological thing happening there that may be counterintuitive, that the call to action is actually sort of the fear and anxiety of the failure to actually do the thing. I think also when you've already sort of metabolize this except the sort of negative outcomes, it doesn't you don't fear them as much because you've actually kind of thought them through.

Annie Duke: [01:07:12] And I just wanna say with Gabrielle's work, it's like it's everywhere. It's like it doesn't - it's true that if you have people in a weight loss program, the people who actually imagine failing and succumbing to the breakthrough donuts or whatever, they're just much more likely to actually lose weight.

Annie Duke: [01:07:27] Then the people who just imagine success, look, you can sort of pick anything at a job, you know, to get A's in your classes, the picket, you're better off doing. This mental contrasting I could get actually does call you to action. More so. But but notice that in both cases, you're setting the same goal. I mean, I think that's the key, is that you do have a destination that you're trying to move toward in both cases. The question is, I think this is your brain and your body sort of interpreting you as already having achieved it or not. And I think that's maybe where the difference is coming from, much of which, by the way, I just said is conjecture, some of which I think Gabriel would say herself. But a lot of that was sort of conjecture on my part of what's going on.

Harpreet Sahota: [01:08:07] It's really, really interesting. So then we're running out of time. But we'd be remiss if we didn't get into the work that you're doing with the Alliance for Decision Education.

Annie Duke: [01:08:15] Oh, thank you.

Harpreet Sahota: [01:08:15] I think that's really cool. I think it's really cool that your class in there that's called habitwise. It seems people like people who are better decision makers also have better habits of mind when it comes to thinking about decisions.

Annie Duke: [01:08:27] So how are you teaching these kids about the rule of habits in decision making?

Annie Duke: [01:08:32] So let me just give a clarification. When we founded the alliance, which was in 2014 initially, who are thinking that we were going to create programs and be and deliver those programs for free into school. So first of all, let me just say what the mission is. And so all of the stuff that we've been talking about really goes under the heading of decision education. That's where cornerman would sit in there. Angela Duckworth, Katie Milkman, Michael Mauboussin would fit in there. Gosh, I'm leaving so many people out. Phil Tetlock. Dan Ariely. Yeah. I don't know if there's something buildOn more. I'm sorry. OK, everybody I like I'll be like at the Oscars, like I'm sorry for any it, but obviously there's lots and lots of people who are doing amazing work thinking about how you can take what we know from behavioral economics, behavioral psychology and apply it into adults and to make that a better decision makers. What hasn't happened is that this very, very rich body of work, there hasn't been a lot done sort of translate it down into K through 12 education. So what we're trying to say is like, oh, if we really get if we offer decision education in the same way that there's social emotional learning now being offered as a way to really produce better decision makers and children, because we think better decisions lead to better lives and better society.

Annie Duke: [01:09:56] And maybe we should spend less time teaching trigonometry, which like - if you're very important, if you're going to become an engineer, but maybe you could decide to do that later in life when you actually decide that's what you want to do -  that we should be teaching more statistics and probability and really teaching people sort of what the decision is. So that's sort of what the mission is of the organization. And we sort of frame it is it's kind of two things we're trying to teach kids to how they figure out what is true and then how they figure out what to do, which is really what we've been talking about today. What's true? What what should you do about it? OK, so when we first found it with that, we're going to create programs. We'll put them into the school. So hopefully it will help kids become better decision makers. And that's what we're going to do with our lives. And we did that.

Annie Duke: [01:10:40] One of those programs was habit wise, which is essentially getting people to think about sort of translating the work of like Charles Duhigg or James Clear with atomic habits and really looking at getting up and looking at the things that have been really good work that's been done on habit formation and sort of what the habit loop is and cues and how you sort of think about your habits and how you figure out which ones you would want to change and which ones you wouldn't want to change.

Annie Duke: [01:11:07] And if you do want to change it, how how would you go about doing that as we created a program for that very popular, by the way, and we did some cultural integrations for other programs that we have to we have like a fantasy football program, which is teaching probabilistic thinking and cognitive bias.

Annie Duke: [01:11:23] We have a program which is called Mindful Choices, which is really about sort of figuring out the emotional component to decision making. Put that into schools, got amazing results in some cases.

Annie Duke: [01:11:34] In one school, the kids who are getting the programs, their math and English scores and science scores went up like between 10 and 15 percent.

Annie Duke: [01:11:41] We weren't teaching math and science and their disciplinary actions were down by like five percent. So great results. So we thought at the time, this is great.

[01:11:51] We'll just go tell other schools about how great our programs are and they'll adopt them. And it turned out that that didn't happen is what they always say is, OK, we really like your programs. That's awesome. But what do you want us to take out of the day? We don't have the limited amount of time to totally get it.

Annie Duke: [01:12:06] And the other thing is like education.

Annie Duke: [01:12:08] It's just really it's a hard boat to turn around. Very bureaucratic, right. So what we said was, OK, we're still going to offer these programs. Would you do if you can look up habit wise on the alliance and find it? And if you're a teacher and you want to actually teach that in your class, you can. But we sort of change the way that we were executing on our. To become a field builder and essentially what we were thinking about actually a lot about the model behind the Statman, social emotional learning and how that became really a movement within education. And it turns out that in those cases where you get these big movements in education, there's an organization or more than one organization that's working behind the scenes to try to create awareness around the field. Like if I think about twenty five years ago, I didn't know social, emotional. It wasn't a term that I knew. So you're trying to sort of define the field, what the boundaries of it are, what the things are that would fit in there in the same way that happen with social emotional learning. And then you try to bring different people like players and stakeholders into that that movement, whether it's the teachers who would be delivering it, whether it's the parents of the students who would be getting it, the businesses that might want to hire people and have them skilled up in this area, colleges who would want to make sure that the students that they're recruited would have really good decisions, girls, so and so forth, and legislatures and people who are policy makers.

Annie Duke: [01:13:32] And you try to get them all to start talking to each other and create a dialog around this field in order to get that to be in schools the same way that now social emotional is pretty ubiquitous within the school system. And there was an organization with social emotional learning that was called Castle that was working for about twenty five years. And it was a good 15 years into that, that all of a sudden everybody started talking about social emotional learning. So we sort of pulled back and decide to model ourselves on that. And we're really sort of Connector's funders really trying to accelerate this field where awareness raisers, we do still offer programs. But a lot of what we're doing is sort of connecting schools that are interested to program the programs, researchers who really want to think about how to apply these things to K through 12 and to see where you can get really good change. We connect them with schools that would be interested in having people come and into studies that were helping to fund academic research around K through 12 in terms of decisions and so on, so forth. So that's that's really what we're doing now. And I'm excited that you ask them.

Annie Duke: [01:14:33] Obviously, I love I love what we do that. 

Harpreet Sahota: [01:14:35] I think you guys are doing did some awesome work there. But thank you on behalf of all the kids that you're impacting this. 

Harpreet Sahota: [01:14:41] Thank you

Harpreet Sahota: [01:14:41] So I know we're running short on time here, but last formal question and we'll do a couple of quick random things. So one hundred years in the future. Annie Duke, what do you want to be remembered for?

Annie Duke: [01:14:52] Oh, yeah. To be remembered for being a good mom. Awesome. That's an easy one.

Harpreet Sahota: [01:15:01] So jump into a real quick random round here. What song do you have on repeat?

Annie Duke: [01:15:06] Oh, my gosh. You know, it's so interesting. I haven't been I just I don't even listen to as much music as I did before. And this is my fault. I should have more intention around this because I sort of watch the news all the time, which is so depressing and awful. So I shouldn't do that. But in terms of songs that I might have on repeat would certainly be basically anything by Jack White. I listen to a lot. There was a period where I was listening to Hamilton on loop and I got to say Lemonade is a pretty strong album, so I'll listen to that on a loop, too.

Harpreet Sahota: [01:15:41] What book are you reading right now?

Annie Duke: [01:15:42] I just finished actually a book called How to Change, which is going to come out in the spring from Katie Milkman. Everybody should keep their eye out for that one. That one. That one is actually I have a change book. It's fabulous. It's really, really good. But you can't I guess you can't get it till the spring. So that's not a nice that's not a nice recommendation, I suppose. Everybody should read The Biggest Bluff and The Psychology of Money.

Harpreet Sahota: [01:16:05] Yeah, I started digging into Maria Konnikova's work and trying to get it on the show. Let's see what happens. I really went after about game theory and all the stuff you did.

Harpreet Sahota: [01:16:15] What do you think's the most bizarre aspect of human nature?

Annie Duke: [01:16:18] Yeah. Gosh bizarre is such an interesting word there. I would say it's that given how freaking uncertain the word world is like, how damn sure we are that we're all right, every single one of us like, I just find that so bizarre.

Harpreet Sahota: [01:16:33] Yeah. So Annie how can people connect with you? Where can they find you online. Where you can get where can they get your books at?.

Annie Duke: [01:16:41] Yeah. So you can find out pretty much anything about me at any dotcom. I can find out how to order my books so that are available at all the usual places.

[01:16:49] I do have a contact form at AnnieDuke.com and I would love it if people reached out to me because it's the biggest contributor to How to Decide. It's actually conversations with people who had written to me. I like to hear from people, listen to me or read my work or whatever. It's where I get I get a tremendous amount of learning out of that. I'm not one hundred percent responding. I that's my goal. I'm about 90 percent on it. So you don't hear back from me. It doesn't mean that I don't appreciate you and love you for having written.

Annie Duke: [01:17:20] And then sometimes things get buried in my email box and I apologize for that. And you can also find me on Twitter, obviously. Annie, do you know that if you check out the alliance, I'd be really appreciative, Annie?

Harpreet Sahota: [01:17:33] Thank you so much for taking time out your schedule to be here. I really appreciate you coming on the show. It has been an honor to speak with you. Thank you.

Annie Duke: [01:17:40] Oh, well thank you for having me.