A guest post by: Willard Zangwill, Ph.D., Professor, University of Chicago, Booth School of Business
Rachel Kaberon, in preparation for the Strategy Management Practices Issues Group discussion of the Chicago Booth Alumni Club, asked me to put together a page or two of thoughts about uncertainty in decision making. Since she had helped me with software I have developed to assist in complex decision making, this was my chance to return the favor. Hence, here are some thoughts that strongly influenced my thinking about uncertainty and how I have tried to suggest how people might better predict the future and make better decisions.
UNCERTAINTY IS RAMPANT.
First is that uncertainty is remarkably uncertain, and our efforts to predict it are likely worse than we often assume. Overconfidence bias is indeed strong. What demonstrated this to me was the outstanding work of Philip Tetlock[i]. He studied how accurate were the predictions of experts and pundits in the political or economic areas. These people were similar to the prognosticators we see on television or other experts discussing what might happen to events in the future. Tetlock examined such predictions for years and studied tens of thousands of them, which was a huge undertaking.
What Tetlock discovered was how bad the predictions were. They were only slightly better than chance. Not the result one might expect, but worse. Too many events seem to unexpectedly occur in the future.
Interestingly, the prognosticators that were most confident and sure of themselves, were wrong more than the more cautious forecasters who hedged and added conditional statements. The confident experts tended to gain more support and attention, as their confidence convinced others, but that did not make them more right.
How could predictions be so faulty? By and large, we tend to think we predict better than we do because if we are wrong, we give ourselves excuses. We suggest that no one could forecast what really happened, or that events no one could have foreseen occurred. That process absolves us of blame and provides exoneration. The net result, however, is that the future is harder to predict than most of us are likely to believe.
Given this conundrum that we have to predict events, but are probably not that good at it, what can be done. Here are a couple of experiments that I have found useful to try to build upon.
As Gary Klein[ii] has noted, Research conducted in 1989 by Deborah J. Mitchell, of the Wharton School; Jay Russo, of Cornell; and Nancy Pennington, of the University of Colorado, found that prospective hindsight—imagining that an event has already occurred—increases the ability to correctly identify reasons for future outcomes by 30%.
The concept is illustrated by the following. Consider some upcoming event, say a presidential election. Then think of reasons why a particular candidate might win.
Now do the following. Assume it is now after the election. And assume it has just been announced that candidate has won by a solid margin. Now think of reasons that triumph occurred. You will likely think of more reasons. In essence, assuming an outcome and carefully imagining it, helps you think of reasons why that outcome might occur. Perceiving those additional reasons then helps as you proceed to analyze the situation.
A much different approach in a study by Armstrong and Green[iii], was also quite helpful for forecasting the future. In brief, they had subjects predict the outcome of past situations that were unknown to the subjects. Since these were past situations, the actual outcome was known, although the subjects did not know those outcomes. After the subjects made their predictions about the outcome of these situations, the accuracy of the predictions were then determined.
At this juncture, the experimenters then changed the situation. They required that the subjects first consider several situations analogous to the one they had to predict; these were analogous situations where the subjects knew the outcomes. Once they considered those several analogous situations, now the subjects were told to predict the situation in question. The success rate went up substantially. In fact, when a group of subjects were involved and they carefully compared analogous situations, the accuracy of the prediction roughly doubled.
The message seems to be this. When we forecast an event, we tend to do that by thinking of some similar event that we know. That similar event we know, gives us ideas about the outcome of the event we are trying to predict. Now take this one step further. If you consider several events roughly similar to the one you are trying to predict, it is like increasing the sample size. The accuracy of your prediction should rise. Moreover, just examining how several situations similar to the one you are considering turned out, is illuminating, and by exposing the complexities of the situation, provides useful insights.
Given the difficulty of predicting the future and the challenges thereof, it might help to broaden our decision-making framework and, in particular, to do more breakthrough thinking as that might provide us with an advantage. Considering breakthrough thinking, as least for most people, good breakthrough ideas seem to occur almost randomly, as we tend to think about an issue and the exciting idea somehow jumps into our minds. But there do seem to be procedures that help them occur more frequently and more when needed. The key insight is to look and examine where the breakthrough idea is more likely to occur.
To illustrate, suppose you cannot find your car keys and have searched all over the house. In frustration, you ask your spouse. He/she replies that they are on your dresser. Despite the mess on your dresser ( not necessarily yours, but certainly mine) you dash over to your dresser and with only a little rummaging, quickly find your keys.
As another example, when they search for oil, they do not put the exploratory well anywhere. But they first conduct detailed geological and seismologic examinations to locate where the oil find is more probable.
The concept for breakthrough ideas is the same. Suppose you have one million possible ideas to search through in order to discover your breakthrough idea. Finding that breakthrough idea from among the million possibilities, is not likely to be easy. This explains why getting breakthrough ideas is usually a challenge, as it required a quite large search to uncover it.
On the other hand, now suppose you obtain some clues as to where that exciting idea might be found that narrows your search down to ten possibilities. You can easily search the ten and, in all likelihood, uncover the breakthrough idea.
The insight is to examine where the breakthrough idea is likely. It is like drilling where the oil is likely, and you will more easily find it. One of the concepts behind the software for decision making I developed takes advantage of this and seeks to suggest where the breakthroughs will be more likely, helping you to more easily discover it.
The uncertainty of the future is probably far greater than most of us assume. Here I have tried to suggest some means that might help reduce that uncertainty and improve decision-making. There are other ways as well, and they should help as you proceed to make difficult decisions for the future.
[i] Philip Tetlock, “Expert Political Judgment: How Good Is It? How Can We Know?” (Princeton)