Cognitive bias mitigation is the prevention and reduction of the negative effects of cognitive biases – unconscious, automatic influences on human judgment and decision making that reliably produce reasoning errors. Coherent comprehensive theories of cognitive bias mitigation are lacking; however, there are debiasing tools, methods, proposals, and other initiatives in academic and professional disciplines concerned with the efficacy of human reasoning, associated with the concept of cognitive bias mitigation, although most address mitigation tacitly rather than explicitly.
A long-standing debate regarding human decision making bears on the development of a theory and practice of bias mitigation. This debate contrasts the rational economic agent standard for decision making versus one grounded in human social needs and motivations. The debate also contrasts the methods used to analyze and predict human decision making, i.e. formal analysis emphasizing intellectual capacities versus heuristics emphasizing emotional states.
Context A large body of evidence has established that a defining characteristic of cognitive biases is that they manifest automatically and unconsciously over a wide range of human reasoning, so even those aware of the existence of the phenomenon are unable to detect, let alone mitigate, their manifestation via awareness only.Ariely, D. (2008). Predictably Irrational: The Hidden Forces That Shape Our Decisions, Harper Collins.Gigerenzer, G. (2006). "Bounded and Rational." Contemporary Debates in Cognitive Science. R. J. Stainton, Blackwell Publishing: 115–133.Gilovich, T. (1991). How We Know What Isn't So: The Fallibility of Human Reason in Everyday Life. New York, NY, The Free Press.Haselton, M. G., D. Nettie, et al. (2005). The Evolution of Cognitive Bias. Handbook of Evolutionary Psychology. D. M. Buss. Hoboken, Wiley: 724–746.Lerher, J. (2009). How We Decide. New York, NY, Houghton Mifflin Harcourt.Nozick, R. (1993). The Nature of Rationality. Ewing, NJ, Princeton University Press.
Real-world effects of cognitive bias There are few studies explicitly linking cognitive biases to real-world incidents with highly negative outcomes. One study,Roberto, M. A. (2002). "Lessons from Everest: The Interaction of Cognitive Bias, Psychological, Safety and System Complexity." California Management Review (2002) 45(1): 136–158. explicitly focused on cognitive bias as a potential contributor to a disaster-level event, examined the causes of the loss of several members of two expedition teams on Mount Everest on two consecutive days in 1996. This study concluded that several cognitive biases were "in play" on the mountain, along with other human dynamics. This was a case of highly trained, experienced people breaking their own rules, apparently under the influence of the overconfidence effect, the sunk cost fallacy, the availability heuristic, and perhaps other cognitive biases. Five people, including both expedition leaders, lost their lives despite explicit warnings in briefings prior to and during the ascent of Everest. In addition to the leaders' mistakes, most team members, though they recognized their leader's faulty judgments, failed to insist on following through on the established ascent rules. In a 2010 MarketBeat study, German researchers examined the role that certain cognitive biases may have had in the 2008 financial crisis. Their conclusion was that the expertise level of stock analysts and traders made them highly resistant to signals that did not conform to their beliefs in the continuation of the status quo. In the grip of strong confirmation bias reinforced by the overconfidence effect and the status quo bias, they apparently could not see the signals of financial collapse, even after they had become evident to non-experts. Similarly, Kahneman, a Nobel Laureate in Economics, reportsKahneman, D. (2011). Thinking, Fast and Slow, Doubleday Canada. in a peer-reviewed study that highly experienced financial managers performed "no better than chance", largely due to similar factors as reported in the study above, which he termed the "illusion of skill".
There are numerous investigations of incidents determining that human error was central to highly negative potential or actual real-world outcomes, in which manifestation of cognitive biases is a plausible component. The Gimli Glider Incident,ID=19830723-0 (1983). "Gimli Glider Accident Report." Aviation Safety Network http://aviation-safety.net in which a July 23, 1983 Air Canada flight from Montreal to Edmonton ran out of fuel 41,000 feet over Manitoba because of a measurement error on refueling, an outcome later determined to be the result of a series of unchecked assumptions made by ground personnel. Without power to operate radio, radar or other navigation aids, and only manual operation of the aircraft's control surfaces, the flight crew managed to locate an abandoned Canadian Air Force landing strip near Gimli, Manitoba. Without engine power, and with only manual wheel braking, the pilot put the aircraft down, complete with 61 passengers plus crew, and safely brought it to a stop. This outcome was the result of skill (the pilot had glider experience) and luck (the co-pilot just happened to know about the airstrip); there were no deaths, the damage to the aircraft was modest, and there were knowledgeable survivors to inform modifications to fueling procedures at all Canadian airports. The Loss of the Mars Climate Orbiter,Stephenson, Arthur G.; LaPiana, Lia S.; Mulville, Daniel R.; Rutledge, Peter J.; Bauer, Frank H.; Folta, David; Dukeman, Greg A.; Sackheim, Robert et al (1999-11-10). "Mars Climate Orbiter Mishap Investigation Board Phase I Report." National Air and Space Administration. which on September 23, 1999 "encountered Mars at an improperly low altitude" and was lost. NASA described the systemic cause of this mishap as an organizational failure, with the specific, proximate cause being unchecked assumptions across mission teams regarding the mix of metric and United States customary units used in different systems on the craft. A host of cognitive biases can be imagined in this situation: confirmation bias, hindsight bias, overconfidence effect, availability bias, and even the meta-bias bias blind spot. The Sullivan Mine Incident,British Columbia Ministry of Energy, Mines and Petroleum Resources: Sullivan Mine Accident Report, May 17, 2006. which occurred on May 18, 2006, when two mining professionals and two paramedics at the closed Sullivan mine in British Columbia, Canada, all specifically trained in safety measures, lost their lives by failing to understand a life-threatening situation that in hindsight was obvious. The first person to succumb failed to accurately discern an anoxic environment at the bottom of a sump within a sampling shed, accessed by a ladder. After the first fatality, three other co-workers, all trained in hazardous operational situations, one after the other lost their lives in exactly the same manner, each apparently discounting the evidence of the previous victims' fate. The power of confirmation bias alone would be sufficient to explain why this happened, but other cognitive biases probably manifested as well. The London Ambulance Service Failures, in which several Computer Aided Dispatch (CAD) system failures resulted in out-of-specification service delays and reports of deaths attributed to these delays. A 1992 system failure was particularly impactful, with service delays of up to 11 hours resulting in an estimated 30 unnecessary deaths in addition to hundreds of delayed medical procedures.Beynon-Davies, P., "Information systems `failure': case of the LASCAD project", European Journal of Information Systems, 1995. This incident is one example of how large computer system development projects exhibit major flaws in planning, design, execution, test, deployment and maintenance.Mann, C. C. (2002). "Why Software is So Bad." Technology Review, MIT, July 2002. Atul Gawande, an accomplished professional in the medical field,Gawande, A. (2010). The Checklist Manifesto: How to Get Things Right. New York, NY, Metropolitan Books. recounts the results of an initiative at a major US hospital, in which a test run showed that doctors skipped at least one of only 5 steps in 1/3 of certain surgery cases, after which nurses were given the authority and responsibility to catch doctors missing any steps in a simple checklist aimed at reducing central line infections. In the subsequent 15-month period, infection rates went from 11% to 0%, 8 deaths were avoided and some $2 million in avoidable costs were saved. Other disaster-level examples of negative outcomes resulting from human error, possibly including multiple cognitive biases: the Three Mile Island nuclear meltdown, the loss of the Space Shuttle Challenger, the Chernobyl nuclear reactor fire, the downing of an Iran Air passenger aircraft, the ineffective response to the Hurricane Katrina weather event, and many more.
Each of the approximately 250 cognitive biases known to date can also produce negative outcomes in our everyday lives, though rarely as serious as in the examples above. An illustrative selection, recounted in multiple studies: Confirmation bias, the tendency to seek out only that information that supports one's preconceptions, and to discount that which does not. For example, hearing only one side of a political debate, or, failing to accept the evidence that one's job has become redundant. Framing effect, the tendency to react to how information is framed, beyond its factual content. For example, choosing no surgery when told it has a 10% failure rate, where one would have opted for surgery if told it has a 90% success rate, or, opting not to choose organ donation as part of driver's license renewal when the default is "No". Anchoring bias, the tendency to produce an estimate near a cue amount that may or may not have been intentionally offered. For example, producing a quote based on a manager's preferences, or, negotiating a house purchase price from the starting amount suggested by a real estate agent rather than an objective assessment of value. Gambler's fallacy (also known as sunk cost bias), the failure to reset one's expectations based on one's current situation. For example, refusing to pay again to purchase a replacement for a lost ticket to a desired entertainment, or, refusing to sell a sizable long stock position in a rapidly falling market. Representativeness heuristic, the tendency to judge something as belonging to a class based on a few salient characteristics without accounting for base rates of those characteristics. For example, the belief that one will not become an alcoholic because one lacks some characteristic of an alcoholic stereotype, or that one has a higher probability to win the lottery because one buys tickets from the same kind of vendor as several known big winners. Halo effect, the tendency to attribute unverified capabilities in a person based on an observed capability. For example, believing an Oscar-winning actor's assertion regarding the harvest of Atlantic seals, or, assuming that a tall, handsome man is intelligent and kind. Hindsight bias, the tendency to assess one's previous decisions as more effective than they were. For example, "recalling" one's prediction that Vancouver would lose the 2011 Stanley Cup, or "remembering" to have identified the proximate cause of the 2007 Great Recession. Availability heuristic, the tendency to estimate that what is easily remembered is more likely than that which is not. For example, estimating that an information meeting on municipal planning will be boring because the last such meeting attended (on a different topic) was so, or not believing a member of Parliament's promise to fight for women's equality because he did not show up to your home bake sale fundraiser for him.