Designing Operator Tasks to Minimize the Impact of Heuristics and Biases

Often times when a person is blamed for “not thinking,” the reality is they were thinking, but were not aware of it. This is the theory of System 1 (i.e., Fast) versus System 2 (i.e., Slow) thinking that explains we are really two people: Our conscious aware selves (System 2 thinking), and a dominant “fast” subconscious making most of our decisions (System 1 thinking) without being consciously aware of it in the moment (to the point that some have argued there is no such thing as “free will”). The heuristics (i.e., mental short cuts) we use to think in System 1 are necessary to make it through a day (it is exhausting to maintain a continuous conscious stream of thought), and often lead to good outcomes. However, System 1 thinking can make us vulnerable to systematic biases (i.e., mental traps) that arise from the use of those heuristics. It is necessary to be aware of the traps System 1 thinking can create, because often times that is our only defense against them. In this respect, “fast thinking” represents one of the fundamental limits to achieving safe operation. In addition to awareness, there is a need where possible to design operator tasks and the interfaces they use to minimize the likelihood of systematic bias occurring when thinking in System 1. Lastly, it would be useful to provide designs that could increase the potential for the operator to engage System 2 thinking (consciousness) when required, which is less susceptible to biases.

This paper proposes a combined approach of discussing the cognitive psychology behind System 1 and System 2 thinking, the types of heuristics we use, the biases that result, and operator task and interface design that can minimize the likelihood of systematic bias. The paper will incorporate the learnings from 5 years of safety critical Task Analysis performed for field and control room tasks. A practical operator response to abnormal situation model will be described that will link the heuristics used and potential biases that may occur, as well as design features to minimize the likelihood of those occurring.