Gerry McGovern // New Thinking

The Optimism Bias, or don’t believe satisfaction surveys

September 25, 2011 -- Gerry McGovern

We need to move to an evidence-based model of understanding people’s needs based on what they actually do, not what they say they do.

The lady had just finished trying to complete 15 tasks on the website. “What did you think of the website,” the organizer asked her. “It was great,” she replied effusively. “I really liked it. Lovely website.” 12 out of 15 of the tasks the lady had tried to complete had been totally unsuccessful.

Time and time again I come across this huge disconnect between what people say and what they do. For example, I have seen high satisfaction scores from employees for an intranet even though it was terrible and employees hardly ever used it. Have you ever heard of the expression “Mark 5 to survive” for employee satisfaction surveys? Basically, many employees game the survey, believing that negative feedback will only get them in trouble.

People are often not cynical on surveys, but are instead positive and optimistic. It’s called the Optimism Bias. “You might expect optimism to erode under the tide of news about violent conflicts, high unemployment, tornadoes and floods and all the threats and failures that shape human life,” Tali Sharot wrote in TIME in May 2011. “Collectively we can grow pessimistic — about the direction of our country or the ability of our leaders to improve education and reduce crime. But private optimism, about our personal future, remains incredibly resilient.”

This is great news. Our future depend on it. However, there are  pitfalls; optimism can lead to irrational exuberance. So, how do we move forward in this ever complex world. “I believe knowledge is key,” Sharot writes. “We are not born with an innate understanding of our biases. The brain's illusions have to be identified by careful scientific observation and controlled experiments and then communicated to the rest of us.”

A defining characteristic of our age is the rise of evidence-based decision making. The more complex the world gets the less we can depend on instincts that were developed and honed during much simpler times. It’s not an admission of weakness for a manager to say: “I don’t know. Let’s get some facts.” What is dangerous is the Steve Jobs Syndrome: the desperate search for the genius who has all the answers.

In general, humans don’t warm to facts and statistics. We love individual stories. Some say that “there are lies, damned lies and statistics.” But where do personal stories lie? Would you believe the personal story of one person who found a particular car unsafe, or statistics that say the same car is safe?

What is so exciting today is that we can get all the statistical data we need. This is driven by two things. Firstly, we spend a lot of time on the Web, where we leave behind massive quantities of data and content recording what they do. Secondly, computer power allows us to cheaply analyze these massive quantities of data.

Today, those who make decisions based on data and not opinion, will make the best decisions.

The Optimism Bias, Tali Sharot (TIME)