Gerry McGovern // New Thinking

Why focus groups don’t work

October 31, 2010 -- Gerry McGovern

The biggest problem in getting to know our customers is that they don’t know themselves.

Years ago, British Airways (BA) decided to introduce a new service for its first class passengers on long haul flights. It was basically a mini fridge full of goodies so that if you woke up in the middle of the night feeling a little hungry, you could get something nice for yourself. The question was: What do we put into this little fridge?

BA dutifully did its market research and assembled several focus groups of first and business class passengers. What would you like, they asked? The response was universal. People wanted fruit or perhaps some light salads. All very good. All very healthy.

On the first flight with the new service an air hostess paused as she noticed the fridge being filled. “What are you doing?” she asked. And the person dutifully explained what was happening. The hostess laughed. “They’re lying!” she said. “They don’t want salads. Listen, I’ve being doing the London to LA route for years, and when they wake up in the middle of the night the last thing on their minds is salads.” “But the focus groups all said …” She shook her head and walked away. A couple of minutes later she came back with some chocolates and cakes. “Please put these in as well,” she said. “Trust me. I know my customer.”

And they did put some chocolate and cakes in and when they checked at the end of the flight, they were all gone and nobody had touched the apples or salads.

The worst way to design a website is to have five smart people in a room drinking lattes. The longer you leave them in the room the worse the design becomes. The next worst way is to have 15 customers in a room drinking lattes. What people say they do and what they actually do are rarely the same thing.

"We hardly ever use focus groups because they just don't work very well at uncovering user needs," stated Christine Perfetti when she worked for At User Interface Engineering. "The biggest problem: what users say in a focus group rarely matches what they do in a real-life setting. Users' opinions about a site or product are very rarely consistent with how they behave when they actually interact with it."

It’s not that people deliberately set out to lie. They’re thinking: ‘I should eat better. I am putting on a bit of weight. Salads are really good for you. It’s ages since I ate an apple.’ But when they wake up in the middle of the night, wipe the sleep from their eyes and open that fridge door their hand reaches out as their mouth mumbles c-h-o-c-a-l-a-t-e-s.

A major challenge for the web manager is thus to get to the chocolates. No point in having lots of salads on your website if nobody really wants them. That was the whole purpose of developing the top task management approach and it’s what the next couple of issues will be about. Helping you separate the chocolates from the salads.

WHITE PAPER: WEB SURVEYS ARE BROKEN: HOW TO FIX THEM (PDF: 2 MB)