A Stanford undergraduate doing a case analysis on using intuition versus systematic analysis wrote me an email last night to get my thoughts on the difference between the two, especially in light of the work that Jeff Pfeffer and I did on evidence-based management. Below is my lightly edited response. This is just off the top of my head (is it mostly intuition?). I would love to hear your thoughts on this distinction — if it is useful, how the two concepts fit together, when one is more useful than the others, and so on:
I don't think that intuition and evidence-based management
are at odds. There are many times when decision-makers don't have very good
data because something is new, the situation has changed (e.g., where do you invest
money right now?), or because what might seem like
intuition is really mindless well-rehearsed behavior that comes from years of
experience at something, so even though people can't articulate the pattern they
recognize, they still are acting on a huge body of experience and knowledge.
And on the very other side of experience there are virtues to the gut reaction of naive people, as those who are not properly
brainwashed may see things and come up with ideas that expertise drives out of
their brains (e.g, that is why Jane Goodall was hired to observe chimps, in
part, because she knew nothing).
The trouble with intuition is that we now have a HUGE
pile of research on cognitive biases and related flaws in decision-making that
show "gut feelings" are highly suspect. Look-up confirmation bias — people have a
very hard time believing and remember evidence that contradicts their beliefs. There
is also the fallacy of centrality, a lot more obscure, but important in that
people — especially those in authority — believe that if something important
happens, they will know about it.
My belief — and it is only partially evidence-based —
is that intuition works best in the hands of wise people (this is all over hard
facts), when people have the mindset to "act on their beliefs, while
doubting what they know," so that they are always looking for
contradictory evidence, encouraging those around them to challenge what they
believe, and constantly updating (but always moving forward), then I think that
intuition — or acting on incomplete information, hunches, conclusions — is
right. Here is one place I've talked about it. Brad Bird of Pixar is a good
example of someone with this mindset, as we learned when we interviewed him for
the McKinsey Quarterly. So is Andy
Grove. I think the most interesting
cases to look at are those where people with a history of good guesses or gut
decisions — what mistakes has Steve Jobs made?
What about Google… indeed, it is interesting that they believed they
were going to crush Firefox with Chrome , but their market share remains modest a year later. My point here isn't to say anything negative about Jobs or Google — they have impressive track records, plus some history of the usual failures that all humans and human organizations suffer from. Rather, my point is that by looking at errors by people and firms that have generally good track records, you can learn a lot about conditions under which judgment fails, because you can rule out the explanation that they generally suffer from judgment.
There is a lot written on intuition and the related topic
of quick assessments — see Blink — and some evidence (although Gladwell
exaggerates about the virtues of snap judgments, as the best are often made by
people with much experience in the domain, but as always he makes wonderful points). Also see this book by David Myers for a balanced
and evidenced perspective on intuition.
My view is that intuition and analysis are not opposing
perspectives, but tag team partners that, under the best conditions, where
hunches are followed and then evaluated with evidence (both quantitative and
qualitative, that is another issue, qualitative data are different than
intuition, and often better) versus when hunches and ingrained behaviors are
mindlessly followed and impervious to clear signs that they are failing.
Work Matters readers: Again, I would appreciate your thoughts, as this is one of those core challenges for every boss and for a lot of behavioral scientists too!