Stanford's James March, perhaps the most prestigious living organizational theorist, wrote me an amazing email when Jeff Pfeffer and were writing Hard Facts, our book about evidence-based management. When I asked him if there were any breakthrough ideas in management, his reply was:
“Most claims of originality are testimony to ignorance and
most claims of magic are testimonial to hubris.” Unfortunately, we are in an era where the evidence to support March's statement is rolling in at remarkable rate. Many of the "geniuses" that have been worshiped in recent years have turned
out to be false gods — just ask all those people who lost billions
with Bernie Madoff.
As much as I love reading Jim Collins' Good to Great for its compelling writing style and great stories, I've always found his claims way overblown. Any study that is based on only 11 companies, out of a sample of 1500, is bound to have a lot of methodological problems. (e.g., How many companies that had Level 5 leaders didn't make the leap? His method makes it impossible to tell.) And it has always bothered me that once his "chimps" (as he called his research team) identified the good and great companies, they went back and interviewed people about the differences. It is sort of like asking players of the team that won the world series or super bowl (and reporters) what the secret to their success was after the series is over, and then comparing their answers to the players on the fourth place team. Compelling research by Berkeley's Barry Staw and others shows that winners and losers offer much different recollections to explain what happen and why, regardless of the facts. In particular, winners report having better leaders, being more persistent, more focused, and more cohesive — even in experiments where there is no real difference between groups, and the winners and losers were simply misled about their performance by some seemingly legitimate person. So I don't know who will win the Superbowl this year, but I can predict what the winning players will say, and so can you. If you want to read a detailed critique of the methods in Good to Great, see The Halo Effect.
The other thing that always bothered me about Good to Great is that — although there are thousands of rigorous peer reviewed studies that are directly on the issues he studies — he never mentions any of them to further bolster or refine his arguments (for example, there is a lot of research on charismatic leadership that he ignores, which shows it is effective under many conditions — most striking to me is that he defined it as, essentially, flashy and arrogant leadership, which isn't how most researchers define and measure it. They they do talk about charismatic leaders being articulate and able to set down a clear vision, but the key thing is the ability to motivate and direct followers' energy — muck like Collins' Level 5 leaders). So this second concern also is reflected in March's quote, as a claim of originality that is testimony to ignorance.
To return to the magic, the evidence that anything that is too good to be true usually is — and there is no such thing as the single magical breakthrough study in business — has been pouring in lately when it comes to Good to Great. Steve Levitt, of Freakonomics fame (and unlike Collins, a researcher with bona fide research training and a world class scholar), did an analysis in The New York Times called "From Good to Great..to Below Average," showing that Collins' 11 "great companies" were as a set performing below the S&P 500. Note also that this even before the shit really the fan, and the financial crisis drove Circuit City into Chapter 11 and Fannie Mae into much deeper trouble.
Just this morning, I got a broadcast email from Academy of Management Perspectives, an academic journal I subscribe to that attempts bridge the gap between academic research and management practice. The next issue contains two articles — both based on data gathered well before the economic meltdown — that further document that lack of staying power among these 11 so-called great companies. I think it is worth printing the entire abstract of each here, as given this book has sold 4.5 million copies. Even if this may be boring to some of you, the ideas in the book are taken as an articles of faith in so much business — and Collins has acted as if the ideas in his book and those 11 companies provide magic for any company. In an ironic twist, he has displayed remarkable personal hubris in his (noble) efforts to undermine leaders who suffer from hubris. His writing and inspiring speeches are moving, but he is selling snake oil, or at least exaggerating the healing powers of some well-known management practices that work pretty well, but are not magical cures.
The first article is by Professors Bruce Renick and Timothy Smunt, and is called "From Good to Great to…" Here is the abstract:
With sales of more than 4.5 million copies, Good to Great
by Jim Collins provides an inspiring message about how a few major
companies became great. His simple but powerful framework for creating
a strategy any organization can use to go from goodness to greatness is
certainly compelling. However, was Collins truly able to identify 11
great companies? Or was the list of great companies he generated merely
the result of applying an arbitrary screening filter to the list of
Fortune 500 companies? To test the durability of his greatness filter,
we conducted a financial analysis on each of the 11 companies over
subsequent periods. We found that only one of the 11 companies
continues to exhibit superior stock market performance according to
Collins' measure, and that none do so when measured according to a
metric based on modern portfolio theory. We conclude that Collins did
not find 11 great companies as defined by the set of parameters he
claimed are associated with greatness, or, at least, that greatness is
not sustainable.
The second article is by Professors Bruce Neindorf and Kristine Beck. It is called "Good to Great, or Just Good?" Here is the abstract:
Good to Great has been on Business Week's best-seller list since its October 2001 release. In Good to Great,
author Jim Collins identified a set of 11 firms as great, then used
them to derive five management principles he believed led to "sustained
great results." We contend that due to two fatal errors, Good to Great
provides no evidence that applying the five principles to other firms
or time periods will lead to anything other than average results. We
explain the two errors and empirically test our contention. When ranked
with the 2006 Fortune 500, the 11 Good to Great firms have an average ranking of 202nd. In addition, in terms of long-term stock return performance, the Good to Great
firms do not differ significantly from the average company on the
S&P 500. Our evidence is consistent with the conclusion that
although the Good to Great firms may be good, they aren't great.
Both these artifices are published in Academy of Management Perspectives, Volume 24, Number 4, 2008.
Although, oddly, my colleague and friend Jeff Pfeffer continually crossed-out most of the text in Hard Facts that took Collins to task for methodological problems and excessive claims. But the key lessons from this book, and so many others, are:
1. As March implies, there are no magical leadership or organizational practices that will quickly propel your organization to the top of the heap. Even the greatest organizations struggle to stay at the top and are led by fallible people who make many mistakes.
2. There is no such thing as a single breakthrough study. The best and most valid conclusions and advice are based on a series of studies that have survived the brutal peer review process and that result in a consistent set of findings. In this regard, an interesting contrast is Chip and Dan Heath's Made to Stick, which is based the weight of the evidence from hundreds of rigorous studies (instead of one that could not survive the peer review process unless the claims were toned way down and the hundreds of past studies that were consistent — and clashed with it — were at least mentioned). I especially point to Made to Stick, and I would add Influence, because they are so well-written that they show you can combine good scholarship with a great read.
3. My main objection, in the end, isn't to the research Collins did — the stories are interesting and I believe that nearly all of the practices that he suggests would make a manager more effective — indeed many if not most are bolstered by more rigorous studies (albeit, even as his research now implies, as signs of competence or even ordinary greatness). My objection is — to use Jim March's words — the hubris and ignorance about the claims about the rigor of the research and the originality of the ideas. There are lots of management books, or parts of management books, that are incredibly useful and inspiring, but don't claim to draw on research. Orbiting the Giant Hairball is a great example. Another is Tom Kelley's masterpiece Art of Innovation. The difference is that these great books don't make excessive claims – Hairball draws on the author's personal story and Tom Kelley draws mostly on what he and his colleagues have done at IDEO.
I suspect that I am not going to make a lot of friends with this email, but as I struggle to identify evidence-based practices –and practices that seem promising, cool, or have been used to good effect in a few places — that help managers and others do their work more effectively, to achieve what might be called "ordinary greatness," I think that — in the spirit of the times — perhaps all of us who are paid to give advice to managers and leaders owe to them, and to all the employees whose jobs depend on their actions, to come clean about the limits of our claims and our knowledge.
We are at a moment in history were we all need a lot more truth and a lot less snake oil.
P.S. Here is a post about another great quote from March, on innovation.
