Category: Evidence-based Management

I was re-reading Guy Kawasaki's Reality Check, and came on a quote that — for me — perfectly summarizes the need for evidence-based management (and the mindset that can make it so hard to implement):

One of the main enemies of truth — and the main causes of persistent but false myths — is what psychologists call confirmation bias — we tend to seek out, believe, and remember facts that support or beliefs, and avoid, disregard, and forget facts that clash with our myths.  The upshot is that we travel through life seeing persistent support for the myths we cling to, despite the evidence.  Simon and Garfunkel summarized this well when they said (I think I remember this accurately):  "A man sees what he wants to see and disregards the rest." This applies to women to, of course.  

One of the best defenses against "the comfort of opinion without the discomfort of thought" is to have strong opinions, weakly held.

P.S. John, thanks for the information and correction, the Simon and Garfunkel song is from The Boxer, and is ""Still, a man hears what he wants to hear;
And disregards the rest."

Like everyone else, I am just tickled by the astounding story about US Airways Flight 1549 and the heroics of Captain Chesley Sullenberger.  Clearly, the guy deserves to be a national hero and he is the model for the pilot that we all want on our flight.  But I noticed something interesting in an AP story today, "The flight was supposed to have been the last leg of a four-trip day.
The crew had begun the day in Pittsburgh, flown to Charlotte, N.C.,
then to LaGuardia, and were to head back to Charlotte in the afternoon."

Immediately, I thought of the research on airplane cockpit crew dynamics and effectiveness done by Harvard's Richard Hackman and others.  Richard is the world's expert on group effectiveness and I am lucky to count him as one of my mentors, as I worked on a group dynamics project that he led when I was in graduate school. I have also seen him now and then over the years.  Some people in the groups area are more well-known in business circles, but Richard is the best, especially if you care about evidence rather than faith-based practices.

One of Richard's biggest research projects was on the dynamics of airline cockpit crews, and he devoted many days — for over a year of his life — sitting in the jump seat of the cockpit and observing and coding the dynamics if the dyad or triad. One of the main lessons that came from this –and related — research is that the less time that a crew has been together, the more group dynamics problems they have and the more mistakes they make.  In his book, Leading Teams (see pages 54 through 59), Richard cites research based on National Transportation Safety Board data indicating that 44% of the errors (called "incidents")that pilots make are on their first flight together and 73% happen on their first day together (although the incident on Flight 1549 happened their first day together, it was their fourth flight). Richard also cites an especially interesting study done at NASA Ames Research Center in flight simulators.  It compared the performance of two different kinds of crews that were put through some very challenging problems — bad weather, closed airports, mechanical problems and so on. These were pairs of pilots, just like Flight 1549.  One kind of crew was fresh, and as often happens on commercial flights, they had never worked together. The other crews had just come off several days on duty, but had flown together during that period –so they had established their group dynamics.  The tired crews made far fewer errors than the fresh crews — because they he had worked out their roles and routines. As Hackman writes, he sometimes has the impulse, when boarding a commercial flight, to stick his head in the cockpit, and ask the crew if it is there first flight together — the odds against an incident are very low even on first flights, but Hackman points out that your risk as a passenger would be far lower if you avoided first flights together. 

To return to the Miracle of the Hudson, besides fitting Hackman's research in general, the AP story provides hints of good group dynamics. Fir example, good groups instantly divide up tasks so the best people are in the best positions. Note that the role switch and quick division of labor reported by AP:

In the cockpit, Sullenberger took over flying from Skiles, who had handled the takeoff, but had less experience in the Airbus.

"Your aircraft," the co-pilot said.

While the pilot quickly leveled the plane off to keep it from stalling
and thought about where to land, Skiles kept trying to restart the
engines. He also began working through a three-page list of procedures
for an emergency landing. Normally, those procedures begin at 35,000 feet. This time, he started at 3,000.

Indeed, to take this a step further, an astounding book by Scott Snook (who was one of Hackman's students) called Friendly Fire
examines why U.S. Black Hawk Helicopters were accidentally shot down over
Northern Iraq in 1994, and one of the contributing factors he
identifies as a cause of this problem was that the crew of the plane
that had made the decision to shoot down the Blackhawks has only two
prior flights together, despite Air Force policy (based partly on
Hackman's research) that, whenever possible, "hard crews," those who
had flown together many times before, had developed themselves into a
team, and were "tight cohesive and stable" should be used whenever
possible, and especially during combat missions.

To go beyond the cockpit, as Hackman's suggests, and I discussed here, there is also evidence from product development teams, top management teams, and surgical teams that being together awhile — or having worked together on teams in the past — is linked to greater performance.  The research on product development teams suggests that they do eventually go stale, but that takes several years — at which time, some "musical chairs,"  rotating people among different teams, may help refresh the teams.  The implication is that familiarity seems to breed performance, not contempt or laziness.  And if you are designing teams, err on the side of assembling them with people who have worked together before and on the side of keeping them together longer.

Of course, there are many many angles on the Miracle on the Hudson, but this one caught my eye because Hackman and Scott Snook make such a big deal out of it.  And it has implications for so many other settings.

I wrote a rave review for my friend and colleague Huggy Rao's book Market Rebels: How Activists Make Or Break Radical Innovations a couple months ago. His book is now for sale on Amazon and everywhere else. 

As I said then:

'The book is full of useful ideas, but perhaps the central one is that,
if you want to mobilize networks of people and markets to embrace and
spread an idea, you need the one-two punch of a "Hot Cause" and "Cool
Solutions."  A hot cause like deaths from tobacco or medical errors can
be used as springboards to raise awareness, spark motivation, and
ignite red-hot outrage.  And naming these as enemies is an important
step in mobilizing a network or market. But creating the heat isn't
enough; the next step needs to be cool solutions.   This doesn't just
mean identifying technically feasible solutions, it also means finding
ways to bind people together, to empower them to take steps that help
solve the problem, and to create enduring commitment to implementing
solutions.'

Market Rebels is a remarkably useful and evidence-based book, it isn't light and fluffy, as it is an academic book. But it is well-written and chock-full with great stories. If you want to learn how a small group of people can have a big impact on networks and markets, this is the book for you.  It is especially relevant entrepreneurs, people in sales and marketing, and social activists — but anyone who wants to spread a message and change behavior on a large scale will find it extremely helpful

Stanford's James March, perhaps the most prestigious living organizational theorist, wrote me an amazing email when Jeff Pfeffer and were writing Hard Facts, our book about evidence-based management. When I asked him if there were any breakthrough ideas in management, his reply was:

“Most claims of originality are testimony to ignorance and
most claims of magic are testimonial to hubris.”  Unfortunately, we are in an era where the evidence to support March's statement is rolling in at remarkable rate. Many of the "geniuses" that have been worshiped in recent years have turned
out to be false gods — just ask all those people who lost billions
with Bernie Madoff. 

As much as I love reading Jim Collins' Good to Great for its compelling writing style and great stories, I've always found his claims way overblown. Any study that is based on only 11 companies, out of a sample of 1500, is bound to have a lot of methodological problems.  (e.g., How many companies that had Level 5 leaders didn't make the leap? His method makes it impossible to tell.) And it has always bothered me that once his "chimps" (as he called his research team) identified the good and great companies, they went back and interviewed people about the differences. It is sort of like asking players of the team that won the world series or super bowl (and reporters) what the secret to their success was after the series is over, and then comparing their answers to the players on the fourth place team. Compelling research by Berkeley's Barry Staw and others shows that winners and losers offer much different recollections to explain what happen and why, regardless of the facts.  In particular, winners report having better leaders, being more persistent, more focused, and more cohesive — even in experiments where there is no real difference between groups, and the winners and losers were simply misled about their performance by some seemingly legitimate person.  So I don't know who will win the Superbowl this year, but I can predict what the winning players will say, and so can you.  If you want to read a detailed critique of the methods in Good to Great, see The Halo Effect. 

The other thing that always bothered me about Good to Great is that — although there are thousands of rigorous peer reviewed studies that are directly on the issues he studies — he never mentions any of them to further bolster or refine his arguments (for example, there is a lot of research on charismatic leadership that he ignores, which shows it is effective under many conditions — most striking to me is that he defined it as, essentially, flashy and arrogant leadership, which isn't how most researchers define and measure it. They they do talk about charismatic leaders being articulate and able to set down a clear vision, but the key thing is the ability to motivate and direct followers' energy — muck like Collins'  Level 5 leaders).   So this second concern also is reflected in March's quote, as a claim of originality that is testimony to ignorance.

To return to the magic, the evidence that anything that is too good to be true usually is — and there is no such thing as the single magical breakthrough study in business — has been pouring in lately when it comes to Good to Great. Steve Levitt, of Freakonomics fame (and unlike Collins, a researcher with bona fide research training and a world class scholar), did an analysis in The New York Times called "From Good to Great..to Below Average," showing that Collins' 11 "great companies" were as a set performing below the S&P 500. Note also that this even before the shit really the fan, and the financial crisis drove Circuit City into Chapter 11 and Fannie Mae into much deeper trouble.

Just this morning, I got a broadcast email from Academy of Management  Perspectives, an academic journal I subscribe to that attempts bridge the gap between academic research and management practice. The next issue contains two articles — both based on data gathered well before the economic meltdown — that further document that lack of staying power among these 11 so-called great companies.  I think it is worth printing the entire abstract of each here, as given this book has sold 4.5 million copies.  Even if this may be boring to some of you, the ideas in the book are taken as an articles of faith in so much business — and Collins has acted as if the ideas in his book and those 11 companies provide magic for any company.  In an ironic twist, he has displayed remarkable personal hubris in his (noble) efforts to undermine leaders who suffer from hubris.  His writing and inspiring speeches are moving, but he is selling snake oil, or at least exaggerating the healing powers of some well-known management practices that work pretty well, but are not magical cures.

The first article is by Professors Bruce Renick and Timothy Smunt, and is called "From Good to Great to…" Here is the abstract:

With sales of more than 4.5 million copies, Good to Great
by Jim Collins provides an inspiring message about how a few major
companies became great. His simple but powerful framework for creating
a strategy any organization can use to go from goodness to greatness is
certainly compelling. However, was Collins truly able to identify 11
great companies? Or was the list of great companies he generated merely
the result of applying an arbitrary screening filter to the list of
Fortune 500 companies? To test the durability of his greatness filter,
we conducted a financial analysis on each of the 11 companies over
subsequent periods. We found that only one of the 11 companies
continues to exhibit superior stock market performance according to
Collins' measure, and that none do so when measured according to a
metric based on modern portfolio theory. We conclude that Collins did
not find 11 great companies as defined by the set of parameters he
claimed are associated with greatness, or, at least, that greatness is
not sustainable.

The second article is by Professors Bruce Neindorf and Kristine Beck. It is called "Good to Great, or Just Good?" Here is the abstract:

Good to Great has been on Business Week's best-seller list since its October 2001 release. In Good to Great,
author Jim Collins identified a set of 11 firms as great, then used
them to derive five management principles he believed led to "sustained
great results." We contend that due to two fatal errors, Good to Great
provides no evidence that applying the five principles to other firms
or time periods will lead to anything other than average results. We
explain the two errors and empirically test our contention. When ranked
with the 2006 Fortune 500, the 11 Good to Great firms have an average ranking of 202nd. In addition, in terms of long-term stock return performance, the Good to Great
firms do not differ significantly from the average company on the
S&P 500. Our evidence is consistent with the conclusion that
although the Good to Great firms may be good, they aren't great. 

Both these artifices are published in Academy of Management Perspectives, Volume 24, Number 4, 2008.

Although, oddly, my colleague and friend Jeff Pfeffer continually crossed-out most of the text in Hard Facts that took Collins to task for methodological problems and excessive claims. But the key lessons from this book, and so many others, are:

1. As March implies, there are no magical leadership or organizational practices that will quickly propel your organization to the top of the heap.  Even the greatest organizations struggle to stay at the top and are led by fallible people who make many mistakes.

2. There is no such thing as a single breakthrough study. The best and most valid conclusions and advice are based on a series of studies that have survived the brutal peer review process and that result in a consistent set of findings. In this regard, an interesting contrast is Chip and Dan Heath's Made to Stick, which is based the weight of the evidence from hundreds of rigorous studies (instead of one that could not survive the peer review process unless the claims were toned way down and the hundreds of past studies that were consistent — and clashed with it — were at least mentioned).  I especially point to Made to Stick, and I would add Influence, because they are so well-written that they show you can combine good scholarship with a great read. 

3. My main objection, in the end, isn't to the research Collins did — the stories are interesting and I believe that nearly all of the practices that he suggests would make a manager more effective — indeed many if not most are bolstered by more rigorous studies (albeit, even as his research now implies, as signs of competence or even ordinary greatness). My objection is — to use Jim March's words — the hubris and ignorance about the claims about the rigor of the research and the originality of the ideas.  There are lots of management books, or parts of management books, that are incredibly useful and inspiring, but don't claim to draw on research.  Orbiting the Giant Hairball is a great example.  Another is Tom Kelley's masterpiece Art of Innovation. The difference is that these great books don't make excessive claims – Hairball draws on the author's personal story and Tom Kelley draws mostly on what he and his colleagues have done at IDEO.

I suspect that I am not going to make a lot of friends with this email, but as I struggle to identify evidence-based practices –and practices that seem promising, cool, or have been used to good effect in a few places — that help managers and others do their work more effectively, to achieve what might be called "ordinary greatness," I think that — in the spirit of the times — perhaps all of us who are paid to give advice to managers and leaders owe to them, and to all the employees whose jobs depend on their actions, to come clean about the limits of our claims and our knowledge. 

We are at a moment in history were we all need a lot more truth and a lot less snake oil.

P.S. Here is a post about another great quote from March, on innovation.

I confess that I am biased when it comes to Hayagreeva (Huggy) Rao's work. We are good friends and work on various projects including an executive program on Customer-Focused Innovation and writing projects, such as our recent article on The Ergonomics of Innovation.   Huggy is great to work with because he not only is deeply smart, he is an unusually broad and open-minded academic.  And, also unlike many academics, nearly everything he says and writes is clear and easy to understand.  If you want to see Huggy at his best, check his new book Market Rebels: How Activists Make Or Break Radical Innovations, which will be published by Princeton next month.  Don't let the academic press put you off, this is an engaging and useful book, showing how innovations ranging from the automobile to micro-brewing spread — and why many innovations (like the Segway) did not. 

The book is full of useful ideas, but perhaps the central one is that, if you want to mobilize networks of people and markets to embrace and spread an idea, you need the one-two punch of a "Hot Cause" and "Cool Solutions."  A hot cause like deaths from tobacco or medical errors can be used as springboards to raise awareness, spark motivation, and ignite red-hot outrage.  And naming these as enemies is an important step in mobilizing a network or market. But creating the heat isn't enough; the next step needs to be cool solutions.   This doesn't just mean identifying technically feasible solutions, it also means finding ways to bind people together, to empower them to take steps that help solve the problem, and to create enduring commitment to implementing solutions.  Huggy focuses on radical innovations in this book, but the logic and general principles can be applied in any setting where a group or organizations wants to mobilize action — be it to solve a social problem or to sell a product.  Indeed, that is why the book received such strong endorsements from Andy Grove (as you can see above) and from Mozilla CEO John Lilly — who praises the book for providing practical ideas that help him run his organization. 

Indeed, Diego Rodriguez, Perry Klebahn, and I are stealing a host ideas from Huggy (both from the book and conversations with him) for our Spring d.school class on Creating Infectious Action. Our theme, which Huggy suggested, is "Kill Gas."  Diego posted about the class last week — don't miss the picture.  Certainly, most of us know that the U.S. dependence on foreign oil is bad for the economy and a national security risk.  And the resulting global warming is bad for the planet.  But — even with Al Gore's impressive accomplishments — it strikes us that this issue could be emotionally hotter.  So we are going to challenge our students — using ideas from Market Rebels and from Huggy directly — to invent and spread solutions that crank-up the emotional heat around this issue and, perhaps, to develop and implement some cool solutions.  We are still designing the class, so we would love suggestions and reactions –and if you are a Stanford master's student, please sign-up for the class.

To return to Huggy's book, it isn't the One Minute Manager, it is a detailed and thoughtful book.  But it is approachable and remarkably useful to any person or company that seeks to spread ideas, products, or practices throughout a marketplace or network. Most university press book sell just a few thousands copies, but — my bias aside — this one deserves much broader attention. 

One of the themes I've been blogging about, and has prompted a lot of response, is how difficult it is to design a financial incentive system that motivates the desired behavior — see my recent post on perverse incentives at Washington Mutual.  The theme in that and related posts, and in so many of your thoughtful comments, is that using financial incentives to motivate individual behavior often backfires because they  lead people to focus narrowly on the rewarded behavior while ignoring other behaviors  that benefit colleagues or the organization, or worse yet, engage in behavior that damages organization organizational performance.  So for example, software engineers who are paid for finding bugs may respond by introducing bugs so they can be paid to fix them — as demonstrated in the Dilbert cartoon I discussed.  Another issue with individual incentives is they sometimes lead to destructive individual competition, or at least, a refusal to help one's colleagues succeed because there is no reward for doing so.

In my last post, I suggested that the problem with money is that at times it is too powerful a motivator, not — as some psychologists have suggested — it is too weak a motivator.   I was focusing on the design problems associated with linking financial rewards to desired individuals behaviors in ways that don't undermine group productivity, group cooperation, learning,  and so on.  I didn't realize, however, that there is a stream of research showing that just getting people to focus on money makes them less cooperative and more individualistic in focus.  Along these lines, I ran into a fascinating set of nine studies packed into a 2006 Science magazine article called The Psychological Consequences of Money by Kathleen D. Vohs and her colleagues.  They used a series of "primes" to turn research subjects' attention to money (showing them lists of words about money, putting piles of monopoly money in front of them, showing them films that talked about money) and then created a host of little challenges, ranging from whether they would ask or gave help while struggling to solve an unsolvable to whether they helped an (apparently) blind person who accidentally dropped a bunch of pencils. Note there we no incentives manipulated in these studies, just a focus on money. 

The pattern of results is pretty scary and has some interesting implications.  Compared to control subjects, those primed to focus on money:

    1. Were less likely to ask others for help
    2. Less likely to give others help
    3. Preferred to work alone
    4. Preferred to play alone
    5. Put more physical distance between themselves as a new acquaintance

The little study with the blind person was especially interesting.  Subjects played Monopoly for 7 minutes and then the board was cleared by the experimenter, and regardless of how the game had gone, they were left with either a pile of $4000 in Monopoly money, $200, or no money.  Then the (apparently) blind person came in and "accidentally" spilled a pile of pencils — those subjects with a big pile of fake money in front of them picked up significantly fewer of the blind person's pencils than those with a small pile or no pile at all.

This is, of course, just a series of contrived experiments. But there are also field studies being done along these lines that tell a similar story.  The implications for how to manage people and use rewards are troubling.  Of course, nearly all of us need to work to earn money to live — so removing money from organizational life is impossible.  But these studies and related research suggest that to the extent money is made more vivid, it makes interdependent work more difficult to accomplish.  Indeed, one of the central concepts Hard Facts is the attitude of wisdom, the notion that wise people have the courage to take action but the humility to doubt what they know.  Wise people also realize that, for collective success to happen, there will be many times when they need to ask others for help and, conversely, that there will also be many times when they ought to offer others their help.  Unfortunately, if these experiments generalize to the real world, they suggest that making money more vivid may, in fact, rob people of their wisdom.

I may be pushing these findings too far, but it is always interesting to think about the implications of research. I would be most curious how others react to this research, and in particular, what you believe the implications are for designing motivation systems for organizations and teams.

P.S. Professor Vohs has a great website as it provides links to pdfs of all her papers and, if you are interested, you can see that she has a fascinating stream of research on money — examining issues such as whether social rejection leads people to shop more and whether focusing on money can reduce subjective physical pain.  She is prolific and imaginative researcher.

The causes of the mortgage meltdown are diverse and complex, and anyone who points to a single cause is almost certainly wrong.  But there is also little doubt that perverse incentives, as economists like to call them, played a role in creating the mess.  I've written about how bad incentives can, over the long haul, drive people to do things that are ultimately destructive to themselves and their organizations — with a little help from a great Dilbert cartoon — and see this great list of examples on Wikipedia. I love this one:

"19th century palaeontologists traveling to China used to pay peasants for each fragment of dinosaur
bone (dinosaur fossils) that they produced. They later discovered that
peasants dug up the bones and then smashed them into multiple pieces to
maximise their payments"

I am not among those psychologists who argues that money is a weak motivator of human behavior.  Rather, as Jeff Pfeffer and I argue in Hard Facts, the problem with using money as a motivator is that it is very difficult to get the incentive system designed so it motivates the right kind of behavior and discourages the wrong kind.  You could argue that the problem with using financial incentives is that they work too well rather than not well enough — causing people to focus their attention narrowly on a small number of things and to forget more subtle and long-term issues.  This is especially  true with individual incentive systems — which have been shown to lead to everything from garbage collectors in Albuquerque driving too fast, driving broken trucks, and missing many collections so they could finish their routes more quickly to schoolteachers in Chicago cheating on standardized tests — giving students the right answers to tests or changing the answers themselves — to get performance bonuses linked to student test scores.

Alas, the once great Washington Mutual bank seems to have fallen, in part, because so much emphasis was placed on writing as many mortgages as possible, fitness of the borrower be damned.  Check out this story in the Sunday New York Times.  Here is a excerpt that shows how their reward system — and misguided culture to supported it — helped bring down this once great bank:

MS. COOPER started at WaMu in 2003 and lasted three and a half years. At first, she was allowed to do her job, she says. In February
2007, though, the pressure became intense. WaMu executives told
employees they were not making enough loans and had to get their
numbers up, she says.

“They started giving loan officers free
trips if they closed so many loans, fly them to Hawaii for a month,”
Ms. Cooper recalls. “One of my account reps went to Jamaica for a month
because he closed $3.5 million in loans that month.”

Although Ms. Cooper couldn’t see it, the wheels were already coming off the subprime bus.

“If
a loan came from a top loan officer, they didn’t care what the
situation was, you had to make that loan work,” she says. “You were
like a bad person if you declined a loan.”

One loan file was
filled with so many discrepancies that she felt certain it involved
mortgage fraud. She turned the loan down, she says, only to be scolded
by her supervisor.

“She told me, ‘This broker has closed over
$1 million with us and there is no reason you cannot make this loan
work,’ ” Ms. Cooper says. “I explained to her the loan was not good at
all, but she said I had to sign it.”

The argument did not end
there, however. Ms. Cooper says her immediate boss complained to the
team manager about the loan rejection and asked that Ms. Cooper be
“written up,” with a formal letter of complaint placed in her personnel
file.

Ms. Cooper said the team manager told her to
“restructure” the loan to make it work. “I said, how can you
restructure fraud? This is a fraudulent loan,” she recalls.

Ms.
Cooper says that her bosses placed her on probation for 30 days for
refusing to approve the loan and that her team manager signed off on
the loan.

Four months later, the loan was in default, she says.
The borrower had not made a single payment. “They tried to hang it on
me,” Ms. Cooper said, “but I said, ‘No, I put in the system that I am
not approving this loan.’ ”

My question: Problems like this crop up over and over again. What can we do to stop them?  Should we stop using individual incentives?  I think that is too extreme, but how do we design individual incentive systems that avert a narrow and misguided focus?  I think part of the answer is supplementing them with other motivators, interesting work, a system that encourages pride and praise, and leaders who set the right example — all that press people to move in the right direction rather than the wrong direction.  But when the money becomes so vivid, people don't even seem to notice long-term dysfunctions, let alone potential ethical lapses. The Enron story was much the same.

I have no magic wand and don't believe that anyone else does.  There are cultures — I think of P&G and Google — that seem to have avoided such evils. But it isn't easy.

I once heard a group of CEOs argue that there would be far fewer ethical lapses and they would be much better at doing the right thing in the long-term if the system was changed so that they reported earnings once a year rather than four times a year.  The world is pretty much going in the opposite direction, of course, with people wanting updated information constantly — but I think it is an interesting thought experiment.

Other thoughts?

Sam Culbert is a crusty and very wise professor from UCLA and author of many books including, most recently, the charming Beyond Bullsh*t: Straight-Talk at Work.  Sam has a fantastic article in today's Wall Street Journal called "Get Rid of the Performance Review!"  I was struck by his proposal, both because it was so well reasoned and because, when I raised a similar question on this blog a few months back, in Performance Evaluations:Do They Do More Harm Than Good?, I was bowled over by both by the number and detail of the responses. In fact the responses were one of the things that led a group of us at the Stanford d.school to launch a year-long project aimed at re-inventing performance evaluations.  I urge you to read Sam's article, as his argument that most evaluations are so destructive that they are beyond repair will resonate with millions of people out there who give and get performance evaluations.

Although an entire industry of consultants, HR professionals, and software firms seem bent on devoting more and more time and money to performance evaluations, all the energy devoted to these things over the years have done little to change Sam's observation about the difference between the promise and the problems:

• The Promise: Performance reviews are supposed to
  provide an objective evaluation that helps determine pay and lets
  employees know where they can do better.
• The Problems:
  That's not most people's experience with performance reviews.
  Inevitably reviews are political and subjective, and create schisms in
  boss-employee relationships. The link between pay and performance is
  tenuous at best. And the notion of objectivity is absurd; people who
  switch jobs often get much different evaluations from their new bosses.

Sam's article is also in the spirit of design thinking, as in many cases, after people have spent years trying to perfect some procedure, gadget, or feature that they — usually mindlessly — accept as something they cannot do without and then a breakthrough happens when some clever person (often someone who isn't an expert in the field) comes along and removes it or unwittingly goes forward and succeeds without it. Then everyone realizes that they never needed it at all. Apple is the master of this approach — you may have seen that their new laptop has no mouse button and, for many years, Apple has had just one mouse button while windows systems have had two.  One of my favorite illustrations of this phenomenon in this article on escape from a submarine, which shows that after decades of trying to develop better gadgets to help people escape from sunken submarines, researchers discovered the the best technology was no technology at all. Actually, as the article shows, they  rediscovered this insight — there was good evidence from incident over 100 years earlier!

So, two insights here. First, I agree completely that performance evaluations are broken and need to reinvented and possibly replaced with something else.  Or, to take an extreme view, perhaps they should be discarded and replaced with nothing other than regular and informal honest feedback in the context of trusting relationships.  Second, there is a design principle here that is always worth remembering: Creative people, often unwittingly, often have a huge impact by removing things that everyone assumes are essential.  Design is as much about what you take away as what you add.

P.S. Check out Pete's great comment, I especially love is quote, 'As George Orwell said, "To see what is in front of your nose requires a constant struggle".'  What a great a great line!

This was the headline on an email
that I got from Geoff Staines last week. 
Geoff works for Vuibert, the publisher of our French translation of Hard Facts, Dangerous Half-Truths and Total
Nonsense (And the French translation of The
No Asshole Rule too, Objectif
Zéro-sale-con). He wrote Jeff Pfeffer and me last week to report that “Faits
et Foutaises dans le management” had won an award last week in Paris called
called “Prix du livre Ressources Humaines.” Apparently, this is an award for
the best business book published in France during the past year.
 

Geoff (who is British, but has lived in France for many years) wrote:

 
“This is the eighth year the
prize has been awarded; the three sponsors are the HR Chair of "Sciences
Po", France's prestigious political science school, Le Monde, the national daily newspaper of reference, and a feisty
headhunter called Syntec. The jury of twelve good men and women was made up of
HR Directors from the private and public sector, professors from Sciences Po,
and journalists from the Le Monde
business and economics section. Sixty titles were originally selected and
whittled down to three finalists. Main criteria were originality and relevance
of the subject-matter, research-based propositions, quality of writing and
pedagogy, and applicability.

        

About three hundred people attended the ceremony in Le Monde's offices.The jury loved the breath of coverage of your
book, the fact-based examples and what they called your unassuming,
non-dogmatic style, and chose it as a fine instance of university-based writing
for the general reader. The president of the jury read out a couple of juicy
bits, and all the copies that were discreetly on sale afterwards went quite
fast. Champagne flowed, as it always does at the drop of a hat, the munchies
were so good people stayed late, and there was a very good overall atmosphere.
I have been doing these things for yonks, and this event stood out.”

I love his description –
especially that word “yonks.”
        

Jeff Pfeffer and I are delighted with the news and even more delighted that the
notion of evidence-based management continues to spread and have impact.  If you are interested in learning more about
evidence-based management, Jeff Pfeffer and I maintain a website, www.evidence-basedmanagement.com.
And we thank the panel of judges, Le Monde and Syntec for the award. We are
honored and would like to thank everyone at Vuibert for their good work.

About two weeks ago I had the privilege of speaking at a conference in San Francisco that was attended by doctors, nurses, hospital administrators, and parents for at least 30 hospitals who were working together to improve the quality of care in Neonatal Intensive Care Units (NICU's).  It was organized by the Vermont Oxford Network, a " Non-profit voluntary collaboration concerned with medical care for newborn infants and their families."  Dr. Jeff Horbar, the CEO of the network, invited me to speak about evidence-based management to the group, and then to lead an interactive "d.school style" session. 

This second part knocked my socks off.  The 250 or so participants sat at round tables, organized by hospital, and most had a doctor or two, some nurses, an administrator or two (including a few CEOs), and parent who an infant that had been in the NICU.  The groups focused on the parent's experience in the NICU, first identifying a list of difficulties with the experience and then brainstorming was to solve one or of these problems, and finally, discussing ways to implement their solutions.  The degree to which everyone in the room was open, and non-defensive about their challenges, and their willingness to take action, and their pure enthusiasm was something to behold.  It was lesson in the power of trust, energy, and the willingness to act. 

More generally, although I realize that the U.S. health care systems is a mess in many ways, after seeing these wonderful people in action, and writing the article about the IHI campaign that saved 100,000 lives in U.S. hospitals, I am getting quite optimistic that there is hope for improving the system, and more generally, hope that some of the most difficult problems that we face in this country — and throughout the world — can be solved by determined, open, and action-oriented people.