Category: Evidence-based Management

A
couple weeks ago, I raved about Better: A
Surgeon’s Notes on Performance, in a post I wrote over at The Working Life ­on Masters of the Obvious. Author and surgeon Atul Gawande provides a
series of compelling essays about how, through one tiny small step at a time,
by gathering, studying, and using evidence, and by focusing on small seemingly unimportant
details, mortality and complication rates can decline steadily in everything
from battlefield injuries, to children suffering from cystic fibrosis, to
halting the spread of polio. It is a
compelling read, and has many implications for what it takes to be a great manager,
not just what it takes to keep improving the quality of medical care throughout
the world. I was especially struck how
the best doctors and hospitals have what Jeff Pfeffer and I have called “the attitude
of wisdom,” they have courage to keep acting on the best knowledge that they
have right now and the humility to doubt what they know, so that when new
information comes along, they can change their beliefs about what works –and
their behavior too.

I
was also taken with Gawande’s suggestion that, if a hospital or medical unit
wants to improve its performance, one of the most effective ways is to study “positive
deviants,” those statistical outliers that are doing far better than the
rest. As I read his stories, especially
about the best versus the average hospitals that treat cystic fibrosis, I was taken with the approach. At the same
time, I realized that it is remarkably similar to what many companies do when
they benchmark: they find the very best performers in their industry – or another
industry – and then try to imitate everything they do as closely as
possible. This method can be useful, but
at the same time, as our work on
evidence-based management shows, it is a risky method if done in a casual
way, without thinking about what you are imitating and why.

If
you are going to try to learn from top performers, there are at least five
pitfalls you need to keep in mind:

1. What seem like characteristics
of top performers may actually not distinguish them at all from poor performers –Don’t just look at
winners, look at winners and losers. This was the main flaw with Peters and
Waterman’s huge best-seller In
Search of Excellence. They only looked at excellent companies, so it
was impossible to tell if what the winners were doing was any different from the
losers!

 2. Watch the correlation is not causation
problem. Everyone learns this in statistics, but a lot of leaders forget it
when they benchmark. Just because something
is associated with performance, doesn’t mean it causes performance. For an enduring example that seems to persist
despite our complaints in the Harvard
Business Review (as well as directly to Bain partners), go to www.bain.com. The very first thing you see is chart the says “Our Client’s Outperform
the Market 4 to 1.” I remain amazed that
the smart people at Bain have had this on their website for so many years. Do they really mean to imply that using Bain
has a huge positive wallop? They have
some bold sounding and meaningless text beneath the chart (e.g., “Companies
that outperform the market like to work with us; we are as passionate about their
results as they are.”). The marketing people seem smart enough to duck
the question of if using Bain really drives these results, because they are
smart enough to know there are so many alternative explanations (e.g., perhaps firm that make more money can better afford an expensive management consultant).  But they aren’t wise enough to take the chart down — I suspect it is one of those sacred cows, something that many people realize is dumb, but are afraid to change.   I also want to emphasize that I am a big fan of Bain, in part,
because I think they are among the most evidence-based of the major consulting
firms, but again, I urge them to take this down – it makes them look bad. 

3. When you compare winners
and losers, beware of “untested” differences.  Just
because every winner you look at does something and every loser you look at doesn’t
do it, isn’t enough – it may just be the result of a bad sample. This is the one of the main problems with Jim
Collins’ best-seller Good
to Great. I find this a compelling read and would like
to think, for example, that firms with level 5 leaders, those who are unselfish and
relentlessly driven to improve firm performance, will trump firms with
selfish and less driven leaders. But note that Collins reaches this conclusion
by comparing his 11 “great” firms to an equally small matched sample of firms
that didn’t make the leap. He fails to
point out that no attempt was made to find firms that also had level 5 leaders, but
failed to make the leap –- and he could have left out thousands of firms from this tiny sample that had level 5 leaders, but didn’t make the leap. Again, I like a lot of things
about this book, but I do wish that Collins wouldn’t hold it up as up as such a
rigorous study, as while I think it has helped a lot companies, it is not a model of a rigorous research, and could only be published in a peer reviewed journal if it made careful links to the prior research that supports his conclusions (something the book doesn’t do) and if he acknowledged the numerous methodological flaws. See The Halo
Effect for a more damning attack. I am not as negative about Good
to Great, as I think it has helped many managers despite the excessive
claims Collins makes about the research.  But the well-crafted critique in The Halo Effect is
worth reading and, frankly, I wish Collins would acknowledge some of these problems.  It would still be a great book — everything has flaws and few books are as compelling as Good to Great. Also, admitting the flaws strikes me as something a level 5 leader would do! 

4. What is good for them might be bad for
you Consider the case of a quality movement. If General Motors had not
massively improved the quality of its cars ands trucks (despite its other persisting
problems), I believe that the company would simply no longer be alive
today. Yes, Toyota continues to be a very tough
competitor, and GM struggles, but their quality has improved massively in the
past decade, and without it, the company would be in far worse shape – if not
out of business. BUT that doesn’t mean
that every company needs a quality movement. Kodak, for example, had a fairly effective quality effort some years
back, but the problem was that it was focused on their soon-to-be obsolete chemical-based film business –so it helped them to be more efficient at doing the wrong.

5. Winners may succeed
despite rather because of some practices. This brings me to my favorite example. It
is very well-documented that Herb Kelleher, who was CEO of Southwest Airlines
during an unprecedented run of growth and profitability in the industry, smoked
a lot of cigarettes and (according to multiple reports, including his own)
drank about a quart of Wild Turkey whiskey per day during this period. If mindless imitation of successful companies
is the key to success, this means that you need to get your CEO to start
smoking and drinking a lot – or to keep it up if he or she is already doing it.
Sounds absurd, doesn’t it? But it is no different than the arguments that
armies of consultants are making right now about GE, Google, and P&G – you should
do it because they do it, and are successful.

In
closing, I want to emphasize that you can learn a lot from “positive deviants.”  But you need to stop and think carefully about why they succeed and what work for your organization. And, consistent with the argument I make
again and again here and elsewhere — following from design thinking —  if you are going to do something new, try a small some cheap
experiments if you possibly can: It is a lot cheaper than rolling out a big
program that turns out to be a bad idea. That is also why, although some mergers are a good idea, it is one of
the organizational changes that I see as most risky because it so difficult to reverse once it is started.  Also, as I’ve written
here before, mergers have high failure rates, despite the success stories
you may hear from your local investment banker.

My
latest post on The Working Life at
Harvard Business Online just came-out. It is called The
War for Talent is Back and starts out:

"Last
week I did a workshop with a group of about 20 CIOs from large companies. Our
discussion focused on what they could do to build a more civilized workplace.
In the course of our conversation, each of these executives emphasized–as I have
read recently in The Economist,
The New York Times,
and BusinessWeek–that
building a workplace that attracts and keeps great people is especially
important now because the job market for skilled people is so hot. I also have
heard similar messages at other companies I have visited recently, including
eBay, Microsoft, Google, SuccessFactors, and Yahoo!, as well as from managers
at companies including Procter & Gamble and Fidelity Investments." €

I develop five suggestions for winning the so-called war for talent
in some detail (see the
post for my arguments); note the first four clash with advice
given by many so-called experts, but are supported by much peer-reviewed research.

1. Superstars are
overrated.

2. Great systems are more
important than great people.

3. Create smaller rather
than larger pay differences between "€œstar" employees and everyone else.

4. The law of crappy people
is probably a myth.

5. The no asshole rule helps.

The
fifth suggestion won’t surprise anyone who reads this blog; but I was surprised
by how vehement these CIOs were about the importance of creating places that
were free of contempt and anger because, when such asshole positioning strikes, it makes it so much harder to attract and keep good people.

P.S. Also see The Waste of Talent.

  It
happened in on October 3, 2002. I was at
a Harvard Business School Press conference in  Cupertino, California, where Clay Christensen of
Innovator’s Dilemma fame was
interviewing Intel’s former CEO and Chairman
Andy Grove. As I point in The
No Asshole Rule, I’ve always admired Grove because – although he is
tough and argues over ideas (and I don’t even agree with a lot of his ideas) — he argues because he cares about testing and developing ever-improving ideas,
not to put people down. Grove is the poster
boy for Intel’s constructive confrontation, which I talk about in this edited
excerpt from Weird
Ideas That Work.  To me, Grove
demonstrates how it is possible to be a tough and effective leader without
being an asshole – in part because he is one of the most honest executives
around, about both his own errors and those made by others.

Grove’s
quote that day was so compelling to me because it conveys so much about the primary
dilemmas that leaders face, and what skilled leaders can do to salvage even
most seemingly impossible situations. [Note
that quote comes from a transcript of the talk that the folks at Harvard gave
me.  I have edited out a few lines, in part,
because Grove made some comments about the Soprano’s TV show that were funny,
but distract from the main point]. Grove
said: 

“None
of us have a real understanding of where we are heading. I don’t. I have senses about it…… But decisions don’t wait, investment decisions
or personal decisions and prioritization don’t wait for that picture to be
clarified. You have to make them when
you have to make them. So you take your
shots and clean up the bad ones later.

And
try not to get too depressed in the part of the journey, because there’s a
professional responsibility. If you are
depressed, you can’t motivate your staff to extraordinary measures. So you have to keep your own spirits up even though
you well understand that you don’t know what you’re doing.”

Then,
Clay Christensen asked, “So how do you work on that part about keeping good
spirits or managing emotional response, leading your team.”  Grove answered:

“Well,
part of it is self-discipline and part of it is deception. And the deception becomes reality —
deception in the sense that you pump yourself up and put a better face on
things than you start off feeling. After
a while, if you act confident, you become more confident. So the deception becomes less of a
deception. But I think it is very
important for you to do two things: act on your temporary conviction as if it
was a real conviction; and when you realize that you are wrong, correct course
very quickly.”

Grove’s
words provide a compact summary of at least four of the core ideas in Hard
Facts, Dangerous Half-Truths, and Total Nonsense: Profiting from Evidence-Based
Management, especially our chapter that asks, “Are Great Leaders in
Control of their Companies?”1.
The last point in his quote is “act on your temporary conviction as if it was a
real conviction; and when you realize that you are wrong, correct course very
quickly,” a nearly perfect compact summary of the most important single idea in
our book: Effective evidence-based management entails adopting the attitude of
wisdom; acting with knowledge while doubting what you know. This means,
as I’ve written here before, that the best leaders have strong
opinions, weakly held. They need strong opinions so that people can
accurately assess an idea, and especially, so they know how to implement it –
which ultimately means that things go wrong and go right, it is possible to
tell WHAT has succeeded failed.

2. The second idea is about confidence and the power of the
self-fulffiling prophecy. Note how Andy says: “And
try not to get too depressed in the part of the journey, because there’s a
professional responsibility. If you are
depressed, you can’t motivate your staff to extraordinary measures. So you have to keep your own spirits up even
though you well understand that you don’t know what you’re doing.” This may disturb some people, but the fact is
that it is impossible to know if a decision will succeed our fail when you make
it; but decisions must be made and implemented, or a company will stagnate or
die.

There
is strong evidence that one of the main ways to increase the odds that a
decision will succeed is to express and feel confidence in it, even if you are
not quite sure, because – as hundreds of studies of the self-fulfilling
prophecy show, believing that something will succeed is one of the best ways to
increase the chances it will succeed. Indeed, if you want to read about another great leader who was fantastic
at presenting public confidence despite private doubt, see David
McCullough’s 1776, on George Washington’s defining year. As Grove adds,
such confidence then becomes authentic, and that further increases the odds
that good things will happen “And the
deception becomes reality — deception in the sense that you pump yourself up
and put a better face on things than you start off feeling. After a while, if you act confident, you
become more confident.”

3. The third
point is the paradox that leaders face: They need to continue doubting what
they have done privately; but if they express too much public doubt, then
people lose confidence. A leader’s self-doubt also undermines the implementation of
current decisions – which increases the odds a good decision will fail and
slows the process of learning why a bad decision is a bad.   At the
same time, if strong opinions are too strongly held, decision-makers have a
tough time admitting failure or even updating current actions to make them more
effective.

S0, to get out of this paradox, effective
leaders need to sustain some private self-doubt and as Michigan’s Karl Weick puts it —  argue if they are right, but listen as if they
are wrong. This requires surrounding
themselves with people who can and will argue over ideas (as Grove loves to do),
people who respect them, but tell them they are wrong (Henry Knox helped play
this role with George Washington).

4. The fourth point is
about how to update and change course. When Grove realizes he has made a mistake (and he
talks about his mistakes openly), he admitted them to yourself and others, and
then took swift public and private action to change course and make repairs –
this avoids focusing on making excuses, pointing fingers,  or trying to hang on to as many elements of
the unwise action as possible (to avoid admitting your imperfections). That is why Grove says “So you take your shots and clean up the bad ones later” and adds “when
you realize that you are wrong, correct course very quickly.” Indeed, as we show in Hard Facts, Dangerous Half-Truths, and Total Nonsense, although
some leaders are loath to admit their mistakes, a growing body of quantitative
research suggests that when they admit that they and their companies have done
something wrong, are going to stop it, and take actions to make things better,
the performance of their organizations over the long haul is superior to
leaders who prefer to point the finger of blame outward rather than
inward.

I don’t
mean to hold up Grove as a perfect or infallible leader; he has certainly made
mistakes, as he is a human being like the rest of us. But I admire this statement
because conveys far more honesty and wisdom than the drivel that comes from most
executives… and note the complete lack of jargon
monoxide.

P.S.
The classic case of a leader that  accepted reason ability and a firm that “acted with
knowledge while doubting what they knew,” of course, was Johnson and Johnson
and CEO James E. Burke’s handling
of the Tylenol murders in the 1980’s. We just  reviewed that case in class this week, astounding story.

Stanford Business School’s Jeff Pfeffer and I published a book last year called Hard Facts, Dangerous Half-Truths, and Total Nonsense: Profiting From Evidence Based Management.  We made the case that using evidence-based practices in organizations was not only possible, but that some of the most the most effective leaders and organizations were already doing it.  I am pleased to report that Jeff testified to congress this week about the implications of evidence principles for on March 8th the Hearing on Federal Personnel Reform. You can read Jeff’s full testimony at our website, www.evidence-basedmanagement.com, where it is the featured article.  I am biased of course, but I think that Jeff does a great job of explaining what evidence-based management is and zeros in on the challenges of implementing pay-for-performance systems (which the Federal government is currently experimenting with and proposing to spread throughout the system). He explains how — although many private sector companies and their consultants have great faith in individual pay-for-performance schemes — these systems are typically less effective and have more negative side-effects than is commonly believed.

P.S. Also check out Joe Mello’s guest column on The Myth of the Mean. Joe is COO of DaVita, one of the companies that we feature as a model of how to turn evidence-based management practices into action. DaVita runs hundreds of kidney dialysis centers and is the industry leader in both quality of patient care and financial performance.

I’ve written before about Karl Weick, a psychologist from The University of Michigan. Karl is one of our most imaginative theorists, and also a gracious person who cares deeply about ideas. I’ve written about his argument that the right attitude for learning and creativity is to "argue as if you are right and listen as if you are wrong," which strikes me as the right path for developing strong opinions, weakly held, as they advocate at the Institute for the Future.

The reactions that I got to Weick’s ideas about arguing and listening reminded me of an exercise that I use in a doctoral class that I teach every few years on "The Craft of Organizational Research." One theme is to help students identify the writings they love most, who their heroes are, who they want to be like when they grow-up — so they can build a career that will enable them to do good work and to enjoy doing it.   As I say to them, my hero is Karl Weick. I pick an excerpt from his 1989 Academy of Management Review article called "Theory Building as Disciplined Imagination" as it weaves together Karl’s keen observational powers, knowledge of empirical research, and his uncanny knack for blending diverse ideas in ways that surprise and delight his readers. 

First, let me set the stage for his quote. Weick was writing about the notion that good theorists should like new evidence that disconfirms their ideas, as it speeds the process of building interesting theory. But then he goes on to say that this doesn’t always happen because, once a theorist has a strong investment in a theory  and has well-organized defenses and ideas about that theory, and is planning to spread the theory, new ideas (especially disconfirming evidence) will likely be experienced as upsetting to him or her — even if they improve the theory — because such interruptions throw a monkey wrench in current plans, causing the person to go through new cognitive effort and threatening to destroy something that he or she has worked hard to build and defend — and  I would add to often lash and destroy the offending ideas and evidence (and perhaps the person who has them).  Then he adds this statement about generalists and specialists:

Generalists, people with moderately strong attachments to many ideas, should be hard to interrupt, and once interrupted, should have weaker, shorter negative negative reactions since they have alternative paths to realize their plans. Specialists, people with stronger attachments to fewer ideas, should be easier to interrupt, and once interrupted, should have stronger,more sustained negative reactions because they have fewer alternative pathways to realize their plans. Generalists should be the the upbeat, positive people in the profession while specialists should be their grouchy, negative counterparts (page 526).

I don’t know about the people that you work with,  but in my field — although I won’t name any names — this theory works pretty well.  The people who have devoted their lives to developing just one idea or a few seem most grumpy to me. And it makes sense.  If you believe in just one idea very strongly, new information either supports it (that is not an interruption, just more signs you are right, so you can hum along), challenges it (so you have to change the one thing you know and love or attack the idea), or it is an irrelevant distraction from the one little would that really matters to you (your one true pet theory, that you have devoted so much effort to building).  This logic not only applies to academics, I’ve heard my wife argue that the grumpiest lawyers are those who are so specialized that "they know almost everything about something that is so narrow and obtuse that it seems like almost nothing,"  these ultra-specialists often have no interest in other law, client’s related business problems,  or what other lawyers do.  And, like ultra-specialists in other fields, they believe that others are really too naive and stupid to understand their precious, well-developed, but very specialized ideas.

I wonder, does Weick’s observation ring true for other occupations?  Note this is an untested hypothesis as far as I know, but it sure is interesting.

I was interviewed for part of the "Valentine’s Day" package by Renee Montagne, the host of National Public Radio’s Morning Edition. It will run two or three times.  As said in my earlier post, the story was about the virtues of love in the workplace.  I acknowledged there are risks — both legal and professional — but there is an argument that, if a couple makes it through the initial period of lust and distraction, then romance can be good for companies. There is evidence that productivity goes down during the early stages of workplace romances, and of course there are other risks as well, especially if they have a messy break-up — which often drives at least one party out of the company.   But once some stability sets in, married couples and domestic partners can be good for organizations, as they end-up with loyal employees who understand the company well.

Southwest Airlines is exhibit one here. They have 1100 married couples — so 2200 of their 35,000 or so employees — and they claim the loyalty and stability helps them a lot.  It was fun and I guess they will cut the 15 minute or so interview to 3 or 4 minutes, which isn’t bad compared to my Today Show experience.

Listen to "Weighing the Pros and Cons of Office Romances" at NPR here

Jeff Pfeffer added an interesting comment the post I did on The Billable Hour Turns People Into Workaholics that I think is most interesting. I repeat:

DeVoe and I started this research to understand how organizational practices "spill over" into other domains.  We have found, using nationally representative survey data, that people paid by the hour (and these are not all, or even most, technical contractors, lawyers, etc., but instead are manufacturing and service workers), are, controlling for many other things, more willing to trade more time for more money and less likely to spend time volunteering.  The research we have done cites an article by Steve Barley (a colleague of Bob’s) and two co-authors who found a paradox:  people who went into contracting to presumably increase their freedom actually acted like they didn’t have any.  This sort of paradox–acting in ways that are inconsistent with our preferences–is something that we continue to pursue in this research project.

So, two points–this is not just about professional or technical contractors or lawyers or accountants–the evidence is that the phenomenon is widespread.  And second, there are other ways to get the same results–namely, priming people (using a sentence descramble task) to think about money or economics terms, or to have people calculate their own hourly wage (regardless of how they are paid).  There is evidence that having people think about money causes them to act in a more independent (not wanting to ask for or give help, not wanting to associate with others) way. 

In light of the pervasive use of economic language in our society, and in light of the prominence of "money," it does sort of make one wonder.  The article on money is in Science and Vohs is one of the co-authors–it is both short and interesting.

P.S.Here is the link to the abstract of the Science article on The Psychological Consequences of Money that Jeff mentions.

My
last post on Jeff Pfeffer’s next book, What
Were They Thinking? reminded me of Jeff’s recent research on how people in
jobs where “time is money” fall into a trap: They start devoting more time
to their jobs, and less time to “unpaid” activities like family, friends, and
leisure. There is a splendid summary of
this research on the Stanford Business School website, Time
is Money When You are Paid by the Hour, “In a series of recent studies,
Pfeffer and doctoral student Sanford E. DeVoe found that people who are used to
being paid by the hour start thinking of time as a commodity almost equal to
cash. And given the choice as to whether they’ll take time or green bills,
they’ll usually take the latter—meaning they’re nearly always willing to put in
more hours to get the pay.”

Writer Marguerite Rigoglioso quotes Pfeffer
further: “Being paid by the hour causes people to endorse the idea that they’d
rather make more money and spend more time at work,” says Pfeffer, the Thomas
D. Dee II Professor of Organizational Behavior. “This shows how a commodified
view of time spills over into how people view their personal and leisure time.”

“Once you’re paid
by the hour,” he says, “you start placing a monetary value on that hour. The
opportunity costs of not working become clearer. People gravitate toward things
that are easier to evaluate, and it’s easier to figure out the value of a paid
hour than it is, say, the value of an hour spent in a leisure activity. So they
chose work over play.”

Pfeffer
and DeVoe also review related research, including one study that found “lawyers
watching their kids play soccer admitted to mentally ticking away lost income
for each minute they stood on the sidelines.”

Ouch… I feel sorry for both the kids and the parents.

Unfortunately,
many professional service firms – especially those like law firms that bill by
the hour – systematically weed out people who don’t think and act this way, as
they are less “profitable.”  This is the
kind of research that gives me chills and makes me wonder if a little less
“productivity” might be a good thing at times.

My co-author Jeff Pfeffer has a great new book coming out in July of this year, called "What Were They Thinking: Unconventional Wisdom About Management."   After writing two books with Jeff — and spending a lot of time in other ways as we are good friends — I love this book because it is so much like sitting down and talking with him.  It contains a series of "rational rants" — that is how I think of Pfeffer, an emotional and persuasive person, but one whose passions are driven by evidence and logic. Indeed, Jeff is one of the most prestigious and influential organizational theorists who has ever lived, and I suspect the most productive who has ever lived. So although his opinions on everything from stock options, to incentive pay, to leadership, to the suspect monetary value of most MBA degrees, to the sometimes questionable ethics of Stanford MBAs are filled with stories and passion, Jeff’s grasp of evidence and logic is always present. So, to me, although his ideas may strike some people as unconventional, what Pfeffer is doing is toppling the dangerous half-truths and total nonsense that so many people believe with facts — which are wrapped in  vivid examples so you can remember them and figure out how to use them (Just as the Heath brothers suggest in Made to Stick).  I don’t agree with every essay in the book — for example Jeff is more accepting of the virtues of assholes than I am — but that is why we look to work together. Indeed, two of our motto’s are "when two people always agree, one of them is unnecessary" and  "the more we fight the better write."

Jeff’s other books (about 15 of them) are all "novels" that weave together disparate ideas into coherent themes. I think of this book as a collection of short stories glued together by the strength of  Pfeffer’s voice, values, and commitment to what is true rather than what makes people comfortable.  I have read every book that Pfeffer has ever written and many of his articles (indeed, people in our field joke that he has written so much that no one has read it all — including Jeff). I think that this is the most fun to read and is filled with ideas to help managers who strive to be humane and effective, to build organizations profit through their people and that develop rather than demean and destroy their "human resources."

I’ve been talking about Made to Stick here for months. As I’ve said, it is one of the most important and useful management books I’ve ever read. I am always stunned at how Chip Heath grabs management audiences of all stripes when he talks about how to develop ideas that will people remember, use, and spread to others.  It is evidence-based and has fantastic stories. The book just came out and Amazon is now shipping it (I bought a lot of them, so I am getting the emails).  I am not the only one who is excited about the book. The national media is too. Check out these stories in USA Today and Time.  AND the Heath brothers are going to by on the Today Show tomorrow morning, Wednesday January 3rd, at some point between 7:30 and 8:00 AM. WOW!  Chip tells me that they have been rehearsing 25 second answers for their 5 minute segment! I can hardly wait to see them.