The strong reactions to Marge’s Asshole Management Metric not only got me thinking about the difficulties of managing nasty people (and the times when there might be a need for people to get nastier to defend themselves, an excellent if unfortunate point that Marge made), it started me thinking about metrics. I have also been thinking about metrics lately because one of my doctoral students has been struggling with the problem of how to measure the effectiveness of user-centered design practices.
Both Hard Facts and The Knowing-Doing Gap review a lot of research about metrics. I confess that wading through the "best practices" claims, theory, and evidence in this area isn’t easy. There is so much written and so many strong beliefs about how performance should be measured and how to motivate people (most unaffected by evidence) that figuring-out how to design an optimal measurement and incentive system is tough, perhaps impossible. We’ve even had compensation consultants tell us that it is such a great area to work in because clients always call you back for more work — because no matter what kind of system you help them design or that they install, it never works quite right.
I can’t claim to have any magic answers either (and don’t believe anyone who tells you they do, like the authors of Topgrading or The War for Talent). But there is one guideline that few companies follow, even though they nearly all know they should (there is a reason we wrote a book called The Knowing-Doing Gap): If you measure and reward people on too many different dimensions, they are pulled so many different different directions (since different criteria are often unrelated or negatively related), that it is simply impossible to adjust 10 — or in some cases — 100 diverse behaviors in response to the system. The result is that that your measurement system becomes useless or worse.
We call this The Otis Redding Problem. Recall the line from his old song: Sitting By the Dock of the Bay, “Can’t
do what ten people tell me to do, so I guess I’ll remain the same.” That’s the problem with holding people,
groups, or businesses to too many metrics: They can’t satisfy or even think about all of them at once, so
they end-up doing what they want or the one or two things they
believe are important or that will bring them rewards (regardless of senior management’s strategic intent). Yet many organizations fail to
implement this well-known and common sense principle.
And the rise of balanced score cards –- which have
many virtues when done right -– have made this problem even worse. As we say in The
Knowing-Doing Gap “In principle, the balanced scorecard makes a great deal of
sense. Rather than just measuring and evaluating
managers on the financial performance of their units, which largely reflects
what has happened in the past, the scorecard emphasizes getting ready for the
future.” But one of the most common unintended side-effects is that people are assessed and rewarded with an absurdly long list of metrics. This happens because so many different groups in the organization want “their” metric to be
measured and linked to incentives and because the groups that design and implement these
systems (often in HR) often don’t have enough
power or will to say “no.” Although HR certainly doesn’t deserve all the blame. I once
went to a talk at Stanford where a supply chain consultant and his client
proudly announced they had just added their 100th metric to the
performance evaluation system (and this was just for supply chain performance). I gently asked the executive who was trying to use the system if it created problems and he admitted that it was confusing and he ended-up focusing on just one or two.
Similarly, we describe another example of the Otis Redding problem in The
Knowing-Doing Gap:
A Harard Business School Case study reported that branch managers at Citibank [now part of Ctigroup]faced the following scorecard measures in 1996:
Financial:
Revenue
Expense
Margin
Strategy Implementation:
Total households
New to bank households
Lost to bank households
Cross-sell, splits, mergers households
Retail asset balances
Market share
Customer Satisfaction
Control:
Audit
Legal/Regulatory
People:
Performance Management
Teamwork
Training/Development
Self
Other
Employee Satisfaction
Standards:
Leadership
Business Ethics/Integrity
Customer
Interaction/Focus
Community Involvement
Contribution to Overall
Business
Each component
of the Scorecard was scored independently into one of three rating
categories: “below par,” “par,” or “above par”.
Pfeffer and I did an interview with a New York-based Citibank branch manager around that time — we met him because bank executives reported that
he was one of their best managers. The manager told us that, although it required a lot of time to
fill-out all the forms and go to all meetings held under the balanced score-card system, he viewed it as a flavor of the month. He described it as just another program that he had to pretend to care about until management tired of it and became enamored with the next fad (which he would have to pretend to care about as well). This manager also told us that – despite all of the hype from executives about the importance of the new balanced-score card system – he believed
that only one thing still mattered at the bank was generating short-term profits. So that
was what he focused on doing, which was why he was so highly regarded.
Part of the problem was that he didn’t believe that management was committed to the system, because he had endured one internal program after another. But this program also wasn’t taken seriously because, when a system tries to measure and link rewards to everything, it means nothing because human beings can only think about and do so many things at once.
The upshot of all this is that I don’t know the secret to designing a successful performance management system
and I am not sure that anyone else does either. But I do know that if you fall
prey to the Otis Redding problem, your system is doomed to fail.
Leave a Reply