Paul L. Caron

Tuesday, January 16, 2018

The Tyranny Of Metrics: 'Not Everything That Is Important Is Measurable, And Much That Is Measurable Is Unimportant'

MetricsWall Street Journal:  A Cure for Our Fixation on Metrics, by Jerry Z. Muller (Catholic University; author, The Tyranny of Metrics (Princeton University Press 2018)):

Measuring results is all the rage in organizations, but it is often wrongheaded and counterproductive.

In recent decades, what I call “metric fixation” has engulfed an ever-widening range of institutions: businesses, government, health care, K-12 education, colleges and universities, and nonprofit organizations. It comes with its own vocabulary and master terms. It affects the way that people talk and think about the world and how they act in it. And it is often profoundly wrongheaded and counterproductive.

Metric fixation consists of a set of interconnected beliefs. The first is that it is possible and desirable to replace judgment with numerical indicators of comparative performance based on standardized data. The second is that making such metrics public (transparency) assures that institutions are actually carrying out their purposes (accountability). Finally, there is the belief that people are best motivated by attaching rewards and penalties to their measured performance, rewards that are either monetary (pay for performance) or reputational (rankings).

But not everything that is important is measurable, and much that is measurable is unimportant. Most organizations have multiple purposes, and that which is measured and rewarded tends to become the focus of attention, at the expense of other essential goals.

Similarly, many jobs have multiple facets, and measuring only a few of them creates incentives to neglect the rest. Almost inevitably, people become adept at manipulating performance indicators. They fudge the data. They deal only with cases that will improve performance indicators. In extreme cases, they fabricate the evidence.

It’s not that measurement is useless or intrinsically pernicious. The challenge is to specify when performance metrics are genuinely useful—that is, how to have metrics without the malady of metric fixation.

Should you find yourself in a position to set policy, here are some questions that you should ask, and the factors that you should keep in mind, in considering whether to use measured performance, and if so, how to use it.

  • What kind of information do you wish to measure? ...
  • How useful is the information? ...
  • Are alternative measurements available? ...
  • What is the metric for? ...
  • What are the costs of getting the data? ...
  • Who develops the measurement? ...
  • Does the measurement create perverse incentives? ...

With measurement as with everything else, recognizing limits is often the beginning of wisdom. Not all problems are soluble, and even fewer are soluble by metrics. It’s not true, as too many people now believe, that everything can be improved by measurement, or that everything that can be measured can be improved.

Book Club, Legal Education | Permalink


Numerocracy, the rule of numbers, it's the curse of modern society . . .

Posted by: mike livingston | Jan 16, 2018 4:11:44 AM

As every accountant knows, be careful what you ask for because you will get what your measure.

Posted by: Dale Spradling | Jan 16, 2018 7:16:17 AM

Metrics can be particularly destructive in healthcare, where I suspect it's becoming the rage among administrators looking to justify their paychecks.

Posted by: Michael W. Perry | Jan 16, 2018 10:04:58 AM