Paul L. Caron

Wednesday, February 28, 2018

The Tyranny Of Metrics

MetricsFollowing up on my previous post, The Tyranny Of Metrics: 'Not Everything That Is Important Is Measurable, And Much That Is Measurable Is Unimportant':  Inside Higher Ed, 'The Tyranny of Metrics':

These days colleges boast about their admissions rankings, their graduation rates, their faculties’ achievements and much more. Many say that the statistics are a tool to promote accountability and improvement.

Jerry Z. Muller disagrees. His new book, The Tyranny of Metrics (Princeton University Press 2018), critiques not only higher education but many parts of society that rely on metrics.

"Gaming the metrics occurs in every realm: in policing, in primary, secondary and higher education; in medicine, in nonprofit organizations; and, of course, in business," Muller writes. "And gaming is only one class of problems that inevitably arise when using performance metrics as the basis of reward and sanction. There are things that can be measured. There are things that are worth measuring. But what can be measured is not always what is worth measuring; what gets measured may have no relationship to what we really want to know."

Q: Some colleges, government agencies and businesses promote tools to evaluate faculty productivity -- number of papers written, number of citations, etc. What do you make of this use of metrics?

A: Here too, metrics have a place, but only if they are used together with judgment. There are many snares. The quantity of papers tells you nothing about their quality or significance. In some disciplines, especially in the humanities, books are a more important form of scholarly communication, and they don’t get included in such metrics. Citation counts are often distorted, for example by including only journals within a particular discipline, thereby marginalizing works that have a transdisciplinary appeal. And then of course evaluating faculty productivity by numbers of publications creates incentives to publish more articles, on narrower topics, and of marginal significance. In science, it promotes short-termism at the expense of developing long-term research capacity.

Book Club, Legal Education | Permalink


Has the author ever worked outside of academia? Every decent size job/company has some way of measuring performance. And some ways of measuring performance ("metrics" is only the current buzz word) aren't terribly smart. Employees are often smarter than managers and learn to hit those numbers that will keep them employed and earn them raises.

Metrics blindly applied by stupid managers produce poor results. Some public companies will punish a head of tax if the tax provision varies from the actual return filed 7 to 8 months later. So, I have reliably been told by a number of heads of tax, they file the provision and correct the return when the IRS auditors arrive. Beating averted.

It is better to measure performance. But only up to a point. Even good ideas become stupid if pushed too far.

Posted by: aircav65 | Feb 28, 2018 12:51:08 PM