Wednesday, February 28, 2018
Following up on my previous post, The Tyranny Of Metrics: 'Not Everything That Is Important Is Measurable, And Much That Is Measurable Is Unimportant': Inside Higher Ed, 'The Tyranny of Metrics':
These days colleges boast about their admissions rankings, their graduation rates, their faculties’ achievements and much more. Many say that the statistics are a tool to promote accountability and improvement.
Jerry Z. Muller disagrees. His new book, The Tyranny of Metrics (Princeton University Press 2018), critiques not only higher education but many parts of society that rely on metrics.
"Gaming the metrics occurs in every realm: in policing, in primary, secondary and higher education; in medicine, in nonprofit organizations; and, of course, in business," Muller writes. "And gaming is only one class of problems that inevitably arise when using performance metrics as the basis of reward and sanction. There are things that can be measured. There are things that are worth measuring. But what can be measured is not always what is worth measuring; what gets measured may have no relationship to what we really want to know."
Q: Some colleges, government agencies and businesses promote tools to evaluate faculty productivity -- number of papers written, number of citations, etc. What do you make of this use of metrics?
A: Here too, metrics have a place, but only if they are used together with judgment. There are many snares. The quantity of papers tells you nothing about their quality or significance. In some disciplines, especially in the humanities, books are a more important form of scholarly communication, and they don’t get included in such metrics. Citation counts are often distorted, for example by including only journals within a particular discipline, thereby marginalizing works that have a transdisciplinary appeal. And then of course evaluating faculty productivity by numbers of publications creates incentives to publish more articles, on narrower topics, and of marginal significance. In science, it promotes short-termism at the expense of developing long-term research capacity.