Inside Higher Education, Can Your Productivity Be Measured? (reviewing Yves Gingras (University of Quebec), Bibliometrics and Research Evaluation: Uses and Abuses (MIT Press, 2016):
“Since the first decade of the new millennium, the words ranking, evaluation, metrics, h-index and impact factors have wreaked havoc in the world of higher education and research.” ... Ultimately, Bibliometrics concludes that the trend toward measuring anything and everything is a modern, academic version of “The Emperor’s New Clothes,” in which — quoting Hans Christian Andersen, via Gingras — “the lords of the bedchamber took greater pains than ever to appear holding up a train, although, in reality there was no train to hold.”
Gingras says, “The question is whether university leaders will behave like the emperor and continue to wear each year the ‘new clothes’ provided for them by sellers of university rankings (the scientific value of which most of them admit to be nonexistent), or if they will listen to the voice of reason and have the courage to explain to the few who still think they mean something that they are wrong, reminding them in passing that the first value in a university is truth and rigor, not cynicism and marketing.”
Although some bibliometric methods “are essential to go beyond local and anecdotal perceptions and to map comprehensively the state of research and identify trends at different levels (regional, national and global),” Gingras adds, “the proliferation of invalid indicators can only harm serious evaluations by peers, which are essential to the smooth running of any organization.”
And here is the heart of Gingras’s argument: that colleges and universities are often so eager to proclaim themselves “best in the world” -- or region, state, province, etc. -- that they don’t take to care to identify “precisely what ‘the best’ means, by whom it is defined and on what basis the measurement is made.” Put another way, he says, paraphrasing another researcher, if the metric is the answer, what is the question?
Without such information, Gingras warns, “the university captains who steer their vessels using bad compasses and ill-calibrated barometers risk sinking first into the storm.” The book doesn’t rule out the use of indicators to “measure” science output or quality, but Gingras says they must first be validated and then interpreted in context. ...
October 25, 2016 in Book Club, Legal Education, Scholarship | Permalink
| Comments (0)