Paul L. Caron

Tuesday, October 13, 2015

Using Publication Metrics To Evaluate Faculty Stifles Innovative Scholarship

MetricsInside Higher Ed, The Costs of Publish or Perish:

Shortly after being awarded the Nobel Prize in Physics in 2013, Peter Higgs, of Higgs boson fame, said he doubted he would have gotten a job, not to mention tenure, in today’s academic system. The professor emeritus at the University of Edinburgh said he simply wouldn’t have been “productive” enough, with academe’s premium on publication metrics. Conversely, said Higgs, working in today’s academic system probably wouldn’t have afforded him the opportunity to identify how subatomic material requires mass.

“It's difficult to imagine how I would ever have enough peace and quiet in the present sort of climate to do what I did in 1964,” he told The Guardian.

The statement resonated with many academic scientists running the funding-collaboration-publication treadmill. But while the negative consequences of the “publish or perish” paradigm, such as innovation costs and decreased attention to teaching and mentoring, are widely acknowledged, there’s been scant data to back them up. So a new study suggesting that publication pressures on scientists lead to more traditional, more likely to be published papers, at the expense of scientific breakthroughs, stands out [Tradition and Innovation in Scientists’ Research Strategies].

“Pursuing innovation is a gamble, without enough payoff, on average, to justify the risk,” the study says. “Nevertheless, science benefits when individuals overcome the dispositions that orient them toward established islands of knowledge … in the expanding ocean of possible topics.” ...

Jacob B. Foster, lead author and professor of sociology at the University of California at Los Angeles, and his co-authors created a database of more than 6.4 million biomedical and chemistry publications from 1934 to 2008. ... Foster and his co-authors, James Evans, an associate professor of sociology at the University of Chicago, and Andrey Rzhetsky, a professor of medicine and human genetics at Chicago, argue that researchers who focus on answering established questions are more likely to see their work published. But while researchers who pursue riskier academic work may not be published as frequently, if published, their work receives more citations. ...

The authors recommend various ways that colleges and universities can promote more innovation, such as not linking job security to productivity, in terms of easy metrics. They say that such a strategy, once proved successful at Bell Labs, where scientists could work on project for a year without being evaluated. ...

Academe should resist “the temptation to outsource judgment of quality to easily countable quantities,” he said. “Top universities emphasize that they are not interested in counting publications or citations -- that colleagues ‘read the work’ when evaluating a case.”

Legal Education | Permalink


Substitute Law professors for scientist and you have the explanation for why most law review articles are never cited for anything of consequence. Law profs cannot afford to take risks so they write what is marketable to 2 and 3d year students.

Posted by: Jeff Harrison | Oct 13, 2015 11:20:38 AM

In accounting we have a saying, "You get what you measure." If you measure papers, you get papers. If you measure innovation, you get innovation. So it pays to think through what you want before you decide how to measure.

Posted by: Dale Spradling | Oct 13, 2015 6:09:33 AM