TaxProf Blog

Paul L. Caron
Dean





Monday, August 23, 2021

U.S. News Abandons Plan To Issue Citations Ranking Of Law Faculty

Hein US NewsIn February 2019, U.S. News announced it would publish a law faculty scholarly impact ranking in 2019 based on 5-year citation data from HeinOnline (FAQ; Updated FAQ; Additional Guidance). In November 2020, U.S. News announced it would be publishing the ranking in 2021. In June 2021, U.S. News abandoned its effort to rank law faculty scholarly impact:

Bloomberg Law op-ed:  Rankings Shift Could Force Big Changes at U.S. Law Schools, by Joshua Fischman (Virginia) & Michael A. Livermore (Virginia):

The U.S. News & World Report rankings are a powerful force in the world of law schools. Deans’ careers can rise or fall on their schools’ rankings, which affect everything from student recruitment to alumni giving to faculty retention. So when U.S. News announced in 2019 that it was considering creating a new ranking for law schools, heads turned across the legal academy.

Bloomberg Law received an email Aug. 19 from U.S. News stating that in June 2021 it decided that it would not proceed with its previously proposed law school scholarly impact ranking [more here]. However, we feel it is important to discuss concerns legal academics have about such proposals.

The crux of the U.S. News proposal was to develop a new measure of a law school’s prestige based on the “impact” of the scholarship produced by its faculty. This impact score would be calculated by counting the number of times a professor’s work was cited by other professors, and perhaps by courts.

Skeptics immediately raised objections. The most general challenge is that judgments of scholarly merit are inevitably subjective and cannot be quantified. Others expressed concern about biases against women, scholars of color, interdisciplinary scholars, and those in less-cited research areas.

But the citation-based rankings have supporters as well. They argue that the new approach will bring some needed objectivity to a system that is biased in favor of the old-school powerhouses and leaves little room for entrepreneurial upstarts to improve their standing.

As with many debates in academia, this might sound like a tempest in a teapot. But if U.S. News changes its ranking system, law schools will be pressured to alter how they recruit and promote faculty.

In particular, law schools will likely focus on professors with the most citations, instead of interdisciplinary credentials, peer-reviewed publications, or diversity. Ultimately, this affects who trains the next generation of lawyers and which ideas are circulated to courts and other legal decision-makers.

In a recent study of the law school lateral hiring market, we show that a focus on citations would result in dramatic changes in law school hiring. We find that the professors recruited into the top law schools are not necessarily the ones with the most citations. The citation counts of law professors who move to the most elite schools in the country—places like Harvard and Yale—are barely distinguishable from the rest of the field.

Joshua Fischman (Virginia) & Michael A. Livermore (Virginia), Empirically Validating Citation Metrics for Legal Scholars: A Market Approach:

Citation counts are a common quantitative metric used by researchers and analysts to assess scholarly output. When U.S. News & World Report announced in 2019 that it was developing a citation-based ranking for law schools, it brought new attention to debates about citations counts in legal scholarship. Supporters of citation metrics argue that they are superior to alternative measures of scholarly reputation, such as surveys. Critics are skeptical that citations serve as a meaningful proxy for scholarly quality and raise concerns that citation metrics could distort the incentives of law professors and faculties. We examine the validity of citation metrics by examining how well they correspond to a “market valuation” of legal scholars in lateral hiring.

We consider two outcomes: whether professors make lateral moves and the rank of the institution where they are hired. Using citation counts derived from the HeinOnline database, we find that citation metrics have a weak association with lateral outcomes. Metrics that mitigate the effect of the highly skewed distribution of citations, such as log citations and the h-index, perform slightly better. Article placements are stronger predictors of lateral outcomes. In particular, articles in top law reviews, top peer-reviewed journals, and online law reviews are all associated with moving to higher-ranked faculties, even after controlling for citations. This implies that citation rankings undervalue these kinds of publications. The divergence between citation counts and professors’ market valuation suggest that citation rankings could significantly distort publication and hiring in the legal academy.

USN1 USN2Prior TaxProf Blog coverage:

https://taxprof.typepad.com/taxprof_blog/2021/08/us-news-abandons-plans-to-issue-citations-ranking-of-law-faculty.html

Law School Rankings, Legal Ed Rankings, Legal Education | Permalink