Monday, January 13, 2020
Following up on my previous coverage (links below): Karen Sloan (Law.com), Dispatches from the Association of American Law Schools' Annual Meeting:
Rankings, Rankings, and More Rankings
In case that headline didn’t tip you off, rankings were a big deal at this year’s annual meeting. More specifically, U.S. News & World Report’s introduction of a “scholarly impact” ranking of faculties appears to have more than a few people in a tizzy. That ranking was a focus of no less than three separate panels over the weekend, two of which featured Bob Morse, U.S. News’ chief data strategist and the person responsible for the law school rankings. Legal academics love to hate on Morse and the U.S. News rankings, primarily for what they see as the outsized influence of the rankings on prospective students and the pernicious effects of playing the rankings game. So Morse is used to facing the slings and arrows of law professors, and frankly, he got away from this year’s AALS relatively unscathed. ...
Here are some of the big concerns raised by academics about the project:
- By using HeinOnline, the ranking will include only work cited in legal journals, and will exclude interdisciplinary work from law professors that appears in non-legal publications at a time when interdisciplinary research is a growing focus.
- The scholarship of clinicians, legal writing faculty and others who aren’t tenured on their campuses will be excluded from the analysis. (Most law schools don’t have tenure for those categories, but some do.)
- Counting citations hurts junior scholars who haven’t had as many years as older colleagues to publish, and certain areas of scholarship, such as tax law, are cited at much lower rates than popular areas such as constitutional law.
- Research shows that women and minorities, on average, are cited at lower rates than white men.
- Students don’t actually care how often a law faculty is cited. They just want good teachers.
The real reason for the mounting anxiety over the scholarly impact ranking, from what I gathered in many hallway discussions with deans and others, is that U.S. News, after a trial period, will make the scholarly impact rankings part of the score a school gets in the overall ranking—which is the number that affects applicants and can get deans fired. If that happens, schools will feel pressure to maximize their scholarly impact scores. That could mean a greater focus on hiring professors with high citation counts, which in turn hurts women and minority candidates; junior scholars; those whose work is largely in interdisciplinary fields; and those who work in lower citation areas of the law. Extending that further, it also disincentivizes law schools from extending tenure to clinicians and legal writing instructors given that they typically publish less than doctrinal faculty.
I also think there is the possibility that the introduction of this new scholarly impact ranking will influence the reputation survey responses that compose 40% of each school’s score in the overall ranking.
Prior TaxProf Blog coverage:
- U.S. News To Publish Law Faculty Scholarly Impact Ranking Based On 2014-2018 Citations (Feb. 13, 2019)
- U.S. News FAQ: Law School Scholarly Impact Rankings (Feb. 14, 2019)
- More Coverage Of The U.S. News Law Faculty Scholarly Impact Rankings (Feb. 15, 2019)
- Robert Anderson (Pepperdine), Some Contrarian Thoughts On The U.S. News Faculty Scholarly Impact Rankings (Feb. 18, 2019)
- Law Prof Commentary On The U.S. News Faculty Scholarly Impact Rankings (Feb. 19, 2019)
- U.S. News Updates FAQ On Law School Scholarly Impact Rankings To Address Inclusion Of Non-Doctrinal Faculty (Feb. 21, 2019)
- Joe Lawprofblawg (Anonymous Professor, Top 100 Law School) & Darren Bush (Houston), A Hilarious (In The Footnotes) Yet Serious (In The Text) Discussion Of Law Reviews And Law Professors (Feb. 26, 2019)
- U.S. News Updates FAQ On Law School Scholarly Impact Rankings To Address Inclusion Of Non-Doctrinal Faculty (Feb. 27, 2019)
- Derek Muller (Pepperdine), Gaming The New U.S. News Citation Rankings (Mar. 6, 2019)
- Jeff Sovern (St. John's), How The U.S. News Scholarly Impact Rankings Could Hurt Niche Subjects (March 11, 2019)
- Ted Sichelman (San Diego), A Defense And Explanation Of The U.S. News 'Citation' Ranking (March 20, 2019)
- U.S. News Offers More Guidance, Meeting With Law School Deans On New Scholarly Impact Rankings (May 2, 2019)
- The U.S. News Citation Ranking Is A 'Rigged Metrics Game' That 'Imperils Legal Academia' (Oct. 10, 2019)
- Changes To U.S. News Law School Rankings Met With Skepticism (Jan. 9, 2020)