Law School Rankings: The Good News, The Bad News And The Ultimate Proof That It Is Flawed

National Law Journal Op-Ed:  Law School Rankings: The Good News, the Bad News and the Ultimate Proof That It Is Flawed, by Alan B. Morrison (Associate Dean, George Washington):

Finally, after two false starts, the U.S. News & World Report law school rankings are out, and one clear positive is that the authors have now told us more clearly what counts and for how much. The newest version has some sensible changes, as well as some problematic elements, but the bottom line is that the dramatic shifts in methodology for the overall rankings prove beyond a doubt that the premise of the endeavor—that these are objective measures—is fatally flawed. ...

These huge changes [in methodology] —in one year—are neither right nor wrong. What they show is how subjective the rankings are. In other words, they are entirely dependent on the personal whims of the U.S. News staff, who are not practicing lawyers, law professors, or law students. This arbitrariness applies not simply to the categories—why are these and only these categories relevant—but why are these the right percentages? The accompanying press release advises students that the rankings are only “one consideration” among many, including cost. But the overall message that the rankings convey is that students should go to the highest-ranked school because one size fits all—why else would they pick the title “Best Law Schools.”

But that cannot be right. Why would a student go to BYU or Alabama or Texas A&M instead of George Washington, unless they had no interest in being in Washington and wanted to practice in the states where those other schools are located? In other words, just because U.S. News “thinks” that the rankings are the best measure of excellence does not mean that students, either individually or even on average, have the same values as U.S. News. It is not that the information that U.S. News provides (or can be found if the rankings are broken down) is without value, it is the combination of these factors, with weights assigned by U.S. News and translated into a single number, that is the most basic flaw of its system. But until someone comes along with a better idea, U.S. News will continue to rule. ...

Ideally, U.S. News would eliminate its single number approach to rankings. However, that is what sells its product, and so that is not likely to change unless someone else comes along with a better alternative. In the interim, it could improve the quality assessment factors by imposing some limits on who can offer their peer reviews of other schools to those who actually have some basis for their conclusions. Finally, now that U.S. News has publicly spelled out its factors and their weights, it should include some further caveats and explanations upfront so that would-be law students would have a proper context for each of them.