Tobin: The Silly U.S. News Law School Rankings

TaxProf Blog op-ed:  The Silly U.S. News Law School Rankings, by Donald Tobin (Maryland; Google Scholar):

It is silly season again!  Last year, after U.S. News announced its new methodology regarding law school rankings, I highlighted how the changed methodology in the rankings, and the changes that U.S. News made to those changes, highlighted how silly these rankings are in the first place.  Schools, based on U.S. News waving a statistical wand, suddenly, at least according to U.S. News, became significantly better or significantly worse overnight [A Preliminary Analysis Of The New U.S. News Law School Rankings].  The new rankings shed light on this silliness once again, further highlighting the uselessness of the rankings.

First the good news!

One thing I predicted last year is that the heavy reliance on bar passage and employment data in the rankings would create tremendous volatility unless U.S. News moved to using some type of moving average. Sure enough, probably to stabilize the rankings, U.S. News has changed its methodology once again, and is now using two-year averages for bar passage and employment data. Good for those of you who like some level of stability in the rankings but surprising that wasn’t considered when they changed the methodology last year.

Second, the rankings continue to rely on data that is harder to manipulate.  Except for the peer assessment and the assessment by lawyers and judges, all the data is public.  This also means that you can ignore U.S. News and create your own rankings.  There is no reason to accept their judgment of the various weights to assign each indicator. You will see that for fun, I have done just that.

Third, the rankings continue to stress some indicators that are really important to students, like employment and bar passage. U.S. News claims that these indicators were used to show whether a school is “offering students a rigorous program of legal education.” This, however, is pure silliness.  Bar passage rates has almost nothing to do with a rigorous program of legal education.  (See the silliness section for more on this).

Now for some bad news, or at least some analysis of how the silliness in the rankings continues.

The U.S. News rankings remind me of Alice in Wonderland.  In Through the Looking Glass, Humpty Dumpty tells us that “When I use a word . . . it means just what I choose it to mean – neither more nor less.” U.S. News appears to be telling us the same thing.  They take mostly publicly available data, add it together, and magically create rankings that students, faculty, and administrators rely on.  They claim the rankings are valid because they say they are valid.  But what expertise do they possess to reach that conclusion?

Now let’s look a little bit at the rankings. Several schools experience rank changes of over fifteen places.  That is a change of over 15 places in one year! Absent a catastrophe or a huge influx in resources, do we really think a law school should change that many places in one year?  This massive variability is a sign that the rankings do not report what U.S. News asserts.  These schools’ program on legal education likely didn’t change much at all over the last year. At least two school increased by 28 places, and one decreased by 37. Yes, one school allegedly moved down 37 spots in one year.

In addition, as U.S. News moves through this fantasy land, they also assert a level of purity and sanctimoniousness that is simply not evident in the results.  Their methodology section is full of sanctimonious comments that assert they are doing this for the students.

Let’s see. Consider the metrics and weighting U.S. News used in creating their rankings.  In the methodology section, they note that employment is important to students, and there is rich employment data available.  In addition, they note employment is an “outcome” measure.  If it is an outcome measure, it must be GOOD!  Finally, “employment is important and that the U.S. News’ Best Law Schools rankings are designed to help.” Thus 60% of the rankings rely on their “successful placement of graduates.” The number is more like 33%, but who is counting.  (I guess I am). U.S. News gets to 60% by including bar passage data, which while important, is not the successful placement of graduates.

Student employment is now 33% of a law school ranking.

But please, please, please students, examine the data. Employment is important to students, so U.S. News says they are doing students a favor by including this rich information in the rankings.  But you know who did the students the favor, the ABA and NALP (National Association of Law Placement).  Both have rich data on employment, data that is far more helpful than the rankings.

For example, take an actual school somewhere in the United States but not near a major legal market.  That school dominates its local legal market but sends very few students outside of its market. The school has terrific employment outcomes but almost all, and I mean almost all, students are employed within the state the school is located.

That school will receive a high ranking from U.S. News because its students are getting employed.  That is great! But if you don’t want to work in state X, the school is probably not a good choice for you. If you do want to work in State X, it is an excellent choice.  There is nothing wrong with this paradigm.  Students often obtain jobs in a local market, and students often chose a school because of its local market.  But just because a school has a high employment rate doesn’t mean it is a great school, let alone that it is a great school for you. It just shows the institution’s strong presence in a particular legal market.

U.S. News’ second factor is bar passage rates, which are also an outcome measure. Bar passage is important because students want to pass the bar.  Of course, they do, and bar passage is extremely important. While bar passage is crucial, the metric U.S. News employs doesn’t reflect a school’s effectiveness in preparing students for the exam. Some top-ranked schools with high bar passage rates pride themselves on NOT teaching students to pass the bar. It is a super important measure for students, but a terrible measure for determining law school quality.  (Though I do think one of the benefits of US News increasing its reliance on bar passage is that schools will invest more in bar prep.) Statistical evidence indicates that success on the bar is far more correlated with entering statistics than the specific program of legal education.  We teach students to be great lawyers and thinkers.  To the extent law schools provide significant assistance to help students pass the bar, that is often individualized attention or specific bar preparation classes. Bar results are very rarely related to the program of legal education. In fact, in my view, one of the most important classes in law school, basic income tax, is not on the bar.  (Go Taxprofs!)

Here are the weights U.S. News uses to create its rankings.

Employment 10 months after graduation (33%), first-time bar passage (18%), ultimate bar passage (7%), peer assessment (12.5%), lawyers and judges (12.5%), LSAT (5%), GPA (4%), Acceptance rate (1%), Student faculty ratio (5%), librarian ratio (2%).

Is that the right measure?  Are they the right weights?  U.S. News is now using public data, so they only have so much to work with.  But should over 60% of a school’s quality be reliant on these types of “outcome” measures.

In my view, U.S. News’ weights and measures is broken.  I have decided to see what would happen if we shifted the weights to what I am calling “A Tax Prof's Rankings.” This weighting is no more, but also no less, valid than the weight U.S. News assigns to each indicator.  It has similar number of academic studies backing it up (none), and it similarly relies on a set of best research practices (cough!).  At least I can promise you that I picked the weights before I ran the numbers and have not altered the results to fit expectations.

My weights and reasons.

I assigned 10% weight to employment.  Employment is an important measure AND it is an outcome measure, but it is a really flawed measure.  One simply cannot capture the nuances in the data.  Ten percent seemed like the right percentage for important data but data that is hard to analyze.  In addition, the NALP and ABA data are available to students, so they have easy access to look at the data more closely.

I also assigned a weight of 15% to the various bar passage statistics used by U.S. News.  I similarly think this is important to students, but question how much an impact law schools have on bar passage, especially at the top-ranked law schools. I have seen some amazingly successful bar passage programs at law schools, but most of those schools are not the very top ranked-schools and bar passage is an essential component of their educational model. I assigned 14% to LSAT and GPA and 6% to the acceptance rate.

I assigned 25% to the peer assessment score and the assessment by lawyers and judges.  To some (including me), this may seem backward.  These reputation scores have built-in biases and create a huge historical drag on the rankings.  But these reputational scores are really the new information that U.S. News is providing to students.  The surveys may be flawed, but they are at least new information.  In addition, one of the questions that students want answered is what do others, including peers but most importantly employers, think about my school. The reputational scores thus answer those questions.

With these modifications, here is what A Tax Prof's Rankings of the top 175 law schools look like (based on U.S. News data):

A Tax Prof's Ranking	U.S. News	School Name	State
1	1	Stanford University	CA
2	1	Yale University	CT
3	4	Harvard University	MA
4	3	University of Chicago	IL
5	8	Columbia University	NY
6	9	New York University	NY
6	4	University of Virginia	VA
8	4	University of Pennsylvania (Carey)	PA
9	9	University of Michigan--Ann Arbor	MI
10	9	Northwestern University (Pritzker)	IL
11	4	Duke University	NC
12	12	University of California, Berkeley	CA
13	14	Cornell University	NY
13	14	Georgetown University	DC
15	13	University of California--Los Angeles	CA
16	16	University of Texas--Austin	TX
17	19	Vanderbilt University	TN
18	16	Washington University in St. Louis	MO
19	20	University of Notre Dame	IN
20	20	University of Southern California (Gould)	CA
20	24	Boston University	MA
21	16	University of Minnesota	MN
22	20	University of North Carolina--Chapel Hill	NC
23	36	University of Wisconsin--Madison	WI
24	20	University of Georgia	GA
25	28	Boston College	MA
26	41	George Washington University	DC
27	33	Fordham University	NY
28	36	William & Mary Law School	VA
29	28	University of Florida (Levin)	FL
30	33	University of Alabama	AL
31	42	Emory University	GA
32	26	Ohio State University (Moritz)	OH
33	42	University of California--Irvine	CA
34	33	Washington and Lee University	VA
35	28	University of Utah (Quinney)	UT
36	25	Wake Forest University	NC
37	26	Texas A&M University	TX
37	28	Brigham Young University (Clark)	UT
37	36	University of Iowa	IA
37	28	George Mason University (Scalia)	VA
41	36	Arizona State University (O'Connor)	AZ
42	36	University of Illinois Urbana-Champaign	IL
43	55	University of California--Davis	CA
44	48	University of Washington	WA
45	48	University of Colorado--Boulder	CO
46	42	Indiana University--Bloomington (Maurer)	IN
47	46	Baylor University	TX
48	48	Florida State University	FL
49	52	Pepperdine University (Caruso)	CA
50	55	University of Arizona (Rogers)	AZ
51	42	Southern Methodist University (Dedman)	TX
51	61	Yeshiva University (Cardozo)	NY
53	55	University of Maryland (Carey)	MD
53	48	Villanova University (Widger)	PA
53	54	Temple University (Beasley)	PA
53	55	University of Connecticut	CT
57	46	University of Kansas	KS
58	52	University of Tennessee--Knoxville	TN
59	61	Loyola Marymount University	CA
60	78	Tulane University	LA
61	68	Northeastern University	MA
62	82	University of California College of the Law, San Francisco	CA
63	66	University of Richmond	VA
64	68	University of San Diego	CA
65	98	American University (Washington)	DC
66	68	University of Houston Law Center	TX
66	82	University of Miami	FL
68	68	Marquette University	WI
69	89	Case Western Reserve University	OH
70	61	University of Missouri	MO
70	78	Loyola University Chicago	IL
72	66	University of South Carolina	SC
73	75	Pennsylvania State University Dickinson Law	PA
73	68	St. John's University	NY
73	55	Wayne State University	MI
76	68	Pennsylvania State University--University Park	PA
76	75	Georgia State University	GA
78	91	University of Pittsburgh	PA
79	61	University of Kentucky (Rosenberg)	KY
80	55	University of Oklahoma	OK
81	94	The Catholic University of America	DC
82	82	University of Nebraska--Lincoln	NE
83	89	University of Denver (Sturm)	CO
83	78	University of Cincinnati	OH
83	130	Howard University	DC
83	75	Drexel University (Kline)	PA
87	78	University of Nevada--Las Vegas (Boyd)	NV
87	61	Seton Hall University	NJ
89	103	Rutgers University	NJ
90	114	Brooklyn Law School	NY
91	82	University of Oregon	OR
91	82	Lewis & Clark College (Northwestern)	OR
93	108	Michigan State University	MI
94	108	University at Buffalo--SUNY	NY
95	108	Illinois Institute of Technology (Chicago-Kent)	IL
96	91	Louisiana State University--Baton Rouge (Hebert)	LA
97	94	Saint Louis University	MO
98	68	Florida International University	FL
99	94	University of New Mexico	NM
100	120	Syracuse University	NY
100	98	Indiana University--Indianapolis (McKinney)	IN
100	127	University of Hawaii--Manoa (Richardson)	HI
103	114	Seattle University	WA
103	120	University of Mississippi (Ole Miss) School of Law	MS
103	82	Drake University	IA
106	114	University of Arkansas--Fayetteville	AR
106	82	Texas Tech University	TX
108	130	Hofstra University (Deane)	NY
108	120	University of Maine School of Law	ME
110	134	DePaul University	IL
111	108	University of Dayton	OH
112	158	Santa Clara University	CA
113	103	Cleveland State University	OH
114	117	University of Missouri--Kansas City	MO
115	98	Stetson University	FL
116	94	Duquesne University (Kline)	PA
117	127	New York Law School	NY
118	103	Mercer University (George)	GA
119	98	University of St. Thomas	MN
120	108	Chapman University (Fowler)	CA
120	117	Albany Law School	NY
122	91	Belmont University	TN
122	103	Samford University (Cumberland)	AL
124	117	West Virginia University	WV
124	150	CUNY School of Law	NY
126	98	University of New Hampshire (Pierce)	NH
127	136	University of Louisville (Brandeis)	KY
128	130	Suffolk University	MA
129	120	Washburn University	KS
129	120	Gonzaga University	WA
131	103	University of Montana (Blewett)	MT
131	120	University of Tulsa	OK
133	145	Willamette University College of Law	OR
133	108	Regent University	VA
135	136	Pace University (Haub)	NY
136	130	Loyola University New Orleans	LA
137	153	Creighton University	NE
138	143	Quinnipiac University	CT
138	140	University of Baltimore	MD
140	143	University of Arkansas--Little Rock (Bowen)	AR
141	120	University of Wyoming	WY
141	127	University of South Dakota (Knudson)	SD
143	134	Campbell University (Wiggins)	NC
143	150	South Texas College of Law Houston	TX
143	140	University of Memphis (Humphreys)	TN
146	145	University of Idaho	ID
147	165	University of San Francisco	CA
147	136	University of Akron	OH
149	145	Southwestern Law School	CA
150	153	St. Mary's University	TX
151	159	University of the Pacific (McGeorge)	CA
152	136	University of Detroit Mercy	MI
153	148	University of Toledo	OH
153	157	Northern Illinois University	IL
155	161	University of Illinois--Chicago	IL
156	168	Vermont Law School	VT
157	148	Elon University	NC
158	161	University of Massachusetts--Dartmouth	MA
159	168	University of North Dakota	ND
160	150	Northern Kentucky University (Chase)	KY
161	153	Ohio Northern University (Pettit)	OH
162	164	Mitchell Hamline School of Law	MN
163	172	University of North Texas--Dallas	TX
163	140	Liberty University	VA
165	178-196	Florida A&M University	FL
166	172	Southern Illinois University--Carbondale	IL
167	159	New England Law Boston	MA
168	168	Touro University (Fuchsberg)	NY
169	172	Roger Williams University	RI
169	165	Widener University--Pennsylvania (Commonwealth)	PA
171	178-196	Western New England University	MA
171	165	Lincoln Memorial University (Duncan)	TN
171	178-196	Widener University--Delaware	DE
174	153	Oklahoma City University	OK
175	178-196	North Carolina Central University	NC

And here are the rankings if we only looked at peer reputation and the reputation by Lawyers and Judges (treating each as 50%).

Reputation	U.S. News	School Name	State
1	1	Stanford University	CA
2	4	Harvard University	MA
3	3	University of Chicago	IL
4	1	Yale University	CT
4	8	Columbia University	NY
6	9	New York University	NY
7	4	University of Virginia	VA
7	9	University of Michigan--Ann Arbor	MI
9	12	University of California, Berkeley	CA
10	4	University of Pennsylvania (Carey)	PA
11	9	Northwestern University (Pritzker)	IL
11	4	Duke University	NC
13	14	Cornell University	NY
13	14	Georgetown University	DC
15	13	University of California--Los Angeles	CA
16	16	University of Texas--Austin	TX
17	19	Vanderbilt University	TN
18	20	University of Notre Dame	IN
19	16	Washington University in St. Louis	MO
20	20	University of Southern California (Gould)	CA
21	20	University of North Carolina--Chapel Hill	NC
22	42	Emory University	GA
23	24	Boston University	MA
23	16	University of Minnesota	MN
25	41	George Washington University	DC
25	36	William & Mary Law School	VA
27	28	Boston College	MA
28	55	University of California--Davis	CA
29	33	Fordham University	NY
29	28	University of Florida (Levin)	FL
31	20	University of Georgia	GA
32	42	University of California--Irvine	CA
33	33	Washington and Lee University	VA
34	36	University of Wisconsin--Madison	WI
34	36	University of Iowa	IA
36	26	Ohio State University (Moritz)	OH
37	25	Wake Forest University	NC
38	36	University of Illinois Urbana-Champaign	IL
39	48	University of Washington	WA
39	42	Indiana University--Bloomington (Maurer)	IN
41	33	University of Alabama	AL
41	36	Arizona State University (O'Connor)	AZ
43	78	Tulane University	LA
44	48	University of Colorado--Boulder	CO
45	82	University of California College of the Law, San Francisco	CA
46	28	Brigham Young University (Clark)	UT
47	55	University of Arizona (Rogers)	AZ
48	52	Pepperdine University (Caruso)	CA
48	82	University of Miami	FL
50	48	Florida State University	FL
51	28	University of Utah (Quinney)	UT
51	55	University of Maryland (Carey)	MD
53	54	Temple University (Beasley)	PA
53	61	Loyola Marymount University	CA
53	98	American University (Washington)	DC
53	130	Howard University	DC
57	28	George Mason University (Scalia)	VA
57	48	Villanova University (Widger)	PA
57	46	University of Kansas	KS
60	26	Texas A&M University	TX
60	55	University of Connecticut	CT
60	46	Baylor University	TX
63	42	Southern Methodist University (Dedman)	TX
63	78	Loyola University Chicago	IL
65	52	University of Tennessee--Knoxville	TN
65	82	University of Oregon	OR
67	61	Yeshiva University (Cardozo)	NY
67	68	University of San Diego	CA
69	89	Case Western Reserve University	OH
70	68	University of Houston Law Center	TX
71	61	University of Kentucky (Rosenberg)	KY
71	55	University of Oklahoma	OK
71	103	Rutgers University	NJ
74	89	University of Denver (Sturm)	CO
75	108	Michigan State University	MI
76	68	Northeastern University	MA
76	61	University of Missouri	MO
76	66	University of South Carolina	SC
76	91	University of Pittsburgh	PA
76	114	Brooklyn Law School	NY
81	66	University of Richmond	VA
82	158	Santa Clara University	CA
83	61	Seton Hall University	NJ
84	75	Georgia State University	GA
85	75	Pennsylvania State University Dickinson Law	PA
85	120	Syracuse University	NY
87	82	Lewis & Clark College (Northwestern)	OR
88	68	St. John's University	NY
88	82	University of Nebraska--Lincoln	NE
88	108	Illinois Institute of Technology (Chicago-Kent)	IL
91	98	Indiana University--Indianapolis (McKinney)	IN
92	78	University of Nevada--Las Vegas (Boyd)	NV
93	68	Marquette University	WI
93	94	The Catholic University of America	DC
93	120	University of Mississippi (Ole Miss) School of Law	MS
96	68	Pennsylvania State University--University Park	PA
96	75	Drexel University (Kline)	PA
96	91	Louisiana State University--Baton Rouge (Hebert)	LA
99	114	University of Arkansas--Fayetteville	AR
100	134	DePaul University	IL
101	78	University of Cincinnati	OH
101	94	Saint Louis University	MO
103	127	University of Hawaii--Manoa (Richardson)	HI
104	120	Gonzaga University	WA
105	130	Hofstra University (Deane)	NY
105	136	University of Louisville (Brandeis)	KY
106	108	University at Buffalo--SUNY	NY
107	55	Wayne State University	MI
108	98	Stetson University	FL
108	130	Loyola University New Orleans	LA
110	94	University of New Mexico	NM
111	114	Seattle University	WA
112	82	Drake University	IA
112	153	Creighton University	NE
112	168	Vermont Law School	VT
115	120	University of Maine School of Law	ME
115	145	Willamette University College of Law	OR
115	161	University of Illinois--Chicago	IL
118	150	CUNY School of Law	NY
118	130	Suffolk University	MA
120	94	Duquesne University (Kline)	PA
120	127	New York Law School	NY
122	117	Albany Law School	NY
122	117	West Virginia University	WV
124	68	Florida International University	FL
125	165	University of San Francisco	CA
126	82	Texas Tech University	TX
126	98	University of New Hampshire (Pierce)	NH
126	145	University of Idaho	ID
129	117	University of Missouri--Kansas City	MO
130	159	University of the Pacific (McGeorge)	CA
131	136	Pace University (Haub)	NY
132	140	University of Baltimore	MD
133	103	University of Montana (Blewett)	MT
134	120	University of Tulsa	OK
135	103	Mercer University (George)	GA
136	120	Washburn University	KS
137	143	University of Arkansas--Little Rock (Bowen)	AR
138	103	Cleveland State University	OH
139	120	University of Wyoming	WY
140	108	University of Dayton	OH
140	143	Quinnipiac University	CT
140	140	University of Memphis (Humphreys)	TN
143	98	University of St. Thomas	MN
144	103	Samford University (Cumberland)	AL
145	127	University of South Dakota (Knudson)	SD
146	150	South Texas College of Law Houston	TX
147	178-196	Widener University--Delaware	DE
148	153	St. Mary's University	TX
149	161	University of Massachusetts--Dartmouth	MA
150	136	University of Akron	OH
151	145	Southwestern Law School	CA
152	148	University of Toledo	OH
153	168	University of North Dakota	ND
154	164	Mitchell Hamline School of Law	MN
155	91	Belmont University	TN
156	108	Chapman University (Fowler)	CA
157	148	Elon University	NC
158	136	University of Detroit Mercy	MI
159	165	Widener University--Pennsylvania (Commonwealth)	PA
160	157	Northern Illinois University	IL
160	178-196	Florida A&M University	FL
160	172	Roger Williams University	RI
163	134	Campbell University (Wiggins)	NC
164	150	Northern Kentucky University (Chase)	KY
165	172	Southern Illinois University--Carbondale	IL
165	178-196	North Carolina Central University	NC
165	178-196	Nova Southeastern University (Broad)	FL
168	108	Regent University	VA
169	168	Mississippi College	MS
170	153	Ohio Northern University (Pettit)	OH
170	172	University of North Texas--Dallas	TX
170	168	Touro University (Fuchsberg)	NY
173	178-196	California Western School of Law	CA
174	159	New England Law Boston	MA
174	178-196	Southern University Law Center	LA

Notice that these rankings probably more closely align with your expectations than the ones in the current rankings. Why? First, peer rankings are heavily influenced by previous U.S. News rankings that create a self-reinforcing cycle. Second, since both of these alternative rankings rely heavily on reputation and expectations are in a sense based on the schools’ reputational score, we would expect these rankings to align more with expectations. Does that make these rankings better?  NO. But they are just as valid as the ones published today.

Let’s take a look at how these alternative rankings play out with a few schools. (Note I have no association with these schools).  Howard, the pre-eminent historically Black law school, is ranked 130 in U.S. News’ Rankings. I don’t know anyone who thinks that ranking reflects the greatness of Howard Law. In the “A Tax Prof's Ranking,” Howard would be ranked 83, and in the modified ranking based on reputation, Howard would be ranked 53. This shows that while the rankings may be “objective” in that it uses numbers, the policy decisions made by U.S. News regarding the weight of the rankings and what measures to include significantly impact the result.  I think the alternative rankings better reflect Howard’s excellence.

Let’s take one more school. American University Washington College of Law is ranked 98 in U.S. News, is ranked 65 in “A Tax Prof's Ranking” and ranked 53 in the peer rankings. You decide, but in my view, the alternative rankings better reflect American’s excellence as well.  (Note: Almost 25 years ago, American hired me to be an adjunct professor.  I am very grateful that they helped launch my journey as a professor. I have not taught for them since 2000).

There are plenty of schools that do worse on these alternative rankings.  And if these rankings were THE rankings, people would argue that reputational scores don’t reflect that excellence of what is going on at the school. They would be correct.  My only response is neither do the factors used by U.S. News, especially with the amount of weight they use on their indicators.

I am not making my calculations public (though they are just statistical Z scores, which is the same method used by U.S. News and you can replicate it in excel).  I don’t want you to check my work, and I am sure based on the limited time I had to generate them that there are some errors. 😊 I am also not making any assertion of validity, just that they have as much validity as what was released today.  (Actually, I am not sure what data from U.S. News is proprietary, so I am not comfortable releasing the underlying data).  So please, take all my calculations with a grain of salt.

One last point, U.S. News appears to be throwing a little bit of a temper tantrum.  They indicate in the methodology that they are not counting reputational submissions by schools that have decided not to provide data to U.S. News.  You might say that this is fair game.  If you don’t provide us with the information we want, you don’t get to vote. But this decision has huge implications for the validity of the rankings.  Has U.S. News done any analysis regarding who the people are who it is not excluding and whether U.S. News’ statistical pool of respondents continues to make up a diverse group of respondents?  (Maybe they have, but they certainly haven’t addressed this publicly). Have they examined if there is a type of school that is more likely to avoid participation?  This tantrum likely skews the results of their survey.

This revenge by U.S. News against those who don’t participate is also inappropriate for an organization that claims its standing to do these rankings through journalism. Do only people who do what a journalist wants count?  Basically, U.S. News is saying that if you don’t give me the private information we want, you don’t get to vote. This doesn’t seem like sound journalism or like an institution interested in getting things right.

In addition, this decision to restrict voting is particularly concerning because U.S. News actually takes the proprietary data it seeks from the schools and then sells it back to schools. It does not need the data to do the rankings.  (I am not sure if U.S. News provides any of this extra data to students).

To prospective students, make analysis of the rankings your first step in being a lawyer who engages in critical analysis.  If you care about job placement, look at the ABA and NALP data.  Determine whether the outcomes are right for you based on your interests.  If you are concerned about bar passage, for example, maybe you are not a great test taker, look at schools that offer strong bar preparation programs. Those factors will be better markers of the school that is right for you.

And take a look at the reputational rankings. For all their faults, and they have a lot of faults, they do represent how some members of the profession might view your school.  They are much less effective if you are interested in a regional school that has a particularly strong reputation in a particular area. But U.S. News doesn’t help you figure that out. Research does.

Finally, pick a strong school where you feel comfortable. Pick a school that is right for you.  You will be happier and more successful, if you pick a school based on thoughtful analysis of the data, and then add in a good ole does of “this place feels right to me.” I wish you lots of luck in your journey.

Donald Tobin is the former dean at the University of Maryland Francis King Carey School of Law. He is now a professor of law at Maryland Carey Law and a member of LSAC Legal Education Consulting.