Paul L. Caron
Dean



Tuesday, July 14, 2015

LSAT Is Poor Predictor Of Law School Grades: 6 LSAT Points = 0.1 LGPA

LSAT (2015)Alexia Brunet Marks (Colorado) & Scott A. Moss (Colorado), What Makes a Law Student Succeed or Fail? A Longitudinal Study Correlating Law Student Applicant Data and Law School Outcomes:

Despite the rise of "big data" empiricism, law school admission remains heavily impressionistic; admission decisions based on anecdotes about recent students, idiosyncratic preferences for certain majors or jobs, or mainly the Law School Admission Test (LSAT). Yet no predictors are well-validated; studies of the LSAT or other factors fail to control for college quality, major, work experience, etc. The lack of evidence of what actually predicts law school success is especially surprising after the 2010s downturn left schools competing for fewer applicants and left potential students less sure of law school as a path to future success. We aim to fill this gap with a two-school, 1400-student, 2005-2012 longitudinal study. After coding non-digitized applicant data, we used multivariate regression analysis to predict law school grades ("LGPA") from many variables: LSAT; college grades ("UGPA"), quality, and major; UGPA trajectory; employment duration and type (legal, scientific, military, teaching, etc.); college leadership; prior graduate degree; criminal or discipline record; and variable interactions (e.g., high-LSAT/low-UGPA or vice-versa).

Our results include not only new findings about how to balance LSAT and UGPA, but the first findings that college quality, major, work experience, and other traits are significant predictors: (1) controlling for other variables, LSAT predicts more weakly, and UGPA more powerfully, than commonly assumed – and a high-LSAT/low-UGPA profile may predict worse than the opposite; (2) a STEM (science, technology, engineering, math) or EAF (economics, accounting, finance) major is a significant plus, akin to 3½-4 extra LSAT points; (3) several years' work experience is a significant plus, with teaching especially positive and military the weakest; (4) a criminal or disciplinary record is a significant minus, akin to 7½ fewer LSAT points; and (5) long-noted gender disparities seem to have abated, but racial disparities persist. Some predictors were interestingly nonlinear: college quality has decreasing returns; UGPA has increasing returns; a rising UGPA is a plus only for law students right out of college; and 4-9 years of work is a "sweet spot," with neither 1-3 or 10 years' work experience significant. Some, such as those with military or science work, have high LGPA variance, indicating a mix of high and low performers requiring close scrutiny. Many traditionally valued traits had no predictive value: typical pre-law majors (political science, history, etc.); legal or public sector work; or college leadership.

These findings can help identify who can outperform overvalued predictors like the LSAT. A key caveat is that statistical models cannot capture certain difficult-to-code key traits: some who project to have weak grades retain appealing lawyering or leadership potential; and many will over- or under-perform any projection. Thus, admissions will always be both art and science – but perhaps with a bit more science.

Wall Street Journal Law Blog, New Study Tries to Predict Law School Grades:

Law school admission test scores are an “overvalued predictor” of law school grades,according to a new study. ...

[T]he magnitude of the predictive power of LSAT is modest compared to how heavily schools weight LSAT scores. A 6-point LSAT difference is enough to make a dispositive difference in where one attends law school and whether one receives a six-figure scholarship – but even that large an LSAT gap really predicts only a modest 0.1 difference in LGPA…

[C]hanges in LSAT do not appear to have increasing or decreasing returns; an X-point difference between a low and very low LSAT predicts the same as an X-point difference between a high and very high LSAT. Thus, contrary to some common assumptions, a “cutoff” driven by fear of an especially low LSAT is unsound: the difference between a 147 and a 152 is the same as the difference between a 157 and a 162.

https://taxprof.typepad.com/taxprof_blog/2015/07/a-longitudinal-study-correlating-law-student-applicant-data-and-law-school-outcomes.html

Legal Education, Scholarship | Permalink

Comments

There has to be some huge methodological error here. My first concern is that all comparisons would have to be between students at the same school, since, of course, a GPA of 3.4 at Cooley is quite different than at Columbia. I doubt that occurred since they are talking about GPA differences with a 6 point LSAT spread, and that type of spread is rare in the modern admissions regime, where the 25-75% numbers are typically 3-4 points apart.

Posted by: JM | Jul 14, 2015 4:26:35 AM

It’s ironic, given all the ligation over affirmative action, that law schools are allowed to use the LSAT as the primary variable for admissions, e.g., 75% of Stanford admits scored 172 or better; or said another way, 75% of Stanford admits are 1%’ers. This is doubly ironic because more and more studies have criticized the validity of standardized tests such as the LSAT. But law schools get away with this, while everybody else has to rotate in the wind.

Posted by: Dale Spradling | Jul 14, 2015 5:36:43 AM

Outside the top 5-10 law schools, the only thing that matters to admissions offices is that they enroll a class that gives them the biggest USNWR rankings boost. The admissions/enrollment process is not about getting the people most likely to succeed in law school. It is about maximizing the median GPA and LSAT of each incoming class. That is all that matters.

Take a look at UVA's admissions chart - (http://uva.lawschoolnumbers.com/stats/1415). A students whose LSAT and GPA both drop a fraction below UVA's targets is automatically rejected. Even before this study came out did anyone really think that there was a big difference between a 167/3.75 and a 168/3.80?

Posted by: Nathan A. | Jul 14, 2015 10:39:31 AM

@JM: Sadly I think you are correct, it looks like this study equates a 3.4 at Cooley with a 3.4 at Harvard. To make matters even worse, grading curves vary widely amongst law schools; a 3.15 is probably in the top 10% at a school with a 'C' average, but in the bottom half of the class at a school with a 'B+' average.

Posted by: Lonnie | Jul 14, 2015 12:33:16 PM

Where are you seeing that? As far as I can tell, it's a study at two schools, Case Western and the University of Colorado. Whether results obtained at those schools would be the same as those obtained at other schools is a different question, but the study does not appear to equate a 3.4 at Cooley with a 3.4 at Harvard.

Posted by: Chuck | Jul 14, 2015 9:01:03 PM

As another new academic year looms, I've been reflecting on my recent 30th (!) reunion at Harvard Law School. Getting in touch with old classmates again reminded me of many good times, but also a lot of stress. I'm curious about how today's situation at various law schools compares to what I experienced at HLS back in the 1980's.


For the last 4 years or so, I've been teaching undergraduates after 12 years of full-time law school teaching. As a result, I'm not completely current on the latest trends in law school pedagogy and culture. I would welcome any news, even anecdotal, pertaining to whether today's students are subjected to the same intense, competitive, highly individualistic approach to legal education that I "enjoyed" back in the day.


When I last was on a law school faculty, there were very few signs that cooperation was encouraged rather than competition, at least at the institutions with which I was most familiar. What are your views of the current situation, at law schools of whatever size, profit/non structure, or prestige level? Is it still mostly recognizable as a vestige from The Paper Chase paradigm, or are we in a new and more interdependent era?

Posted by: John Charles Kunich | Jul 15, 2015 10:42:11 AM

Without a measure for socioeconomic class -- parents' education level, profession, etc, -- this study, while sure to make for good had wringing, tells us nothing useful with regard to addressing underlying causes.

Posted by: Jeffrey Harrison | Jul 19, 2015 8:23:00 AM

When I was in law school many of the grades were given for subjective reasons. I tended to do very well in classes on financial and tax law where there was an objectively right answer in the test, and less well in those classes where the test was quite subjective, and the professor didn't like my politics.
My LSAT score? 169.

Posted by: Isab | Jul 19, 2015 8:56:24 AM

I'm curious at how at Professor Kunich's 30th Harvard Law School reunion how he defends his schilling for the predatory garbage law schools that he does in the name of "diversity" and "accessibility" and all his other long-winded klaptrap that he spouts to keep Ivy League educated lawyers employed teaching at schools that they would never let their kids attend.

Posted by: Cent Rieker | Jul 19, 2015 2:22:35 PM