LSAT Is Poor Predictor Of Law School Grades: 6 LSAT Points = 0.1 LGPA

Alexia Brunet Marks (Colorado) & Scott A. Moss (Colorado), What Predicts Law Student Success? A Longitudinal Study Correlating Law Student Applicant Data and Law School Outcomes, 13 J. Empirical Legal Stud. 205 (2016):

Despite the rise of "big data" empiricism, law school admission remains heavily impressionistic; admission decisions based on anecdotes about recent students, idiosyncratic preferences for certain majors or jobs, or mainly the Law School Admission Test (LSAT). Yet no predictors are well-validated; studies of the LSAT or other factors fail to control for college quality, major, work experience, etc. The lack of evidence of what actually predicts law school success is especially surprising after the 2010s downturn left schools competing for fewer applicants and left potential students less sure of law school as a path to future success. We aim to fill this gap with a two-school, 1400-student, 2005-2012 longitudinal study. After coding non-digitized applicant data, we used multivariate regression analysis to predict law school grades ("LGPA") from many variables: LSAT; college grades ("UGPA"), quality, and major; UGPA trajectory; employment duration and type (legal, scientific, military, teaching, etc.); college leadership; prior graduate degree; criminal or discipline record; and variable interactions (e.g., high-LSAT/low-UGPA or vice-versa).

Our results include not only new findings about how to balance LSAT and UGPA, but the first findings that college quality, major, work experience, and other traits are significant predictors: (1) controlling for other variables, LSAT predicts more weakly, and UGPA more powerfully, than commonly assumed – and a high-LSAT/low-UGPA profile may predict worse than the opposite; (2) a STEM (science, technology, engineering, math) or EAF (economics, accounting, finance) major is a significant plus, akin to 3½-4 extra LSAT points; (3) several years' work experience is a significant plus, with teaching especially positive and military the weakest; (4) a criminal or disciplinary record is a significant minus, akin to 7½ fewer LSAT points; and (5) long-noted gender disparities seem to have abated, but racial disparities persist. Some predictors were interestingly nonlinear: college quality has decreasing returns; UGPA has increasing returns; a rising UGPA is a plus only for law students right out of college; and 4-9 years of work is a "sweet spot," with neither 1-3 or 10 years' work experience significant. Some, such as those with military or science work, have high LGPA variance, indicating a mix of high and low performers requiring close scrutiny. Many traditionally valued traits had no predictive value: typical pre-law majors (political science, history, etc.); legal or public sector work; or college leadership.

These findings can help identify who can outperform overvalued predictors like the LSAT. A key caveat is that statistical models cannot capture certain difficult-to-code key traits: some who project to have weak grades retain appealing lawyering or leadership potential; and many will over- or under-perform any projection. Thus, admissions will always be both art and science – but perhaps with a bit more science.

Wall Street Journal Law Blog, New Study Tries to Predict Law School Grades:

Law school admission test scores are an “overvalued predictor” of law school grades,according to a new study. ...

[T]he magnitude of the predictive power of LSAT is modest compared to how heavily schools weight LSAT scores. A 6-point LSAT difference is enough to make a dispositive difference in where one attends law school and whether one receives a six-figure scholarship – but even that large an LSAT gap really predicts only a modest 0.1 difference in LGPA…

[C]hanges in LSAT do not appear to have increasing or decreasing returns; an X-point difference between a low and very low LSAT predicts the same as an X-point difference between a high and very high LSAT. Thus, contrary to some common assumptions, a “cutoff” driven by fear of an especially low LSAT is unsound: the difference between a 147 and a 152 is the same as the difference between a 157 and a 162.