When Old Habits Die Hard: A Comment On Sander And Steinbuch's 'Mismatch And Bar Passage'

Following up on Friday's post, Richard Sander (UCLA), Law-School “Mismatch” Is Worse Than We Thought:  Sherod Thaxton (UCLA), When Old Habits Die Hard: A Comment on Sander and Steinbuch's “Mismatch and Bar Passage”:

Mismatch theory—which posits that race-conscious admissions policies harm racial minorities by admitting students into challenging schools where they cannot succeed—has figured prominently in the debate over affirmative action during the last half century. The recent challenges to Harvard University’s and the University of North Carolina’s consideration of race as a factor in admissions decisions have argued, inter alia, that universities “can eliminate this harmful mismatch and allow students to excel at schools for which they are most prepared by eliminating the use of racial preferences.” The persistence of the idea of mismatch in both policy discussions and litigation is puzzling given that the empirical evidence in favor of mismatch is not only sparse, but has been deemed unreliable by the vast majority of analysts who have rigorously investigated the matter. In an effort to preserve the legal and political relevance of mismatch in light of these mounting critiques by many of the most prominent statisticians and social scientists in academia, proponents of mismatch posit that the limitations of available data hamper the precise measurement of key variables and that better data would reveal stronger support for mismatch.

Richard Sander and Robert Steinbuch’s “Mismatch and Bar Passage: A School-Specific Analysis” presents a statistical analysis of the “law school mismatch hypothesis” in an effort to explain racial differences in the likelihood of passing the bar examination on the first attempt.

Sander and Steinbuch (S&S) claim their analysis is an improvement on prior studies of mismatch—which relied on data from the Law School Admission Council’s (LSAC) Bar Passage Study (BPS)—because their data are (a) more recent (the BPS is nearly 25 years old) and (b) permit the construction of a school-specific measure of mismatch that was not possible with the BPS. These improvements, notwithstanding, several conceptual and methodological mistakes previously identified in the empirical literature on mismatch analyzing the BPS data are present in S&S's study of school-specific data, thereby calling into question S&S’s conclusion that “the mismatch effects in their models can therefore account for the large disparities in bar passage across racial lines.” This paper identifies several problems with S&S’s empirical analysis and attempts to propose a sensible solution, or set of solutions, to each problem. Properly unpacking the organizational dynamics contributing to racial differences in law students' bar exam performance is key to assisting law schools in their continuing efforts to reassess their practices to ensure that all students can maximally contribute to and gain from from the academic experience.

(Hat Tip: Victoria Schwartz)