Letter to State Bar of California (Aug. 25, 2017):
We write as deans of 19 ABA-accredited California law schools to provide comment on the 2017 Standard Setting Study and related options for the California Bar Examination cut score. [Fn: Signatories to this letter include the Deans of all but two ABA-accredited California law schools (Davis and Whittier).] ...
We will start with our conclusion. Having reviewed the Standard Setting Study, several critical evaluations of that Study, and having heard testimony at the public hearing, we continue to believe that our earlier view, expressed by 20 deans of ABA-accredited California law schools in February of this year, remains correct. California’s atypically high cut score is not justifiable and it has very high costs. We appreciate the Bar’s efforts to begin to study this set of issues, but unfortunately, given its serious flaws, the Standard Setting Study simply cannot provide a sound basis for setting a cut score. We believe that sound studies should be conducted to set a cut score, but properly doing those studies will take time — not merely a couple of months, which forces important decisions to be made in a somewhat scattershot and insufficiently careful way, but likely a year or more. The fact that this area is understudied in general makes the need for studies significant but also elevates the difficulty level of pursuing such studies in a methodologically sound manner.
In the interim, we believe that the California cut score should be set somewhere within the national average — the national average would be between 133 and 136 and, even if the Supreme Court wanted to choose a cut score on the higher end of the national range, that would be 137 to 139. We reach this conclusion for a variety of reasons.
First, the Standard Setting Study has serious flaws in both design and execution. To our knowledge, every independent expert who has reviewed the Standard Setting Study, including the two retained by the State Bar, has pointed out serious methodological flaws in the design and conduct of the study and in the presentation and analysis of the data. ... Several well-done studies — rather than one deeply flawed one — should properly inform the setting of the cut score, but the simple reality is that those studies will take considerable time to do well. ...
[W]e recognize that determining an appropriate cut score uninformed by any valid social science poses challenges. We appreciate that if, as we believe, the current standard setting study does not provide any genuinely legitimate basis for determining a cut score, one might therefore argue that there is no evidentiary basis for making an interim change, and the right answer would therefore be simply to wait several more years for additional, appropriately careful studies before taking any step toward change. However, we vehemently disagree with this position. Compared to bar-takers in every other state in the country (with the exception of Delaware, which has a total of just a few hundred bar takers per year) there is no question that a series of significant harms flow from California’s current atypically high cut score. These harms include the additional costs and fees borne by repeat-exam takers; the significant reduction in theracial and socioeconomic diversity of California’s lawyers; and the challenges faced by experienced lawyers who wish to relocate to California and — notwithstanding successful multi-year practice experience that strongly suggests that they meet a minimum competence standard — are more likely to fail than to pass the California Bar Exam.
Given our positions as deans, we wish to point out that the current cut score also hurts California ABA-accredited law schools in significant ways that we would hope would be relevant to the Bar and to the Court. As a consequence of the atypically high cut score, more students at California’s ABA-accredited law schools fail the bar the first time than similarly situated students elsewhere. In addition to the significant hardships for the students themselves, this means that fewer of our students can be employed as lawyers at graduation or at 10 months afterwards. In addition, if California maintains its current cut score and the ABA pursues the accreditation standard it currently has under consideration,3 several California schools will be disproportionately affected relative to comparable schools elsewhere. Moreover, every one of us — even those of us who come from schools viewed as among the best in the nation — sees, firsthand, the way that false negatives are not simply statistics. We witness the painful phenomenon of students we know to be competent to practice law being told otherwise simply because of California’s exceptionally high cut score. Finally, we know how many skills that go well beyond those tested on the bar exam assist students in achieving true professional competence.
We also want to highlight particularly critical consequences of setting the cut score too high: the effects on access to justice and the pass rates of under-represented minority populations. This is shown in Table 5 of the memo circulated by the Bar. That table indicates that a reduction of the cut score on the July 2016 exam from 1440 to the Bar’s recommended alternative of the interim cut score of 1414 would have resulted in increased passage rates of 12.5% for Blacks, 10.6% for Hispanics, and 8.6% for Asians, compared to 7.2% for Whites. In response to a request made to the State Bar, we understand that data will be made available next week on how these groups would have performed had the cut rate been set at 139 (as recommended by some commentators), 135 (the most common cut score) and 133 (New York’s cut score). We also refer to the information presented at the State Bar’s public hearing in San Francisco on August 15 by Dean Anthony Niedwiecki of Golden Gate University School of Law, and subsequently submitted as written public comment to the State Bar, about the decisions made both in New York and Illinois rejecting recommendations to set higher cut scores based on studies showing the adverse impact on minority applicants. We believe that the evident harm done to minority applicants by high cut scores should weigh heavily in the State Bar and Supreme Court’s policy decision, especially given the Study’s acknowledgement that there is no empirical evidence that the California’s high cut score has made our state’s attorneys more competent or resulted in fewer discipline cases than in other states.
The decision to employ a 1440 cut score in California back in 1985 simply did not have a careful, empirically grounded basis to support it. We readily acknowledge that even now, we do not yet have a careful, empirically grounded basis to support any given cut score — not 1330, not 1360, not 1390, not 1410, and not 1440. It would be a significant conceptual mistake to believe that in the absence of appropriate validity data, doing nothing is somehow more justified than a change. Rather, given that standard setting is a policy decision, we believe the right approach is to look at the known benefits and costs of the various possible cut scores, while also recognizing that we do not have all the information we would wish for to inform a choice.
Given this state of affairs — that the current cut score is not justifiable and causes harm and that the flawed Standard Setting Study does not provide a valid basis for setting the cut score — we continue to believe that the best option is to set an interim cut score that is selected from the range of scores that are closer to the national average. That average would be between 133 and 136 or, if the Bar is concerned to set the cut score on the high end, it could choose between 137 and 139. Ultimately setting the cut score is a policy judgment and, while awaiting the results of validly conducted studies that can inform that judgment, choosing a cut score more consistent with the wisdom of general practice is entirely defensible. It is perhaps worth noting that several other states — indeed two of California’s neighbors — have recently come to a similar conclusion. In Oregon, the only state apart from Delaware and California to have a cut score above 140, the Supreme Court adopted the recommendation of its Board of Bar Examiners and reduced that state’s cut score from 142 to 137. Nevada, one of four states at 140 (currently the highest cut score outside of California and Delaware), reduced its cut score to 138 as a result of a regular re-examination of the exam by its Board of Bar Examiners and the Supreme Court.
Prior TaxProf Blog coverage:
- July 2016 California Bar Exam (Nov. 22, 2016)
- More On The California Bar Exam (Dec. 5, 2016)
- Who Is To Blame For UC-Hastings 'Horrific' 51% Bar Pass Rate? (Dec. 7, 2016)
- Anderson On The California Bar Exam Carnage: Law Schools Need To 'Cull The Herd Of Bloated Tenured Faculties' (Dec. 8, 2016)
- July 2016 California Bar Exam Results: Nine Law Schools (Including UC-Hastings) Are At Risk Of Failing ABA's Proposed New Bar Passage Accreditation Standard (Dec. 13, 2016)
- Anderson: The Top Ten Myths About the Bar Exam (Dec. 15, 2016)
- California Law School Deans Blast State Bar's Historically Low Pass Rate On July 2016 Exam (Dec. 16, 2016)
- Muller: The Complete Collapse Of Bar Passage Rates In California (Dec. 16, 2016)
- Boise & Morriss: The Shameful Truth Is That Many Law Schools Have Admitted Students With Low LSAT Scores To Prop Up Tuition Revenue And Now Seek To Avoid Accountability For The Ensuing Poor Bar Passage Results(Dec. 19, 2016)
- Anderson: Deans, Denial, And The California Bar Exam (Dec. 20, 2016)
- Despite 'Horrific' 51% Bar Passage Rate, New UC-Hastings Dean Says School Is Poised To 'Catapult Into National Preeminence' Using NYU As A Model (Dec. 21, 2016)
- California Law School Bar Pass Rates Recalculated For New York: Stanford, UCB, USC > NYU; UCI, UCLA > Columbia; Chapman, Loyola, McGeorge, Pepperdine, Santa Clara, UCD, USD > Fordham (Dec. 22, 2016)
- More On The July 2016 California Bar Exam (Dec. 26, 2016)
- California’s New Bar Exam Format And ABA’s Proposed 75% Bar Passage Requirement Will Adversely Impact Diversity, Women, And Access To The Legal Profession (Jan. 30, 2017)
- Deans Of 20 Of California's 21 ABA-Accredited Law Schools Ask State Supreme Court To Lower Bar Exam Pass Score (Feb. 2, 2017)
- ABA 'Overwhelmingly' Rejects 75% Bar Passage Requirement (Feb. 7, 2017)
- California Law Deans Take Bar Exam Complaints To Lawmakers; State Bar Director Admits There Is 'No Good Answer' For High MBE Pass Score (Feb. 15, 2017)
- Beyond The Cut Score: Piercing The Veil Of The California Bar Exam's Validity (Mar. 2, 2017)
- Lawmakers Ask Calif. Chief Justice To Cut State Bar Exam Score (Mar. 9, 2017)
- UC-Hastings Dean: The California Bar Exam Flunks Too Many Law School Graduates (Mar. 27, 2017)
- Anderson, Merritt & Muller Debate: Is The Bar Exam Broken? (May 30, 2017)
- Deans Push To Lower California Bar Pass Score, But Lawyers With Lower Scores Are More Likely To Be Disciplined Or Disbarred (May 31, 2017)
- Anderson, Merritt & Muller On The Push To Lower California's Bar Pass Score (June 4, 2017)
- California Supreme Court Strips Authority Of Bar Examiners To Set Cut Score; Lower Cut Score May Apply Retroactively To July Test-Takers (July 12, 2017)
- NY Times: California Supreme Court Moves To Make Bar Exam Easier To Pass (July 14, 2017)
- Allard: The California Bar Exam Is The Tip Of The Lawyer Licensing Iceberg (July 20, 2017)
- California Bar Cut Score, Lawyer Salaries Are Second Highest In Country: Anti-Competitive Or Pro-Consumer Protectionism? (July 24, 2017)
- Anderson, Faigman Debate: Should California Lower Its Bar Exam Cut Score? (July 31, 2017)
- California Considers Lowering Passing Score On July Bar Exam (Aug. 1, 2017)
- More Law School Deans Call For Lowering California Bar Exam Cut Score (Aug. 21, 2017)