TaxProf Blog

Editor: Paul L. Caron, Dean
Pepperdine University School of Law

Sunday, December 30, 2018

Study: Students' Entering Credentials And Law School Performance Account For Only 20%-50% Of Decline In California Bar Exam Pass Rates

California Bar ExamThe Recorder, California's Bar Exam: How Schools Fared and What Questions a New Analysis Didn't Answer:

A state bar-commissioned study concludes that California law school students’ undergraduate credentials and law school performance accounted for up to 50 percent of the decline in bar exam scores and passing rates between 2013 and 2017 [Performance Changes on the California Bar Examination: Part 2 (Dec. 20, 2018) (113 pages)].

But much of the slump in pass rates on the July tests during that four-year period cannot be accounted for even after culling data from 11 participating American Bar Association-approved law schools, according to the report, dated Dec. 20 and conducted by Research Solutions Group.

The 113-page report is unlikely to offer much guidance for law school deans, bar officials, students and lawmakers seeking answers for the decade-long decline in California bar exam pass rates, which plummeted to 40.7 percent—a 67-year low—for the July 2018 sitting. ...

The Research Solutions Group report found that students’ final law school grade-point averages had the strongest predictive relationship to their success on the bar exam, followed by their first-year law school GPAs, their Law School Admission Test scores and their undergraduate GPA. Mean LSAT scores of students taking the bar exam edged down from 159.4 in 2013 to 157.1 in 2017.

The report also noted that percentage of women and ethnic minorities taking the bar exam increased by six percentage points each during that four-year period. The average age of test-takers also rose from 28.9 in 2013 to 2017.

Older students logged slightly lower scores on the written portion of the exam, reducing their chances of passing by a small amount, the report found. The increased number of women taking the test had no correlation to success rates and “this study reconfirmed that racial/ethnic minorities with equivalent credentials to whites will tend to earn the same scores on the [bar exam] and have the same probability of passing,” the report concluded.

State Bar of California Press Release, State Bar of California Releases Report Exploring Changes in Student Performance on the California Bar Exam:

Today the State Bar released the fourth report in a series of studies designed to examine the decline in pass rates on the California Bar Exam. The report, Performance Changes on the California Bar Examination, uses detailed data on over 7,000 students who took the Bar Exam in July 2013, 2016, and 2017. Between 2013 and 2016, the pass rate on the exam fell from 56 percent to 43 percent. In 2017, the pass rate rebounded slightly, to 50 percent; it declined again in July 2018 to 40.7 percent.

This report is unique in that it utilized individual data on student characteristics to explore the causes of, or contributors to, the decline in the pass rate on the California Bar Exam. Individual data included information on students’ undergraduate grade point average (GPA), undergraduate major, LSAT score, final law school GPA, course work in law school, and basic demographic information. The study examined these characteristics in relation to Bar Exam performance—both pass/fail outcomes and scores—and found that:

  • Changes over time in the characteristics of exam takers accounted for only some of the decline in Bar Exam performance during the study period.  The study found that these changes accounted for between 20 and 50 percent of the decline in Bar Exam performance during the study period, depending on the performance metric chosen and year compared;
  • The two student characteristics that most strongly predicted performance on the California Bar Exam were a student’s GPA in the final year of law school and a student’s LSAT scores;
  • Changes over time in entering credentials (undergraduate GPA and LSAT) and law school credentials (final law school cumulative GPA) contributed roughly equally to that portion of the decline in Bar Exam performance attributable to changes in student characteristics during the study period;
  • The proportion of test takers who were ethnic minorities or female grew slightly over the study period, but the study found no correlation between these demographic characteristics and pass/fail outcomes, among students with similar abilities; and
  • Ultimately, the study was unable to account for a substantial amount of the decline in pass rates, concluding that other unexamined factors have contributed to the decade-long decrease in Bar Exam performance.

The State Bar undertook this and the other studies in response to the California Supreme Court’s direction to conduct a “thorough and expedited” study of the California Bar Examination. In December 2017, the State Bar completed a report to the Court and the Legislature on the other three studies.  

The State Bar also recently launched a year-long study of practicing attorneys to document the work that lawyers currently do and ensure that the Bar Exam aligns with the modern practice of law.

State Bar of California, Performance Changes on the California Bar Examination: Part 2 (Dec. 20, 2018) (133 pages)

New Insights from a Collaborative Study with California Law Schools

Entering law school credentials, law school experiences and grade point averages of 7,563 students from eleven California ABA-approved law schools were linked to their results on the 2013, 2016 and 2017 bar examinations to determine their interrelationships and potential impact on the continuing decrease in examination scores and passage rates

Cal Bar Chart

Cal Bar Chart 2

Cal Bar Chart 3 Cal Bar Chart 4

Cal Bar Chart 5 Cal Bar Chart 6

UpdateJuly 2018 California Bar Exam Results

https://taxprof.typepad.com/taxprof_blog/2018/12/study-students-entering-credentials-and-law-school-performance-accounts-for-50-of-decline-in-califor.html

Legal Education | Permalink

Comments

Ok, then the decline is attributable to poor teaching, or over-emphasis on non-essential material. I have to assume that if the test itself got harder, law school administrators would be screaming this from the hills.

Posted by: JM | Dec 31, 2018 7:20:11 AM

“this study reconfirmed that racial/ethnic minorities with equivalent credentials to whites will tend to earn the same scores on the [bar exam] and have the same probability of passing,” the report concluded.

Aye, there's the rub. "Equivalent credentials" is a hidden bugaboo. Not in explaining the lower rate of passage over the years, but to point out that equivalent credentials means a lot.

Posted by: Chris Pastel | Dec 31, 2018 5:47:44 PM

I blame the legalization of marihuana.

Posted by: Juan Pelota | Dec 31, 2018 6:19:58 PM

Much of the felt necessity to pass the exam has been gradually draining out of the economic circumstances of many, likely most, law school students: young people living at home, alternative employment availability, few starting families young (no need to support anyone else from lawyer earnings), and the resulting lack of felt urgency to study hard and pass …. would seem to explain a lot of it.

Posted by: Boyd J Ready | Dec 31, 2018 6:34:31 PM

"The proportion of test takers who were ethnic minorities or female grew slightly over the study period, but the study found no correlation between these demographic characteristics and pass/fail outcomes, among students with similar abilities; and"

Hmmmm. This appears to be designed to mislead. Note the last part: "with similar abilities". What if some groups are being admitted with lower abilities?

Posted by: James | Jan 1, 2019 9:43:32 AM

1. "Mean LSAT scores of students taking the bar exam edged down from 159.4 in 2013 to 157.1 in 2017."

"Edged down: is a pretty weaselly way to describe what is, in fact, nearly a 10 percentile point drop. A 159.4 is nearly the 80th percentile among test-takers; a 157.1 is about the 70th percentile. It's hardly meaningless.


2. From very lightly skimming the report (hey, I spent some of my magic million law dollars in Ibiza over New Years, as do we all), all 11 participating law schools in the study are in California. OK, so what percentage of CA test-takers are from CA-sited, ABA approved law schools? Let's not forget that a very large number of out-of-state law school grads take the CA Bar, and of course the state has like dozens of CALS approved law schools and distance learning options and all the rest of it. A bit of googling isn't revealing a lot in terms of overall numbers, but if it turns out that only, say, 40% or even 50% of CA bar exam takers are from ABA-approved law schools in California, it diminish the importance of the study's conclusions.

Posted by: Unemployed Northeastern | Jan 1, 2019 10:38:51 AM