TaxProf Blog

Editor: Paul L. Caron, Dean
Pepperdine University School of Law

Wednesday, May 10, 2017

Merritt:  The Bar Exam Is Broken

AALSAALS Faculty Perspectives: Validity, Competence, and the Bar Exam, by Deborah Jones Merritt (Ohio State):

The bar exam is broken: it tests too much and too little. On the one hand, the exam forces applicants to memorize hundreds of black-letter rules that they will never use in practice. On the other hand, the exam licenses lawyers who don’t know how to interview a client, compose an engagement letter, or negotiate with an adversary.

This flawed exam puts clients at risk. It also subjects applicants to an expensive, stressful process that does little to improve their professional competence. The mismatch between the exam and practice, finally, raises troubling questions about the exam’s disproportionate racial impact. How can we defend a racial disparity if our exam does not properly track the knowledge, skills, and judgment that new lawyers use in practice?

We can’t. In the language of psychometricians, our bar exam lacks “validity.” We haven’t shown that the exam measures the quality (minimal competence to practice law) that we want to measure. On the contrary, growing evidence suggests that our exam is invalid: the knowledge and skills tested by the exam vary too greatly from the ones clients require from their lawyers.

We cannot ignore the bar exam’s invalidity any longer. Every legal educator should care about this issue, no matter how many of her students pass or fail the exam. The bar exam defines the baseline of our profession. If the exam tests the wrong things, we have a professional obligation to change it. ...

Some legal educators have raised concerns about the bar exam because an increasing number of their students are failing. I am not part of that group. Law schools have an obligation to prepare students to satisfy our profession’s definition of minimum competence. We cannot change that definition simply because graduates find it harder to meet.

The problems with our bar exam, however, date back decades—encompassing years with high pass rates as well as low ones. An exam’s pass rate tells us little about the test’s validity. Rather than worry about pass rates, legal educators should focus on validity. Most important, we must develop a definition of minimum competence that tracks the real work of new lawyers.

This will not be an easy task for law schools. We will have to examine our assumptions about law practice and lawyering competence. If we want bar examiners to change their approaches, we may have to revise parts of our own educational model. The work, however, comes at a good time. Our profession is struggling to define itself in the face of changing technologies, business practices, and client needs. If we more fully identify our professional competencies, teach students to achieve those competencies, and develop a valid licensing system, we will help build a stronger profession.

Legal Education | Permalink


"the exam licenses lawyers who don’t know how to interview a client, compose an engagement letter, or negotiate with an adversary."

Law schools are uniquely unqualified to change this reality because for the last several decades, they have insisted that the more experience lawyers have, the less desirable they are as law professors (and for the most part are hired after 1-2 years at a corporate law firm where they maybe had two hours' worth of client contact in two years, and essentially just did legal research and memo writing all day). As a result, I would wager that the majority of prawfs are as hopeless at negotiation, client consultation, letter-drafting, and all the rest of it as the fresh bar exam passer is. And lest we forget, the biggest trend in law professor hiring today is to find JD-PhD holders who generally have never practiced... and if someone is a real superstar, even possessing a legal education is not a prerequisite for teaching law. The John Milton scholar Stanley Fish comes to mind.

Posted by: Unemployed Northeastern | May 10, 2017 11:58:49 AM

It is funny that legal academics are so far removed from practice that they can't even properly IDENTIFY what a true practice-ready competence is. Merritt regurgitates the usual ones: client interview, negotiation, engagement letter, etc. Can throw in the mythical "taking a deposition" into that lot as well. In fact, these are all skills that most law school graduates WILL have because they just require analytic thinking rather than specific knowledge.

Some true "practice ready" skills that nearly all law grads lack:

1. How to file a complaint (cover sheet, jurisdiction, filing fee)
2. Service of process (special process server or sheriff)
3. Filing return of service with Court;
4. Obtaining prejudgment attachment and perfecting it with registry
5. Notices of Appearance/Withdrawal
6. Timing of appeals
7. Engaging an expert (and knowing when its necessary)
8. Reading a docket
9. Verifying exhibits with Affidavits
10. COLLECTION A JUDGMENT (supplementary process, etc.)

Posted by: JM | May 10, 2017 12:30:30 PM

Everyone else said it all. When I was a 3L working on a pro bono clinic case, I asked our Civil Procedure professor how to file an amended complaint and she was absolutely clueless. That's when I learned how to use practice guides.

Posted by: Chris P. | May 10, 2017 5:34:24 PM

JM, just for the record, I teach two clinics: one in which the students and I prosecute misdemeanors and another (in a different county and semester) in which we defend indigents charged with misdemeanors. Our list of "practice ready" skills is a little different from your civil-side one, but it's analogous. The students learn how to issue subpoenas, provide discovery (or hound the prosecutor for it), read dockets, identify prior convictions, keep in touch with homeless clients, etc. This is one reason why clinical education is so useful: Even though the specifics differ among practice areas and courthouses, the students learn that all of these skills are not just technicalities; they're essential parts of representing clients. Mastering these skills in one area gives students confidence that they can do the same in other areas--and alerts them to the need to do so.

We could test these specifics on the bar exam, but I think that's less important than testing the other skills I identify. I agree with you that some of those other skills (interviewing, negotiating, fact gathering, etc.) are intellectual skills, and they draw upon “analytic thinking.” But these less-taught skills differ from the ones taught in most law school classes—the thinking is not the same. In fact, most law students are very poor interviewers because they want to be too analytic, zeroing in on the points they find relevant before a client or witness has told their story. Students and lawyers can learn to improve these thinking skills, just as they learn to improve the somewhat different thinking skill of analyzing a judicial opinion. The New Hampshire experiment described in my essay illustrates this. Thanks for engaging with the column!

Posted by: Deborah Merritt | May 10, 2017 5:52:28 PM

If people are flunking, it must be the test's fault

Posted by: mike livingston | May 10, 2017 9:34:30 PM

Again, I'm not trying to be a smart ass, but what is the purpose of the bar exam? Is to ensure only qualified law graduates are allowed to practice? Or is the purpose of the exam restraint of trade?

If the exam is meant to screen out unqualified applicants, has anyone defined what is a qualified applicant?

Posted by: Dale Spradling | May 11, 2017 5:53:59 AM

Mike, you've misunderstand Debby's article. She's been rather vocal about where the fault lies for falling bar passage rates, and who should be held accountable.

Posted by: Kyle McEntee | May 11, 2017 6:29:35 AM

When I sat for the California Bar exam just over a decade ago, I found the performance portion, in particular, to be a pretty good test of what I do now as a successful practicing attorney. Is the bar exam perfect? No. Are the sudden criticisms of the exam by those in legal academia/administration of late highly suspect in light of the current state of legal education? Hell yes!
All of a sudden, the exam is a problem when schools correspondingly lower admissions standards and admit large numbers of folks who wouldn't have gained admission to a single school a decade ago, and who, on paper, are clearly at high risk of not passing any bar exam anywhere. (I'm referring primarily to those Deans that signed the letter asking California to lower it's pass score, not Prof. Merritt in particular).

The bar exam is not the problem, folks. Law schools in their present form are the problem, and they are damaging MY PROFESSION...It is infuriating)!

Posted by: Anon | May 11, 2017 10:04:23 AM