AALS Faculty Perspectives: Validity, Competence, and the Bar Exam, by Deborah Jones Merritt (Ohio State):
The bar exam is broken: it tests too much and too little. On the one hand, the exam forces applicants to memorize hundreds of black-letter rules that they will never use in practice. On the other hand, the exam licenses lawyers who don’t know how to interview a client, compose an engagement letter, or negotiate with an adversary.
This flawed exam puts clients at risk. It also subjects applicants to an expensive, stressful process that does little to improve their professional competence. The mismatch between the exam and practice, finally, raises troubling questions about the exam’s disproportionate racial impact. How can we defend a racial disparity if our exam does not properly track the knowledge, skills, and judgment that new lawyers use in practice?
We can’t. In the language of psychometricians, our bar exam lacks “validity.” We haven’t shown that the exam measures the quality (minimal competence to practice law) that we want to measure. On the contrary, growing evidence suggests that our exam is invalid: the knowledge and skills tested by the exam vary too greatly from the ones clients require from their lawyers.
We cannot ignore the bar exam’s invalidity any longer. Every legal educator should care about this issue, no matter how many of her students pass or fail the exam. The bar exam defines the baseline of our profession. If the exam tests the wrong things, we have a professional obligation to change it. ...
Some legal educators have raised concerns about the bar exam because an increasing number of their students are failing. I am not part of that group. Law schools have an obligation to prepare students to satisfy our profession’s definition of minimum competence. We cannot change that definition simply because graduates find it harder to meet.
The problems with our bar exam, however, date back decades—encompassing years with high pass rates as well as low ones. An exam’s pass rate tells us little about the test’s validity. Rather than worry about pass rates, legal educators should focus on validity. Most important, we must develop a definition of minimum competence that tracks the real work of new lawyers.
This will not be an easy task for law schools. We will have to examine our assumptions about law practice and lawyering competence. If we want bar examiners to change their approaches, we may have to revise parts of our own educational model. The work, however, comes at a good time. Our profession is struggling to define itself in the face of changing technologies, business practices, and client needs. If we more fully identify our professional competencies, teach students to achieve those competencies, and develop a valid licensing system, we will help build a stronger profession.