Paul L. Caron

Monday, September 2, 2019

The Future Of The Bar Exam

Phase 1

Kyle McEntee (Law School Transparency), The Future Of The Bar Exam:

The National Conference of Bar Examiners (NCBE) released its Testing Task Force’s Phase 1 report this week. The report summarizes what people throughout the legal profession think about the current licensing process and exam. While readers will not be surprised by the general sentiments — people are generally dissatisfied with the current bar exam — they may be surprised that the NCBE published such forceful critiques. ...

Phase I involved listening sessions with lawyers, judges, examiners, and educators. Phase II involves a practice analysis to “gather current, empirical data on the knowledge, skills, abilities, other characteristics, and technologies that newly licensed lawyers use to accomplish the job tasks they perform.” Phase III will involve turning the results of Phase I and II into exam design recommendations by the end of 2020. ...

That brings us to the Phase I report and some common threads throughout all the listening sessions.

  • The exam tests both too much and too little. That is, it should emphasize more lawyering skills and less subject matter knowledge.
  • The exam should utilize more writing, less multiple-choice, and additional methods like simulation.
  • Jurisdictions should consider breaking the exam into different parts over more than just a few days. ...

We operate in a world of standardized tests for some good reasons, and the bar exam is no exception. Not only do state regulators expect the exam to be valid, but also to be reliable and thus able to yield stable scores over time. Often, however, test-makers sacrifice validity for reliability because the principal means for achieving reliability is through multiple-choice questions that don’t quite reflect what people in the real world do. I’m hopeful that the bar exam of the future will focus significantly more on validity. ...

The only fair reason to limit licensure to those who pass the bar is to protect the public. If the bar exam does not measure minimum competence, it needs to change or be eliminated. Fortunately the NCBE seems poised to oversee that change in the near future.

Update:, Ditch Multiple Choice? Simplify the Test? Ideas Abound on Reforming the Bar Exam

Legal Ed News, Legal Education | Permalink


Makes sense. But fairly comparing an enormous set of written answers is hard. The grading instructions or algorithms need to be public, or a lucky few and/or insiders will have an inappropriate advantage.

(Took me awhile to figure out most fancy law school - but not necessarily tax LLM - exams are in form essays, but in essence check-lists looking for a sentence about every point covered in the class that sort of relates to the prompt, rather than particularly much about the debatable-looking bits. And simple sentences might work better with moonlighting human graders, or overly complicated verbiage and legalese keywords for computers programmed to think that's smart...)

Posted by: Anand Desai | Sep 2, 2019 6:22:40 AM