TaxProf Blog

Editor: Paul L. Caron
Pepperdine University School of Law

Wednesday, April 12, 2017

The Debate Over The Impact Of Different MBE Cut Scores In Different States

MBESuzanne Darrow Kleinhaus (Touro), UBE-Shopping: An Unintended Consequence of Portability? (Mar. 2016):

Getting our students ready for the UBE, may require more than just learning the law; it also means learning in which jurisdiction you should take it. While there is not much that is new about the UBE’s individual components — the Multistate Essay Examination (MEE), the Multistate Performance Test (MPT) and the Multistate Bar Examination (MBE) — what is new is that where you take the UBE may make the difference between passing and failing. This is possible because of the convergence of bar exam test practices of “portability,” “relative grading,” and “scaling” of scores.

By adopting the UBE, jurisdictions agree to weight the MEE at 30%, the MPT at 20%, and the MBE at 50% in determining an examinee’s score. As a result, the UBE, as currently administered, leads to the situation where the same skill level could result in different UBE scores depending on where the candidate takes the exam, fails to ensure that the scores used to grant licensure in a UBE jurisdiction are sufficiently reliable for high-stakes testing when it is possible to achieve different outcomes on the same test by the same candidate if taken in different UBE jurisdictions, results in a “portable” score but not an “accurate” one because the written score — 50% of the total — depends on the strength of the applicant pool in the jurisdiction where the candidate wrote the exam, presents a candidate with the opportunity to “UBE shop” and “game the system” by taking the UBE in a jurisdiction where the same essay and MPT performance would result in a higher score and then transferring that inflated score for admission in a “harder” jurisdiction, makes it possible for a candidate to file a discrimination lawsuit challenging his or her UBE results.

Mark A. Albanese (Ph.D.), The Testing Column: Let the Games Begin: Jurisdiction-Shopping For the Shopoholics (Good Luck With That), The Bar Examiner, Sept. 2016:

It has been argued that variation in how Uniform Bar Examination (UBE) scores are produced across jurisdictions presents opportunities for examinees to shop around for a UBE jurisdiction that gives them the best shot at passing the bar exam. A number of concerns related to grading consistency, fairness, and transparency have been raised, as well as a proposal that centralizing the grading process for the written portion of the exam would be a remedy to these ills. In addressing these concerns, I will start with what we know about the variability in grading among jurisdictions, assess the likelihood that examinees could benefit from shopping around for a more favorable jurisdiction in which to take the UBE, and close with what could be gained from centralizing the grading process. ... 

[I]f examinees think that they can find a jurisdiction that will give them a leg up when it comes to the grading of their essays and want to put the time into finding it, well, that is the American way. Our economy thrives on people who shop till they drop. It has made America great. So if you shopaholics know what characteristics a jurisdiction must have to let you play the game, and if you have the unknowable knowledge of how jurisdictions will perform on the administration when you plan to sit for the bar exam, and if you would rather jurisdiction-shop than use the time for study, shop away; there are 37 jurisdictions, 14 of which are UBE jurisdictions, that have no limit on the number of times you can take the bar exam. Of course, they may not be the ones you are shopping for, but they will be there for you when you have finished shopping.

Suzanne Darrow Kleinhaus (Touro), A Reply to the National Conference of Bar Examiners: More Talk, No Answers, so Keep on Shopping (Mar. 2017):

In Let the Games Begin: Jurisdiction-Shopping for the Shopaholics (Good Luck With That), Mark Albanese defends the National Conference of Bar Examiners’ grading practices as essential to assuring reliability given the variability in grading between UBE jurisdictions. In addressing the claim that it is possible to achieve different outcomes on the same test by the same candidate if taken in different UBE jurisdictions, he describes how NCBE monitors jurisdiction variation to ensure grading consistency. Those of us concerned, however, with the possibility that the jurisdiction in which a candidate takes the Uniform Bar Examination (UBE) may make the difference between passing and failing will not find Mark Albanese’s explanations satisfactory. Contradictory and confusing perhaps, but not responsive. Rather than answer the question of whether it is possible for the same person to be found “competent” to practice law in one UBE jurisdiction and “incompetent” in another when it is the same person with the same skill level writing the same exam, NCBE deflects and disguises — and despite a lot of words and numbers — avoids it completely.

This reply identifies the significant flaws in Dr. Albanese’s defense of NCBE’s scoring practices. These practices include standardizing the written scores to the subset of MBE scores that come only from that jurisdiction and standardizing written scores to multiple choice scores. The way that the written raw scores are standardized is itself a problem for two reasons and will be addressed. In so doing, the underlying question becomes clear: why would NCBE and Dr. Albanese defend admittedly defective practices — practices that are antithetical to the bar exam’s objective of determining an individual’s minimum competency for the practice of law?

Legal Education | Permalink


I hate to interrupt the navel-gazing by the Ivy League prawf brigade, but law firms and most legal employers hire locally. If a Touro grad decides to take the bar in like Kansas or New Mexico (or whatever states have the lowest MBE scores), they quite likely aren't going to be hired by a firm out there. If you didn't attend HYSCCN or very similar, the utility of your JD diminishes very rapidly away from the city/state where you attended law school. Fact. Maybe Touro should focus more on rebuilding its incoming LSAT and GPA and less on making weak justifications on why every state should have the same (low) MBE passing score.

Posted by: Unemployed Northeastern | Apr 12, 2017 1:56:16 PM