Paul L. Caron

Tuesday, July 14, 2020

Predictive Test Scores And Diploma Privilege

Michael Abramowicz (George Washington), Predictive Test Scores and Diploma Privilege:

The International Baccalaureate program, which credentials high-school students who take college-level classes, canceled exams this year because of COVID-19. But that did not stop the program from granting exam grades. Exam grades were given based on a predictive algorithm. According to Wired, "The system used signals including a student's grades on assignments and grades from past grads at their school to predict what they would have scored had the pandemic not prevented in-person tests."

Unsurprisingly, this is controversial. And without the precise formula or algorithm that the program used to calculate grades, it is difficult to assess. But the core idea makes a lot of sense: Measure students with a system of decentralized grades, then normalize the grades from different schools or teachers based on how predictive those grades have been of some other performance measure. At least, this seems like a reasonable approach if it is impossible to give the test and we still need to distinguish students. Use the best information available, including historical data.

Indeed, it would be an improvement over current practice if U.S. News and World Report normalized students' grades in this way. How would this work? In effect, a predicted LSAT score would be calculated based on one's GPA, given the GPA distribution and LSAT distribution at one's undergraduate school. So, if one received a 75th percentile GPA, that would be translated into a 75th percentile LSAT score for the same school. With enough data, the prediction might be based on GPA across majors, to account for tougher grading in some departments than others. A student would receive an actual LSAT score too, but this approach would make comparisons of GPAs across schools much more meaningful. ...

If such a system can be used to normalize high school grades and college grades, such a system could also be used for law school grades to determine whether students can be admitted to the bar.

There has recently been a push for diploma privilege. The justification is the COVID-19 pandemic, and there is a powerful argument that this is not a good time for standardized testing. But if one accepts the claim that the bar exam helps to protect clients from poorly qualified lawyers, diploma privilege is difficult to countenance. Why should lawyers' welfare be placed above clients'? ...

A compromise on diploma privilege would be to use a predictive grading approach. Bar examiners could create a simple, school-specific model forecasting bar exam performance as a function of law-school grades. Then, they might decide that any student who is predicted more likely than not to pass the exam receives grade-based diploma privilege. Or bar examiners might choose other thresholds. If one is particularly concerned about false positives (students admitted to the bar who would not have passed the exam), the examiners might demand a higher percentage, and if examiners are particularly concerned about false negatives (students whose grades predict failure but who would have passed the exam), one might demand a lower percentage. The core point is that states should use the information available to them, at least if it is infeasible to generate information in the form of a bar exam.

Coronavirus, Legal Ed News, Legal Education | Permalink