Klein: Of Prisoner's Dilemmas And Straw Men — A Response To Blackman, Adler, And Krauss On Law School Grading

Following up on my previous posts on law school grading policies for the Spring 2020 semester in the wake of the coronavirus (links below):

Diane Klein (La Verne), Of Prisoner's Dilemmas and Straw Men: A Response to Blackman, Adler, and Krauss on Law School Grading:

Dramatic (probably temporary) changes to grading policies are afoot in America's law schools, and in higher education more generally, in response to COVID-19, the all-online transition, and the seismic disruption taking place in education across the United States. ...

Fa lively debate has been taking place about whether law schools should adopt mandatory pass/fail grading for spring 2020, leave their grading systems alone, or do something in the middle.  A midsemester transition to all pass/fail recommends itself to many faculty members and administrations, based on a sense of compassion for students, and in acknowledgement of the extraordinary dislocation created by the circumstances we are all in.  Led by Dorf on Law's eponymous founder, Prof. Michael Dorf, Cornell Law was among the first to decide to adopt mandatory pass/fail, as they announced on March 16, 2020, and an informal survey suggests that other elite institutions have largely followed suit.

This has triggered a somewhat predictable backlash from some conservatives among the legal professoriate. Josh Blackman of South Texas College of Law [here, here, and here] and Prof. Jonathan Adler of Case Western both blogged about it at The Volokh Conspiracy, and Prof. Michael Krauss of George Mason University Law wrote in Forbes. All take something like a "tough love" approach; all, in my view, minimize the magnitude of what is unfolding; mistakenly fixate on grades as the sole way for professors to convey their assessment of students (to students themselves and to potential employers); and (perhaps most surprisingly!) miss some of the problematic game-theoretic dimensions of what is taking place, leading to their support for an even worse option than no change at all: individualized opt-in to pass/fail.

The costs and benefits of an across-the-board switch to pass/fail grading (by whatever name) are, I think, pretty clear. Professor Noah Zatz of UCLA has clearly and compellingly articulated many of them. This switch immediately and dramatically lowers the pressure and stress on both students and faculty. Knowing that some students will experience much greater disruption than others, that some students will themselves fall ill, perhaps seriously; some will lose family members; some have returned to living environments in which their ability to study will be greatly worsened; some are much more reliant on school-based resources and services no longer available, the shift to pass/fail functions as an equalizer. It also greatly reduces the long-term professional consequences for those students very significantly negatively impacted by coronavirus, and removes any unearned advantage that would be enjoyed by students whose academic lives turn out to be minimally disrupted (likely to be predominantly traditional-age affluent students who remain healthy and have few non-academic responsibilities). It thereby allows all students to cope with the crisis as needed, continue their legal education, and prevent non-academic disparities from having an even larger influence than they already do, both on short-term performance and long-term prospects.

There are, perhaps, some good arguments to be made for retaining traditional grading - but Profs. Blackman, Adler, and Krauss do not make any of them. ...

The nation is facing an unprecedented health crisis, and nearly all of America's almost 20 million college students are now taking part in a great experiment in all-online education.  This is an arrangement the overwhelming majority did not sign up (or go into debt) for, but in which they are participating nonetheless. Throughout higher education, faculties and administrations are considering whether their usual grading policies should be revised this semester in light of these changes.  Most are free to focus on their own institution, its mission, and its students.  But the uniquely competitive and fiercely hierarchical law school ecology and economy add some complications to this analysis.  In this decision, as in so many, law schools are also constantly looking "up," at what the highest-ranked schools are doing, and looking "sideways," at what their competitors are doing, in order to insure that nothing they do disadvantages them or their students. ...

With all this in mind, I am still not entirely sure what I think is the best approach: that all schools, regardless of rank, switch to mandatory pass/fail; or that some should retain standard grading but with an explicit commitment to reduce certain expectations of students.

What schools should not do, however, is take a "neither fish nor fowl" approach, allowing individual students to opt-in to pass/fail, either at will or on the basis of an individualized demonstration of having been impacted by COVID-19.

Perhaps in the grip of worries about placing their students at fancy firms in the summer of 2021, or about falling behind their peer schools in employment statistics, some first-tier but not T-14 schools, some top-ranked schools, and some midrange schools as well, have adopted or are considering "hybrid" approaches of some kind.  This seems like an attempt to "game" a system in which a school ranked roughly in the top 20 hopes to give its students an advantage over T-14 students in the highly-competitive job search to come, and so on, down the line.  The idea is that students who get good grades will be advantaged vis-a-vis T-14 students with no grades from spring 2020, and those who go pass/fail will be no worse off. ...

Any approach which requires students to show they are "eligible" for the pass/fail option has the additional defect of inadministrability (don't these law and econ folks care about efficiency, at all?).  The circumstances that would qualify a student for this relief could happen at any time, up to the very day of an exam, and would present the student with an extremely difficult strategic choice and little time or information with which to make it wisely.  Does the strategic student go pass/fail when their parent or spouse is diagnosed with COVID-19? Or not until they are on a ventilator?  Are these the kind of questions we want students and administrators to be asking themselves?  Is that prospect really more appealing than the possibility that a mandatory pass/fail system will result in some small number of students who might "'game' the system or take advantage" (and how, exactly, will they do that?).

Individualized approaches and exceptions to generally-applicable policies are appropriate in ordinary circumstances, when schools can estimate with reasonable accuracy how many students in a given year will face the kind of catastrophic circumstances that make withdrawal, rescheduling of exams, or other remedies appropriate.  This is nothing like that, and suggesting it can or should be treated the same way is a sort of fantastical denial.

But bad as the Adler/Blackman approaches are, worse yet are systems which allow students themselves to decide to opt in.  Without a doubt, this is the worst approach for faculty.  Courses would still be taught and assessed as usual, thus maximizing both faculty labor and the chance that faculty members will spend many, many hours grading 100 essay exams with care and precision, only to find out afterwards that 90% of the class is taking it pass/fail.

If students are permitted to wait until after grades are awarded, this will result in maximal "gaming" of the system (not that Prof. Adler seems to mind, strangely, since he suggests it), perhaps as those school administrations intend.  It seems safe to assume that almost every student whose spring 2020 GPA is lower than their cumulative GPA up to that point will take the pass/fail option, likely "breaking" the curve in every class and putting the entire school's GPAs on an escalator.  Students who have been enrolled in the same classes as one another will not have the same number of grades.  Knowing that the pass/fail option is available after grades are awarded would also seem to have the most deleterious effects on motivation over the course of the term.

In adopting an approach like this, administrations seem to have reasoned that if students who do "badly" are the only ones to shift to pass/fail, the school itself will be better off competitively.  It will produce as many students with high GPAs as before, and fewer with lower GPAs.  Everyone at Freerider Law School wins! Right? Wrong.  First, this only works if other schools don't do the same thing - what is sometimes called the "race to the bottom," and in this case, is just another name for grade- (or GPA-) inflation.  In this case, once enough institutions ranked higher than Freerider LS go pass/fail, it is hard to imagine employers won't see through this obvious tactic. In addition, since employers typically interview more than one student from a given school, the pass/fail designation will speak for itself.  But equally to the point, the massive unpredictability that lies ahead makes it impossible to assert with any confidence how the world will look to firms and others hiring for summer 2021, making "strategic" behavior even more difficult.

But worst of all is an approach, currently being contemplated by Notre Dame, in which students would be required, now, to opt in to pass/fail grading for all their spring 2020 courses, anonymously.  It imposes all the costs on faculty already described, and creates all the same freerider problems.  But the defects don't end there.

This policy is also worse for students.  How, exactly, are students supposed to determine, now, whether this is a rational choice to make?  Keep in mind that it is, by definition, a high-stakes choice.  If it weren't, if one semester of law school grades (especially the spring semester of the 1L year) didn't matter so much, schools wouldn't be bothering to change their grading policies at all.

So we are asking our least-experienced law students to make a highly impactful, complex strategic decision, with long-term implications for their career, under what most game-theorists recognize are the worst conditions: incomplete and asymmetric information, time pressure, and stress.  What becomes of the overconfident student, or the one who is not well-informed about risks, whose life is upended in late April, when it's too late to change this irrevocable election?  Students would not be obligated to share their decisions with one another, but each piece of such information is very valuable in an asymmetrical information "game" like this.

Such a situation creates a bizarre prisoner's dilemma, pitting students against their classmates under circumstances of nearly maximal uncertainty.  If you believe many of your fellow students will go pass/fail, and reduce their efforts accordingly, this might be the ideal time to study hard, end up at the top of the curve, and move up dramatically in your class ranking.  But is such a belief rational, under the circumstances?  Only if you can reliably predict your own situation, a thing no one is currently in a position to do.  Not only would choosing well require knowing (or guessing accurately) about one's own likely performance (something 1Ls are notoriously poor at even under ordinary circumstances), it also requires knowing (or guessing accurately) about both the performance and the choices likely to be made by others.  The game-theoretic dimensions of this situation are eye-crossing for most people in the best of times; there is no justification for demanding it of those least able to do so, to try and solve it now.

Gov. Andrew Cuomo might have been talking about Professors Adler, Blackman, and Krauss, when he said on March 25, 2020, "You're missing the magnitude of the problem, and the problem is defined by the magnitude."  In assessing the impact of COVID-19 on all Americans, including law students and their professors, the arguments for retaining existing grading schemes seem to rely mostly on "missing the magnitude of the problem."  Law students and law faculties alike are ill-served by fetishizing grading and ignoring other available methods of assessment that are meaningful for students and employers, now and in the months to come.  In the end, those who got there first, and imposed mandatory pass/fail for spring 2020, probably got it right.

TaxProf Blog coverage of law school grading policies in Spring 2020:

• Karen Sloan (Law.com), Law Schools Adopt Pass-Fail Grades As They Move Online Amid COVID-19 (Mar. 19, 2020)
• Jon Adler (Case Western), Grading In The Time Of Coronavirus: For Most Law Schools, Pass/Fail Is A Mistake (Mar. 23, 2020)
• Josh Blackman (South Texas), Law Schools Should Not Abandon Standard Grading Policies For All Students Due To The Coronavirus (Mar. 23, 2020)
• Josh Blackman (South Texas), Law Professors Quietly Oppose Pass/Fail Grading (Mar. 25, 2020)
• Josh Blackman (South Texas), Law Students Quietly Oppose Pass/Fail Grading (Mar. 26, 2020)
• Karen Sloan (Law.com), 'People Are Pissed': Pass/Fail Grading Controversy Roils Law Schools (Mar. 26, 2020)
• Karen Sloan (Law.com), Law Students Share Anxiety, Support On Reddit Over Grading Policies In Wake Of Coronavirus (Mar. 27, 2020)
• Noah Zatz (UCLA), The Case For Mandatory Pass/Fail Grading In Spring 2020 (Mar. 28, 2020)
• Derek Muller (Pepperdine), Thoughts About (And Mostly Against) Pass-Fail Law School Grading During Covid-19 (Mar. 29, 2020)
• Michael Krauss (George Mason), Should Law Schools Grade Pass-Fail This Semester? (Mar. 29, 2020)
• Diane Klein (La Verne), Of Prisoner's Dilemmas And Straw Men: A Response To Blackman, Adler, And Krauss On Law School Grading (Mar. 29, 2020)

For complete TaxProf Blog coverage of the coronavirus, see here.