Paul L. Caron

Monday, May 7, 2018

Why We Must Stop Relying On Student Evaluations Of Law School Teaching — Like The University Of Oregon Is Doing

Chronicle of Higher Education op-ed:  Why We Must Stop Relying on Student Ratings of Teaching, by Michelle Falkoff (Northwestern Law School):

Few academics will be surprised to hear that more evidence has come out showing that student evaluations of teaching are often biased.

The latest study [Gender Bias in Student Evaluations], released this year by the American Political Science Association, found that the "language students use in evaluations regarding male professors is significantly different than language used in evaluating female professors." The study also showed that "a male instructor administering an identical online course as a female instructor receives higher ordinal scores in teaching evaluations, even when questions are not instructor-specific."

Table 1

Kristina Mitchell, one of the study’s authors, summarized its findings in Slate last month and concluded: "Our research shows they’re biased against women. That means using them is illegal." Academic institutions must stop giving an inordinate amount of weight to student evaluations when making employment decisions, she argued, until the institutions can account for, address, and eliminate bias.

Unfortunately, there’s no consensus on how best to do that, and gender isn’t the only kind of bias at issue. Still, it’s time for academic institutions to do better on this front. ...

[I]t’s time to stop relying primarily on one approach — in this case, student evaluations of teaching — and move to a more holistic strategy in which multiple factors contribute to a more accurate, consistent, and well-rounded assessment.

I was convinced of that by my experience as director of the program in communication and legal reasoning at Northwestern University’s law school.

A majority of the program’s faculty members are women, and our primary responsibility involves teaching a required first-year course ("Communication and Legal Reasoning") on legal analysis, writing, and research. Students don’t get to choose their professor, and they receive extensive critical feedback during the semester before they fill out their course evaluations. Perhaps not surprisingly, they often use the evaluation to vent — which typically involves lashing out at the professor.

In my own program, I’ve found alternative methods of assessing teaching to be extremely effective: watching faculty members teach (whether via video or in person), reviewing their course materials, reading faculty self-evaluations, and meeting with them one-on-one to discuss performance.

With a clear sense of how faculty members perceive their own courses, student feedback is easier to contextualize. It becomes possible to determine whether student concerns are legitimate or just typical of a demanding course (as first-year law-school classes tend to be).

Holding instructors to high standards is important, and student feedback is relevant. But if academic institutions do not take steps to assess teaching more holistically, they run the risk of losing talented faculty members for reasons that are not only inappropriate but may well be illegal. Moving beyond reliance on student evaluations may take more time and effort, but it will also help us ensure that we are helping instructors succeed while eliminating the possibility that bias will play a role in making or breaking their careers.

Daily Emerald, University of Oregon Course Evaluations Will Likely Change Soon:

The University of Oregon is changing course evaluations to make them more useful and eliminate implicit bias. The changes, if approved, will remove the number-based rating systems that are in place and replace them with more specific questions that require written answers.

“Those scores have nothing to [do] with how much a student learns in a class and that’s really problematic because our intent as [a] university is to try to get better at teaching,” said Bill Harbaugh, Teaching Evaluation Task Force co-chairman and Senate vice president. ...

The decision to improve evaluations came after Harbaugh discovered a study that showed trends of discrimination in college course evaluations. “If this information is not telling us what is good teaching then we have to wonder why we are collecting it and fix the way we are collecting it,” Harbaugh said. ...

In addition to students evaluating teachers, UO has teachers observe and review each other’s classes. “It’s not very well organized, so different professors use different systems for doing that, and we suspect that those evaluations are not really any more accurate than the student evaluations,” Harbaugh said.

UO is also planning to improve the way these evaluations are done by creating specific questions about the teacher’s performance during the class. “I think most faculty are enthusiastic about the idea of replacing numerical scores,” Harbaugh said.

AroundtheO, Senate, Provost Plan Changes to the Teaching Evaluation Process

Prior TaxProf Blog coverage:

Legal Education, Teaching | Permalink


Great! Now do a similar study based on political orientation, if you can find enough conservatives to achieve a big enough n.

Posted by: Nom de Blog | May 7, 2018 5:47:48 AM

If student evaluations are biased and therefore illegal, then so are the law school exams that professors administer and anonymously grade since White and Asian students consistently perform better than underprivileged minority students on these exams. Professors must find a new tool for evaluating students that yields outcomes that are even across race and gender. Maybe grades should be determined based only on the professors subjective evaluation of class participation. I am sure that won't yield any blowback.

Posted by: JM | May 7, 2018 9:50:33 AM

Hold the phone! Are you saying chairs and deans should be doing faculty evaluations? That won't give them enough time to do, well, whatever it is they do in all those meetings.

Posted by: Dale Spradling | May 8, 2018 5:30:59 AM