Chronicle of Higher Education op-ed: Why We Must Stop Relying on Student Ratings of Teaching, by Michelle Falkoff (Northwestern Law School):
Few academics will be surprised to hear that more evidence has come out showing that student evaluations of teaching are often biased.
The latest study [Gender Bias in Student Evaluations], released this year by the American Political Science Association, found that the "language students use in evaluations regarding male professors is significantly different than language used in evaluating female professors." The study also showed that "a male instructor administering an identical online course as a female instructor receives higher ordinal scores in teaching evaluations, even when questions are not instructor-specific."
Kristina Mitchell, one of the study’s authors, summarized its findings in Slate last month and concluded: "Our research shows they’re biased against women. That means using them is illegal." Academic institutions must stop giving an inordinate amount of weight to student evaluations when making employment decisions, she argued, until the institutions can account for, address, and eliminate bias.
Unfortunately, there’s no consensus on how best to do that, and gender isn’t the only kind of bias at issue. Still, it’s time for academic institutions to do better on this front. ...
[I]t’s time to stop relying primarily on one approach — in this case, student evaluations of teaching — and move to a more holistic strategy in which multiple factors contribute to a more accurate, consistent, and well-rounded assessment.
I was convinced of that by my experience as director of the program in communication and legal reasoning at Northwestern University’s law school.
A majority of the program’s faculty members are women, and our primary responsibility involves teaching a required first-year course ("Communication and Legal Reasoning") on legal analysis, writing, and research. Students don’t get to choose their professor, and they receive extensive critical feedback during the semester before they fill out their course evaluations. Perhaps not surprisingly, they often use the evaluation to vent — which typically involves lashing out at the professor.
In my own program, I’ve found alternative methods of assessing teaching to be extremely effective: watching faculty members teach (whether via video or in person), reviewing their course materials, reading faculty self-evaluations, and meeting with them one-on-one to discuss performance.
With a clear sense of how faculty members perceive their own courses, student feedback is easier to contextualize. It becomes possible to determine whether student concerns are legitimate or just typical of a demanding course (as first-year law-school classes tend to be).
Holding instructors to high standards is important, and student feedback is relevant. But if academic institutions do not take steps to assess teaching more holistically, they run the risk of losing talented faculty members for reasons that are not only inappropriate but may well be illegal. Moving beyond reliance on student evaluations may take more time and effort, but it will also help us ensure that we are helping instructors succeed while eliminating the possibility that bias will play a role in making or breaking their careers.
Daily Emerald, University of Oregon Course Evaluations Will Likely Change Soon:
The University of Oregon is changing course evaluations to make them more useful and eliminate implicit bias. The changes, if approved, will remove the number-based rating systems that are in place and replace them with more specific questions that require written answers.
“Those scores have nothing to [do] with how much a student learns in a class and that’s really problematic because our intent as [a] university is to try to get better at teaching,” said Bill Harbaugh, Teaching Evaluation Task Force co-chairman and Senate vice president. ...
The decision to improve evaluations came after Harbaugh discovered a study that showed trends of discrimination in college course evaluations. “If this information is not telling us what is good teaching then we have to wonder why we are collecting it and fix the way we are collecting it,” Harbaugh said. ...
In addition to students evaluating teachers, UO has teachers observe and review each other’s classes. “It’s not very well organized, so different professors use different systems for doing that, and we suspect that those evaluations are not really any more accurate than the student evaluations,” Harbaugh said.
UO is also planning to improve the way these evaluations are done by creating specific questions about the teacher’s performance during the class. “I think most faculty are enthusiastic about the idea of replacing numerical scores,” Harbaugh said.
AroundtheO, Senate, Provost Plan Changes to the Teaching Evaluation Process
Prior TaxProf Blog coverage:
- Student Evaluations Hinge on "Course Easiness and Professor Sexiness" (May 24, 2005)
- University of Iowa Pays $226k to Law Prof Accused of Forging Student Evaluations (Apr. 9, 2007)
- Sometimes, Student Evaluations Are Right: We Really Do Suck (Nov. 23, 2009)
- How to Boost Your Student Evaluations (Apr. 14, 2010)
- Do Attractive Professors Get Better Student Evaluations? (Oct. 24, 2011)
- Prof Denied Tenure for Using Socratic Method (Nov. 1, 2011)
- Brooklyn Law School to Permit Dismissal of Tenured Faculty for Lack of Collegiality or Poor Student Evaluations (Apr. 26, 2013)
- WSJ: When Students Rate Professors, Standards Drop (Oct. 30, 2013)
- Scholars Critique Validity of Student Evaluations: 'Consumer Satisfaction' ≠ 'Product Value' (Sept. 22, 2014)
- Duquesne Says Law Prof Was Denied Tenure Due to Poor Teaching, Not Discrimination (Dec. 5, 2014)
- Does Professor Who Is Denied Tenure Have The Right To See Student Evaluations Of Her And Her Colleagues? (Apr. 1, 2015)
- Measuring Teaching Excellence: Shifting From Delivery Of Course Content To Impacting Student Lives (Apr. 23, 2015)
- Nobel Laureate: Replace Student Evaluations With Best Practices (July 1, 2015)
- University of Chicago Law School Study: Better Scholars Are Better Teachers (July 17, 2015)
- NY Times Debate: Is It Fair To Rate Professors Online? (Dec. 16, 2015)
- Harvard Business Review: How Gender Bias Corrupts Performance Reviews, And What To Do About It (Apr. 14, 2017)
- American University Provost Denies Tenure To Latina Prof Based On Standard Deviation Of Her Student Evaluations, Rather Than Relying On The 'Tyranny Of The Mean' (July 12, 2017)
- Should Student Evaluations Rate A Professor's Inclusiveness? (Sept. 25, 2017)
- Why Do Law Profs Provide Peer Review Of Scholarship, But Not Teaching? (Oct. 5, 2017)
- Students Rate Male Professors More Highly Than Female Professors, Even When They Teach Identical Courses (Apr. 5, 2018)
- More Evidence That Student Evaluations Are Biased Against Women (Apr. 17, 2018)