Paul L. Caron

Tuesday, August 20, 2019

Student Evaluations Are Unreliable And Biased Against Female Professors

London School of Economics and Public Policy, Student Evaluations of Teaching Are Not Only Unreliable, They Are Significantly Biased Against Female Instructors:

A series of studies across countries and disciplines in higher education confirm that student evaluations of teaching (SET) are significantly correlated with instructor gender, with students regularly rating female instructors lower than male peers. Anne Boring, Kellie Ottoboni and Philip B. Stark [Student Evaluations of Teaching (Mostly) Do Not Measure Teaching Effectiveness] argue the findings warrant serious attention in light of increasing pressure on universities to measure teaching effectiveness. Given the unreliability of the metric and the harmful impact these evaluations can have, universities should think carefully on the role of such evaluations in decision-making.


The sign of any connection between SET and teaching effectiveness is murky, whereas the associations between SET and grade expectations and between SET and instructor gender are clear and significant. Because SET are evidently biased against women (and likely against other underrepresented and protected groups) — and worse, do not reliably measure teaching effectiveness — the onus should be on universities either to abandon SET for employment decisions or to prove that their reliance on SET does not have disparate impact.

Legal Ed News, Legal Ed Scholarship, Legal Education | Permalink


The only way I'll ever be convinced this is accurate is to have a professor deliver the exact same lecture, once dressed up as a dude and then again dressed up as a gal (wearing makeup and prosthetics).

If you show me that the gal gets lower scores for the same lecture, differing only in appearance, then I'll be a believer.

Until then, call me deeply skeptical about this voodoo, junk research.

Posted by: Anon | Aug 20, 2019 4:48:25 AM

Yup - dividing subjective judgments into numerical scores just helps honest people be thorough. I do think the narratives add value, and should be posted publicly if the reviewers want: the more meaningful ones are likely self evident. Kudos, authors, for checking your own study against a firmer standard of teacher skill (grades on same exam by different section students.)

I was shocked by the amount of sexism in my New York law school. In my Southern college I might have occasionally heard some professor or teaching assistant was pretty but I don't remember even that. Up there women professors' looks (and these were just normal office looking folks) were constantly discussed - sometimes in a creepy way - and acted on as the reason to pack a couple otherwise dry courses. It also seemed to drive temporary-professor selection. Older, less made up ladies got not-that-hushed mean gossip.

Protip: To the extent this "is what it is," submitting a department store style studio portrait to the faculty directory rather than the standard snapshot in front of the bookshelf makes a huge difference.

Posted by: Anand Desai | Aug 20, 2019 5:51:52 AM