Thursday, January 27, 2022
Vladimir Kogan, Brandon Genetin, Joyce Chen & Alan Kalish (Ohio State), Students' Grade Satisfaction Influences Evaluations of Teaching: Evidence from Individual-level Data and an Experimental Intervention:
Student surveys are widely used to evaluate university teaching and increasingly adopted at the K-12 level, although there remains considerable debate about what they measure. Much disagreement focuses on the well-documented correlation between student grades and their evaluations of instructors. Using individual-level data from 19,000 evaluations of 700 course sections at a flagship public university, we leverage both within-course and within-student variation to rule out popular explanations for this correlation. Specifically, we show that the relationship cannot be explained by instructional quality, workload, grading stringency, or student sorting into courses. Instead, student grade satisfaction — regardless of the underlying cause of the grades — appears to be an important driver of course evaluations.
We also present results from a randomized intervention with potential to reduce the magnitude of the association by reminding students to focus on relevant teaching and learning considerations and by increasing the salience of the stakes attached to evaluations for instructor careers. However, these prove ineffective in muting the relationship between grades and student scores.
Inside Higher Ed, No Satisfaction on Student Ratings of Instruction:
Yet another study is challenging the idea that student evaluations of teaching reliably measure what they’re intended to measure: instructional quality.
The new study, available now as a preprint, builds on the well-documented correlation between students’ grades and how they rate teachers (i.e., students who earned better grades in a course tend to rate those instructors more highly than peers who got lower grades). After testing and eliminating other possible drivers of this correlation, the study ultimately asserts that it’s not about instructional quality, workload or grading stringency or leniency.
Instead, the paper says, student grade satisfaction drives this correlation.