Saturday, September 22, 2007
Interesting front-page article in the Weekend Wall Street Journal: Some of MIT's Arithmetic Hasn't Been Adding Up, by Keith J. Winstein (emphasis added):
[F]or some time, MIT ... says it wasn't properly calculating the average SAT scores of its freshmen. Those scores are closely scrutinized as a barometer of college quality. They are part of the formula used by U.S. News & World Report's influential annual rankings of schools.
When MIT dropped this year to seventh place from a three-way tie for fourth, its student newspaper, the Tech, asked why. In response, MIT revealed that its latest numbers factored in the SAT scores of non-native English speakers -- and that the school had excluded them for years. The change contributed to a 16-point drop in MIT's average SAT scores between 2005 and 2006. The reported SAT average was inflated by six points in 2005 and four in 2004. The school says it isn't sure the scores ever were correct before this year.
"We were not at all trying to do this in any way to increase our rankings," says interim admissions dean Stuart Schmill.
Excluding the test scores of foreign students -- which tend to be lower than those of U.S. students in reading -- is one of many tricks that have plagued the U.S. News numbers. These days, the magazine asks schools to certify that international students who provided test scores are included, and deducts points for those who don't. MIT said it did. ...
In the end, says Robert J. Morse, director of data research at U.S. News, a number of fluctuations -- including an increase in class sizes -- caused the school's drop in the rankings. Says Mr. Shmill: "It was a pretty harmless error, or we wouldn't be talking about it."
Why is it that when schools admit to "inadvertent" errors in the data they report to U.S. News, the errors invariably increased the school's ranking? For more, see the MIT student newspaper: Errors, Ambiguities Plague U.S. News Rankings; Data Uncertain, by Gregory N. Price,