Inside Higher Ed, Rankings Noise:
What would it take for a well-regarded institution -- such as the University of Rochester, and a few dozen more like it -- to be among U.S. News & World Report’s top 20 national universities? Hundreds of millions of dollars and a prayer, according a new peer-reviewed paper co-written by a former Rochester provost and his staff.
The study [Modeling Change and Variation in U.S. News & World Report College Rankings: What would it really take to be in the Top 20?], published by the journal Research in Higher Education, argues that small movements in the rankings are simply “noise” and that any kind of sustained upward movement is both immensely expensive and nearly impossible. ...
The paper found that small movements up or down in the rankings are more or less irrelevant. For most universities in the top 40, any movement of two spots or less should be considered noise, the paper said. For colleges outside the top 40, moves up or down of four spots should be thought of as noise, too. “For example, a university ranked at 30 could be 95 percent conﬁdent that its rank will fall between 28 and 32, and only when the rank moves beyond those levels can a statistically signiﬁcant change be claimed,” the paper said.
Colleges that want to move any farther have a hard row to hoe, the paper argues, because “meaningful rank changes for top universities are difﬁcult and would occur only after long-range and extraordinarily expensive changes, not through small adjustments.”...
[T]he paper, by authors who have all worked at Rochester, adds to the ruminating by administrators over the U.S. News list with several arguments, including in-depth exploration of what it would take for Rochester, consistently in the mid-30s on the list, to break into the top 20. Emory University, Georgetown University and the University of California at Berkeley currently tie for No. 20. ...
If it wanted to move into the top 20, Rochester would have to do a lot on several of the various factors U.S. News uses to rank colleges. To move up one spot because of faculty compensation, Rochester would have to increase the average faculty salary by about $10,000. To move up one spot on resources provided to students, it would have to spend $12,000 more per student. Those two things alone would cost $112 million a year.
To get into the top 20, Rochester would also have to increase its graduation rate by 2 percent, enroll more students who were in the top 10 percent of their high school graduating class, get more alumni to give, cut the acceptance rate and increase the SAT and ACT scores of incoming students. Some of those things, like offering aid money to highly qualified students, might further increase the expense.
But that’s not all, the paper argues. Rochester would still have to do well in the rankings magazine’s “beauty contest.” Because 15 percent of the ranking is based on reputation among other administrators, even massive expenditures year after year and huge leaps in student quality and graduation would not be enough. The reputation score as judged by its peers would need to increase from 3.4 to 4.2 on a scale of 5, something that has only a .01 percent chance of happening, the paper said.