Colin Diver: The Rankings Farce

Chronicle of Higher Education Op-Ed:  The Rankings Farce, by Colin Diver:

‘U.S. News’ and its ilk embrace faux-precise formulas riven with statistical misconceptions. ...

On July 1, 2002, I became president of Reed College in Portland, Ore. As I began to fill the shelves in my office with mementos from my previous life as a law-school dean, I could feel the weight already lifting from my shoulders. “I’m no longer subject to the tyranny of college rankings,” I thought. “I don’t need to worry about some news magazine telling me what to do.”

Seven years before my arrival at Reed, my predecessor, Steven S. Koblik, decreed that Reed would no longer cooperate with the annual U.S. News Best Colleges rankings. ...

There is a growing cottage industry of college evaluators, many spurred by the commercial success of U.S. News. I call it the “rankocracy” — a group of self-appointed, mostly profit-seeking journalists who claim for themselves the role of arbiters of educational excellence in our society. It wasn’t just the U.S. News rankings that were incompatible with Reed’s values. Virtually the whole enterprise of listing institutions in an ordinal hierarchy of quality involves faux precision, dubious methodologies, and blaring best-college headlines. To make matters worse, the entire structure rests on mostly unaudited, self-reported information of dubious reliability. In recent months, for example, the data supporting Columbia’s second place U.S. News ranking have been questioned, the University of Southern California’s School of Education has discovered a “history of inaccuracies” in its rankings data, and Bloomberg’s business-school rankings have been examined for perceived anomalies. ...

I came by my rankings aversion honestly. In 1989, I became the dean of the University of Pennsylvania’s law school. The next year, U.S. News began to publish annual rankings of law schools. Over the next nine years of my deanship, its numerical pronouncements hovered over my head like a black cloud. During those years, for reasons that remained a complete mystery to me, Penn Law’s national position would oscillate somewhere between seventh and 12th. Each upward movement would be a cause for momentary exultation; each downward movement, a cause for distress.

My admissions dean reported that prospective applicants were keenly attuned to every fluctuation in the annual pecking order. So were my alumni. If we dropped from eighth to 10th, alumni would ask what went wrong. If we moved up to seventh, they would ask why we weren’t in the top five. Each year, Penn’s president would proudly present to the Board of Trustees a list of the university’s schools whose ranking numbers had improved. (She’d make no mention of those whose numbers had slipped.) ...

The rankings of U.S. News and its followers typically produce gruel. A careful look at the “recipes” for preparing these rankings shows why. ...

What we have, in sum, is a group of popular rankings that simplify the complexity of evaluating a college’s performance by arbitrarily selecting a collection of measures, many of which overlap substantially, and then assigning equally arbitrary weights in order to purée them together into a single offering. The result is a tasteless mush.

Colin Diver, Breaking Ranks: How the Rankings Industry Rules Higher Education and What to Do About It (Johns Hopkins University Press 2022):

Some colleges will do anything to improve their national ranking. That can be bad for their students—and for higher education.

Since U.S. News & World Report first published a college ranking in 1983, the rankings industry has become a self-appointed judge, declaring winners and losers among America's colleges and universities. In this revealing account, Colin Diver shows how popular rankings have induced college applicants to focus solely on pedigree and prestige, while tempting educators to sacrifice academic integrity for short-term competitive advantage. By forcing colleges into standardized "best-college" hierarchies, he argues, rankings have threatened the institutional diversity, intellectual rigor, and social mobility that is the genius of American higher education.

As a former university administrator who refused to play the game, Diver leads his readers on an engaging journey through the mysteries of college rankings, admissions, financial aid, spending policies, and academic practices. He explains how most dominant college rankings perpetuate views of higher education as a purely consumer good susceptible to unidimensional measures of brand value and prestige. Many rankings, he asserts, also undermine the moral authority of higher education by encouraging various forms of distorted behavior, misrepresentation, and outright cheating by ranked institutions. The recent Varsity Blues admissions scandal, for example, happened in part because affluent parents wanted to get their children into elite schools by any means necessary.

Explaining what is most useful and important in evaluating colleges, Diver offers both college applicants and educators a guide to pursuing their highest academic goals, freed from the siren song of the "best-college" illusion. Ultimately, he reveals how to break ranks with a rankings industry that misleads its consumers, undermines academic values, and perpetuates social inequality.

Inside Higher Ed, ‘Breaking Ranks’ With ‘U.S. News’