WSJ: The Unraveling Of The U.S. News Rankings

Wall Street Journal, The Unraveling of the U.S. News College Rankings:

Yale Law School Dean Heather Gerken floated the idea past a small circle of colleagues. She had a sleepless night and queasy morning. And then, on Nov. 16, she started the revolt.

“The U.S. News rankings are profoundly flawed,” Ms. Gerken said in a letter that day. And with that, Yale Law pulled out.

Within three months, more than 40 law schools—about 20% of the programs that U.S. News ranks—said they would also end their cooperation and no longer share data with the publication, including 12 of the top 14. A wave of medical schools, led by No. 1 Harvard Medical School, followed. At the undergraduate level, the Rhode Island School of Design (No. 3 among regional universities in the North) and Colorado College (No. 27 in the latest measure of national liberal-arts colleges) withdrew last month.

The rebellion, which has thrown into tumult the most famous source of college rankings for generations of would-be students, was decades in the making.

College presidents, deans and institutional research officers said they have regularly raised concerns with the company’s chief data strategist in letters, on phone calls, at conferences and during in-person meetings at the company’s Washington, D.C., headquarters. The rankings, they argued, were opaque, favored the wealthiest schools and promoted practices that didn’t benefit students. They warned against simplifying something as complex as an education into an ordinal rank. ...

The move has led to a moment of crisis for U.S. News, once a publisher of a weekly news magazine that competed with Time and Newsweek. It began ranking colleges 40 years ago and graduate schools soon after. The rankings quickly grew into the definitive guide for prospective students and families desperate to simplify an otherwise complicated calculation of which school was the best choice. ...

Critics say that there are myriad ways for schools to goose the numbers. Inaccurate data, including submissions with transposed numbers and efforts to creatively tally alumni donations or research expenditures, are unearthed every few months and detailed on U.S. News’s own website.

In 2021, the former dean of Temple University’s business school was convicted on fraud charges related to a scheme in which he submitted inflated student data. He was sentenced to more than a year in prison.

Last year, the University of Southern California’s Rossier School of Education said a report it had commissioned by a law firm found administrators omitted information from the school’s submission to boost placement dating back to at least 2013. Columbia University admitted in September to reporting incorrect figures in multiple categories. The school dropped to No. 18 from No. 2 after U.S. News turned to other public sources for data, rather than to the school, in the latest ranking. ...

At a January presentation to law schools, Robert Morse, U.S. News’s chief data strategist, disclosed that he didn’t commit to a particular mathematical model until after receiving schools’ data, according to Ian Ayres, an economist and Yale Law School professor who attended the event. Once that information was in hand, Dr. Ayres said, the team ran simulations giving various factors different weights to see the potential outcomes before deciding on a final method. 

Dr. Ayres said that approach violates fundamental social-science research standards in which the methodology is specified ahead of time to prevent anyone from reverse-engineering a preferred result. ...

About 16% of undergraduate programs ranked by U.S. News didn’t provide the publication with data last year, up from 7% in 2015. Most of them hover near the bottom of the lists. 

Administrators at multiple schools in the middle of the pack say they’d be more inclined to pull out if one of the top colleges, such as Princeton University (No. 1 among national universities for 21 of the past 23 years) or Williams College (No. 1 among national liberal-arts colleges for the past 20 years straight), went first.

Both schools declined to comment.