Paul L. Caron

Thursday, December 6, 2018

U.S. News Rejects Democratic Senators' Demand To Change Its Rankings Methodology

U.S. News Generic RankingsChronicle of Higher Education, In Unusual Letter, Democratic Senators Ask ‘U.S. News’ to Change Emphasis of College Rankings:

A handful of Democratic senators want an influential ranker of colleges to reconsider what’s important in higher education.

Specifically, the six senators wrote in a letter to U.S. News & World Report, compiler of the most prominent college rankings in the country, that more weight should be given to institutions that open their doors to students from underrepresented backgrounds.

“We urge U.S. News to use its influential platform to better align its rankings with the three longstanding goals behind federal financial aid: improving college access, supporting student success, and providing every talented student a pathway to economic stability and meaningful participation in our country’s economic, social, and civic life,” they wrote in the letter, released on Monday.

College leaders, too, have warned that the nation’s obsessive focus on the ubiquitous rankings may carry significant costs. U.S. News appears to have gotten the message, at least in part. It tweaked this year’s rankings to give greater weight to the graduation rates of students who receive Pell Grants. But the new measures are only a sliver of the statistics that determine an institution’s ranking.

The magazine also removed from the formula a college’s acceptance rate and reduced the importance of students’ standardized-test scores and high-school class standing, the type of markers more commonly associated with students who come from wealthier families.

The senators acknowledged the “modest improvements” the publication had made in its ranking formula, but wrote that more needed to be done.

U.S. News & World Report, U.S. News Responds to Senators on Social Mobility in the Best Colleges Rankings:

Dear Senators Coons, Booker, Schatz, Harris, Murphy and Baldwin:

U.S. News & World Report shares your dedication to preparing America’s students for a diverse and global workforce. For nearly 40 years, we have evaluated 1,400 colleges and universities for the benefit of students and their families, who rightly want to know what they will get out of a college degree. With tuition and fees at public four-year universities tripling over 20 years and student loans costing American families $1.5 trillion collectively, we think that measuring academic excellence and rewarding schools that enroll, retain and graduate their students, regardless of economic status, is a necessary and apolitical undertaking.

The U.S. News Best Colleges rankings are a reflection of the current higher education landscape. Our practice is to change our methodology when new or better data becomes available – or there is broad consensus among educators about measures of academic quality. Social mobility is one of a number of such factors.

Our barrier to measuring social mobility has always been data – we cannot measure data we don’t have. Nor can we make fair comparisons among hundreds of schools. As Sen. Coons will recall, this is something we discussed in detail with him in a meeting in December, 2016. (For background on the issues we discussed, I’ve attached a copy of our correspondence from September, 2016.)

But for many years, we have used the best available data in our rankings to show how well schools enroll and graduate students from low-income families. In the 1990s we began incorporating Pell Grant data into our methodology as part of our graduation rate performance measure. This evaluates how well a school graduates its students in the federal standard of six years taking into account characteristics of the student body, including the proportion of students on Pell Grants. The intent has been to create a level playing field by not penalizing schools that enroll low-income students who often have academic challenges.

When the federal government mandated that schools report Pell Grant graduation rates, we finally could make a more useful comparison of the actual graduation rates of students from low-income families. In addition to adding social mobility to our methodology, we also published a list of the colleges that promote social mobility so that families could easily access this valuable information.

The new data has caused meaningful changes in our rankings. Clearly some schools are doing a better job at this than others. Schools such as UCLA and Howard have benefitted by moving up in the rankings. Many schools that serve low-income populations do not fare so well. Interestingly, so-called elite schools often perform quite well on this measure, as there is a clear correlation between the wealth of a school and its ability to serve all of its students. We will continue to study this data, take feedback, and make refinements as necessary.

There is an active debate about how to best measure quality in education, in which we play a part. As always, we engage college and university officials and leaders in education policy and quality metrics on an ongoing basis. Those discussions could include members of the Senate – on an informal basis, of course. Despite some of the intemperate rhetoric in your letter (U.S. News has no reason to “acknowledge” or “correct” our past work; it speaks for itself), I think there is room for a useful and cordial exchange of ideas. This exchange would also clear up some evident misunderstandings about the higher education industry.

Law School Rankings, Legal Education | Permalink


The way to get USNWR to change its methodology is to put on your adult pants and actually get a wee bit technical to dissect, publicly and loudly, just how BS the USNWR methodology is and how gameable the rankings are. Note how few safeguards there are on the data, and how infrequently incorrect information submissions get called out (example: every year like 12 to 18 law schools *accidentally* report their average ANNUAL student debt instead of the average three-year total; USNWR has never held any of them to account). The university where I went to law school made it its unofficial mission to move up the rankings some years ago; in about the space of a decade they climbed more than 100 slots. Back in the 1990s Caltech wound up as #1 one year and USNWR swiftly rejiggered the methodology to ensure that HYPS would be some combination of #1-4 forever. There are numerous takedowns of the veracity and soundness of the rankings by education policy wonks and statisticians. Etc. etc., etc. USNWR rankings are built on a foundation of sand; the college-curious public needs to be reminded of this at every turn.

Posted by: Unemployed Northeastern | Dec 7, 2018 10:14:58 AM

So they want US News to rank schools based on how many of the right sorts of minorities are students (African American and Latino, but not East Asian, I presume), the availability of remedial classes, and how close the cost is for free.

That sounds like a great idea.

Posted by: Jablonski | Dec 7, 2018 6:18:38 AM

Politicians always think that getting elected makes them instant experts on everything.

Posted by: Ted Quick | Dec 7, 2018 4:06:54 AM