NY Times: U.S. News Ranked Columbia No. 2, But A Math Professor Has His Doubts

Following up on my previous post, Columbia Math Professor Questions University's #2 U.S. News Ranking: 'A Web Of Illusions':

New York Times, U.S. News Ranked Columbia No. 2, but a Math Professor Has His Doubts:

Everyone knows that students buff their résumés when applying to college. But a math professor is accusing Columbia University of buffing its own résumé — or worse — to climb the all-important U.S. News & World Report rankings of best universities.

Michael Thaddeus, who specializes in algebraic geometry at Columbia, has challenged the university’s No. 2 ranking this year with a statistical analysis that found that key supporting data was “inaccurate, dubious or highly misleading.”

In a 21-page blistering critique on his website, Dr. Thaddeus is not only challenging the rating but redoubling the debate over whether college rankings — used by millions of prospective college students and their parents — are valuable or even accurate.

Columbia said it stood by its data. Officials said there was no accepted industry standard for the data that goes into college rankings — every rankings project does it differently — and they strove to meet the technical requirements as set by U.S. News. But, they said, the university was not necessarily defending the process. ...

“I think the majority of institutions would be happy if the rankings went away,” said Colin Diver, a former president of Reed College, who has a book coming out about college rankings [Breaking Ranks: How the Rankings Industry Rules Higher Education and What to Do About It (Johns Hopkins University Press Apr. 22, 2022)]. ,,,

Asked about Dr. Thaddeus’s analysis, U.S. News & World Report did not address the details, but said that it relied on schools to accurately report their data. ...

Dr. Thaddeus has not done a systematic analysis of universities other than Columbia, but he does have a bigger agenda.

He believes that all rankings are “inherently suspect,” he said, because they are based on information from the institution being ranked.

There is little, if any, independent monitoring. “Who has the power to audit the books of these organizations that make the reports?” he said. “It’s frightening. There’s almost nothing.”

Dr. Thaddeus found discrepancies in important criteria that go into the ratings: class size, the proportion of faculty with the highest degree in their field, the percentage of faculty who are full-time, the ratio of students to faculty and the amount of spending on instruction. Those categories make up about one-fifth of the ranking formula used by U.S. News.

Columbia claimed that 100 percent of its faculty had “terminal degrees,” the highest in their field; Harvard, for instance, claimed 91 percent, he said.

By poring through the 958 full-time faculty members of Columbia College listed on its website (the only public list he could find), Dr. Thaddeus came up with 69 people (he has since corrected it to 66) whose highest degree, if any, was a bachelor’s or master’s degree (not including a master of fine arts) or a degree that was not in the field that they were teaching. ...

Dr. Thaddeus, who has taught at Columbia for 24 years, has made a hobby of provoking his employer. He said he had been radicalized by the experience of being chair of the math department from 2017 to 2020, when he discovered how secretive the university was. Since then, he has challenged the administration on subjects like administrative bloat, the management of its endowment and, now, the rankings.

Chronicle of Higher Education, Columbia Is Ranked No. 2 by ‘U.S News.’ A Professor Says Its Spot Is Based on False Data.:     

The Chronicle sent Thaddeus’s work to Tim Chartier, a professor of mathematics at Davidson College who previously attempted to reverse-engineer U.S. News for a broad review. Whether or not his exact figures are correct, Thaddeus’s analysis pointed to some important questions about the data Columbia submits to U.S. News, Chartier said.

“Michael recognizes his analysis may not be foolproof,” Chartier wrote in an email. “The fundamental issue isn’t the absolute accuracy of his numbers. One has to see a fundamental, rather blaring flaw, for the numbers to replicate the U.S. News & Report numbers.”

He explained an example. Columbia’s percentage of under-20-student classrooms is higher than any other institution in the top 100 national universities, and four percentage points higher than the two runners-up. Those exceptional numbers, in addition to how much lower Thaddeus’s estimate is, “does make one stop and want to ask what’s under the hood of Columbia’s calculation,” Chartier wrote. In response to this claim, Columbia pointed to small classes that all undergraduates must take as part of a core humanities curriculum.

Poets & Quants, Columbia Claims It Deserves Its No. 2 Rank; One Of Its Own Profs Disagrees:

A faculty member at Columbia for 24 years, Thaddeus found a number of significant differences between the numbers Columbia sent U.S. News and his own estimates based on publicly available data directly from the university. One example: U.S. News reports that 83% of undergraduate classes at Columbia have fewer than 20 students. Using data from the online directory of classes, Thaddeus estimated the true proportion is between 63% and 67 %. ...

Another example: U.S. News reports that Columbia boasts the ninth-highest spending per student out of 392 colleges. Part of that high spending is due to university counting of patient care under instructional costs. Thaddeus notes that Columbia reported to the government that instructional spending in 2019–20 was slightly over $3.1 billion. “This is a truly colossal amount of money,” he writes. “It works out to over $100,000 annually per student, graduates, and undergraduates alike. It is by far the largest such figure among those filed with the government by more than 6,000 institutions of higher learning. It is larger than the corresponding figures for Harvard, Yale, and Princeton combined."