TaxProf Blog

Editor: Paul L. Caron, Dean
Pepperdine University School of Law

Monday, October 24, 2016

Weapons Of Math Destruction: How U.S. News' 'Craven' Decision To Eschew Affordability In Its Rankings Increases Inequality

WeaponsThe Traps of Big Data (reviewing Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy (2016)):

As O’Neil defines it, a weapon of math destruction, or WMD, has three elements: Opacity, Scale, and Damage. Combined, these factors create traps with feedback loops, capturing victims in systems they can’t understand and can’t escape, all the while exploiting them. Of the three, Scale seems the most pernicious element, enabling Damage.

After a critique of value-add theory for teachers and a refresher course on the 2008 credit-default swaps and mortgage-backed securities that led to the Great Recession’s financial meltdown, O’Neil produces a trenchant analysis of the U.S. News & World Report college rankings. After reading this chapter about how higher education has become captured by a big data system, the limitations and difficulties of the Impact Factor seem downright charming.

Whatever U.S. News & World Report measures is what colleges seek to optimize. The almost-blind pursuit of improving these metrics increases marketing spending, leads to investments in new buildings and stadiums, and provokes a number of other expensive endeavors that ultimately end up costing students more without adding educational value (unless you count acquired cynicism as part of a formal education). Combined with the well-documented cuts in state funding of universities, all of this ends up on students’ backs at public and private colleges and universities.

Predictably, anything that can be measured can be manipulated, and universities are not above these hijinks. Noting how universities both game the system and have been caught cheating outright, O’Neil speaks to the unintended consequences, craven choices, and inherent unfairness of the rankings. For example, if affordability had been built in to the rankings from the beginning, Harvard and Yale likely would have been low-ranked, which would have kept the U.S. News & World Report rankings from having face validity in the early days, limiting their uptake and commercial success. Affordability had to be downplayed for the metrics to appear to match pre-existing notions of how universities ranked. Affordability measures have not been included ever since.

Sometimes, universities go too far in their efforts to manipulate the rankings. Some of the scandals O’Neil mentions were new to me. For instance, Baylor University once paid the fee for its incoming freshman to retake the SAT, so that Baylor could generate higher scores in its reporting to U.S. News. Bucknell and Claremont McKenna sent false data, inflating the SAT scores of their incoming freshmen. Iona College, in 2011, fudged numbers about test scores, acceptance and graduation rates, freshman retention, student-faculty ratio, and alumni giving. Other scandals in the chapter range from corrupt to sadly hilarious.

The U.S. News & World Report WMD is opaque because 25% of the score — a portion that is weighted to have even more influence — is based on subjective opinions of peers and students and alumni. It has scale, because everyone uses it. And it does damage by driving inequality in higher education. This happens for a number of reasons — the higher cost of college driven by the competition for rankings; the data-driven exercise of applying to universities, which only well-off students and families can really ace; and the emphasis on test scores and other academic measures, which students from marginalized areas or low-income areas struggle to achieve. In short, the system informed by U.S. News & World Report’s rankings ensures that the rich remain on top of the heap.

Big data in higher education is also eroding the concept of a “safety school,” as these schools proactively reject high-achieving students because they are unlikely to attend or matriculate, and because actively rejecting them drives up their standings. Combined with pressures at top schools to have low acceptance rates, this can leave good students with no options — their “reach” schools reject them, their “safety” schools reject them, and they end up caught in a data-driven no-man’s land.

There are consulting services designed to help universities to treat their student body like an investment portfolio, balancing assets (top students) and liabilities (students who need financial aid). These add yet more costs. As O’Neil writes:

If you think about it, the college admissions game, while lucrative for some, has virtually no educational value. The complex and fraught production simply re-sorts and reranks the very same pool of eighteen-year-olds in newfangled ways. . . . All of them, from the rich to the working class, are simply being trained to fit into an enormous machine — to satisfy a WMD.

Law School Rankings, Legal Education | Permalink


In general, I'm a big Cathy O'Neil fan. I enjoy listening to her weekly on the Slate Money podcast. That said, while the book raises important issues for 21st century US consumers, I wasn't a huge fan of it overall. Lacking detail? Too short? Too mass-market? I dunno. It was a quick read, but wasn't my favorite.

Posted by: juniper | Oct 24, 2016 5:17:22 AM

Don't forget that the top schools routinely recruit high-scoring students despite the fact that there are so few slots available for them. This gives them a low rejection rate and a high spot on the U.S. News list. (Source: "Crazy U")

Oddly enough, my son's gotten one offer from a school in Florida when he was named a National Merit Semifinalist. If he's named a Finalist, they'll give him a free ride.

Posted by: Uma Thurmond's Feet | Oct 24, 2016 7:44:19 AM

If you are looking for the best of the best, affordability is not a valid measure. Should we exclude Mercedes and BMW from the list of best cars because most people cannot afford one? The best university has the top faculty, most competitive admissions process and best qualified students (among other things). There is a reason there are separate lists for best value colleges and most affordable.

Shocking news flash - the best things in life are expensive, and not everyone can afford them. Here's another - life isn't fair.

Posted by: todd | Oct 24, 2016 8:12:14 AM

A better scheme would allow users to select the weight they assign to each variable and rank schools for them personally based on that. Some might weight cost high. Some might weigh it low.

Even telling the ranking scheme to favor a certain range of SAT scores, high or low, would benefit applicant decision-making. Since professors tend to aim their teaching to the average student, someone learns best if they are around the class average not well above or below it.

Remove this one-size-fits-all ranking, and law schools won't be able play games with the ranking.

Posted by: Michael W. Perry | Oct 24, 2016 1:28:11 PM