Inside Higher Ed, Academic 'Moneyball':
MIT management professors push data-based model they say is more predictive of an academic's future research success than traditional methods of peer review in tenure.
Michael Lewis’s 2003 book, Moneyball — later made into a movie starring Brad Pitt — tells the story of how predictive analytics transformed the Oakland Athletics baseball team and, eventually, baseball itself. Data-based modeling has since transcended sport. It’s used in hiring investment bankers, for example. But is academe really ready for its own “moneyball moment” in terms of personnel decisions?
A group of management professors from the Massachusetts Institute of Technology think so, and they’ve published a new study [Tenure Analytics: Models for Predicting Research Impact)] on a data-driven model they say is more predictive of faculty research success than traditional peer-based tenure reviews. In fact, several of the authors argue in a related essay [‘Moneyball’ for Professors?] in MIT Sloan Management Review that it’s “ironic” that “one of the places where predictive analytics hasn’t yet made substantial inroads is in the place of its birth: the halls of academia. Tenure decisions for the scholars of computer science, economics and statistics — the very pioneers of quantitative metrics and predictive analytics — are often insulated from these tools.”
Many professors oppose the use of bibliometrics in hiring, tenure and promotion decisions, saying that scholarly potential can’t be captured in a formula most often applied to pursuits with a bottom line, like winning games or playing the stock market. Such a system inevitably will be “gamed" by academics, critics say, and time-consuming but ground-breaking research will be sidelined in favor of “sure things” in terms of publishing -- the academic equivalent of clickbait.
But in Sloan Review, the researchers argue that making data-based personnel decisions is in the public interest. “These decisions impact not just the scholars’ careers but the funding of universities and the overall strength of scientific research in private and public organizations as well,” they say. “On an individual level, a tenured faculty member at a prestigious university will receive millions of dollars in career compensation. At a broader scope, these faculty will bring funding into the universities that house them. The National Science Foundation, for instance, provided $5.8 billion in research funding in 2014, including $220 million specifically for young researchers at top universities.”
Bringing predictive analytics to any new industry means “identifying metrics that often have not received a lot of focus” and “how those metrics correlate with a measurable definition of success,” they note. In the case of academics, they say, important metrics are not just related to the impact of scholars’ past research but also details about their research partnerships and how their research complements the existing literature.
Erik Brynjolfsson (MIY) & John Silberholz (MIT), ‘Moneyball’ for Professors?:
Using analytics to improve hiring decisions has transformed industries from baseball to investment banking. So why are tenure decisions for professors still made the old-fashioned way?
How can we use information about an employee’s current performance to predict future performance? That’s one of the key questions addressed by the ever-growing use of predictive analytics as a human resources (HR) tool. The application of predictive analytics to the management of baseball teams — brought to life in the book (2003) and then movie (2011) “Moneyball” — made vivid the ways that data-based modeling can be used for more accurate talent acquisition and deployment. The idea that metrics could guide strategy by supporting intuitive decision making has created a boom in the use of predictive analytics in the HR industry.
Ironically, one of the places where predictive analytics hasn’t yet made substantial inroads is in the place of its birth: the halls of academia. Tenure decisions for the scholars of computer science, economics, and statistics — the very pioneers of quantitative metrics and predictive analytics — are often insulated from these tools.
That may soon change. A study we conducted with Dimitris Bertsimas and Shachar Reichman, published in Operations Research (Tenure Analytics: Models for Predicting Research Impact), finds that data-driven models can significantly improve decisions for academic and financial committees. In fact, the scholars recommended for tenure by our model had better future research records, on average, than those who were actually granted tenure by the tenure committees at top institutions. ...
[A]cademic decision-making processes rely mainly on subjective assessments of candidates. We believe, though, that if analytics is given the opportunity to complement the tenure decision-making process by offering improved predictions about candidates’ future performance and scholarly research, businesses and the public will be better served by the academic community. Given the stakes, we think it is time for a “Moneyball moment” in academia.
For my perspective, see What Law Schools Can Learn from Billy Beane and the Oakland Athletics, 82 Texas L. Rev. 1483 (2004). Prior TaxProf Blog coverage:
- MIT: Moneyball and Faculty Rankings (Jan. 21, 2014)
- Moneyball, Volleyball, and Faculty Productivity (Dec. 9, 2009)
- Paging Billy Beane: Scholarly Productivity Lowers Reputation But Raises Salary (Nov. 30, 2009)
- Bill Belichick, Moneyball Savant? (Nov. 17, 2009)
- Somin: Can Moneyball Strategies Still Work for Law Schools? (July 29, 2009)
- Livingston: Rahm Emanuel and the Future of Law Schools (July 27, 2009)
- Who Is the Shane Battier of Your Faculty? (Feb. 15, 2009)
- Henderson: Law Professor Free Agency and "School-Specific" Capital (Aug. 14, 2008)
- The Nadir of Rankings Puffery: Quinnipiac's WSJ Ad (June 4, 2008)
- Is Moneyball Passé? (Apr. 22, 2008)
- Individualized Law School Rankings (Mar. 10, 2008)
- Advice for Erwin Chemerinsky from Caron & Henderson (Sept. 24, 2007)
- Erwin Chemerinsky as Billy Beane? (Sept. 19, 2007)
- Associate Moneyball (Aug. 7, 2007)
- Provost Billy Beane (June 19, 2007)
- A Moneyball Guide to Law Schools (June 8, 2007)
- Moneyball Redux: The Tax Prof Exchange (Apr. 22, 2007)
- Should Deans Have Their U.S. News Cake and Eat It Too? (Apr. 2, 2007)
- Applying Moneyball Principles to Law Schools (Feb. 23, 2007)
- The First Moneyball Law School? (Dec. 14, 2006)
- What Constitutes Law School "Success"? (Nov. 15, 2006)
- Application of Moneyball Principles in Faculty Hiring (Sept. 28, 2006)
- Moneyball and Faculty Hiring (Sept. 21, 2006)
- Ranking Law Professors and Heart Surgeons (July 6, 2006)
- Top B-School Deans Are Top Scholars (Apr. 26, 2006)
- Lessons from the U.S. News Rankings: Are Law Schools Overinvesting in Scholarship? (Apr. 5, 2006)
- National Review on George Mason Law School (Feb. 28, 2006)
- New Student Ranking of Law Schools (Nov. 7, 2005)
- Tax Prof Fantasy League (Nov. 4, 2005)
- Top University Presidents Are Top Scholars (Oct. 21, 2005)
- The Faculty Salary Game (Sept. 19, 2005)
- Law Faculty Hiring: Pedigree or Performance? (Sept. 15, 2005)
(Hat Tip: Michael Helfand.)