Wednesday, July 31, 2013
[A]s a rule, I’m not a fan of empirical studies, and the recent “value of a law degree” debate shows why. ...
The bigger picture, however, is this: the words “empirical study” give the impression (at least to me) that the authors will “prove” something. Therefore, empirical studies should be more reliable than anecdotes, and should be more useful than other tools such as logic, analogies, and even economic-based reasoning. And I think that in some cases, they are. ...
But in other instances, such as the “value of a law degree” debate, empirical studies prove little, and merely result in near-endless back-and-forth where each side of the debate preaches to their respective choir. No one will be persuaded. No one will switch camps. Everyone will cite the study, or the criticisms of the study, in support of the conclusion they’ve already reached. ...
Not that any of this is bad. It's just that, even though we tend to perceive them as more, empirical studies are often just arguments that cloud the underlying issue with statistical complexity and the illusion of certainty, thereby creating dozens of sub-debates and detracting from the real issue. ...
Further, for purposes of argument, these empirical studies can actually be far less persuasive than other tools, including the simple anecdote. But, admittedly, anecdotes, analogies, economic reasoning, and even logic aren’t always ideal for every type of debate. And I just haven’t figured out when empirical studies are helpful and when they’re worse than useless. Does it depend on the topic? On the empirical study’s level of complexity? On the number of assumptions built into the model? I just don’t know.
Prior TaxProf Blog coverage:
- What Is the Economic Value of a Law Degree -- $1 Million? (July 17, 2013)
- More on The $1 Million Value of a Law Degree (July 18, 2013)
- NY Times, WaPo, Others Debate The $1 Million Value of a Law Degree (July 18, 2013)
- What Is the Economic Value of a Law Degree -- $1 Million or $100,000? (July 19, 2013)
- More on The $1 Million Value of a Law Degree (July 20, 2013)
- Merritt on The $1 Million Value of a Law Degree (July 21, 2013)
- Diamond on The $1 Million Value of a Law Degree (July 22, 2013)
- Tamanaha: How 'The Million Dollar Law Degree' Study Systematically Overstates Value (July 23, 2013)
- Tamanaha: How the 'Million Dollar Law Degree' Study Understates Risk (Part I) (July 24, 2013)
- Tamanaha: Why the “Million Dollar Law Degree” Study Fails (Final Post) (July 25, 2013)
- Rasmusen: Critics of The Economic Value of a Law Degree Are Making the Paper Better (July 25, 2013)
- Pasquale and Simkovic Respond to Tamanaha (July 25, 2013)
- The American Lawyer: Paper on Law Degree's Economic Value a Non-Sequitur (July 26, 2013)
- Simkovic Responds to Tamanaha (Part 3) (July 26, 2013)
- Tamanaha: Short Term Versus Long Term Perspective (July 27, 2013)
- Harper, Diamond: The $1 Million Value of a Law Degree: Distraction, Astronomy, or Astrology? (July 28, 2013)
- Simkovic Responds to American Lawyer Op-Eds (July 29, 2013)
- Diamond Responds to Tamanaha (July 29, 2013)
- Rasmusen: The Economic Value of a Law Degree and the 'Typical' Law Student (July 29, 2013)
- Simkovic Responds to Tamanaha (Part 4) (July 30, 2013)
- Campos on The Economic Value of a Law Degree: The Problem With Back in the Day (July 30, 2013)
- Leiter's Reflections on The Economic Value of a Law Degree (July 30, 2013)
- Tamanaha Responds to Leiter (and Leiter Replies) (July 31, 2013)