Wednesday, July 31, 2013
The $1 Million Value of a Law Degree and the Limits of Empirical Studies
The Legal Watchdog: The Limits of Empirical Studies, by Michael D. Cicchini:
[A]s a rule, I’m not a fan of empirical studies, and the recent “value of a law degree” debate shows why. ...
The bigger picture, however, is this: the words “empirical study” give the impression (at least to me) that the authors will “prove” something. Therefore, empirical studies should be more reliable than anecdotes, and should be more useful than other tools such as logic, analogies, and even economic-based reasoning. And I think that in some cases, they are. ...
But in other instances, such as the “value of a law degree” debate, empirical studies prove little, and merely result in near-endless back-and-forth where each side of the debate preaches to their respective choir. No one will be persuaded. No one will switch camps. Everyone will cite the study, or the criticisms of the study, in support of the conclusion they’ve already reached. ...
Not that any of this is bad. It's just that, even though we tend to perceive them as more, empirical studies are often just arguments that cloud the underlying issue with statistical complexity and the illusion of certainty, thereby creating dozens of sub-debates and detracting from the real issue. ...
Further, for purposes of argument, these empirical studies can actually be far less persuasive than other tools, including the simple anecdote. But, admittedly, anecdotes, analogies, economic reasoning, and even logic aren’t always ideal for every type of debate. And I just haven’t figured out when empirical studies are helpful and when they’re worse than useless. Does it depend on the topic? On the empirical study’s level of complexity? On the number of assumptions built into the model? I just don’t know.
Prior TaxProf Blog coverage:
- What Is the Economic Value of a Law Degree -- $1 Million? (July 17, 2013)
- More on The $1 Million Value of a Law Degree (July 18, 2013)
- NY Times, WaPo, Others Debate The $1 Million Value of a Law Degree (July 18, 2013)
- What Is the Economic Value of a Law Degree -- $1 Million or $100,000? (July 19, 2013)
- More on The $1 Million Value of a Law Degree (July 20, 2013)
- Merritt on The $1 Million Value of a Law Degree (July 21, 2013)
- Diamond on The $1 Million Value of a Law Degree (July 22, 2013)
- Tamanaha: How 'The Million Dollar Law Degree' Study Systematically Overstates Value (July 23, 2013)
- Tamanaha: How the 'Million Dollar Law Degree' Study Understates Risk (Part I) (July 24, 2013)
- Tamanaha: Why the “Million Dollar Law Degree” Study Fails (Final Post) (July 25, 2013)
- Rasmusen: Critics of The Economic Value of a Law Degree Are Making the Paper Better (July 25, 2013)
- Pasquale and Simkovic Respond to Tamanaha (July 25, 2013)
- The American Lawyer: Paper on Law Degree's Economic Value a Non-Sequitur (July 26, 2013)
- Simkovic Responds to Tamanaha (Part 3) (July 26, 2013)
- Tamanaha: Short Term Versus Long Term Perspective (July 27, 2013)
- Harper, Diamond: The $1 Million Value of a Law Degree: Distraction, Astronomy, or Astrology? (July 28, 2013)
- Simkovic Responds to American Lawyer Op-Eds (July 29, 2013)
- Diamond Responds to Tamanaha (July 29, 2013)
- Rasmusen: The Economic Value of a Law Degree and the 'Typical' Law Student (July 29, 2013)
- Simkovic Responds to Tamanaha (Part 4) (July 30, 2013)
- Campos on The Economic Value of a Law Degree: The Problem With Back in the Day (July 30, 2013)
- Leiter's Reflections on The Economic Value of a Law Degree (July 30, 2013)
- Tamanaha Responds to Leiter (and Leiter Replies) (July 31, 2013)
I think there are two answers to this. First, when I used to crunch numbers for corporations, I used to set up models so that underlying assumptions could be changed quite easily, and then we could see how it rippled through future projections. That way, you could do side-by-side comparisons. I'm sure something like that could be done here to replace the rigidity of the models and their claims. But second, former attorney Jeff Winger has a better approach for these profs: “I could cite quotes or dig up statistics, but those are just words and numbers. I think we could have a little more fun, if I expressed myself in song.” Wouldn't you like to hear Steve Diamond singing a dramatic performance about the value of law school? That would be tops on my list.
Posted by: Michael Cicchini | Aug 1, 2013 8:13:11 AM
So -- what -- we're going with straightforward Humean anti-empiricism? Perhaps we should try a priori reasoning instead -- surely that will comport with reality! We either attempt to use existing evidence to determine what will happen (empiricism) or we're reading chicken entrails.
Posted by: anon | Aug 1, 2013 6:59:02 AM
Cichinni hits the nail about the way empirical studies can be twisted to fit any viewpoint. For example, consider Leiter's recent claim on this blog that critics of the million-dollar-law-degree study don't grasp present value concepts of valuation. Present value analysis is useful but far too malleable to furnish persuasive evidence of the value of a law degree, especially when the million dollar value apparently rests on assumptions regarding income decades in the future.
Posted by: Jake | Jul 31, 2013 1:50:50 PM
"We either attempt to use existing evidence"
Like the employment and salary results for the classes of 2008 through 2013?
Posted by: Unemployed_Northeastern | Aug 1, 2013 2:31:19 PM