TaxProf Blog

Paul L. Caron
Dean





Wednesday, July 31, 2013

The $1 Million Value of a Law Degree and the Limits of Empirical Studies

The Legal Watchdog:  The Limits of Empirical Studies, by Michael D. Cicchini:

[A]s a rule, I’m not a fan of empirical studies, and the recent “value of a law degree” debate shows why. ...

The bigger picture, however, is this: the words “empirical study” give the impression (at least to me) that the authors will “prove” something. Therefore, empirical studies should be more reliable than anecdotes, and should be more useful than other tools such as logic, analogies, and even economic-based reasoning.  And I think that in some cases, they are. ...

But in other instances, such as the “value of a law degree” debate, empirical studies prove little, and merely result in near-endless back-and-forth where each side of the debate preaches to their respective choir. No one will be persuaded. No one will switch camps. Everyone will cite the study, or the criticisms of the study, in support of the conclusion they’ve already reached. ...

Not that any of this is bad. It's just that, even though we tend to perceive them as more, empirical studies are often just arguments that cloud the underlying issue with statistical complexity and the illusion of certainty, thereby creating dozens of sub-debates and detracting from the real issue. ...

Further, for purposes of argument, these empirical studies can actually be far less persuasive than other tools, including the simple anecdote. But, admittedly, anecdotes, analogies, economic reasoning, and even logic aren’t always ideal for every type of debate. And I just haven’t figured out when empirical studies are helpful and when they’re worse than useless. Does it depend on the topic? On the empirical study’s level of complexity? On the number of assumptions built into the model? I just don’t know.

Prior TaxProf Blog coverage:

https://taxprof.typepad.com/taxprof_blog/2013/07/the-1-million-1.html

Legal Education | Permalink

TrackBack URL for this entry:

https://www.typepad.com/services/trackback/6a00d8341c4eab53ef01910475444a970c

Listed below are links to weblogs that reference The $1 Million Value of a Law Degree and the Limits of Empirical Studies:

Comments

"We either attempt to use existing evidence"

Like the employment and salary results for the classes of 2008 through 2013?

Posted by: Unemployed_Northeastern | Aug 1, 2013 2:31:19 PM

I think there are two answers to this. First, when I used to crunch numbers for corporations, I used to set up models so that underlying assumptions could be changed quite easily, and then we could see how it rippled through future projections. That way, you could do side-by-side comparisons. I'm sure something like that could be done here to replace the rigidity of the models and their claims. But second, former attorney Jeff Winger has a better approach for these profs: “I could cite quotes or dig up statistics, but those are just words and numbers. I think we could have a little more fun, if I expressed myself in song.” Wouldn't you like to hear Steve Diamond singing a dramatic performance about the value of law school? That would be tops on my list.

Posted by: Michael Cicchini | Aug 1, 2013 8:13:11 AM

So -- what -- we're going with straightforward Humean anti-empiricism? Perhaps we should try a priori reasoning instead -- surely that will comport with reality! We either attempt to use existing evidence to determine what will happen (empiricism) or we're reading chicken entrails.

Posted by: anon | Aug 1, 2013 6:59:02 AM

Cichinni hits the nail about the way empirical studies can be twisted to fit any viewpoint. For example, consider Leiter's recent claim on this blog that critics of the million-dollar-law-degree study don't grasp present value concepts of valuation. Present value analysis is useful but far too malleable to furnish persuasive evidence of the value of a law degree, especially when the million dollar value apparently rests on assumptions regarding income decades in the future.

Posted by: Jake | Jul 31, 2013 1:50:50 PM