Paul L. Caron

Saturday, August 3, 2013

Merritt: Small Sample Sizes Undermine Claims of 'The Economic Value of a Law Degree'

Deborah Jones Merritt (Ohio State), Small Samples:

I haven’t been surprised by the extensive discussion of the recent paper by Michael Simkovic and Frank McIntyre. The paper deserves attention from many readers. I have been surprised, however, by the number of scholars who endorse the paper–and even scorn skeptics–while acknowledging that they don’t understand the methods underlying Simkovic and McIntyre’s results. An empirical paper is only as good as its method; it’s essential for scholars to engage with that method.

I’ll discuss one methodological issue here: the small sample sizes underlying some of Simkovic and McIntyre’s results. Those sample sizes undercut the strength of some claims that Simkovic and McIntyre make in the current draft of the paper. ...

The sample of JD graduates ... is much smaller. Those totals range from 282 to 409 for the four panels, yielding a total of 1,342 lawyers. That’s still a substantial sample size, but Simkovic and McIntyre need to examine subsets of the sample to support their analyses. To chart changes in the financial premium generated by a law degree, for example, they need to examine reported incomes for each of the sixteen years in the sample. Those small groupings generate the uncertainty I discuss here. ...

As a professor who has taught Law and Social Science, I think the critics of the Simkovic/McIntyre paper raised many good questions. Empirical analyses need testing, and it is especially important to examine the assumptions that lie behind a quantitative study. ...

I hope to write a bit more on the Simkovic and McIntyre paper; there are more questions to raise about their conclusions. I may also try to offer some summaries of other research that has been done on the career paths of law school graduates and lawyers. We don’t have nearly enough research in the field, but there are some other studies worth knowing.

Prior TaxProf Blog coverage:

Legal Education | Permalink

TrackBack URL for this entry:

Listed below are links to weblogs that reference Merritt: Small Sample Sizes Undermine Claims of 'The Economic Value of a Law Degree' :