I haven’t been surprised by the extensive discussion of the recent paper by Michael Simkovic and Frank McIntyre.
The paper deserves attention from many readers. I have been surprised,
however, by the number of scholars who endorse the paper–and even scorn
skeptics–while acknowledging that they don’t understand the methods
underlying Simkovic and McIntyre’s results. An empirical paper is only
as good as its method; it’s essential for scholars to engage with that
I’ll discuss one methodological issue here: the small sample sizes
underlying some of Simkovic and McIntyre’s results. Those sample sizes
undercut the strength of some claims that Simkovic and McIntyre make in
the current draft of the paper. ...
The sample of JD graduates ... is much smaller. Those totals range
from 282 to 409 for the four panels, yielding a total of 1,342 lawyers.
That’s still a substantial sample size, but Simkovic and McIntyre need
to examine subsets of the sample to support their analyses. To chart
changes in the financial premium generated by a law degree, for example,
they need to examine reported incomes for each of the sixteen years in
the sample. Those small groupings generate the uncertainty I discuss
As a professor who has taught Law and Social Science, I think the
critics of the Simkovic/McIntyre paper raised many good questions.
Empirical analyses need testing, and it is especially important to
examine the assumptions that lie behind a quantitative study. ...
I hope to write a bit more on the Simkovic and McIntyre paper; there
are more questions to raise about their conclusions. I may also try to
offer some summaries of other research that has been done on the career
paths of law school graduates and lawyers. We don’t have nearly enough
research in the field, but there are some other studies worth knowing.