TaxProf Blog

Editor: Paul L. Caron, Dean
Pepperdine University School of Law

Friday, December 2, 2011

Shultz & Zedeck: Predicting Lawyer Effectiveness -- A New Assessment for Use in Law School Admission Decisions

Marjorie M. Shultz (UC-Berkeley, Law) & Sheldon Zedeck (Vice Provost, UC-Berkeley), Predicting Lawyer Effectiveness: A New Assessment for Use in Law School Admission Decisions, 36 Law & Soc. Inquiry 620 (2011):

The problem of how to define merit and achieve equity in high stakes selection is deeply contested in the many arenas of our society where scarce resources and privileges are allocated. Law school admission is one such arena. The Law School Admission Test (LSAT) dominates current admission decisions for law school, but it assesses only a few competencies and has a large adverse impact on minority applicants. Moreover, the LSAT and its associated Index Score focus on law school performance and are not strongly predictive of lawyer performance.

Innovative exploratory research by two UC Berkeley faculty (Marjorie Shultz, Law and Sheldon Zedeck, Psychology) has demonstrated that on-the-job professional effectiveness of lawyers can be predicted. The new Shultz-Zedeck tests, developed based on models from employment selection and promotion and the field of industrial psychology, identify and assess many factors not measured by the LSAT that are vital to lawyer efficacy, such as problem solving, advocacy, practical judgment, and communication skills. Exploratory research conducted with participation of more than 5000 law grads suggests that tests can be developed and validated that will predict professional performance (as appraised by peers and supervisors). Equally important, performance on these new tests shows no significant race, ethnic or gender differences. This research has the potential to diversify the pool of students admitted to law schools and to do so on the basis of a principled definition of merit rather than on the basis of more politically and legally controversial methods of race-conscious affirmative action. The next step is a national validation study to ready this new approach to law school admission for adoption by the nation’s law schools.

http://taxprof.typepad.com/taxprof_blog/2011/12/shultz-zedeck.html

Legal Education, Scholarship | Permalink

TrackBack URL for this entry:

http://www.typepad.com/services/trackback/6a00d8341c4eab53ef015393adc698970b

Listed below are links to weblogs that reference Shultz & Zedeck: Predicting Lawyer Effectiveness -- A New Assessment for Use in Law School Admission Decisions:

Comments

Note the sample: "Exploratory research conducted with participation of more than 5000 law grads..." All of the sample subjects already had done well enough on the LSAT and college grades to be admitted to law school, and did well enough once they were there to graduate. People who could not reason well from the kinds of information you work with in law school had long since been stripped out from the potential sample. It is the purpose of the LSAT to give information about this particularly important aspect--of many--that will be required. Even if the new tests are useful for predicting success in the job AFTER success in law school, this does not mean they can be substituted for the LSAT.

The appropriate test would be a sample of 5000 students who would like to apply to law school: Have them take the LSAT but don't use it for admittance; accept them all, give them the new test too, and see who succeeds in law school training and then becomes successful in the profession. Then see how well success (both in law school and ultimately as a lawyer if you graduate) is predicted by (a) LSAT only, (b) new test only, (c) LSAT + new test.

Marjorie Shultz and Sheldon Zedeck have not done very well in this practical reasoning exercise.

Posted by: Charlie | Dec 3, 2011 6:39:45 PM

the left will take over administration of this, then adjudge conservatism as a legal profession disqualifying psychiatric disorder.

Posted by: apetra | Dec 3, 2011 6:49:12 PM

"...has a large adverse impact on minority applicants..." isn't the term that has been used for years now "disparate impact".

Of course it has an adverse impact on ANYONE who doesn't score well. The test can't tell the color of anyone taking it. Blacks will score lower on any test measuring "Q" or intelligence.

It really isn't important if we have 14% black lawyers in the legal pool. Fewer people are going to hire black lawyers when their lives or money is being contested. Why? Most black lawyers are NOT as effective or capable as non-black lawyers.

Notice I said most, not all. One of these days we'll stop playing games that claim we are all equal in mental or physical ability. When will this happen? Never in our lifetime.

We will continue to pretend that blacks should share equally in being lawyers, doctors, engineers, physicists, chemists, CEOs et al. Sorry, all of us are NOT CREATED EQUAL except in the eyes of God and the Declaration of Independence.

The Declaration of Independence states that we also are entitled to "Life, Liberty and the Pursuit of Happiness" NOT to Equality of Outcomes.

There is no special exception for ANYONE.

Posted by: jgreene | Dec 3, 2011 7:11:27 PM

Why not use an auction?
0% tuition discount (all forms paid by the University)
5% tuition discount (all forms paid by the University)
10% tuition discount (all forms paid by the University)
etc. until the seats are full (plus 10% for shrinkage).
The student can get the tuition money from anyone, anyhow,
on any terms, just not from the po' ol' University.

Posted by: Law Prof | Dec 3, 2011 7:15:43 PM

No matter how you slice it it is still balogna. OK, let us see how this works. Just use this method of selection and see how many of the selected individuals can get through the filter of a typical bar exam. Let us see how all this "problem solving, advocacy, practical judgment, and communication skills." business works out.

Posted by: Griggs v. DukePower | Dec 3, 2011 7:23:38 PM

I was skeptical about this at first, but then I saw that studies have been done, so now I'm a believer.

True, the studies were merely exploratory research that suggests that tests can be developed that will correlate very well with professional performance and hardly at all with sex, race or ethnicity. This would only be possible, of course, if professional performance correlates hardly at all with sex, race or ethnicity, but we know this is the case. Next, we should similarly reform our criminal justice system, since we know that criminal performance correlates hardly at all with sex, race or ethnicity.

Posted by: LTEC | Dec 3, 2011 7:38:25 PM

But politically-correct left/liberal Pyschology types will simply develop a test that gives high scores for a "social justice" disposition along with everything else and, presto, only folk who think the "right" (i.e., Left) way will be qualified to get into Law school because, see, there's an "objective test" that said so.

Posted by: Anonymous | Dec 3, 2011 8:23:31 PM

1. Hooray! Berkeley academics have concocted, um, discovered a nondiscriminatory screening exam.

2. ...Equally important, performance on these new tests shows no significant race, ethnic or gender differences. This research has the potential to diversify the pool of students admitted to law schools and to do so on the basis of a principled definition of merit rather than on the basis of more politically and legally controversial methods of race-conscious affirmative action. The next step is a national validation study to ready this new approach to law school admission for adoption by the nation’s law schools.

Note the tone of scientific objectivity and dispassionate inquiry. No preconceived aganda here!

3. There are major political forces behind "diversity". Mirabile dictu, this research, as summarized in the post, tells those forces exactly what they want to hear.

4. Despite my blatant skepticism above, I concede the results might be valid.

5. My skepticism is reinforced by the experience of working in the military-industrial complex when it was performing "objective analysis" of Reagan's proposals for missile defense.

Posted by: gs | Dec 3, 2011 8:37:44 PM

There are several reasons to be very leery of this idea, if not outright terrified, not least of which is the fact it is advocated by UC-Berkeley professors. Experience at a minimum tells us such advocates are rarely friends of the theories of liberty, law and governance as championed by our Founding Fathers.

But there are other reasons. One is related to the same reason I think that, by and large, jury consultants do little more than sell us the emperor's new clothes: this almost religious faith in determinism. Put another way, they don't believe in either free will or individuality. Just as jury consultants believe we are all slaves to our attitudes and circumstances, and are incapable of following the law or unwilling to abide by a judge's instructions, Shultz and Zedeck seem to suggest that psychological traits, not intellectual capacity are the best predictors of future "success" (which I notice is not defined) in law.

Also, measuring for psychological traits is not as readily quantifiable as intelligence. They're famously "mushy", and capable of being massaged into virtually any form the sculptor desires, which is I am sure much of this system's allure for the "deciders".

Such mushy tests are also famously easy to abuse. For example, who would say that "caring" is not important for a lawyer? (Now I suspect many would, and a very good argument can be made supporting that attitude, but few in this day and age would ever publicly say they didn't simply ooze care and concern.) Then the lords of the new Emo-LSAT will just slip a "caring" section in to determine that vital legal characteristic. And who would deny that political/cultural liberals typically show more concern than conservatives? And because we know being rational is such a vital trait for lawyers, shouldn't we examine whether applicants are "rational" atheists or "irrational" persons of faith? You see where this goes - it would be child's play to build in bias which rewards the "correct" social, political or cultural attitudes.

This proposal is just affirmative action in drag, gussied up as a "test"

Posted by: Sardondi | Dec 3, 2011 9:22:11 PM

"...the field of industrial psychology..."

I couldn't finish. Laughing too hard.

Posted by: Hucbald | Dec 3, 2011 9:43:03 PM

"but it assesses only a few competencies and has a large adverse impact on minority applicants. Moreover, the LSAT and its associated Index Score focus on law school performance and are not strongly predictive of lawyer performance.

There is a good reason to focus on law school performance. Law school is what the applicant is applying for admission to. Applicants are not applying for admission to practice law. If they cannot pass law school they will not practice law no matter what their psychological profile.

Posted by: willis | Dec 4, 2011 4:02:08 AM

Hate to say this, but psychological tests are of incredibly dubious value once they're known to have high stakes attached to them. Once the test takers figure this out, they'll game the test. The LSAT is fairly hard to game, these tests won't be.

Posted by: Jehu | Dec 4, 2011 2:09:56 PM

Honestly, it's hard to believe this is possible and one wonders what kind of chicanery is behind it. Clearly such a test would not be based on objective and verifiable competencies or knowledge, as pretty much every objective test-based assessment has the unfortunate side effect of having disparate impact on some groups (since human population groups are not interchangeable).

Posted by: Unbelievable | Dec 4, 2011 6:07:04 PM