Paul L. Caron

Tuesday, April 11, 2017

Henderson:  U.S. News Eliminates The Rankings Advantage Of The GRE, But Harvard Has Started A 'Quant' Arms Race For Diverse Students Who Will Thrive In A Transformed, Tech-Driven, Disrupted Legal Profession

GRE.USNEWS.LSATThe Legal Whiteboard: The GRE and the Revised US News Ranking Methodology, by William Henderson (Indiana):

When I initially learned that Harvard Law would start accepting the GRE as an alternative to the LSAT, I viewed it through the prism of the US News & World Report ranking and concluded that it was a very good thing for Harvard and all of legal education. Aggressive rankings management has led to tremendous over-reliance on the LSAT. By using on the GRE, I reasoned, Harvard would have sufficient test score information to assess a candidate's intellectual capacity while also obtaining the freedom to use other admissions methods to explore the larger and more diverse universe of candidates who are destined to become great leaders and lawyers.  

My thinking is crudely sketched out in the diagram below.


If Harvard Law was trying to get around U.S. News rankings formula, the USN chief strategy officer, Bob Morse, saw it coming.  ...

[W]hen US News released its 2018 law school rankings, something that got zero attention was a very significant change in the USN rankings methodology. Here is the description of the selectivity factor that was previously limited to the LSAT:

Median LSAT and GRE scores (0.125): These are the combined median scores on the Law School Admission Test of all 2016 full-time and part-time entrants to the J.D. program. For the first time, U.S. News used median GRE scores in combination with LSAT scores for this indicator if they were reported for a law school's 2016 entering class. The University of Arizona was the only school that reported both LSAT and GRE scores to U.S. News for its 2016 entering class.

This is the same methodology that US News uses for colleges that admit based on both the ACT and SAT.  USN converts the medians to their percentile equivalents and weights them based on their proportion in the total enrolled student population. This year, only the University of Arizona had such a combined median. ...

[T]his methodology change means that the GRE strategy won't be an effective way to break free of the ranking's powerful influence on admissions decisions.  Which led me to ask, "Is Harvard Law's move on the GRE still a game-changer for legal education?"

I think the answer is yes, but for reasons that are not as obvious as potential bump is a school's USN ranking.  The contours of a potentially powerful GRE strategy are laid out in an article I wrote for, Underestimate Harvard Law's New Admissions Strategy at Your Own Risk[:]

I see the contours of a strategy where Harvard would pull away from the rest of legal education, including Stanford Law School and Yale Law School.

This strategy is grounded in history. Because of the 100 percent adoption of the case method as the basis for instruction starting nearly a century ago, an innovation pioneered by Harvard dean Christopher Columbus Langdell, it is not an exaggeration to say that all of legal education is built on the Harvard model. Someday, this paradigm will be replaced by something else. When it occurs, the only way Harvard retains its hallowed position is by leading its own demolition crew. The leadership challenge is in picking the right set of ideas and the right time. Yet, Harvard’s position is unique, as the school historically at the forefront of the teaching method, because other law schools will be nervous about hanging on to a very old model while the original architect is clearly moving on.

Students and employers decide when this shift will occur, not the law schools. And the strategy analysis is not complicated. To be the undisputed market leader, a law school must dominate in two categories: enrolling the best students and teaching the most relevant curriculum. By pulling Lever 1 (students), Harvard Law will eventually be ideally situated to pull Lever 2 (curriculum). ...

In legal education, the most widely accepted proxy for intelligence is the LSAT. For students admitted in the fall of 2015, the median LSAT scores at the three highest-ranked law schools in the U.S. News & World Report rankings were 173 (Yale), 172 (Harvard) and 171 (Stanford), all above 98th percentile of all test-takers and thus not meaningfully different from one another.

Yet, the LSAT is what psychometricians call a univariate test. It measures only one dimension—in this case, verbal reasoning ability. If you are a prospective employer trying to build an organization in the age of Big Data, however, this single measure is likely to be inadequate. Instead, you want a workforce that is equally able in quantitative reasoning. For Harvard Law School, that is a very compelling reason to start using the GRE, as the GRE reports separate scores for verbal and quantitative ability, each on a standardized scale of 130 to 170.

Imagine the reaction of employers when Harvard Law boasts that 200 of its entering students—more than the entire entering classes of Yale and Stanford—scored in the 98th percentile on the quantitative section of the GRE. Between July 1, 2015, and June 30, 2016, over 584,000 candidates took the GRE compared to 105,000 for the LSAT. Harvard’s move could easily touch off a “quant” arms race among the elite national law schools, as the subset of law school applicants who possess relatively high quantitative ability — and thus aren’t going to law school to avoid math — see a way to separate themselves from the pack. ...

In addition to measuring both verbal and quantitative reasoning, the GRE has a remarkably diverse test-takers pool. Among U.S. citizens who took the GRE between July 2015 and June 2016, 26,000 were Hispanic, 24,000 were black, and 19,000 were Asian—numbers three to five times larger than the LSAT test-taker population. The GRE pool is also very international. Among the 99,500 test-takers who took the GRE in India, over 50,000 expressed a preference to pursue their graduate studies in the United States. ...

In the individual consumer market, such companies as LegalZoom, Avvo, Modria and others are finding ways to engineer costs and inconvenience out of a wide array of common legal problems. Likewise, in the corporate legal services space, such growing fields as legal operations and legal managed services are assembling teams of lawyers and allied professionals to figure out ways to drive up quality while driving down costs and cycle time.

At some point in the future — and I doubt it is too far off — legal education will begin to acknowledge this paradigm shift. Law schools will teach data, process and technology along with doctrinal law. Students will use design-thinking to evaluate various parts of a legal system and suggest improvements that could form the basis for a business. Diversity will be expanded to include perspectives from multiple countries. Virtually all law school grads will consider themselves legally trained professionals, but only a subset will aspire to be lawyers who provide one-on-one consultative legal services to a client. ...

If Harvard Law’s leadership wanted to set this plan in motion, a logical first step would be to expand the admissions policy to include GRE scores, as the GRE provides more information, a bigger talent pool, and access to global diversity. The benefits of the Harvard brand will likely attract additional exceptional students. In turn, Harvard’s large class size (550+ per year), deep pockets, large faculty and committed alumni base will give rise to curricular experimentation that other elite law schools could not easily match without placing a much larger bet.

Within a year or two, employers will be clamoring for other law schools to adopt the new Harvard Law model. When that happens, the rest of legal education will dutifully fall in line.

Law School Rankings, Legal Education | Permalink


I think the motivation behind the GRE ploy is a revenue one, rather than a rankings advantage one.

The pool of bodies available within a particular school's desired LSAT percentile range is small and finite these days. What the GRE does, in theory is expand the pool of tuition paying bodies within the same acceptable LSAT percentile range thereby adding revenue without adversely affecting the test score medians for rankings purposes. That seems to have been the U. of Arizona plan, anyway. Harvard may have different motives.

Schools knew GRE percentiles would be easily translated into the same percentile conversion methodology that U.S. News has used historically for LSATs.

The interesting question is how the GRE folks at a particular percentile level will compare against LSAT takers in the same percentile level in terms of law school grades, attrition, and more importantly, bar passage. Stay tuned.

Posted by: Anon | Apr 11, 2017 12:11:01 PM

Lot of ad hoc reasoning and analysis going on here for what is clearly an admissions scam aimed at providing justification for lowering standards without actually taking a rankings hit. For lower ranking schools, the goal will be simply to admit as many low iq students as possible but hide the stats of what is happe ing. At Harvard and Yale, it will be obfuscate the massive mismatch that clearly transpires at their schools.

Posted by: Bobby | Apr 11, 2017 9:13:15 AM

It’s a shame that law schools will not innovate until Harvard leads the way. Over 10% of the class of 2015 was unemployed 9 months after graduation. Many graduates turned to short term, part time, and non-professional jobs. Low ranking schools had significantly worse employment outcomes (for instance, 28% of Cooley grads were unemployed). With the economy at full employment, it is clear that employers do not value the skills taught by law schools. Moreover, bar passage rates have plummeted across the country because law schools admitted less qualified students to maintain enrollment.

Rather than innovate to meet the needs of employers, law schools innovate to maximize profit (conditional scholarships with section stacking) and avoid ABA sanction (coerce students to forgo taking the bar). You would think that law schools with low bar passage rates would hire PhDs in education to craft a curriculum to get their students to pass the bar. You would expect these law schools to meet with employers and find out what skills are in demand.

Instead, the schools blame others for low bar passage rates (of course the school is never at fault). Academics make faulty claims that the BLS reports another banner year for the legal profession, while failing to mention the BLS foresees strong competition for jobs because there are more grads than jobs available.

Posted by: anon JD/MD | Apr 11, 2017 9:07:58 AM