Without Any Transparency In The Process, ABA Legal Ed Council Approves Changes To Employment Report And Classification Of Law-School-Funded Positions That Erode Transparency

At its June 1-2 meeting, the ABA Council for the Section of Legal Education and Admissions to the Bar approved a proposal to completely eviscerate the steps it approved in 2015 to assure greater transparency in reporting law-school-funded positions.  Indeed, the Council went even further, changing the rules to make it impossible for anyone to discover what number/percentage of a law school’s graduates are in law-school-funded positions, so long as those positions pay $40,000. 

The Council did this with no notice, no chance for comment, and no presentation of possible concerns associated with this change.  Rather, it simply approved a proposal purporting to simplify reporting of employment outcomes that was submitted by one Council member, Paul Mahoney, whose law school was among several that would benefit from the reclassification of law-school-funded positions. 

More significantly, in approving the proposal, the Council also approved several other changes in reporting of employment outcomes that merit much more discussion.  These changes, discussed below, were not meaningfully discussed in the proposal, nor do they appear to have been meaningfully discussed by the Council in approving the proposal.  Once again, there was no notice of these changes, no chance for comment, and no presentation of possible concerns associated with these changes.

It pains me to write this, as I hold the members of the Council in high regard and believe the Council has done a very good job over the last several years navigating legal education through uncharted waters, particularly with its emphasis on increased transparency regarding conditional scholarships and employment outcomes. 

In this instance, however, the Council’s laudable desire to support simplification in reporting of employment outcomes meant that a number of other policy considerations that merit much more attention and thoughtful deliberation did not get due consideration prior to the Council taking action that effectively erodes transparency.

The Council should rescind its action, and send out the proposed changes for notice and comment and for consideration by the Standard’s Review Committee, which can give due consideration to intended and unintended consequences in recommending an appropriate set of changes regarding the reporting of employment outcomes.

Summer 2015 – Prolonged Process Results in Changes Designed to Promote Transparency

In 2015, after multiple proposals from the Data Policy and Collection Committee, refined significantly in response to comments received following public notice and opportunity to comment for law schools and other concerned constituents, and after multiple hearings, at which the Council also considered a proposal by Paul Mahoney, the Council approved a proposal from the Data Policy and Collection Committee to categorize law-school-funded positions in a separate category “above the line”  on the Employment Summary Report. 

Thus, in the top grouping on the Employment Summary Report, law-school-funded positions were listed separately from bar passage required positions and JD advantage positions.  (This changed the pre-existing approach in which law-school-funded positions had been allowed to be categorized as bar-passage-required positions or JD advantage positions “above the line” while being delineated separately below the line.  The approved change retained a “below the line” classification in which law-school-funded positions were separately delineated as bar passage required or JD advantage positions. To see the difference, compare a given law school's Employment Summary Report for 2014 and 2016.)

June 2017 – Truncated “Ex Parte” Process Results in Reversal of Course to Obscure Identification of Law-School-Funded Positions

At its June 2017 meeting, however, the Council, without any notice and without any opportunity to comment for law schools or other concerned constituents who might be opposed to the changes, without any effort to consult the Standards Review Committee (into which the Data Policy and Collection Committee has been merged), and without any consideration of implementation challenges associated with reversing course in the middle of a data collection period for the Class of 2017, approved a renewed proposal from Paul Mahoney which has the effect of eliminating transparency relating to the reporting of law-school-funded positions. 

The changes the Council approved actually are worse than the pre-existing environment prior to the Council’s action in 2015.  Prior to the Council’s 2015 action, the listing below the line of law-school-funded positions indicating whether such positions were full-time, long-term bar passage required or whether they were in another category allowed observers (including prospective law students) to “do the math.”  That is, anyone looking at the data could “subtract” the number of law-school-funded full-time, long-term bar passage required positions shown “below the line” from the total of full-time, long term bar passage required positions shown “above the line.”

By approving the proposal at its June meeting, however, the Council NOT ONLY eliminated the “above the line” delineation of law-school-funded positions, it also excluded jobs with salaries of $40,000 or more for which the law school is providing some or all of the funding from even being classified as “law-school-funded” positions.  Instead, the Council approved classifying such positions as full-time, long-term bar passage required positions “above the line” and then subclassifying such positions as public interest or government positions “below the line.” They are now completely excluded from the new “Employed by school” category (which the proposal doesn’t actually define explicitly, but implicitly would be law-school-funded positions with salaries less than $40,000).  This is the antithesis of transparency.

Why Does this Matter?

The premise for the change the DPCC recommended and the Council approved in 2015, was that law-school-funded positions were different in kind, because they were not provided by the legal employment market, but were provided solely as a result of a law school subsidy to compensate in whole or in part those graduates in positions for which graduates of other law schools are not allowed to compete.   

The DPCC also was concerned by significant growth in law-school-funded positions that were classified as full-time, long-term bar passage required positions even though many of these new positions were receiving a stipend of only $1500 to $2000 per month according to information gathered by NALP. 

-For the Class of 2012, there were 517 law-school-funded full-time, long-term bar passage required positions, roughly 2% of all full-time, long-term bar passage required positions. 

-For the Class of 2013, there were 774 law-school-funded full-time, long-term bar passage required positions, roughly 2.9% of all full-time, long-term bar passage required positions. 

-For the Class of 2014, there were 831 law-school-funded full-time, long-term bar passage required positions, roughly 3.2% of all full-time, long-term bar passage required positions. 

This represents a 60% increase in the number of law-school-funded full-time, long-term bar passage required positions in two years.

Notably, for the Class of 2014, the 25 law schools with 10 or more law-school-funded, full-time, long-term bar passage required positions totaled 676 such positions, making them responsible for 81% of the law-school-funded positions classified as full-time, long-term bar passage required positions.  Although law-school-funded full-time, bar passage required positions were only 3.2% of all such positions, across these 25 law schools, the 676 full-time, long-term bar passage required positions that were law-school-funded represented 10.7% of all such full-time, long-term bar passage required positions obtained by graduates of those 25 law schools.

Again, why does this matter?  Prior to 2015, the USNews rankings counted these law-school-funded positions as if they were full-time, long-term bar passage required positions.  This allowed the 25 law schools with significant numbers of graduates in law-school-funded, full-time, long-term bar passage required positions to significantly improve their employment outcomes for rankings purposes and for reporting employment outcomes to prospective students.    In the same year the Council adopted its changes to reporting of law-school-funded positions for the Class of 2015, however, USNews modified its methodology to provide less weight to graduates in such law-school-funded positions.

Notably, after the USNews changed its methodology and the Council approved the changes to the definition and classification of law-school-funded positions, the number of law-school-funded full-time, long-term bar passage required positions dropped dramatically – to fewer than 400 for the Class of 2015 and only 344 for the Class of 2016. 

Several Other Problems with the Proposal

While the reclassification of law-school-funded positions may be the most disturbing component of the Council’s action in approving the proposal at its June meeting, it is not the only disturbing aspect of the Council’s action.

The proposal consisted of a short memorandum focused on two things.  First, the proposal suggested simplifying the Employment Summary Report by consolidating all short-term or part-time positions into one new category that is “NOT full-time, long-term.”  This has the effect of making four columns into two columns on the Employment Summary Report, which is significant simplification.  This might be a good idea.  It is not clear that prospective law students or other observers have paid much attention to whether a non-full-time, long-term position is short-term rather than part-time.  Rather, they appear to be focused on whether a position is full-time, long-term, or not.  So this suggestion might be a wonderful idea once everyone has had a chance to think through things a little and make sure there aren’t any unintended consequences.

Second, the proposal suggested the change described above regarding the reclassification (and obfuscation) of law-school-funded positions.

What is even more troubling, however, is that the revised Employment Summary Report submitted as part of the proposal also included several other changes that were not discussed in any detail in the proposal or apparently at the Council.  The revised Employment Summary Report:

1. Eliminates seven “above the line” employment categories, merging them into two new categories. The revised Employment Summary Report as approved
   1. consolidates “Employed-Professional,” “Employed-Non-Professional,” and “Employed Undeterminable” into one new “Employment Other Category” – as if there is no difference between working at a Starbucks or a Wendy’s and working as a professor at a community college; and
   2. consolidates “Unemployed-Start Date Deferred,” “Unemployed-Seeking,” “Unemployed-Not Seeking” and “Employment Status Unknown” into one new “Unemployed or status unknown” – as if there is no relevance to the number of graduates for whom a law school could not find employment information or no relevance to the difference between a stay-at-home dad who is not looking for employment and someone hoping to find a position as a lawyer somewhere;
2. Eliminates the line “above the line” that lists total graduates, making it impossible for someone looking to know whether all graduates are accounted for or what percentage of graduates are in any given category;
3. Eliminates five “below the line” employment categories relating to firm size, merging them into two new categories. The revised Employment Summary Report as approved
   1. consolidates “11-25,” “26-50” and “51-100” into one new 10-100 category – as if there is no difference whether someone is in a 12-lawyer firm or an 85-lawyer firm; and
   2. consolidates “101-250,” and “251-500” into one new 100-500 category – as if there is no difference between a 115-lawyer law firm and a 450-lawyer law firm; and
4. Consolidates two judicial clerkship categories “clerkships – state and local” and “clerkships – other” into one new combined category.

As further evidence of the Council’s lack of attention to detail in approving the revised Employment Summary Report, no one apparently flagged the fact that the combined law-firm categories mentioned in item 3 above have overlapping numbers.  Thus, it is not clear which category one belongs in if someone works for a 10-lawyer firm or a 100-lawyer firm.

Making Changes Without Real Engagement with Consequences/Concerns

These many additional changes, which were not discussed in any detail in the proposal, or apparently during the Council deliberations, are problematic for several reasons.

            Inaccurate Statements

The proposal doesn’t discuss any of these proposed changes in detail, but dismisses the significance of any of these changes with one sweeping (if inaccurate) statement –

“Every cell that has disappeared through consolidation with another in the simplified report accounted for less than 3% of the employment outcomes for the Class of 2016; most accounted for less than 1%.  The simplified report accordingly loses no important and useful information and is substantially more user-friendly and understandable.”

This is just not accurate.  First, the listing of the total number of graduates is very important and useful information and its elimination does not make the revised Employment Summary Report more user-friendly and understandable.  Second, two categories that are eliminated as separate categories in the revised Employment Summary Report account for more than 3% of employment outcomes for the Class of 2016 – the Unemployed Not Seeking category had 3,271 representing 8.8% of graduates, and the Law Firm – 11-25 full-time, long-term category had 1,644 representing 4.4% of graduates. 

Failure to Consider Importance of Identifying Outliers

More significantly, however, the proposal fails to address the outliers in each category for which this is particularly relevant information.  For example, the Employment Status Unknown category only had 616 total people, or 1.7% of graduates in the Class of 2016.  Yet, there were 18 law schools – nearly 10% of law schools – for which Employment Status Unknown was at least 5%, with six law schools above 10% topping out at 18.6%.  But apparently this is not information prospective law students should have available to them.

The same is true in the Law Firm – 11-25 full-time, long-term category.  This category had 1,644 total people or 4.4% of all graduates in the Class of 2016.  But fully 80 law schools had 5% or more of their graduates in this category of which seven law schools had 10% or more of their graduates in this category, topping out at 12.4%.  Again, this apparently is not information prospective law students should have available to them.

No Consideration of Value of Longitudinal Data

More significantly, however, the consolidation of categories takes an unusually short term view of relevant data.  Let’s go back three years to the Class of 2013.  At that time, unemployed seeking represented 11.2%, with employment status unknown at 2.3% and unemployed not seeking at 1.8% -- totaling 15.3%.  For the Class of 2016, those numbers are now 8.8%, 1.7% and 1.3%, respectively – totaling 11.8%.  While the combined total has decreased by 3.5%, the percentage unemployed seeking actually only declined by 2.4%.  Consolidating these categories, however, would have obscured the fact that more of the improvement was attributable to reductions in unemployed not seeking and employment status unknown than in a reduction in unemployed seeking. 

No Consideration of Consistency with NALP Reporting

These types of decisions require careful attention and careful thought, particularly given the desire for as much consistency as possible between ABA reporting requirements and NALP reporting requirements. The existing structure of the Employment Summary Report largely tracks the data categories that NALP has used for many years precisely to provide consistency over time AND to provide greater consistency across reporting structures to simplify the work of career services professionals.  In approving the proposal, the Council did not appear to give any consideration to these concerns.

Failure to Reconcile Employment Questionnaire and Employment Summary Report

One other thing that is almost incomprehensible is that the proposal justified the changes in the Employment Summary Report because they would make it easier for career services personnel.  “Moreover, it will decrease the substantial burden we have put on career services offices, . . .”

In reality, however, none of the changes in the Employment Summary Report actually change the way in which career services personnel are being asked to report employment outcomes in the Employment Questionnaire for the Class of 2017. 

Indeed, suggesting they saw no meaningful connection between these two decisions, the Council, immediately prior to approving the proposed revised Employment Summary Report had approved the 2017 Employment Questionnaire, which incorporates none of the consolidated categories contained in the revised Employment Summary Report as approved by the Council.

Thus, for this year, the “simplified” Employment Summary Report will provide no meaningful “simplification” benefit to career services personnel, but will simply mean that important information is more difficult to access.

Barry Currier, Managing Director of the Section, in a recent memo to the Council about implementing these changes, acknowledges these difficulties created by the Council’s action in June.  Currier notes that for the Class of 2017, the ABA plans to run duplicate models, continuing to gather and report all of the data gathered for the Class of 2016 and prior years, along with the data reflected in the revised Employment Summary Report.  In his memo, Currier “also recommend[s] that the Council task the SRC with revising the questionnaire to align the data we collect with the new form so that next year we would ask schools to collect and be responsible for only the data points that are publicly reported. Reducing the amount of work that CSO staff must do collect and document data is one of the benefits of a simpler and streamlined public report.” 

This memo essentially acknowledges the major headaches involved in trying to rewrite the Employment Questionnaire and the accompanying instructions and definitions that would be needed to implement the revised Employment Summary Report.  By recommending that the SRC be tasked with revising the Employment Questionnaire, Currier is essentially asking the Council to do what it should have done BEFORE VOTING ON THE PROPOSAL.  The SRC is well-equipped to consider the policy implications of changing some aspects of the Employment Questionnaire and Employment Summary Report, is well-positioned to gather public comment from concerned constituencies, and has experience in weighing the importance of consistency between ABA reporting requirements and NALP reporting requirements.  It also is keenly aware of the value of designing the information gathering process to simplify the workload of those in career services offices as much as possible.  This is what should have happened in the first instance. 

The Council’s desire to simplify reporting and thereby ease the burden on law school career services professional is laudable.  But its failure to provide a meaningful process for airing the complicated issues associated with gathering and reporting employment outcomes information is most unfortunate. 

The Council would be well-advised to suspend implementation of the proposal until the SRC conducts a thorough review, including providing opportunity for public comment.

Rankings Gambit Disguised as Simplification

Again, why does this matter? While this blog posting has highlighted a number of procedural and policy concerns associated with the proposal, as it relates to law-school-funded positions, the proposal amounts to little more than a rankings gambit, something the Council may not have appreciated fully in voting to approve the proposal. 

USNews decided to discount law-school-funded positions in its rankings methodology in 2015.  This has impacted modestly the 14 law schools with 10 or more law-school-funded full-time, long-term bar passage required positions in the Class of 2016 who combined were responsible for roughly 80% of all law-school-funded full-time, long-term bar passage required positions for the Class of 2016.  At these 14 law schools, ten of which are ranked in the top-20 in USNews, law-school-funded full-time, long-term bar passage required positions represented between 2.8% and 11.8% of Class of 2016 graduates.  Having the Council reclassify these positions as full-time, long-term bar passage required positions benefits these 14 law schools to the detriment of the vast majority of other law schools that cannot afford to subsidize post-graduate employment opportunities in this way.  The Council’s decision is an “end run” around USNews, making it impossible for USNews to “discount” law-school-funded positions because they no longer will be identified as law-school-funded positions. 

In making its recommendations in 2015, the DPCC recognized that some of these law-school-funded positions that law schools were classifying as full-time, long-term bar passage required positions, particularly those with salaries in excess of $40,000 per year, might well be good work experiences for graduates transitioning into careers in public interest work or the government.  But the DPCC (and I assume, the Council) still believed these jobs were different in kind than other full-time, long-term bar passage required positions because they are not provided by the market and are not open to competition from graduates of all law schools.  These distinctions were the foundation for the DPCC’s recommendation and the Council’s decision to make law-school-funded positions transparent in their own “above the line” category after an extended dialogue and much opportunity for public comment. 

To reverse this decision two years later, with no dialogue and no opportunity for public comment, and no consultation with the Standards Review Committee, is a travesty that serves to benefit a handful of elite law schools while making it harder for prospective law students and observers to fully understand the extent to which reported employment outcomes truly are provided by the market or are being subsidized by the law school. 

To do so at the same time as several other changes are made for which there was little discussion of implementation challenges or policy consequences makes the decision all the more unfortunate.

Petition

I have created a petition, set forth below, requesting that the Council suspend implementation of the proposal and assure careful delineation of all proposed changes required to implement the proposal together with notice and opportunity to comment on all proposed changes before any such changes are implemented.  If you share my concerns, please sign the petition and email it to Barry Currier at [email protected] with a copy to me at [email protected] prior to the upcoming Council meeting on August 10-12 in New York.

Petition Requesting That the ABA Council for the Section of Legal Education and Admissions to the Bar Suspend Implementation of the Proposal Approved at its June Meeting Regarding Changes to the Employment Summary Report and the Classification of Law-School Funded Positions

At its June 1-2 meeting, the ABA Council for the Section of Legal Education and Admissions to the Bar approved a proposal submitted on May 30 to dramatically change the format of the Employment Summary Report.   

Most significantly, the proposal called for a reversal in policy regarding the treatment of law-school-funded positions from the policy and definitions the Council approved nearly unanimously in 2015 (after notice and an opportunity to comment had been provided to law schools and others).  The proposal calls for the elimination of the separate “above the line” classification of law-school-funded positions, and the reclassification of some positions so that they no longer count as law-school-funded positions, eliminating transparency for such positions.

The Council considered the proposal at its June 1-2 meeting without presenting the proposal to the Standards Review Committee for comment and without notifying the broader community and requesting comment.  In other words, in what could be considered a “contested matter” given the extensive dialogue that took place in 2015, the Council basically heard the proposal, which primarily benefits a handful of law schools, as an “ex parte” request for changes to the policy, while providing those who might be opposed no opportunity to be heard.

Somewhat remarkably, in fact, the Council approved the proposal immediately after approving the Employment Questionnaire (including instructions and definitions), even though the Council’s approval of the proposal effectively requires significant rewriting of the Employment Questionnaire it had approved moments earlier.    

Some aspects of the proposal may make sense after due consideration in terms of simplifying the reporting of graduate employment outcomes.  That said, the complete reversal of policy regarding law-school-funded positions, and the elimination, without discussion, of several other categories of employment outcomes that are being consolidated, highlight the profound need for more deliberate process with notice and opportunity to comment.

Accordingly, we, the undersigned, concerned about transparency and consistency in reporting law school graduate employment outcomes, hereby:

1) petition the Council to suspend implementation of the proposal until at least the Class of 2018, and direct the Section to implement for the Class of 2017, the Employment Questionnaire as approved at the June meeting, together with the Employment Summary Report used for the Class of 2016, and

2) petition the Council to direct the Standards Review Committee to

a. delineate all of the changes in the Employment Questionnaire that would be necessary to implement the proposal, and

b. provide notice of all proposed changes to the Employment Questionnaire and Employment Summary Report and an opportunity to comment on the proposed changes before the Council takes any further action to implement the proposal.

Name ________________________ Affiliation/Position_____________________________

Date _________________________ Contact Email ________________________________

Email your signed form to Barry Currier at [email protected] with a cc. to Jerry Organ at [email protected].