Larry Cunningham (Vice Dean, St. John's), Collecting Ultimate Bar Passage Data: Weighing the Costs and Benefits:
Friday afternoon the ABA Associate Deans’ listserv received a message from William Adams, Deputy Managing Director of the ABA. In it, he described a new process for collecting data on bar passage. A copy of the memo is on the ABA website. This change was authorized at the June 2017 meeting of the Council. Readers may remember that the June meeting was the one that led to a major dust-up in legal education, when it was later revealed that the Council had voted to make substantial (and some would say, detrimental) changes to the Employment Questionnaire. When this came to light through the work of Jerry Organ and others, the ABA wisely backed off this proposed change and indicated it would further study the issue.
The change that the ABA approved in June and announced in greater detail on Friday is equally problematic. In the past, schools would report bar passage as part of the Annual Questionnaire process. The bar passage section of the questionnaire asked schools to report first-time bar passage information. If a school was going through a site visit, it would also report this information on the Site Evaluation Questionnaire. If a school could not demonstrate compliance with Standard 316 with first-time bar passage, it was asked to show compliance using ultimate bar passage in the narrative section of the SEQ, specifically question 66, or as part of an interim monitoring or report-back process, described here (page 6).
Now, per the ABA, all schools — even those that can show that their graduates meet the minimums of Standard 316 through first-time passage data—must track, collect, and report ultimate bar passage information going back two years. (There is a phase-in process as outlined in the memo.) Hypothetically, let us assume that a school always has a pass rate of 80% (for sake of argument with 100% of graduates reporting) in a state with a consistent average of 75%. The school is in compliance with Standard 316, but it must nevertheless track the 20% of graduates who did not pass on the first attempt to see if they passed on subsequent attempts.
I have several problems with this change.
As with the Employment Questionnaire issue, this change to the collection of bar passage data was done without notice and comment. While notice and comment is not required under the ABA rules for changes to questionnaires, a more open dialogue with schools would have likely highlighted the issues I raise below. Not all of us have the time to scour the ABA’s website for agenda and minutes for the various entities involved with the accreditation process (Council, Accreditation Committee, Standards Review Committee). A change this significant should have been done with input from those of us — deans and vice/associate deans — who are on the front lines of ABA compliance.
From a substantive perspective, the new change in data collection adds significant burdens without much benefit to the accreditation process. Tracking graduates two years out will not be easy, particularly for schools in states that do not release data to schools or the public on bar passage. This is on top of the employment data that is collected every year, which is a significant undertaking if done correctly. Compliance with ABA Standards, state and federal Department of Education rules, state court rules (e.g., New York, which has a number of quasi-accreditation rules, such as its Skills Requirement and Pro Bono Requirement), and regional accreditors is increasingly taking up much of the work of associate deans of law schools. Time spent on compliance and accreditation is time that could otherwise be spent managing our institutions, helping students, or teaching. ...
I have great respect for the staff of the Managing Director’s Office. In these types of matters, they are the proverbial messenger, so I don’t fault them. I have three requests of the Council, however:
- First, the Council should give greater thought to the costs of data collection, particularly where it’s unclear whether or how such data will translate to assessing compliance with the existing standards. The Council has done a terrific job of streamlining the data collected for site visits, but more work can be done on the AQ.
- Second, the Council should withdraw its proposed implementation of the section of the new Bar Passage Questionnaire that asks all schools to report ultimate passage until these issues can be more fully aired.
- Finally, if significant changes are proposed to data questionnaires in the future, the Council should engage in a more open and collaborative process with the law schools and the broader legal education community to get feedback.