Monday, April 27, 2009
Last week, I blogged the report from U.S. News that it is looking into allegations "that a few school might have submitted incorrect data to U.S. News," as well as reports that Brooklyn and Nebraska may be two of those schools. Florida now can be added to the list.
Florida's Dean sent an email to its Board of Trustees, which included this charge:
There are a number of things in the rankings this year that do not appear correct. Today US News announced that it was starting to look at possible errors in this year’s results. At this time, we believe that there are errors in the information for some law schools in our vicinity in the rankings that have had a material effect upon the results. I will try to keep you posted.
Florida's Dean sent an email to the student body, which reported this error in the Florida data:
I must add that U.S. News does not measure many of the things which make us good, such as our new facilities, our robust array of outside speakers and visitors, our fundraising successes, and the achievements of our students in a variety of extramural, co-curricular, and student organization settings. I must also add that it is well known that many schools game the system and that over half of the information upon which U.S. News relies cannot be verified. Although U.S. News has assured us that this did not affect the rankings, it is worth noting that U.S. News published incorrect information (due to an error in my office) for our entering class GPA at the 75th and 25th percentiles; both are actually almost one-tenth of a point higher. This underscores the problems with verification and accuracy that are pervasive in the U.S. News ratings process.
For a timely post on the Faculty Lounge, see US News Rankings and the Data Disclosure Dance
One partial answer to Judge Pollak’s call for a better focused ranking system could be a more extensive disclosure of data by US News. Under the current system, each law school provides US News the data it requests. This component data is not made broadly available by either US News or the law schools. US News then notifies each school about how its own data is converted into points which result in its overall score. Disclosure by US News of every school’s component points and the underlying data used to create component points would enable users to better evaluate factors that are important to them individually. ...
More disclosure should also lead to better reliability. ... [T]he legal education community ought to take taking very seriously its obligation of providing accurate data to US News. Such an obligation already exists in the ABA accreditation standards (specifically Interpretation 509-4). US News requires deans to sign their school’s submission. Maybe the ABA should pile on and require a Sarbanes Oxley like certification to it that the law school is playing fair with US News.