Saturday, October 14, 2006
This article, very much a work-in-progress, will ultimately be in four Parts. Part I describes both U.S. News' methodology and problems involved in replicating it. Part II explores the reliability and validity of U.S. News' overall scores and rankings. Part III will focus on the problem of managing rankings. Part IV, finally, will suggest ways in which the rankings might be improved. The posted draft includes Part I, most of Part II, and the introductory paragraphs of Part III. Its descriptions, analyses, and conclusions are based on U.S. News' published descriptions of its 2006 computations, telephone conversations with U.S. News' staff clarifying those descriptions, and a spreadsheet I have written that approximately replicates those computations.
I am posting this paper early in the drafting process for two reasons. First, the silly season is already upon us. U.S. News' questionnaires have been mailed. Respondents may find the posted draft, particularly its discussion of reputational surveys, helpful in thinking about how to respond. Second, because of the unusual nature of what I am attempting, early feedback would be of great benefit. Although I have never received substantive comments on anything I have ever posted on SSRN, I am hoping that this will be an exception.