Understanding the U.S. News & World Report “Best Colleges” 2007


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Understanding the U.S. News & World Report “Best Colleges” 2007

  1. 1. Understanding the U.S. News & World Report “Best Colleges” 2007: How it Works, Problems, and Implications for Institutional Research Matthew J. Hendrickson Ball State University
  2. 2. Introduction: Ranking Systems <ul><li>Many companies have made attempts at trying to provide the college “consumer” with guides for selecting a institution. These systems were created to aid in making “one of the most important decisions of your life” (USNWR-primer, 2007) </li></ul><ul><ul><li>U.S. News, Insider’s Guide, Fiske, Peterson’s, Princeton Review, College Blue Book, etc. </li></ul></ul><ul><li>The most controversial is U.S. News & World Report (USNWR) because it offers a ranking system that the others do not. </li></ul>
  3. 3. How the Rankings Work <ul><li>Uses a factor system to assign ranks to institutions in different categories. </li></ul><ul><li>Surveys 8 types of programs associated with student learning. </li></ul><ul><li>Relies on quantitative measures of academic quality. </li></ul><ul><li>Divides institutions into categories based on mission and region (if necessary). </li></ul><ul><ul><li>The Carnegie Foundation for Advancement of Teaching system is used to determine the categories. </li></ul></ul><ul><li>Gather info for up to 15 indicators of academic excellence. </li></ul><ul><li>Lastly, they are ranked against each other to create a numerical standing. </li></ul>
  4. 4. U.S. News & World Report: Factor Weights <ul><li>These weights were selected based on years of reporting about education, on reviews of research about education, and after consultation with experts in higher education (USNWR-cofaq_brief, 2007). </li></ul><ul><li>More weight has been placed on the outcome measures versus the input measures. </li></ul>
  5. 5. Definitions Morse & Flanigan, 2007 <ul><li>Peer Assessment (25%): assessed by peer surveys completed by the administration. </li></ul><ul><li>Retention (20%): measures average six-year graduation rates and freshman retention. </li></ul><ul><li>Faculty Resources (20%): considers compensation, terminal degree, student/faculty ratio, class size, and percent full-time faculty. </li></ul><ul><li>Student Selectivity (15%): based on acceptance rates, high school class standing, and SAT/ACT scores. </li></ul><ul><li>Financial Resources (10%): uses average expenditures per-student. </li></ul><ul><li>Graduation Rate Performance (5%): considers actual versus expected six-year graduation rates. </li></ul><ul><li>Alumni Giving (5%): percentage of alumni donating to the institution. </li></ul>
  6. 6. Problems with the Rankings <ul><li>Does not include all institutions. </li></ul><ul><li>Methodology changes frequently, making it difficult to compare previous versions to the most current version. </li></ul><ul><li>Only ranks the top 50% of institutions. The lower 50% is placed into two different tiers, neither of which are numerically ranked. </li></ul><ul><li>If an institution does not provide the data, USNWR will fill in that data, or use the previous data supplied by the institution. </li></ul>
  7. 7. Justifications for Rankings From Morse & Flanigan (2007) unless otherwise noted. <ul><li>Peer Assessment: assumes that those in the field are the most knowledgeable about their competition and the status of higher education. </li></ul><ul><li>Retention: the larger the number of returning freshman and those who graduate the university shows the institution to offer classes and services the students need to succeed. </li></ul><ul><li>Faculty Resources: the more satisfied students are about their contact with professors, the more likely they will learn, making it more likely they will graduate. </li></ul><ul><li>Student Selectivity: a institution’s academic atmosphere and abilities are limited to the students who attend. </li></ul><ul><li>Financial Resources: it is assumed that a generous per-student spending indicates that a college can offer a wide variety of programs and services. </li></ul><ul><li>Graduation Rate Performance: if a institution has a low graduation rate, there must be other issues impacting the institution’s overall ability. </li></ul><ul><li>Alumni Giving Rate: as an indirect measure, it assumes the more alumni who donate to the institution, the more satisfied they were with their experience. </li></ul>
  8. 8. Cheating the Rankings Ehrenberg (2005) and Woodbury (2003) <ul><li>Peer Assessment: Talk up your institution to others. Host a 1-2 year spending “boost” of rankings to increase opinion. </li></ul><ul><li>Retention: Relax standards. Spend retention money on other factors to boost overall opinion. </li></ul><ul><li>Faculty Resources: Pay tenure faculty high salaries and have instructors teach many low level courses (faculty/student ratio). When selecting course size, do not allow more than 19 or less than 100 students. </li></ul><ul><li>Student Selectivity: Increase application numbers. Pre-acceptance statements and the “safety school”. Reporting of SAT/ACT optional, but only accept high scores. Accept as few students as possible. Go for high “yield.” </li></ul><ul><li>Financial Resources: Have faculty get outside grants, spend institution funds elsewhere. </li></ul><ul><li>Graduation Rate Performance: This is taken care of through the “fixes” proposed in the retention and student selectivity sections. </li></ul><ul><li>Alumni Giving Rate: Have as many alumni donate as possible; get them to donate ANYTHING to increase your percentage. </li></ul><ul><li>General: Accept only non-traditional students. </li></ul>
  9. 9. Implications for IR <ul><li>Pros </li></ul><ul><ul><li>The creation of a standardized Common Data Set. </li></ul></ul><ul><ul><li>Allows for competition for increasing educational methods. </li></ul></ul><ul><ul><li>Provides information to potential college applicants about specific institutions. </li></ul></ul><ul><li>Cons </li></ul><ul><ul><li>Upward pressure on tuition due to “spending races.” </li></ul></ul><ul><ul><li>Competition for rankings can create negative emphasis. </li></ul></ul><ul><ul><li>Penalizes schools for not participating. </li></ul></ul><ul><ul><li>There have been criticisms about the use and validity of the USNWR rankings. </li></ul></ul>
  10. 10. Selected References: <ul><li>Carnegie Foundation for Advancement of Teaching, Classification System </li></ul><ul><li>http://www.carnegiefoundation.org/about/sub.asp?key=18&subkey=405#4 </li></ul><ul><li>Ehrenberg, R.G. (2005). Method or madness? Inside the U.S. News & World Report college rankings. Journal of College Admission, Fall, 29-35. </li></ul><ul><li>Hamrick, F. A., Schuh, J. H., & Shelley, M. C. (2004, May 4). Predicting higher education graduation rates from institutional characteristics and resource allocation. Education Policy Analysis Archives, 12, 19. Retrieved [12/8/2006] from http://epaa.asu.edu/epaa/v12n19/. </li></ul><ul><li>Morse, R.J. & Flanigan, S. (2007). How we do the rankings. http://www.usnews.com/usnews/edu/college/rankings/about/07rank_brief/php </li></ul><ul><li>Stuart, D.L. (1995). Reputational rankings: Background and Development. In D. Walleri, & M.K. Moss (Eds.), New Directions for Institutional Research: Evaluating and Responding to College Guidebooks and Rankings (13-20). Jossey-Bass, San Francisco, Winter 1995, No. 88. </li></ul><ul><li>U.S. News & World Report, Best Colleges 2007. http://www.usnews.com/usnews/edu/college/rankings/about/cofaq_brief.php </li></ul><ul><li>http://www.usnews.com/usnews/edu/college/rankings/about/primer_brief.php </li></ul><ul><li>http://www.usnews.com/usnews/edu/college/rankings/rankindex_brief.php </li></ul><ul><li>Woodbury, R.L. (2003). How to make your college no. 1 in U.S. News & World Report …and lose your integrity in the process. New England Board of Higher Education, Spring, 18-20. </li></ul>