Leveraging Comparative Analysis for Institutional Decision Making NJAIR Annual Conference April 17 th , 2009 The College o...
BENCHMARKING: WHAT IS IT AND WHO DO WE COMPARE TO?   Leveraging Comparative Analysis for Institutional Decision Making
What Forces are Driving the Need for Using Data for Institutional Decision Making <ul><li>Internal </li></ul><ul><ul><li>F...
What is Benchmarking? <ul><li>Benchmarking is an ongoing, systematic process for measuring and  comparing the work process...
Why Benchmark? <ul><li>Identification of “Best Practices”  </li></ul><ul><ul><li>Academic </li></ul></ul><ul><ul><li>Opera...
How Do We Select the Institutions? Adapted from (Teeter & Brinkman 2003 in  The Primer for Institutional Research, AIR ) C...
Types of Peers <ul><li>Definitional </li></ul><ul><ul><li>Have similar identifiers expressing the essential nature of the ...
Reasons for PEER ANALYSIS ??? <ul><li>Compare </li></ul><ul><li>Complain </li></ul><ul><li>Assess </li></ul><ul><li>Boast ...
Some Common Peer Characteristics <ul><li>Affiliation (Public vs. Private) </li></ul><ul><li>Carnegie Classification </li><...
<ul><li>Data & Statistics & Judgment (Hybrid approach) </li></ul><ul><li>Data & Statistics (Cluster Analysis) </li></ul><u...
Sources of Comparison Data To Help Identify Peers <ul><li>Carnegie Foundation </li></ul><ul><li>National Student Clearingh...
WHAT TYPE OF COMPARISON DATA IS AVAILABLE? Leveraging Comparative Analysis for Institutional Decision Making
Sources of Comparison Data Recruitment & Retention <ul><li>Noel-Levitz National Enrollment Management Survey </li></ul><ul...
Sources of Comparison Data Student Engagement <ul><li>National Survey of Student Engagement (NSSE) </li></ul><ul><ul><li>S...
Sources of Comparison Data Student Learning <ul><li>Collegiate Learning Assessment (CLA) </li></ul><ul><li>ACT Collegiate ...
Sources of Comparison Data  Financial Operations <ul><li>NACUBO Endowment Study </li></ul><ul><li>NACUBO Tuition Discounti...
Sources of Comparison Data Satisfaction <ul><li>Student Satisfaction Surveys </li></ul><ul><ul><li>Noel-Levitz Student Sat...
Analytical Tools ( Software/Services ) <ul><li>Proprietary Software </li></ul><ul><ul><li>MS Excel </li></ul></ul><ul><ul>...
Big Picture Initiatives/Projects Available <ul><li>Council for Independent Colleges </li></ul><ul><ul><li>CIC KIT </li></u...
PSEUDO CASE STUDY    Leveraging Comparative Analysis for Institutional Decision Making
Comparative Data for Internal Analysis: Case Study Example <ul><li>Using comparative data to answer institutional specific...
Using Admissions and FASFA Data <ul><li>Admissions interview data </li></ul><ul><li>Extraction of enrolled and not enrolle...
Internal Data Combined with Student Tracker Last Name SAT Combined HS GPA EFC Enrolled College Name College State 2yr/4yr ...
Example Output- “Win/Loss” Ratio YOUR INSTITUTION Competitor Comparison Using FAFSA and NSC Data # on  # on  Mean  # Enrol...
Hypothesis <ul><li>Are institutional aid policies in line with other institutions? </li></ul><ul><ul><li>Is there a signif...
Average Aid by Institution Percentage receiving any financial aid Average federal grant  Average state/local grant  Averag...
INTEGRATING COMPARATIVE ANALYSIS WITH PLANNING   Leveraging Comparative Analysis for Institutional Decision Making
Reporting Comparative Data  <ul><li>Standard comparative reports </li></ul><ul><ul><li>Externally processed </li></ul></ul...
Yellow Bars- Represent Aspiration Institutions Dark Blue Bars- Represent Peer-Like Institutions Aqua Bars- Represent Peer-...
Example Institutional Dashboard  Summary Dashboard Fall 2008 Part-Time UG’s FR  Applicants Endowment/Reserves Total Gifts ...
Conclusion/Discussion <ul><li>Comparative Analysis provides context for institutional data with respect to decision making...
Upcoming SlideShare
Loading in …5
×

Leveraging Comparative Analysis - Presentation by Robert ...

574 views
536 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
574
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Go through this quickly!!
  • Leveraging Comparative Analysis - Presentation by Robert ...

    1. 1. Leveraging Comparative Analysis for Institutional Decision Making NJAIR Annual Conference April 17 th , 2009 The College of New Jersey Robert Miller, Centenary College Chad May, The Richard Stockton College of NJ
    2. 2. BENCHMARKING: WHAT IS IT AND WHO DO WE COMPARE TO? Leveraging Comparative Analysis for Institutional Decision Making
    3. 3. What Forces are Driving the Need for Using Data for Institutional Decision Making <ul><li>Internal </li></ul><ul><ul><li>Finite Resources </li></ul></ul><ul><ul><li>Competition for Students </li></ul></ul><ul><li>External </li></ul><ul><ul><li>Increased Accountability </li></ul></ul><ul><ul><li>Increased Call for Transparency </li></ul></ul><ul><ul><li>Students as Consumers </li></ul></ul>
    4. 4. What is Benchmarking? <ul><li>Benchmarking is an ongoing, systematic process for measuring and comparing the work processes of one organization to those of another, by bringing an external focus to internal activities, functions, or operations (Kempner 1993). </li></ul><ul><li>Practitioners at colleges and universities have found that benchmarking helps overcome resistance to change, provides a structure for external evaluation, and creates new networks of communication between schools where valuable information and experiences can be shared (AACSB 1994). </li></ul><ul><li>Benchmarking is a positive process, and provides objective measurements for baselining (setting the initial values ), goal-setting and improvement tracking , which can lead to dramatic innovations (Shafer & Coate 1992). </li></ul>
    5. 5. Why Benchmark? <ul><li>Identification of “Best Practices” </li></ul><ul><ul><li>Academic </li></ul></ul><ul><ul><li>Operational </li></ul></ul><ul><li>Provide context for institutional data </li></ul><ul><li>Goal setting and measurement </li></ul><ul><li>Institutional Planning </li></ul>
    6. 6. How Do We Select the Institutions? Adapted from (Teeter & Brinkman 2003 in The Primer for Institutional Research, AIR ) Competitor Aspiration Predetermined Peer Natural (i.e. athletic conference, regional group) Traditional (i.e. based on history) Jurisdictional (state or local jurisdiction) Classification-based (i.e. Carnegie Commission, AAUP, US News, etc.)
    7. 7. Types of Peers <ul><li>Definitional </li></ul><ul><ul><li>Have similar identifiers expressing the essential nature of the institution </li></ul></ul><ul><li>Informational </li></ul><ul><ul><li>Hold practical knowledge of a desired process, outcome, accomplishment </li></ul></ul><ul><li>Analytical </li></ul><ul><ul><li>Provide realistic and practical benchmarks for internal and external review </li></ul></ul><ul><li>Nonsensical </li></ul><ul><ul><li>Have “…no meaning or [convey] no intelligible ideas”;”…absurd or contrary to good sense” </li></ul></ul><ul><ul><li>*Webster’s Seventh New Collegiate Dictionary. G.&C. Merriam Company, Springfield, Massachusetts. 1967. </li></ul></ul><ul><ul><li>Adapted from a presentation given at the NEAIR 2002 Conference </li></ul></ul>
    8. 8. Reasons for PEER ANALYSIS ??? <ul><li>Compare </li></ul><ul><li>Complain </li></ul><ul><li>Assess </li></ul><ul><li>Boast </li></ul><ul><li>Improve </li></ul><ul><li>Fund </li></ul><ul><li>Evaluate </li></ul>
    9. 9. Some Common Peer Characteristics <ul><li>Affiliation (Public vs. Private) </li></ul><ul><li>Carnegie Classification </li></ul><ul><li>Financials (endowment, tuition, assets, liabilities, expenses, revenue) </li></ul><ul><li>Enrollment and Staffing Levels </li></ul><ul><li>Selectivity (SAT, Acceptance rates) </li></ul><ul><li>Academic Programs (majors and degrees) </li></ul><ul><ul><li>IPEDS PAS System can generate a comparison group automatically using the information above </li></ul></ul>
    10. 10. <ul><li>Data & Statistics & Judgment (Hybrid approach) </li></ul><ul><li>Data & Statistics (Cluster Analysis) </li></ul><ul><li>Data & Judgment (Threshold Approach) </li></ul><ul><li>Judgment (Panel Review) </li></ul>Strategies of Developing Peer/Aspirant List (Adapted from Teeter & Brinkman 2003 in The Primer for Institutional Research, AIR)
    11. 11. Sources of Comparison Data To Help Identify Peers <ul><li>Carnegie Foundation </li></ul><ul><li>National Student Clearinghouse- StudentTracker </li></ul><ul><li>Integrated Postsecondary Education Data System (IPEDS) Peer Analysis System, Dataset Cutting tool, Executive peer tool, etc. </li></ul><ul><ul><li>http://nces.ed.gov/IPEDS/ </li></ul></ul>
    12. 12. WHAT TYPE OF COMPARISON DATA IS AVAILABLE? Leveraging Comparative Analysis for Institutional Decision Making
    13. 13. Sources of Comparison Data Recruitment & Retention <ul><li>Noel-Levitz National Enrollment Management Survey </li></ul><ul><li>Consortium for Student Retention Data Sharing </li></ul><ul><ul><li>Data on retention rates & graduation rates </li></ul></ul><ul><li>IPEDS Peer Analysis System </li></ul><ul><li>The College Board Admitted Student Questionnaire (ASQ and ASQ plus) </li></ul><ul><li>ACT, Inc </li></ul>
    14. 14. Sources of Comparison Data Student Engagement <ul><li>National Survey of Student Engagement (NSSE) </li></ul><ul><ul><li>SPSS Syntax Files </li></ul></ul><ul><li>UCLA’s Higher Education Research Institute Surveys </li></ul><ul><ul><li>Cooperative Institutional Research Program (CIRP) </li></ul></ul><ul><ul><li>Your First College Year </li></ul></ul><ul><ul><li>College Senior Survey </li></ul></ul><ul><li>Education Benchmarking Inc. (Resident Student Assessment, First Year Initiative Survey, etc.) </li></ul>
    15. 15. Sources of Comparison Data Student Learning <ul><li>Collegiate Learning Assessment (CLA) </li></ul><ul><li>ACT Collegiate Assessment of Academic Proficiency (CAAP) </li></ul><ul><li>ETS Measurement of Academic Proficiency and Progress </li></ul>
    16. 16. Sources of Comparison Data Financial Operations <ul><li>NACUBO Endowment Study </li></ul><ul><li>NACUBO Tuition Discounting Study </li></ul><ul><li>Voluntary Support of Education </li></ul><ul><ul><ul><li>Fundraising results </li></ul></ul></ul><ul><li>IPEDS Finance Survey (Peer Analysis System) </li></ul><ul><li>Guidestar (990 data for non-profits) </li></ul>
    17. 17. Sources of Comparison Data Satisfaction <ul><li>Student Satisfaction Surveys </li></ul><ul><ul><li>Noel-Levitz Student Satisfaction Inventory </li></ul></ul><ul><ul><li>ACT Survey of Student Opinions </li></ul></ul><ul><ul><li>In-house surveys </li></ul></ul><ul><li>Employee Satisfaction Surveys </li></ul><ul><ul><li>HERI Faculty Survey </li></ul></ul><ul><ul><li>Harvard University (Collaborative On Academic Careers in Higher Education survey) </li></ul></ul><ul><li>Alumni Surveys </li></ul><ul><ul><li>ACT Alumni Survey and Alumni Outcomes Survey </li></ul></ul>
    18. 18. Analytical Tools ( Software/Services ) <ul><li>Proprietary Software </li></ul><ul><ul><li>MS Excel </li></ul></ul><ul><ul><li>SPSS/SAS/STATA and other Stat packages </li></ul></ul><ul><ul><li>Rapid Insight Analytics / Data Integration </li></ul></ul><ul><ul><li>Tableau- Visual Analysis Software </li></ul></ul><ul><li>Proprietary Services (Internet based applications) </li></ul><ul><ul><li>AGB Benchmarking Service </li></ul></ul><ul><ul><li>Peer Analysis System (PAS) </li></ul></ul><ul><ul><li>Dataset Cutting Tool </li></ul></ul><ul><ul><li>Executive Peer Analysis Tool (create your own data feedback report) </li></ul></ul><ul><ul><li>CUPA- Data on Demand Services </li></ul></ul><ul><ul><li>Voluntary Support of Education (CAE)- Data tool </li></ul></ul><ul><ul><li>AAUP Faculty Compensation data published in Academe </li></ul></ul><ul><ul><li>JMA Associates </li></ul></ul>
    19. 19. Big Picture Initiatives/Projects Available <ul><li>Council for Independent Colleges </li></ul><ul><ul><li>CIC KIT </li></ul></ul><ul><ul><li>This tool provides information relating to enrollment, staffing, admissions, and financial aid. </li></ul></ul><ul><ul><li>Key feature: allows you to conduct comparative analysis using schools with similar financial resources. </li></ul></ul><ul><ul><ul><ul><li>Sample of CIC KIT Tool </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>http://www.cic.edu/projects_services/infoservices/kit.asp </li></ul></ul></ul></ul></ul><ul><li>CIC FIT Tool </li></ul><ul><ul><li>While the KIT tool provides traditional indicators such as acceptance rate, yield rate, and faculty counts, the FIT tool provides detailed financial comparisons </li></ul></ul><ul><ul><li>Ratio analysis for overall institutional health </li></ul></ul><ul><ul><li>Sample of CIC FIT Tool </li></ul></ul><ul><ul><ul><li>http://www.cic.edu/projects_services/infoservices/fit/index.asp </li></ul></ul></ul>
    20. 20. PSEUDO CASE STUDY Leveraging Comparative Analysis for Institutional Decision Making
    21. 21. Comparative Data for Internal Analysis: Case Study Example <ul><li>Using comparative data to answer institutional specific questions </li></ul><ul><li>Common Question for IR professionals </li></ul><ul><ul><li>Who are students choosing over us and who are students choosing us over?(i.e. the win/loss question) </li></ul></ul>
    22. 22. Using Admissions and FASFA Data <ul><li>Admissions interview data </li></ul><ul><li>Extraction of enrolled and not enrolled students </li></ul><ul><ul><li>Analysis of fields to identify what other institutions students sent their FAFSA data to- they can list up to six </li></ul></ul><ul><ul><li>Send batch files to the National Student Clearinghouse using the StudentTracker service </li></ul></ul><ul><ul><ul><li>Return file from NSC shows enrollment history of your non-enrolling admitted students </li></ul></ul></ul><ul><ul><li>Match NSC return file data to other institutional data </li></ul></ul>
    23. 23. Internal Data Combined with Student Tracker Last Name SAT Combined HS GPA EFC Enrolled College Name College State 2yr/4yr Public/Private Major Smith L 2.2 12000 0 LCCC PA 2yr Public Business Smith L 2.4 2300 1 Moravian PA 4yr Public Business Smith L 2.3 7800 0 Desales PA 4yr Private Business Smith L 2.5 1200 0 PSU PA 4yr Public Business Smith L 2.9 18000 0 PSU PA 4yr Public Business Smith L 2.2 11000 1 Moravian PA 4yr Private Business Smith L 2.6 9000 1 Moravian PA 4yr Private Business Smith L 2.4 11000 0 LCCC PA 2yr Public Business Smith L 3.7 3000 0 Albright PA 4yr Private Business Smith L 2.3 7000 1 Moravian PA 4yr Private Business Smith L 2.3 12000 1 Moravian PA 4yr Private Business Smith L 3.2 19000 0 Scranton PA 4yr Private Business Smith L 3.4 14000 0 Albright PA 4yr Private Business
    24. 24. Example Output- “Win/Loss” Ratio YOUR INSTITUTION Competitor Comparison Using FAFSA and NSC Data # on # on Mean # Enrolled Estimated % FAFSA FAFSA 2004-05 2005-06 2006-07 3-Year NSC Data Enrolled Institution Name 2006-07 2007-08 Rank Rank Rank Rank Public 2007-08 2007-08 Your Institution 2499 2585           805 31.1% Competitor1 198 205 1 1 1 1 Y 180 87.8% Competitor2 113 138 8 6 2 5.3 Y 60 43.5% Competitor3 147 128 3 2 3 2.7 35 27.3% Competitor4 108 120 5 7 4 5.3 95 79.2% Competitor5 130 106 2 3 5 3.3 Y 100 94.3% Competitor6 117 102 5 5 6 5.3 10 9.8% Competitor7 106 95 7 8 7 7.3 18 18.9% Competitor8 130 93 4 3 8 5 25 26.9% Competitor9 80 85 11 10 9 10 6 7.1% Competitor10 78 85 13 11 9 11 Y 12 14.1% … .. … .. … .. … .. … .. … .. … .. … .. … .. … ..
    25. 25. Hypothesis <ul><li>Are institutional aid policies in line with other institutions? </li></ul><ul><ul><li>Is there a significant difference in EFC of enrolling and non enrolling business students? </li></ul></ul><ul><ul><ul><li>Internal analysis </li></ul></ul></ul><ul><ul><li>Is there a significant difference in the institutional grant aid awarded to enrolling and non enrolling business students? </li></ul></ul><ul><ul><ul><li>Internal analysis </li></ul></ul></ul><ul><ul><li>How does grant aid compare between our institution and other institutions? </li></ul></ul><ul><ul><ul><li>IPEDS PAS </li></ul></ul></ul>
    26. 26. Average Aid by Institution Percentage receiving any financial aid Average federal grant Average state/local grant Average institutional grant Average student loan Albright College 89 4736 3327 12354 10185 DeSales University 99 3542 3284 9408 2984 Moravian College 92 4508 3528 11124 6603 University of Scranton 91 4000 3776 12102 5326 Case Study College 64 8752 3688 18463 7532
    27. 27. INTEGRATING COMPARATIVE ANALYSIS WITH PLANNING Leveraging Comparative Analysis for Institutional Decision Making
    28. 28. Reporting Comparative Data <ul><li>Standard comparative reports </li></ul><ul><ul><li>Externally processed </li></ul></ul><ul><ul><ul><li>Faculty Compensation Report (Academe) </li></ul></ul></ul><ul><ul><ul><li>IPEDS Feedback Report </li></ul></ul></ul><ul><ul><ul><li>University of Delaware Study of Instructional Costs and Productivity </li></ul></ul></ul><ul><ul><ul><li>NSSE, HERI, and other survey instruments </li></ul></ul></ul><ul><ul><li>Internally Processed </li></ul></ul><ul><ul><ul><li>Dashboards and/or report of Key Indicators report(s) </li></ul></ul></ul><ul><ul><ul><li>Competitors report and Tuition/Fee Comparison report </li></ul></ul></ul><ul><ul><ul><li>Other IR reports </li></ul></ul></ul><ul><li>Ad-hoc comparative reports </li></ul><ul><ul><li>Retention- where are our students going? </li></ul></ul><ul><ul><li>Graduation Rate Study </li></ul></ul><ul><ul><li>Internal analysis of survey data (comparison of student satisfaction) </li></ul></ul>
    29. 29. Yellow Bars- Represent Aspiration Institutions Dark Blue Bars- Represent Peer-Like Institutions Aqua Bars- Represent Peer- Below Institutions Orange line across represents the target institution
    30. 30. Example Institutional Dashboard Summary Dashboard Fall 2008 Part-Time UG’s FR Applicants Endowment/Reserves Total Gifts 10,000 2,000 Full-Time UG’s Graduate Students 1,000 (fall) 97.6% % UGs in-State 68% 6-year Graduation Rate Diversity Enrollment 21% 81.6% First-year Retention SR Stdnt Satisfaction 85% Gifts to Capital & Endwmnt 20,000 $ $ 60% 37.0% 74%ile 1250 % FR Acceptances Yield (% Enrolled) H.S. Avg. Rank Avg. SAT- Regular % % % % % % Return on Endowment /Reserves Portfolio Spending Rate Unrestricted Annual Fund Gifts (change) Positive Variance Student Revenue Reliance Debt coverage ratio % UG Alumni Participation Number $1,000 donors Faculty Student Aid UG Student/Faculty Ratio Full-Time Faculty UG Class Size >=30 UG Class Size <10 Taught by FT Faculty FT Faculty W/ Term. Deg. Discount Rate 60% Inst. Financial Aid as % of Operating Budget % of FT Students w/ Financial Need KEY Change : Higher Lower None Importance of Change: Green = better Red = worse Yellow = neutral 55% % Resident Stds. (FT) % FT Faculty w/ Tenure 50% 60% 29% 74.7% 85% 1017 51.4% Current Benchmark Gross Cost to raise $1 FY 2006 % Plant Reinvestment Rate (excludes current construction projects) 1.5-2% 65% Another Indicator 15,000 65% 15,000 40% Student Body Admission Finance (FY 08) Advancement (FY08)
    31. 31. Conclusion/Discussion <ul><li>Comparative Analysis provides context for institutional data with respect to decision making/planning/and assessment. </li></ul><ul><li>There is a significant amount of data already available. Much of which is almost ready-made for dissemination. </li></ul><ul><li>If you do not do the comparative analysis someone else will. (students, government, parents, etc.) </li></ul>

    ×