Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

iMBA Initial (unofficial) Evaluation Plan


Published on

This is a presentation that I had developed to propose an evaluation and assessment plan for a new degree program at the University of Illinois - the iMBA (online MBA) degree. This was prior to any job offer and was NEVER implemented as an official evaluation plan. The purpose of sharing here is to show self-generated work outlining a proposed assessment and evaluation plan.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

iMBA Initial (unofficial) Evaluation Plan

  2. 2. BACKGROUND • iMBA • Online, stackable MBA program • Offered through UIUC • True UIUC degree • Low-Cost • ~$20,000 • Compared to other online programs at ~$75,000-$100,000
  3. 3. BACKGROUND • iMBA, cont. • Approved in 2015 • Starting in Spring 2016
  4. 4. COURSERA PARTNERSHIP • “Students” • Individuals enrolled through UIUC and taking the prescribed courses on Coursera • Credit and MBA recipients • “Participants” • Individuals not paying but consuming content provided on Coursera • No credit or MBA receipt
  5. 5. IS IT A “MOOC MBA?” YES AND NO • Yes • Offered for free, with open enrollment, to anyone interested who has Internet access • No • The MBA itself is not open, nor free-of-charge, hence will not be “massive” compared to the free offering
  6. 6. POTENTIAL KNOWLEDGE GAINS • MOOCs/Non-MOOCs • Compare curricula, students/participants, progress, learning with identical content • Graduate-level MOOCs • Very few truly open graduate-level courses • Differences in demographics, engagement, success • MOOC-based learning • Is learning any different in MOOC spaces due to the massive size of courses and the diversity of interaction?
  7. 7. ASSESSMENT AND EVALUATION ASSESSMENT • Improvement-focused • Engages faculty, content creators, administrators, and assessment personnel as “partners” in improvement • Cyclical, focusing on areas that can be improved and of top priority/need PROGRAM EVALUATION • Decision-focused and often finite in length • Compares variables to expectations/baselines • Informs decision-makers and administrators • Typically descriptive and easier-to- understand
  8. 8. ASSESSMENT AND EVALUATION, CONT. • Similarities • Data-based process • Extensive overlap on process/methods • Which to pick? • Both! • Evaluation: Inform decision-makers and ensure that expectations are being met • Assessment: Work with practitioners and faculty to ensure top-quality content and instruction
  9. 9. STEP 1: ESTABLISH PRIORITIES AND OUTCOMES • August-December 2015 • Individual/Group Meetings • Dean/Associate Deans • Director of eLearning • Select faculty • iMBA-specific staff • Graduate College Dean/Associate Deans • Provost office representatives
  10. 10. PLAN, CONT. • Purpose of Meetings: To identify key priorities of leadership, hone expected outcomes, and garner buy-in from all levels • Product of Meetings: Rough outline of expected outcome variables, logic models, and basic data strategies
  11. 11. LOGIC MODELS • A Good Assessment Starts With a Complete Understanding of Process • STEP ONE: FYE Logic Model – Inputs – Activities/Outputs – Outcomes • Short-Term (0-24 months) • Medium-Term (2- 5 years) • Long-Term (6+ years) – Impacts (Abstract)
  12. 12. LOGIC MODELS • Complete Understanding of Process • STEP ONE: FYE Logic Model – Inputs – Activities/ Outputs – Outcomes • Short-Term (0- 24 months) • Medium-Term (2-5 years) • Long-Term (6+ years) – Impacts (Abstract) ASSESSMENT * Assessment measures all steps of the process, except for the “abstract concepts”
  13. 13. LOGIC MODELS, CONT. • Purpose • Establishes necessary variables that can and should be measured • Identifies connection points between inputs, activities, and expected outcomes • Analyses can then answer whether there is that connection and how strong the connections are • Establishes organizational unity on priorities and inquiry
  14. 14. STEP 2: DEVELOP ASSESSMENT AND EVALUATION PLAN • November-December 2015 • Identify activities, data, sources, and next steps • Variables established from logic models and data source (Banner, Coursera instruments, surveys, etc.) • Collection plan (pre-course, post-course, instrument) • Quantitative and qualitative data are both key to activities • Rough analytical planning • Regression of outcomes to demographic/input variables • Comparison of iMBA students, participants, and traditional MBA students • Approaches for semester-to-semester program evaluation
  15. 15. STEP 3: EXECUTE PLAN • January-May 2016 • First iteration of survey instruments, evaluative approaches, etc. • High level of qualitative data collection on satisfaction, progress, etc. • Faculty • Students • Participants
  16. 16. STEP 4: INITIAL REPORTING • May-August 2016 • First semester report • First quantitative analyses • PR/Marketing • Interim survey of faculty and administrators • Address satisfaction with iMBA
  17. 17. SUMMER 2016 DELIVERABLES • Semester Report • Baseline Report • Used for comparative purposes in follow-up analyses • PR Materials • Brochures and publications with statistics, anecdotes, etc. • Scholarly Publications, as accepted • Peer-reviewed journal articles • Conference presentations
  18. 18. FUTURE YEARS • Assessment Cycles • Academic years • Focus on 2-3 issues of highest priority • Evaluation Cycles • Focus on shifts year-to-year • Highlight major changes, improvements, issues • Priority will be automating the process so metrics are available in real-time and require little human resources after initial implementation
  19. 19. IDEAL HUMAN RESOURCES • Evaluation and Assessment Specialist, 1.0 FTE • Liaison with decision-makers • Craft assessment/evaluation plan • Carry-out initial and advanced data collection and analyses • Graduate Research Assistants, 0.5 FTE • As needed, can provide detail on specific research questions, carry-out basic analyses • Data Specialist, 0.25 FTE • Develop needed data queries • Work on visualization tools