• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Evaluating higher education learning outcomes presentation to ces
 

Evaluating higher education learning outcomes presentation to ces

on

  • 416 views

The Higher Education Quality Council of Ontario (HEQCO) has a number of projects focused on defining and measuring learning outcomes working with Ontario's colleges and universities in partnership ...

The Higher Education Quality Council of Ontario (HEQCO) has a number of projects focused on defining and measuring learning outcomes working with Ontario's colleges and universities in partnership with international organisations. This presentation provides an overview of how learning outcomes are increasing being viewed as a means to evaluate higher education quality, and presents the challenges and successes of developing, demonstrating and assessing higher education learning outcomes in Ontario.

Statistics

Views

Total Views
416
Views on SlideShare
416
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Evaluating higher education learning outcomes presentation to ces Evaluating higher education learning outcomes presentation to ces Presentation Transcript

    • MARY CATHARINE LENNONSENIOR RESEARCH ANALYSTANDRICHARD WIGGERSEXECUTIVE DIRECTOR, RESEARCH AND PROGRAMSPRESENTATION TO:CANADIAN EVALUATION SOCIETY ANNUAL CONFERENCETORONTO, ONTARIOJUNE 10, 2013EVALUATING HIGHEREDUCATION LEARNINGOUTCOMESInforming the Future of Higher Education
    • WHAT IS THE HIGHER EDUCATIONQUALITY COUNCIL OF ONTARIO?• An independent agency of provincial government• Mandate: To conduct research and offer policyadvice to government on access, quality andaccountability of the postsecondary educationsystem• Influence based on evidence and best practices
    • TRADITIONAL MEASURES OF PSE‘QUALITY’• Inputs: funding, library holdings, studentsreceiving merit scholarships, etc.• Outputs: retention rate, graduation rate, credittransfer agreements, etc.• Proxies: student evaluations, studentsatisfaction, labour market outcomes, etc.3Informing the Future of Higher Education
    • WHAT CAN LEARNING OUTCOMESDO?• Promote quality in three primary ways:– create transparency for the public, employers andstudents,– support institutional improvement through bettercoordination and improved teaching and learning,and– enhance system level quality assurance andaccountability.4Informing the Future of Higher Education
    • WHAT IS THE VALUE OF IDENTIFYINGLEARNING OUTCOMES?Hattie, J. (2009). The Black Box of Tertiary Assessment: An Impending Revolution. In L. H. Meyer, S. Davidson, H. Anderson, R. Fletcher, P.M. Johnston, & M.Rees (Eds.), Tertiary Assessment & Higher Education Student Outcomes: Policy, Practice & Research (pp.259-275). Wellington, New Zealand: Ako AotearoaA study synthesizing:800 meta-analyses50,000+ studies200+ million studentsfound that explicit outcomes andassessment has one of the largest effectson learning…
    • 0 0.2 0.4 0.6 0.8 1 1.2 1.4Student self-assessmentFormative evalution to instructorExplicit objectives and assessmentReciprocal teachingFeedbackSpaced vs. mass practiceMetacognitive strategiesCreativity programsSelf-questioningProfessional developmentProblem solving teaching…Teaching qualityTime on taskComputer assisted instructionEffect size (performance gain in σ)
    • HEQCO LEARNING OUTCOMESPROJECTS• ‘Measuring the Value of a Postsecondary Education’conference and book• HEQCO initiated a number of projects to examininggeneric, sector-level, and discipline-specific learningoutcomes– ‘Tuning’: Identifying and Measuring Sector-BasedLearning Outcomes in Postsecondary Education– Assessment of Higher Education Learning Outcomes(AHELO): Civil Engineering Strand– Collegiate Learning Assessment (CLA)/CollegeCollegiate Learning Assessment (CCLA)• Learning Outcomes Assessment ConsortiumInforming the Future of Higher Education
    • MEASURING THE VALUE OF APOSTSECONDARY EDUCATIONConference and book• What do we mean by ‘quality’education?• What do courses and programspromise to deliver and do theysucceed?• What do we know aboutimproving learning outcomes,and is reform possible?8Informing the Future of Higher Education
    • TUNING:• Establish clear statements of what studentsshould know and be able to do• Intended to promote mobility, credit transferand degree recognition, transparency• Life and Health, Physical Science, Social Sciences• 2 year diploma, 3 year diploma, 4 yearBachelors, Research Masters9 Informing the Future of Higher Education
    • 10 Informing the Future of Higher Education
    • COLLEGIATE LEARNING ASSESSMENT• Written assessment of ‘generic skills’• Examines critical thinking, analytic reasoning,problem solving, and written communicationskills gained from first to final year.• Provides information on the ‘value-add’ of theinstitution11Informing the Future of Higher Education
    • ASSESSMENT OF HIGHER EDUCATIONLEARNING OUTCOMES (AHELO)AHELO seeks to determine if there are internationallyappropriate standard tests which measure what final-year undergraduate students know and can do.Primary questions• Is it possible to have agreement on expected learningoutcomes?• Is it possible to implement a test across cultures andlanguages?• Are the assessments valid and reliable?12Informing the Future of Higher Education
    • AHELO• Three strands:– Generic: Assessment of analytical skills, writtencommunication, leadership ability, etc.– Discipline specific: Assessment of context knowledgeand students’ capacity to use or apply theknowledge• Economics and Civil Engineering– Context: Surveys of student, faculty and institutioncharacteristics13Informing the Future of Higher Education
    • CIVIL ENGINEERING STRANDPARTICIPANTSAustraliaOntarioColombiaEgyptJapanMexicoRussiaSlovakiaAbu Dhabi14Informing the Future of Higher Education
    • INTERESTING PRELIMINARY FINDINGS15Informing the Future of Higher Education500520540560580600620Among thetopstudentsAboveaverageAboutaverageBelowaverageAmong thebottomstudentsMean scores by student assessment ofown performance500520540560580600620Excellent Good Fair PoorMean scores by level of satisfaction withentire undergraduate experience
    • PRELIMINARY FINDINGS FROM AHELO• In Ontario– There is considerable interest from programs in jurisdictional andinternational comparison– Institutions have been prompted to reflect on their curriculum design anddelivery, and on the assessment techniques they employ– Faculty members have been prompted to reflect on their own methodsand re-evaluate what they require of students• At the OECD– Theoretically and practically possible to assess common student learningoutcomes internationally– Conversations continue on the ultimate goals, purpose and possibilities ofsuch information– Determining the value of the data is key to assessing the success of thefeasibility study16Informing the Future of Higher Education
    • BENEFITS OF LARGE SCALEASSESSMENTS• Provides ‘unbiased’ indication of teaching andlearning quality• Useful in benchmarking programs, institutions,and systems• Supports transferability and mobility17Informing the Future of Higher Education
    • CHALLENGES OF LARGE SCALEASSESSMENTS• Faculty Resistance• Institutional concern with student level datacollection• Student recruitment– Cumbersome, expensive• Validity of results– Appropriate sample? Student effort on low-stakestest?18Informing the Future of Higher Education
    • LEARNING OUTCOMES CONSORTIUM• Durham College - Student success e-portfolio: Essentialemployability skills project• Humber College - Develop rubrics to assess skills• George Brown College - Building capacity to measureessential employability skills• University of Toronto - Development of analytic rubricsfor learning outcomes assessment• University of Guelph - A pilot project for thedevelopment of an online learning outcome assessmentstrategy• Queen’s University - Learning outcomes assessment andprogram improvement19Informing the Future of Higher Education
    • For more information please visitwww.heqco.ca20Informing the Future of Higher Education