Assessment of Higher Education      Learning Outcomes        Hamish Coates     coatesh@acer.edu.au acer.edu.au/highereduca...
Rationales for                   change   Building                         Building capacity and                     asses...
Rationales for   change
What is the “best”  university in yourcountry? How do you       know?
An experiment                 95%                65%
“Standards”, not “standardisation”                 Generalising thePerformance                principles of good          ...
Commitment to                    effectiveness                                EffectivenessLittle data   Happiness data   ...
Data fortransparency
Plan            Data for                           leadershipImprove              Act          Evaluate                   ...
Enhanced management, evaluation and qualitymonitoring – to do more, better, with less    Higher education growing in signi...
Building assessmentcollaborations
Are valid and reliable cross-cultural/-institutional/- linguistic comparisons of HE learning outcomes scientifically and p...
„Strands‟ of testingGeneric Skills       Economics            Engineering                  Contextual dimension           ...
Framework and Tuning AHELO / QAA frameworks Curriculum documents                               item development Accreditat...
Linguistically diverse… AHELO          Arabic    Dutch       English    Finishworld map         Flemish      Italian      ...
Building capacity andimprovement
Institutional                 Change                positioning                                             horizons Syste...
Buildingcommunities
Buildingawareness
Reporting resultshttp://oecdbetterlifeindex.org
Assessment           Build frameworks                               collaborations                                   Produ...
HowPreparation              Establish                                                      assessment              coordin...
Assessment of Higher Education      Learning Outcomes        Hamish Coates     coatesh@acer.edu.au acer.edu.au/highereduca...
Hamish coates noma tues
Upcoming SlideShare
Loading in …5
×

Hamish coates noma tues

257 views
200 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
257
On SlideShare
0
From Embeds
0
Number of Embeds
38
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Hamish coates noma tues

  1. 1. Assessment of Higher Education Learning Outcomes Hamish Coates coatesh@acer.edu.au acer.edu.au/highereducation
  2. 2. Rationales for change Building Building capacity and assessmentimprovement collaborations
  3. 3. Rationales for change
  4. 4. What is the “best” university in yourcountry? How do you know?
  5. 5. An experiment 95% 65%
  6. 6. “Standards”, not “standardisation” Generalising thePerformance principles of good assessment will encourage diversification and excellence Minimum standard Score distributions for eight groups (institutions, faculties, subjects, etc.)
  7. 7. Commitment to effectiveness EffectivenessLittle data Happiness data data Elite Mass Universal
  8. 8. Data fortransparency
  9. 9. Plan Data for leadershipImprove Act Evaluate Hunch
  10. 10. Enhanced management, evaluation and qualitymonitoring – to do more, better, with less Higher education growing in significance and Shaping scale: clear rationales for increasing output complexities and ensuring quality outcomesHuge cost and competitive pressures – evidence-basedmanagement helps expand provision with quality Further reliance on flawed, simplistic rankings will constrain growth and prosperity – robust multidimensional perspectives required Internationalisation pervades all facets of teaching and learning, and graduate outcomes – international perspectives are vital
  11. 11. Building assessmentcollaborations
  12. 12. Are valid and reliable cross-cultural/-institutional/- linguistic comparisons of HE learning outcomes scientifically and practically feasible? AHELO at a glance An international feasibility study – not a ‘pilot’ Conceived ~2005, announced 06, scoping 06-09, work 10-13 >300 individuals involved directly ~270 HEIsFor students near end of ‘bachelor degree’, also faculty, nationalmanagers, institutional coordinators, policymakers, expert advisors, etc. Up to 30,000 leaders, faculty, students Design, develop, validate and evaluate Many thousands of public frameworks, tests, infrastructure and processes stakeholders ~$30m total cost HEI reports (not for systems), along with international reports and materials, and recommendations for a main study
  13. 13. „Strands‟ of testingGeneric Skills Economics Engineering Contextual dimension National Context Information Institution Context Instrument Faculty Context Instrument Student Context Instrument
  14. 14. Framework and Tuning AHELO / QAA frameworks Curriculum documents item development Accreditation systems Discipline research Item creation• Outcome specification• Document analysis • Gather existing materials • Qualitative testing• Consultation • Item workshops • Quantitative testing• Synthesis, review • Technical review • Operationalisation • Framework mapping Instrument • Adaptation, translation validation Framework creation • Verification Authentic, hybrid item types ‘Above content’ reasoning Focus on ‘common core’
  15. 15. Linguistically diverse… AHELO Arabic Dutch English Finishworld map Flemish Italian Japanese Korean Norwegian Russian Slovak Spanish + Abu Dhabi…
  16. 16. Building capacity andimprovement
  17. 17. Institutional Change positioning horizons System Facultymonitoring improvement Assessment collaborationsStakeholder Policyengagement research Learner feedback
  18. 18. Buildingcommunities
  19. 19. Buildingawareness
  20. 20. Reporting resultshttp://oecdbetterlifeindex.org
  21. 21. Assessment Build frameworks collaborations Produce items Analyse and report Select and useTypes of Definitions Tasks Processes Resultssharing
  22. 22. HowPreparation Establish assessment coordinating centre Translate and collaborations validate instruments work Assessment Select students Engage institutions Administer secure Train institution test coordinators Score responses Prepare for testing Verify and provide Prepare multilevel Reporting data benchmarking reports Distribute reports to institutions Interpretation for monitoring and improvement
  23. 23. Assessment of Higher Education Learning Outcomes Hamish Coates coatesh@acer.edu.au acer.edu.au/highereducation

×