OECD Feasibility Study            for an international            Assessment of Higher            Education Learning      ...
 AHELO rationale Feasibility study design and operationalisation Progress to date Emerging insights National perspect...
AHELO rationale        Shared vision of OECD Education Ministers (2006)        Shift from quantity to quality        Despi...
 AHELO rationale Feasibility study design and operationalisation Progress to date Emerging insights National perspect...
Goals of the feasibility studyAssess whether we can measure whatundergraduate students know and can do upongraduation …… a...
Design parameters for the                    feasibility studyArtificial distinction between 4 strands of work To examine...
A range of geographic, linguistic and              cultural backgrounds involved                                          ...
 AHELO rationale Feasibility study design and operationalisation Progress to date Emerging insights National perspect...
Work undertaken in 2 phases                                      Generic Skills       Economics           Engineering     ...
Quick facts on AHELO fieldwork Data collection from February to June 2012 17 countries involved in 25 strand replication...
 AHELO rationale Feasibility study design and operationalisation Progress to date Emerging insights National perspect...
Emerging insights Cultural adaptation and translation followed established  guidelines Assessment frameworks have been v...
Next steps Feasibility study report (Volume 1): December 2012 AHELO conference: 14-15 March 2013 in Utrecht Feasibility...
AHELO sponsors
For more information, visitwww.oecd.org/edu/aheloKarine.Tremblay@oecd.orgDiane.Lalancette@oecd.org
Upcoming SlideShare
Loading in …5
×

OECD Feasibility Study for an international Assessment of Higher Education Learning Outcomes (AHELO) - Karine Tremblay and Diane Lalancette

2,754 views

Published on

Published in: Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,754
On SlideShare
0
From Embeds
0
Number of Embeds
1,313
Actions
Shares
0
Downloads
21
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

OECD Feasibility Study for an international Assessment of Higher Education Learning Outcomes (AHELO) - Karine Tremblay and Diane Lalancette

  1. 1. OECD Feasibility Study for an international Assessment of Higher Education Learning Outcomes (AHELO) IMHE General Conference 18 September 2012Karine Tremblay and Diane Lalancette, OECD
  2. 2.  AHELO rationale Feasibility study design and operationalisation Progress to date Emerging insights National perspectives: the Mexican and Russian experiences
  3. 3. AHELO rationale Shared vision of OECD Education Ministers (2006) Shift from quantity to quality Despite huge progress in quality assurance, institutional quality remains largely unknown  No perfect proxy  Reputation race  Rankings biased  Satisfaction culturally sensitive  Labour market outcomes problematic So Information vacuum filled by available informationwhat? Learning outcomes need to be taken into account  Defining them  Incorporating them in quality assurance  Measuring them (AHELO)
  4. 4.  AHELO rationale Feasibility study design and operationalisation Progress to date Emerging insights National perspectives: the Mexican and Russian experiences
  5. 5. Goals of the feasibility studyAssess whether we can measure whatundergraduate students know and can do upongraduation …… across diverse countries, languages, culturesand types of institutions Test the science of the assessment Test the practicality of implementation
  6. 6. Design parameters for the feasibility studyArtificial distinction between 4 strands of work To examine different types of learning outcomes and different testing approaches Generic skills and 2 different disciplines to prove the concept A research strand exploring the issue of value-added measurementA proof of concept approach Not the perfect (nor final) instruments Ensuring diversity of participants (types of institutions, countries, languages, cultural backgrounds)An improvement-driven approach Rich collection of contextual data (from students, faculties, leadership) To go beyond diagnosis tool towards analysis of “what works”
  7. 7. A range of geographic, linguistic and cultural backgrounds involved • • •• • •• ••• • • • Observers • •• • • Bahrein ••• • Brazil Saudi ArabiaGeneric Skills •• SingaporeColombia EngineeringEgypt Abu DhabiFinland Australia EconomicsKorea Canada (Ontario) Belgium (Fl.)Kuwait Colombia Egypt •Mexico Egypt ItalyNorway Japan MexicoSlovak Republic Mexico NetherlandsUnited States Russian Fed. Russian Fed.(CT, MO, PA) Slovak Republic Slovak Republic
  8. 8.  AHELO rationale Feasibility study design and operationalisation Progress to date Emerging insights National perspectives: the Mexican and Russian experiences
  9. 9. Work undertaken in 2 phases Generic Skills Economics Engineering Frameworks Framework Framework Framework Phase 1 –Initial proof of concept Instrument development & Generic Skills Economics Engineering small-scale Instrument Instrument Instrument Jan 2010- validation June 2011 Contextual dimension surveys Phase 2 – Scientific feasibility Project management, & proof of Implementation survey operations and analyses of results practicality Mar 2011- Dec 2012 Where we are now
  10. 10. Quick facts on AHELO fieldwork Data collection from February to June 2012 17 countries involved in 25 strand replications Data collected from over  23,000 students  4,900 faculties  270 institution coordinators 1,000 test sessions and 20,000 computers involved Scoring completed in June 2012 Analysis of results and findings underway Final report by December 2012
  11. 11.  AHELO rationale Feasibility study design and operationalisation Progress to date Emerging insights National perspectives: the Mexican and Russian experiences
  12. 12. Emerging insights Cultural adaptation and translation followed established guidelines Assessment frameworks have been validated in all strands Students and faculty random sampling was conducted in the What can we learn majority of participating institutions now from the data? International online testing generally ran smoothly in most countries ... answers in Scorer training was conducted in all participating countries December 2012 Independent quality monitoring of fieldwork activities indicated that most countries followed survey procedures
  13. 13. Next steps Feasibility study report (Volume 1): December 2012 AHELO conference: 14-15 March 2013 in Utrecht Feasibility study report (Volume 2): April 2013
  14. 14. AHELO sponsors
  15. 15. For more information, visitwww.oecd.org/edu/aheloKarine.Tremblay@oecd.orgDiane.Lalancette@oecd.org

×