0
Developing evidence-based strategies to support improvedorganisational performance in teaching and learningBuilding Future...
The challenge                 Achieving synergies                      between          individual improvement of practice...
Plan of presentation• The context – an inefficient tertiary system which works  well for many learners but very poorly for...
The context …• Significant advances in course and qualification  completion rates, but still wide variation:    o   Course...
Since its establishment in 2007, Ako Aotearoa hasprovided $5m + funding for 190 projects toimprove tertiary teaching and l...
What we’ve learnt …•   We work within a system that allows great teaching and    learning to happen•   We are not very str...
Variability of learner outcomes is the immediate problem,  but is not the only fixExcellence                              ...
Why is it so important?It’s about moral purpose ......• Tertiary education offers opportunities for empowerment    •   for...
Why is evidence of learner benefit fundamental?                                         October 2012
Variation in qualification completion rates for Level 1-3students within public tertiary providers                        ...
The policy - research - practice disconnect      Practice                             Policy                              ...
Evidence-based enhancement of teaching andlearningPartnership and continuing dialogue betweenprofessional practitioners an...
Testing assumptions and effective dialogue relies onrobust, easily understood evidence• For 70% + traditional practice wil...
Characteristics of sound evidence • Focussed on student achievement and outcomes    o For the individual learner    o For ...
Meaningful critical self-reflectionExperienced practitioners often do this intuitively andinformally….“   Well that went w...
Evidence-based Enhancement of Teaching andLearning                           A Tertiary Practitioner’s Guide              ...
Organisations need to do this more purposefully andformally What works? What works most effectively? Can we make it busine...
So what do we do?• Systematise the informal evidence  –   Students’ stories have validity  –   Student contributions to cl...
So what do we do?• Develop new sources of evidence  –   Student engagement – e.g. AUSSE  –   Critical incident questionnai...
An example of robust and rich evidence collection:• Collects data from students about how students are learning• Has a ver...
October 2012
So what do we do?• Locate evidence and projects firmly in an  organisational context.                                     ...
Guide to change projects • Creating sustainable   change to improve   outcomes for tertiary   learners       (Alkema, 2012...
Changes to our project funding model • Ako Aotearoa funds change projects    • i.e. we fund evidence-based change projects...
Logic sequence for sustainable improvement  SHARE              IdentifyEXTERNALLY            need /         Investigate op...
So do Ako Aotearoa projects benefit learners?                                        October 2012
Impact evaluation of our own projects in providinglearner benefit:• 63 successfully completed small projects• Interest gen...
Pragmatic choices Evidence that identifies the most effective interventions when providers need to make choices           ...
Exemplar project: Dedicated education unit • Project: Piloted 2 DEUs at   Middlemore hospital using action   research • ht...
Exemplar project: Dedicated education unit • What’s changed? They are now business as usual   • 9 DEUs established, 2 more...
The key output from the dedicated educationunit project is not the research report It’s the practice guide                ...
Exemplar project: ITO assessment systems • Project: examined assessment systems for industry   training • http://akoaotear...
Exemplar projects: ITO assessment systems • What’s changed?   • Principles have contributed to organisational wide     pra...
Professional development workshops andresources                                   October 2012
Organisation                   Explicit aspirations for their learners     Mutual                                         ...
The last big question:            Are we supporting change           to improve what we do now                         or ...
The changing nature of 21st century tertiary education                                             October 2012
So what are the characteristics of a highperforming professional tertiary teacher?These will stay the same:•   Empathy•   ...
Plus what for the future?                            October 2012
What are the critical shifts?• A focus beyond the boundaries of courses and their  programmes – tighter integration betwee...
Where is this thinking coming from? Wide range of different conversations about 21st education and educators, but more rec...
What are the norms of 21st century education?• Focus on successful outcomes for all learners in their  classes• Highly cul...
What are the changes in skill set for the faculty?  More specialist (e.g. learning design / assessment /  curriculum devel...
What are the changes in skill set for the tertiaryeducation organisation? It will host, support and develop 21st century e...
What are the changes in skill set for the tertiaryeducation organisation? It will become a learning organisation          ...
Organisation                   Explicit aspirations for their learners     Mutual                                         ...
Thank youwww.akoaotearoa.ac.nz                        October 2012
Upcoming SlideShare
Loading in...5
×

NTLT 2012 Peter Coolbear Keynote Presentation to Conference

516

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
516
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • There is a disconnect between research and practice. Research is often not practice focussed, because it is too theoretical or merely descriptive or written primarily for other researchers or, quite simply, insufficently robust. However good, practice focussed research may not reach - or be resisted by - providers and practitioners for many reasons – they are unaware of it, it is inaccessible, it is inaccessible to budget holders/decision makers, it is uncomfortable, it is too easy to dismiss as “not applicable to us”, it is too expensive to implement change, etc., etc.Policy may have little impact on practice because it’s development has not been referenced to any practice context, it’s implementation is not practice informed. Most policy, if it has impact on practice, impacts in unintended ways.The research-policy connect is often weak. There is still too much first-principles policy development or politically driven policy. This is exacerbated by the prevalance of rushed, under-resourced implementational policy development.
  • Transcript of "NTLT 2012 Peter Coolbear Keynote Presentation to Conference"

    1. 1. Developing evidence-based strategies to support improvedorganisational performance in teaching and learningBuilding Futures: Hindsight – Insight – ForesightNational Learning and Teaching Conference, Nelson October 2012Peter Coolbear
    2. 2. The challenge Achieving synergies between individual improvement of practice and organisational change for the benefit of tertiary learners October 2012
    3. 3. Plan of presentation• The context – an inefficient tertiary system which works well for many learners but very poorly for others• The need for evidence-based improvement• What constitutes good evidence?• Ensuring our projects contribute to sustainable change• Changing expectations of 21st century vocational educators October 2012
    4. 4. The context …• Significant advances in course and qualification completion rates, but still wide variation: o Course completions <50% - 100% o Qualifications 12% - 100%• Lack of parity of success for Māori and Pacific learners – what’s the gap in your organisation?• Low progression to success at higher levels of study• In creasing lack of confidence about consistent academic standards• The inevitability of continued change October 2012
    5. 5. Since its establishment in 2007, Ako Aotearoa hasprovided $5m + funding for 190 projects toimprove tertiary teaching and learning• Projects designed to support evidence-based improvement of practice We’ve learnt a lot! October 2012
    6. 6. What we’ve learnt …• We work within a system that allows great teaching and learning to happen• We are not very strategic about improving tertiary teaching• We do not do enough to share good practice• We work within a system that allows the mediocre (or worse)• We work in a system that is highly fragmented• The NZ research base in tertiary education is still quite weak and has limited impact on practice October 2012
    7. 7. Variability of learner outcomes is the immediate problem, but is not the only fixExcellence Led by the profession and organisations, enabled by government Led by governmentThreshold of What also needs to be doneacceptability What the Minister sees as the present state October 2012
    8. 8. Why is it so important?It’s about moral purpose ......• Tertiary education offers opportunities for empowerment • for individuals • for whanau • for communities • for New Zealand as a wholeIt’s also about New Zealand’s international reputation October 2012
    9. 9. Why is evidence of learner benefit fundamental? October 2012
    10. 10. Variation in qualification completion rates for Level 1-3students within public tertiary providers 90% 80%Qulaification Completion Rate 70% 60% 50% 40% 30% 20% 10% 0% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Providers Data for 2009 (TEC) October 2012
    11. 11. The policy - research - practice disconnect Practice Policy October 2012
    12. 12. Evidence-based enhancement of teaching andlearningPartnership and continuing dialogue betweenprofessional practitioners and their organisations • High performing practitioners and tertiary education organisations continually test their assumptions about how and how effectively their learners are learning October 2012
    13. 13. Testing assumptions and effective dialogue relies onrobust, easily understood evidence• For 70% + traditional practice will not change without evidence that something else is better [aid theory]• Organisational decisions to redirect resources will (hopefully!) not be made without evidence [no theory] October 2012
    14. 14. Characteristics of sound evidence • Focussed on student achievement and outcomes o For the individual learner o For the employer o For the community • Uses robust, transparent, purposeful and well validated data collection techniques • Uses mixed methods (qualitative + quantitative) • Is action focussed October 2012
    15. 15. Meaningful critical self-reflectionExperienced practitioners often do this intuitively andinformally….“ Well that went well: there was some really great interaction in that class and the formative assessment demonstrated that the group really understood those key concepts”“ There’s some feedback here from their employers that Joe and Lisa had some difficulties setting the levels on that formwork correctly, yet both breezed through the class exercise. Clearly I need to do something different to help them translate theory into practice.” October 2012
    16. 16. Evidence-based Enhancement of Teaching andLearning A Tertiary Practitioner’s Guide to Collecting Evidence of Learner Benefit Individuals Organisations October 2012
    17. 17. Organisations need to do this more purposefully andformally What works? What works most effectively? Can we make it business as usual and sustainable? • What’s the RoI? • Is there anything we can stop doing if we do this instead? • What are my levers to ensure this happens in the way I intend? October 2012
    18. 18. So what do we do?• Systematise the informal evidence – Students’ stories have validity – Student contributions to class discussion – Student preferences for different activities – Employer feedback – Alumni feedback October 2012
    19. 19. So what do we do?• Develop new sources of evidence – Student engagement – e.g. AUSSE – Critical incident questionnaires – Research based in cognitive psychology – Graduate surveys – Employer surveys October 2012
    20. 20. An example of robust and rich evidence collection:• Collects data from students about how students are learning• Has a very strong research base• Used by all Australian Universities and now all New Zealand Universities + pilots in NZ ITPs and PEPs• International benchmarking against the NSSE used in North America• Most highly validated, data-rich student survey instrument available to tertiary education• Student satisfaction is just one dimension of thirteen others• Specifically designed to start organisational conversations. October 2012
    21. 21. October 2012
    22. 22. So what do we do?• Locate evidence and projects firmly in an organisational context. October 2012
    23. 23. Guide to change projects • Creating sustainable change to improve outcomes for tertiary learners (Alkema, 2012) October 2012
    24. 24. Changes to our project funding model • Ako Aotearoa funds change projects • i.e. we fund evidence-based change projects with a high potential to benefit learners. • Co-funding model - 50% contribution (or more) by the organisation • i.e. we require organisational commitment to change practice • We will measure the impact of projects • We expect to create PD resources from high impact projects October 2012
    25. 25. Logic sequence for sustainable improvement SHARE IdentifyEXTERNALLY need / Investigate options opportunity Change becomes Student success Create BAU on course(s) intervention Better Student student response: do Practice change outcomes they like it? October 2012
    26. 26. So do Ako Aotearoa projects benefit learners? October 2012
    27. 27. Impact evaluation of our own projects in providinglearner benefit:• 63 successfully completed small projects• Interest generated• Impact on practice• Evidence of learner benefit associated with the work• Impact on project team• Outcomes hierarchy.... direction of travel October 2012
    28. 28. Pragmatic choices Evidence that identifies the most effective interventions when providers need to make choices October 2012
    29. 29. Exemplar project: Dedicated education unit • Project: Piloted 2 DEUs at Middlemore hospital using action research • http://akoaotearoa.ac.nz/nursing-deu Vision was to change the way nurses were supported as they learnt October 2012
    30. 30. Exemplar project: Dedicated education unit • What’s changed? They are now business as usual • 9 DEUs established, 2 more in 2013 (covers between 80-90% of nursing students at MIT) • exploring possibility of multi-disciplinary unit • Key features • Robust design, strong organisational buy- in, focused on change for learners, broad influence October 2012
    31. 31. The key output from the dedicated educationunit project is not the research report It’s the practice guide October 2012
    32. 32. Exemplar project: ITO assessment systems • Project: examined assessment systems for industry training • http://akoaotearoa.ac.nz/projects/ito-workplace- assessment-structures Vision was to create system level change in assessment practice October 2012
    33. 33. Exemplar projects: ITO assessment systems • What’s changed? • Principles have contributed to organisational wide practice in 5 ITOs (impacts ~23,000 learners) • Workshops and new project with BCITO illustrate how change can be achieved • Key features • Issue of critical importance, research based principles, structure created to embed within organisational practice October 2012
    34. 34. Professional development workshops andresources October 2012
    35. 35. Organisation Explicit aspirations for their learners Mutual Mutuallypreparedness to Evidence congruent innovate and about what expectationschange practice works Practitioners Individual (and team) aspirations for their own practice and for their learners October 2012
    36. 36. The last big question: Are we supporting change to improve what we do now or are we fundamentally changing the way we do things? October 2012
    37. 37. The changing nature of 21st century tertiary education October 2012
    38. 38. So what are the characteristics of a highperforming professional tertiary teacher?These will stay the same:• Empathy• Reflection-in-action• Critical self reflection-on-action• Emancipation October 2012
    39. 39. Plus what for the future? October 2012
    40. 40. What are the critical shifts?• A focus beyond the boundaries of courses and their programmes – tighter integration between programmes of study and work• Changing expectations about Return on Investment• Changing expectations about inclusivity• New technologies allow greater learner autonomy and much greater expectations around meaningful student centricity• Open source materials will change the dynamics of delivery October 2012
    41. 41. Where is this thinking coming from? Wide range of different conversations about 21st education and educators, but more recently: • DEANZ Conference • Metro ITPs Graduate Diploma in Tertiary Education • HERDSA Graduate Attributes • Te Ara Whakamana • Tertiary Teaching Excellence Awards portfolios • Ako Aotearoa’s work on its own business model October 2012
    42. 42. What are the norms of 21st century education?• Focus on successful outcomes for all learners in their classes• Highly culturally competent• Capability focus rather than content / competence focus: “ I teach less content so that my students learn more” Zoe Jordens October 2012
    43. 43. What are the changes in skill set for the faculty? More specialist (e.g. learning design / assessment / curriculum development / open source retrieval) The 21st Century tertiary educator will spend much less time developing resources and much more time facilitating learning and providing facilitative feedback to learners The 21st Century tertiary educator’s practice will be increasingly framed by how their students use on-line technologies The 21st century educator’s practice will be informed by the latest understandings in cognitive psychology October 2012
    44. 44. What are the changes in skill set for the tertiaryeducation organisation? It will host, support and develop 21st century educators who are dual professionals It will own (and, in turn, expect employee ownership of) successful learner outcomes It will seek to disestablish the resource management – educational performance divide October 2012
    45. 45. What are the changes in skill set for the tertiaryeducation organisation? It will become a learning organisation October 2012
    46. 46. Organisation Explicit aspirations for their learners Mutual Mutuallypreparedness to Evidence congruent innovate and about what expectationschange practice works Practitioners Individual (and team) aspirations for their own practice and for their learners October 2012
    47. 47. Thank youwww.akoaotearoa.ac.nz October 2012
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×