Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

NTLT 2012 Peter Coolbear Keynote Presentation to Conference


Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

NTLT 2012 Peter Coolbear Keynote Presentation to Conference

  1. 1. Developing evidence-based strategies to support improvedorganisational performance in teaching and learningBuilding Futures: Hindsight – Insight – ForesightNational Learning and Teaching Conference, Nelson October 2012Peter Coolbear
  2. 2. The challenge Achieving synergies between individual improvement of practice and organisational change for the benefit of tertiary learners October 2012
  3. 3. Plan of presentation• The context – an inefficient tertiary system which works well for many learners but very poorly for others• The need for evidence-based improvement• What constitutes good evidence?• Ensuring our projects contribute to sustainable change• Changing expectations of 21st century vocational educators October 2012
  4. 4. The context …• Significant advances in course and qualification completion rates, but still wide variation: o Course completions <50% - 100% o Qualifications 12% - 100%• Lack of parity of success for Māori and Pacific learners – what’s the gap in your organisation?• Low progression to success at higher levels of study• In creasing lack of confidence about consistent academic standards• The inevitability of continued change October 2012
  5. 5. Since its establishment in 2007, Ako Aotearoa hasprovided $5m + funding for 190 projects toimprove tertiary teaching and learning• Projects designed to support evidence-based improvement of practice We’ve learnt a lot! October 2012
  6. 6. What we’ve learnt …• We work within a system that allows great teaching and learning to happen• We are not very strategic about improving tertiary teaching• We do not do enough to share good practice• We work within a system that allows the mediocre (or worse)• We work in a system that is highly fragmented• The NZ research base in tertiary education is still quite weak and has limited impact on practice October 2012
  7. 7. Variability of learner outcomes is the immediate problem, but is not the only fixExcellence Led by the profession and organisations, enabled by government Led by governmentThreshold of What also needs to be doneacceptability What the Minister sees as the present state October 2012
  8. 8. Why is it so important?It’s about moral purpose ......• Tertiary education offers opportunities for empowerment • for individuals • for whanau • for communities • for New Zealand as a wholeIt’s also about New Zealand’s international reputation October 2012
  9. 9. Why is evidence of learner benefit fundamental? October 2012
  10. 10. Variation in qualification completion rates for Level 1-3students within public tertiary providers 90% 80%Qulaification Completion Rate 70% 60% 50% 40% 30% 20% 10% 0% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Providers Data for 2009 (TEC) October 2012
  11. 11. The policy - research - practice disconnect Practice Policy October 2012
  12. 12. Evidence-based enhancement of teaching andlearningPartnership and continuing dialogue betweenprofessional practitioners and their organisations • High performing practitioners and tertiary education organisations continually test their assumptions about how and how effectively their learners are learning October 2012
  13. 13. Testing assumptions and effective dialogue relies onrobust, easily understood evidence• For 70% + traditional practice will not change without evidence that something else is better [aid theory]• Organisational decisions to redirect resources will (hopefully!) not be made without evidence [no theory] October 2012
  14. 14. Characteristics of sound evidence • Focussed on student achievement and outcomes o For the individual learner o For the employer o For the community • Uses robust, transparent, purposeful and well validated data collection techniques • Uses mixed methods (qualitative + quantitative) • Is action focussed October 2012
  15. 15. Meaningful critical self-reflectionExperienced practitioners often do this intuitively andinformally….“ Well that went well: there was some really great interaction in that class and the formative assessment demonstrated that the group really understood those key concepts”“ There’s some feedback here from their employers that Joe and Lisa had some difficulties setting the levels on that formwork correctly, yet both breezed through the class exercise. Clearly I need to do something different to help them translate theory into practice.” October 2012
  16. 16. Evidence-based Enhancement of Teaching andLearning A Tertiary Practitioner’s Guide to Collecting Evidence of Learner Benefit Individuals Organisations October 2012
  17. 17. Organisations need to do this more purposefully andformally What works? What works most effectively? Can we make it business as usual and sustainable? • What’s the RoI? • Is there anything we can stop doing if we do this instead? • What are my levers to ensure this happens in the way I intend? October 2012
  18. 18. So what do we do?• Systematise the informal evidence – Students’ stories have validity – Student contributions to class discussion – Student preferences for different activities – Employer feedback – Alumni feedback October 2012
  19. 19. So what do we do?• Develop new sources of evidence – Student engagement – e.g. AUSSE – Critical incident questionnaires – Research based in cognitive psychology – Graduate surveys – Employer surveys October 2012
  20. 20. An example of robust and rich evidence collection:• Collects data from students about how students are learning• Has a very strong research base• Used by all Australian Universities and now all New Zealand Universities + pilots in NZ ITPs and PEPs• International benchmarking against the NSSE used in North America• Most highly validated, data-rich student survey instrument available to tertiary education• Student satisfaction is just one dimension of thirteen others• Specifically designed to start organisational conversations. October 2012
  21. 21. October 2012
  22. 22. So what do we do?• Locate evidence and projects firmly in an organisational context. October 2012
  23. 23. Guide to change projects • Creating sustainable change to improve outcomes for tertiary learners (Alkema, 2012) October 2012
  24. 24. Changes to our project funding model • Ako Aotearoa funds change projects • i.e. we fund evidence-based change projects with a high potential to benefit learners. • Co-funding model - 50% contribution (or more) by the organisation • i.e. we require organisational commitment to change practice • We will measure the impact of projects • We expect to create PD resources from high impact projects October 2012
  25. 25. Logic sequence for sustainable improvement SHARE IdentifyEXTERNALLY need / Investigate options opportunity Change becomes Student success Create BAU on course(s) intervention Better Student student response: do Practice change outcomes they like it? October 2012
  26. 26. So do Ako Aotearoa projects benefit learners? October 2012
  27. 27. Impact evaluation of our own projects in providinglearner benefit:• 63 successfully completed small projects• Interest generated• Impact on practice• Evidence of learner benefit associated with the work• Impact on project team• Outcomes hierarchy.... direction of travel October 2012
  28. 28. Pragmatic choices Evidence that identifies the most effective interventions when providers need to make choices October 2012
  29. 29. Exemplar project: Dedicated education unit • Project: Piloted 2 DEUs at Middlemore hospital using action research • Vision was to change the way nurses were supported as they learnt October 2012
  30. 30. Exemplar project: Dedicated education unit • What’s changed? They are now business as usual • 9 DEUs established, 2 more in 2013 (covers between 80-90% of nursing students at MIT) • exploring possibility of multi-disciplinary unit • Key features • Robust design, strong organisational buy- in, focused on change for learners, broad influence October 2012
  31. 31. The key output from the dedicated educationunit project is not the research report It’s the practice guide October 2012
  32. 32. Exemplar project: ITO assessment systems • Project: examined assessment systems for industry training • assessment-structures Vision was to create system level change in assessment practice October 2012
  33. 33. Exemplar projects: ITO assessment systems • What’s changed? • Principles have contributed to organisational wide practice in 5 ITOs (impacts ~23,000 learners) • Workshops and new project with BCITO illustrate how change can be achieved • Key features • Issue of critical importance, research based principles, structure created to embed within organisational practice October 2012
  34. 34. Professional development workshops andresources October 2012
  35. 35. Organisation Explicit aspirations for their learners Mutual Mutuallypreparedness to Evidence congruent innovate and about what expectationschange practice works Practitioners Individual (and team) aspirations for their own practice and for their learners October 2012
  36. 36. The last big question: Are we supporting change to improve what we do now or are we fundamentally changing the way we do things? October 2012
  37. 37. The changing nature of 21st century tertiary education October 2012
  38. 38. So what are the characteristics of a highperforming professional tertiary teacher?These will stay the same:• Empathy• Reflection-in-action• Critical self reflection-on-action• Emancipation October 2012
  39. 39. Plus what for the future? October 2012
  40. 40. What are the critical shifts?• A focus beyond the boundaries of courses and their programmes – tighter integration between programmes of study and work• Changing expectations about Return on Investment• Changing expectations about inclusivity• New technologies allow greater learner autonomy and much greater expectations around meaningful student centricity• Open source materials will change the dynamics of delivery October 2012
  41. 41. Where is this thinking coming from? Wide range of different conversations about 21st education and educators, but more recently: • DEANZ Conference • Metro ITPs Graduate Diploma in Tertiary Education • HERDSA Graduate Attributes • Te Ara Whakamana • Tertiary Teaching Excellence Awards portfolios • Ako Aotearoa’s work on its own business model October 2012
  42. 42. What are the norms of 21st century education?• Focus on successful outcomes for all learners in their classes• Highly culturally competent• Capability focus rather than content / competence focus: “ I teach less content so that my students learn more” Zoe Jordens October 2012
  43. 43. What are the changes in skill set for the faculty? More specialist (e.g. learning design / assessment / curriculum development / open source retrieval) The 21st Century tertiary educator will spend much less time developing resources and much more time facilitating learning and providing facilitative feedback to learners The 21st Century tertiary educator’s practice will be increasingly framed by how their students use on-line technologies The 21st century educator’s practice will be informed by the latest understandings in cognitive psychology October 2012
  44. 44. What are the changes in skill set for the tertiaryeducation organisation? It will host, support and develop 21st century educators who are dual professionals It will own (and, in turn, expect employee ownership of) successful learner outcomes It will seek to disestablish the resource management – educational performance divide October 2012
  45. 45. What are the changes in skill set for the tertiaryeducation organisation? It will become a learning organisation October 2012
  46. 46. Organisation Explicit aspirations for their learners Mutual Mutuallypreparedness to Evidence congruent innovate and about what expectationschange practice works Practitioners Individual (and team) aspirations for their own practice and for their learners October 2012
  47. 47. Thank October 2012