Download as PDF, PPTX
Uploaded byEduSkills OECD
Surveying administrative innovations in tertiary education: experience from Australia and NZ
The document presents an OECD 2019 study on administrative innovations in tertiary education across 39 Australian and 6 New Zealand universities, focusing on measuring innovation developed by administrative staff. It employs an online and mailed survey of 1,516 senior managers to benchmark innovation methods, outcomes, and obstacles, while comparing results to other public sector organizations. Key findings emphasize the importance of an inclusive innovation culture, collaboration, and design-thinking methods in enhancing innovation success and reducing the likelihood of abandoned initiatives.
More Related Content
Similar to Surveying administrative innovations in tertiary education: experience from Australia and NZ
Surveying administrative innovations in tertiary education: experience from Australia and NZ
- 1. Anthony Arundel (Dominique BowenButchart & Sarah Gatenby-Clark) Surveying administrative innovations in tertiary education: experience from Australia and NZ OECD 2019 Study funded by the LH Martin Institute, University of Melbourne
- 2. Online and mailedsurvey OECD 2019 • 39 Australian universities • 6 New Zealand universities
- 3. Reasons for thissurvey • Experiment with measuring innovations developed by administrative staff in universities – Builds on recent research on measuring innovation in the public sector • Collect data of value for benchmarking (across universities and over time) – Methods that managers use to innovate – Outcomes and obstacles OECD 2019
- 4. Comparison with OECD“Measuring Innovation in Education” OECD AUS-NZ Survey Compare innovation in education to other sectors Main interest is comparing innovation in universities to other public sector organizations Identify use of specified educational practices over time Innovations defined by type only (service, etc), asks about a most important innovation Construct metrics to evaluate effect of educational practices on outcomes Focuses on the process for administrative innovations: how innovations are developed, plus outcomes and obstacles Evaluate drivers of innovation in educational practices (not yet) Drivers plus the effect of a ‘inclusive innovation culture’ Develop an appropriate methodology (not yet) Part of ongoing work on measuring innovation in the public sector
- 5.
- 6. Survey methods • Questionscognitively tested in 13 face-to-face interviews with Senior Managers at seven universities. • Questionnaire sent to 1,516 senior managers in 10 functional areas (Library services, governance, IT services, etc.) – Not sent to senior executive or middle level managers • 573 respondents (37.8% response rate) • Responses from all targeted universities (45 in total) • Questions refer to the respondent’s “area of responsibility” • Reference period of two years. OECD 2019
- 7. + Innovation culture Revisions -Risk aversion OECD 2019
- 8. OECD 2019 Function of the unit General information Innovation environment Innovation methods Useof design thinking methods Two year reference period for all questions
- 9. OECD 2019 Most important innovation Abandoned or under-performing innovation Obstaclesto innovation Most important innovation = greatest expected impacts on the respondent’s area of responsibility, university, students, or staff
- 10. Percent respondents reporting innovationsby type 28% 48% 50% 71% 91% 91% 0% 20% 40% 60% 80% 100% Marke ng / communica on Organisa onal Services / products Processes Any innova on Any innova on (Australian public sector) OECD 2019
- 11. Examples of themost important innovations • New form of therapy for university students. • Enrichment program for high-achieving high school students. • Customized website to provide career development strategies to international students. • Online suite of resources to assist students in managing social media. • Mobile app to allow students to manage their courses, lectures and tutorials from a smartphone. OECD 2019
- 12. OECD 2019 Source ofthe idea for the most important innovation, percent of respondents 3 6 6 24 30 31 0 5 10 15 20 25 30 35 Academics or students at your university Consultants, businesses or other universi es Other Staff that report to the respondent or other managers Respondent VC, Council or Senior Execu ve
- 13. ‘Percent respondents givinga ‘high importance’ rating to innovation drivers OECD 2019
- 14. OECD 2019 Methods tosupport innovation
- 15. Widespread use of‘good practice’ innovation methods • 52% of respondent’s staff involved in brainstorming meetings to develop ideas for innovations. • 61% of respondents delegate responsibility for an innovation to an individual. • 73% of respondents report collaborating on their most important innovation. • Majority of respondents use design-thinking methods. OECD 2019
- 16. Percent respondents reportinguse of design-thinking methods Most of these methods involve ‘co-creation’ with end users of the innovation
- 17. OECD 2019 Does yourworkplace have an inclusive innovation culture?
- 18. OECD 2019 49% ofrespondents agree that their “Senior Executive support a positive innovation culture that includes all staff.” (inclusive innovation culture)
- 19. OECD 2019 Percent respondentsby university agreeing that their Senior Executive supports an inclusive innovation culture
- 20. Share of staffinvolved in three innovation support methods by agreement with an inclusive innovation culture OECD 2019 41% 39% 46% 51% 52% 58% 0% 10% 20% 30% 40% 50% 60% Working groups to develop or implement an innova on Training for how to use an innova on Mee ngs to brainstorm ideas for innova ons Agree that there is an inclusive innova on culture Disagree
- 21. Correlation between aninclusive innovation culture & the use of design thinking methods Design-thinking method Odds ratio p Conduct project user or focus groups 1.9 .029 Surveys of potential users 2.2 .004 “Ease-of-use” surveys 2.7 .001 Pilot tests of an innovation 1.9 .033 Post-implementation studies to identify problems 2.1 .012 OECD 2019 Results from logistic regressions that control for innovation type, reasons for innovating, restructuring, number of staff, and function Comparison between ‘agree’ with an inclusive culture versus ‘disagree. Evidence for a dose-response effect for all methods.
- 22. OECD 2019 Outcomes (canbe positive or negative) 1. University’s brand or reputation 2. Simpler or faster processes 3. Increase in revenue 4. Employee working conditions 5. Student experience 6. Teaching and learning 7. Research 8. Reduction in costs (Results for positive benefits problematic – lack of time?)
- 23. Correlation between aninclusive innovation culture and “major positive effects” from the most important innovation Positive effect on: Odds ratio p Simpler or faster processes 1.9 .08 Employee working conditions - ns Student experience 2.2 .099 Reduction in costs - ns University’s brand or reputation .38 .017 OECD 2019 Results from logistic regressions that control for use of design thinking methods, number of staff, source of the idea, type of innovation. High number of ‘too early to tell’ and ‘not relevant responses’
- 24. OECD 2019 The useof design-thinking methods and collaboration strongly increase the probability that the most important innovation is a novel process or service
- 25. OECD 2019 What doesn’t work? •Abandoned or underperforming innovations • Presence of resource obstacles • Negative outcomes for the most important innovation
- 26. OECD 2019 The absenceof an inclusive innovation culture doubles the probability of an abandoned or underperforming innovation A budget cut increases the probability of abandonment or underperformance by 60%.
- 27. The odds ofreporting each of three obstacles that are measures of a lack of resources (skills, funding and time) decreases substantially in the presence of an inclusive culture (Odds of 0.32, 0.24 & 0.12). OECD 2019
- 28. OECD 2019 The mainfactor increasing the reporting of all resource obstacles is when innovation is driven by a crisis requiring an urgent response (Odds of 2.2 to 2.8).
- 29. Factors correlated withone or more negative effects from the most important innovation (MII) • The absence an inclusive culture increases the odds of a negative effect from the MII by 2.5 times. • When the idea for the MII is obtained from the Senior executive versus the respondent, the odds of a negative effect is increased by 1.9 times. – (respondent better informed or tries harder?) OECD 2019
- 30. Conclusions • We canmeasure administrative innovations, innovation activities and obstacles reported by university managers. – Beneficial outputs are harder to measure (insufficient time?) – Problem of self-reported outcomes • What works: collaboration, inclusive culture, design-thinking methods, other innovation support methods, sufficient resources – Similar success factors as in the private sector and in other public sector innovation surveysOECD 2019
- 31. Conclusions: An inclusive innovationculture – Large impact on the use of ‘best practice’ innovation support methods such as design- thinking. – Substantially decreases the probability of an abandoned or under-performing innovation and negative effects from a most important innovation. – No effect on novel innovations. – Positive but not robust effect on a few beneficial outcomes of the most important innovation. OECD 2019
- 32. Further information University surveyresults: https://www.oecd.org/sti/102%20- %20ARUNDEL%20innovation%20in%20universities.pdf Overview of measuring innovation in the public sector: Arundel A, Bloch C, Ferguson B. Advancing innovation in the public sector: aligning innovation measurement with policy goals, Research Policy, 2918, doi.org/10.1016/j.respol.2018.12.001. (Open access) OECD 2019