Virtual schools success criteria draft final
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Virtual schools success criteria draft final

  • 779 views
Uploaded on

Draf

Draf

More in: Design
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
779
On Slideshare
779
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
6
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Success Criteria for Virtual Schools - Pick&Mix Professor Paul Bacsich Sero Consulting Ltd and Matic Media LtdiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 1
  • 2. Topics1. Introduction, disclaimers and acknowledgements2. History of Pick&Mix in universities3. Why is he telling us this?4. Pick&Mix5. More recent history: application to colleges and high schools6. Reflections on this processiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 2
  • 3. 1. Introduction, disclaimers and acknowledgementsiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 3
  • 4. Who is this talk for?Want to know about comparing your virtual school with other ones?Want to know about the “tradecraft” of benchmarking and quality reviews?Want to adapt or update a benchmarking system?Want to learn some of the underlying principles of such schemes?iNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 4
  • 5. Pick&Mix is an “Open Educational Methodology” for benchmarking online learning, developed by Paul Bacsich and available for all to develop and modify, but the names “Pick&Mix” and “ELDDA” are reserved for the “main sequence” of development Thanks to many, including UK HE Academy, JISC, EU Lifelong Learning Programme (Re.ViCa and VISCED), Manchester Business School, University of Leicester and Sero Consulting Ltd for supportiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 5
  • 6. 2. History of Pick&Mix in universitiesiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 6
  • 7. Benchmarking online learning At national level, started in 2004-05 in UK and New Zealand – Soon spread to Australia – Not closely linked initially to quality agenda At European level, developments include E-xcellence and UNIQUe – Some earlier work from OBHE, ESMU etc – Later, developments in other projects – Increasingly, links made to quality agenda and to critical success factorsiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 7
  • 8. Pick&Mix history Initial version developed in early 2005 in response to a request from Manchester Business School for an international (largely US) competitor study Since then, refined by literature search, discussion, feedback, presentations, workshops, concordance studies and four phases of use – sixth and seventh phases now Forms the basis of the wording of the Critical Success Factors scheme for the EU Re.ViCa project and now being used to develop a similar scheme for Virtual Schools in VISCED iNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 8
  • 9. Benchmarking e-learning (UK) Foreseen in HEFCE 2005 e-learning strategy for universities in England (later for Wales via HEFCW) Higher Education Academy (HEA) oversaw it Four phases – 82 institutions – 5 methodologies Two consultant teams – one run by myself My team benchmarked over 40 institutions using 4 methodologies Including 24 using Pick&Mix – now well over 30iNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 9
  • 10. What HEFCE wanted“Possibly more important is for us [HEFCE] to help individual institutions understand their own positions on e-learning, to set their aspirations and goals for embedding e-learning – and then to benchmark themselves and their progress against institutions with similar goals, and across the sector” iNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 10
  • 11. Methodologies in UK HE Of the five methodologies used in the UK from 2005 on, only one survives that is actively under development, refined annually, public domain and available for supervised or self- applied use in institutions, via funded projects and commercially:Pick&Mix In other countries’ HE systems: – eMM is in a similar situation in New Zealand – Quality Matters is widespread in US – ACODE is used somewhat in Australia – On the continent of Europe there are a few methodologies but fostered purely by EU-funded projects iNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 11
  • 12. Pick&Mix overview Focussed on online learning, not general pedagogy Draws on several sources and methodologies – UK and internationally (including US – especially Quality on the Line) and from college sector Not linked to any particular style of online learning (e.g. distance or on-campus or blended) Oriented to institutions with notable activity in online learning Suitable for desk research as well as “in-depth” studies Suitable for single- and multi-institution studiesiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 12
  • 13. 3. But why is he telling us this? BECAUSE IT CAN BE USED FOR COLLEGES AND SCHOOLS ALSOiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 13
  • 14. 4. Pick&Mix Criteria and metricsiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 14
  • 15. Criteria Criteria are “statements of practice” which are scored into a number of performance levels from bad/nil to excellent It is crucial that these statements are in the public domain – to allow analysis & refinement The number of criteria is also crucial:  24 Pick&Mix originally had a core of 20 – based on analysis from the literature (ABC, BS etc) and experience in many senior mgt scoring meetingsiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 15
  • 16. Pick&Mix: 20 core criteria Removed any not specific to online learning – Including those in general quality/accreditation schemes Careful about any which are not provably success factors Left out of the core were some criteria where there was not yet UK consensus Institutions will wish to add some to monitor their KPIs and objectives. Recommended no more than 6. – Pick&Mix now has over 70 supplementary criteria to choose from – more can be constructed or taken from other schemes These 20 have stood the test of four phases of benchmarking with only minor changes of wording – originally 18 - two were split to make 20iNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 16
  • 17. Pick&Mix ScoringUse a 6-point scale (1-6) – 5 (cf Likert, MIT90s levels) plus 1 more for “excellence”Contextualised by “scoring commentary”There are always issues of judging progress especially “best practice”The 6 levels are mapped to 4 colours in a “traffic lights” system – red, amber, olive, greeniNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 17
  • 18. Pick&Mix System: summaryHas taken account of “best of breed” schemesOutput and student-oriented aspectsMethodology-agnostic but uses underlying approaches where useful (e.g. Chickering & Gamson, Quality on the Line, MIT90s)Requires no long training course to understandiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 18
  • 19. P01 “Adoption” (Rogers)1. Innovators only2. Early adopters taking it up3. Early adopters adopted; early majority taking it up4. Early majority adopted; late majority taking it up5. All taken up except laggards, who are now taking it up (or retiring or leaving)6. First wave embedded, second wave under way (e.g. BYOD-learning after e-learning)iNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 19
  • 20. P10 “Training”1. No systematic training for e-learning2. Some systematic training, e.g. in some projects and departments3. Institution-wide training programme but little monitoring of attendance or encouragement to go4. Institution-wide training programme, monitored and incentivised5. All staff trained in VLE use, training appropriate to job type – and retrained when needed6. Staff increasingly keep themselves up to date in a “just in time, just for me” fashion except in situations of discontinuous changeiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 20
  • 21. Supplementary criteria - examples IT reliability Market research Competitor research IPR Help Desk Management of student expectations Student satisfaction Web 2.0 pedagogyiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 21
  • 22. 5. More recent history Adaptation to colleges and high schools (both virtual and blended)iNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 22
  • 23. Virtual post-secondary institutionsThe EU Re.ViCa project 2007-09 (Review of Virtual Campuses) did a great deal of work to refine and “Europeanise” the Pick&Mix criteria, focussing on – Critical Success Factors for virtual campusesA slightly revised scheme was produced, oriented to institutions where distance learning was the prevalent approachiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 23
  • 24. Distance Learning Benchmarking Club 2009-11In summary: Universities, encouraged by a UK-funded project, formed a group to benchmark themselves: – University of Leicester (UK) – Royal Swedish Institute of Technology (KTH) – Lund University (Sweden) – University College Gotland (Sweden) – Thompson Rivers University (Canada)iNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 24
  • 25. VISCED 2011-12A further iteration is in final stages to produce a scheme of Critical Success Factors for virtual schools – With initial focus to European virtual schools – Since these are smaller and few have ever failed, this is a hard taskiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 25
  • 26. Draft key success factors for European virtual schools1. Usability (for all participants)2. Strategy (for online learning)3. Recruitment and training (fused?)4. Evaluation (of programmes) – often informal but effective5. Reliability (of system)6. Leadership (with knowhow, down and up – fused?)7. Organisation (formal and informal)8. Learning outcomes (often individualised)d9. Use of resources (not key in universities)10. Market research? iNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 26
  • 27. 6. Reflections on this processiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 27
  • 28. Too many concepts (Critical) Benchmarking Success Factors Accreditation Standards /approval Quality /kitemarking Online learning is only a small part of the quality process – how can agencies and assessors handle five variants of the concept across many separate methodologies?iNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 28
  • 29. My view for ENQA – the pyramid Critical Success Leadership level Factors, a selection of: Benchmarking, which Senior managers split into sub-criteria: Quality, split into Criteria are placed at different layers in the pyramid Detailed pedagogic depending on their “level” guidelines ----------iNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 29
  • 30. AdaptabilityThe transition of a basically university- originated methodology to schools is feasible, particularly for larger high schools (virtual or blended)The sectors are not as different as territorial experts like to believeBut the methodology has to be theoretically sound and well-researchedThere are other extensions - eg for OER !iNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 30
  • 31. Merging US and EU practice? But could we cross the transatlantic divide? There were some discussions between EU institutions and QM over a common HE system (UNESCO would love that) For schooIs, we need to look at iNACOL guidelines – meeting after the Symposium Since Pick&Mix was heavily influenced by US experience (Ehrmann; Quality on the Line) there should be hope – at first sight it looks promising A hierarchical (pyramid) approach will be key iNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 31
  • 32. A view after 7 years of Pick&Mix Methodologies do not survive without regular updating by a design authority – this is difficult in a leaderless group context Forking of methodologies needs dealt with by folding updates back to the core system – otherwise survival is affected Complex methodologies do not survive well A public criterion system allows confidence, transparency, and grounding in institutions Sectoral boundaries can be overcomeiNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 32
  • 33. ReferencesThe Pick&Mix system is detailed athttp://www.matic-media.co.uk/benchmarking/PnM-2pt6-beta3-full.xlsxA key paper on the international aspects is:“Benchmarking E-Learning in UK Universities: Lessons from and for the InternationalContext”, in Proceedings of the ICDE conference M-2009, Open Praxis – http://www.openpraxis.com/files/Bacsich%20et%20al..pdfA specific chapter on the UK HE benchmarking programme methodologies is:“Benchmarking e-learning in UK universities – the methodologies”, inMayes, J.T., Morrison, D., Bullen, P., Mellar, H., and Oliver, M.(Eds.)Transformation in Higher Education through Technology-Enhanced Learning,York: Higher Education Academy, 2009 –http://www.heacademy.ac.uk/assets/documents/learningandtech/Transforming-07.pdf iNACOL Virtual Schools Symposium 2012, New Orleans – October 2012 33