Quality, benchmarking and success factors in virtual schools
Upcoming SlideShare
Loading in...5
×
 

Quality, benchmarking and success factors in virtual schools

on

  • 485 views

Presentation to a workshop joint with Walter Kugemann to the EFQUEL Academy just prior to the EFQUEL Innovation Forum

Presentation to a workshop joint with Walter Kugemann to the EFQUEL Academy just prior to the EFQUEL Innovation Forum

Statistics

Views

Total Views
485
Views on SlideShare
484
Embed Views
1

Actions

Likes
0
Downloads
4
Comments
0

1 Embed 1

http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Quality, benchmarking and success factors in virtual schools Quality, benchmarking and success factors in virtual schools Presentation Transcript

    • Quality in Virtual Schools and Colleges – a view from VISCED Paul Bacsich (paul.bacsich@sero.co.uk) VISCED – www.virtualschoolsandcolleges.info1 EFQUEL Academy
    • Introduction2 EFQUEL Academy
    • Topics• Historical overview• The Pick&Mix system• Supplementary criteria, local criteria and slices• Scoring, evidence and documentation3 EFQUEL Academy
    • Benchmarking and Quality in e- learning: My part of the story Overview and history4 EFQUEL Academy
    • Benchmarking e-learning (UK)• Foreseen in HEFCE e-learning strategy• Higher Education Academy oversaw it• Four phases – 82 institutions (2005-07)• Justified entry to Pathfinder and Enhancement programmes - and useful for JISC bids (Curriculum Design etc)• Was leveraged into refresh of learning and teaching strategy (Leicester etc)5 EFQUEL Academy
    • What is benchmarking? (HEFCE)“*for+ individual institutions [to] understand their own positions on e-learning, to set their aspirations and goals for embedding e- learning… to benchmark themselves and their progress against institutions with similar goals, and across the sector [world]”Can replace e-learning by “distance learning”, “OER”, “podcasting” ,etc6 EFQUEL Academy
    • Methodologies: Pick&Mix • First used for Manchester Business School 2005 • Has been used in 36 institutions, mainly new universities and university colleges – Including 4 very diverse institutions in Wales (2007-09) – And 5 in the Distance learning benchmarking Club (UK, Sweden, Canada) plus two more in UK (2009-12) – And basis for the Critical Success Factors schemes in two EU projects: Re.ViCa (2007-09: developments fed back into Pick&Mix) and now VISCED (2011-12) – Also an OER version of the scheme ready for use • Wording slightly revised for schools and colleges7 EFQUEL Academy
    • Pick&Mix Criteria and metrics8 EFQUEL Academy
    • Criteria• Criteria are “statements of practice” which are scored into a number of performance levels from “bad” to “best” to “excellent”• These statements are in the public domain – to allow analysis, refinement etc (cf.OER)• Pick&Mix has over 80 criteria… – But typically a set of 20-30 is chosen – this range is based on analysis from the literature (ABC, BS etc) and experience in many scoring meetings9 EFQUEL Academy
    • Pick&Mix Metrics• Use a 6-point scale (1-6) – ignore 6 if you wish – 5 (cf Likert, MIT90s levels) plus 1 more for “excellence”• Backed up by continuous metrics where possible• Also contextualised by narrative• The 6 levels are mapped to 4 colours in a “traffic lights” system (red, amber, olive, green)10 EFQUEL Academy
    • Pick&Mix Three sample criteria11 EFQUEL Academy
    • P01 “Adoption” (Rogers) – not used now1. Innovators only2. Early adopters taking it up3. Early adopters adopted; early majority taking it up4. Early majority adopted; late majority taking it up5. All taken up except laggards, who are now taking it up (or retiring or leaving)6. First wave embedded, second wave under way (e.g. m-learning after e-learning)12 EFQUEL Academy
    • P10 “Training”1. No systematic training for e-learning2. Some systematic training, e.g. in some projects and departments3. Institution-wide training programme but little monitoring of attendance or encouragement to go4. Institution-wide training programme, monitored and incentivised5. All staff trained in VLE use, training appropriate to job type – and retrained when needed6. Staff increasingly keep themselves up to date in a “just in time, just for me” fashion except in situations of discontinuous change13 EFQUEL Academy
    • P05 “Accessibility” – tough one1. e-learning material and services is not accessible2. Much e-learning material and most services conform to minimum standards of accessibility3. Almost all e-learning material and services conform to minimum standards of accessibility4. All e-learning material and services conform to at least minimum standards of accessibility, much to higher standards5. e-learning material and services are accessible, and key components validated by external agencies6. Strong evidence of conformance with letter & spirit of accessibility in all countries where students study 14 EFQUEL Academy
    • Pause for questions15 EFQUEL Academy
    • Extensions, local criteria and slices16 EFQUEL Academy
    • Extensions• Pick&Mix started with 20 core criteria – but now there are over 80 – and a different core (2.0)• There is room for more supplementary criteria – and further analyses are always being done to ensure topicality• Web 2.0 aspects are already covered• But it is mainly the institutions who have kept and will help to keep the system up to date• And some EU and other projects17 EFQUEL Academy
    • Supplementary criteria - examples• They include: – IT reliability – Market research, competitor research – IPR – Help Desk – Management of student expectations – Student satisfaction18 EFQUEL Academy
    • Slices• As well as benchmarking the whole institution it is wise in large institutions to look at a few “slices”:• Useful to give a context to scores• Slices need not be organisational or subject-based – Distance learning (is a natural one) – Thematic or dimensional like HR, costs…19 EFQUEL Academy
    • Scoring• Scores are usually in the range 1-5, point 6 is exceptional and cannot be planned for• Focus not only on best practice but also bad practice and real practice• Do not try to ensure that your institution scores 5 overall• 6 sometimes has aspects of 120 EFQUEL Academy
    • Use in Distance Learning(such as Virtual Schools) details EFQUEL Academy 21
    • Distance learning• A variant “mood” of the default Pick&Mix was developed to benchmark distance learning• This was used in the Distance Learning Benchmarking Club: – University of Leicester – Thompson Rivers University – Lund University – Royal Swedish Technical Institute (KTH)• and also at the University of Gotland• Most recently at University of Northampton EFQUEL Academy 22
    • Evidence and documentation• Normally institutions want a final report but but the aim is NOT to recreate the QAA-style SED or DATs – the report is an outcome of the benchmarking process not the start of it• Have a “file” for each criterion• Some narrative will be needed• Institutions normally group criteria according to their own L&T strategy or in terms of “owning” departments – We also supply some standard groupings based on MIT90s but few use these23 EFQUEL Academy
    • Documentary support• There are many reports available – many but not all are public, others are available to those participating in Pick&Mix• The HE Academy wiki “was” a good source of information and the Academy blog structure “was to be” maintained indefinitely…. Not now• We store all Pick&Mix key material separately from the HE Academy• There are now many papers and a bibliography is available24 EFQUEL Academy
    • Ongoing work• Evolving Pick&Mix into a benchmarking and quality scheme for virtual schools: – With specialist help from EFQUEL – And a research team at KU Leuven – And correlation with iNACOL scheme in US• Version of Pick&Mix ready to benchmark OER uptake (developed with Bieke Schreurs, OUNL)• Variant moods for accessibility and employability (Wales) EFQUEL Academy 25
    • Thank you for listening email for this work is paul.bacsich@sero.co.uk Benchmarking E-Learning in UK Universities: Lessons from and for the International Context , published in http://www.openpraxis.com/files/Bacsich%20et%20al..pdf http://www.virtualcampuses.eu/index.php/Bibliography_of_benchmarking and http://www.mendeley.com/groups/1075191/benchmarking-e-learning/26 EFQUEL Academy