LINQ 2013
Innovations and Quality: The Future of Digital Resources
Rome, May 16-17, 2013
EVALUATING THE QUALITY OF
DIGITAL...
Overview
1. Open education: a brief history
2. Criteria: the ACTIONS model
3. Applying the model to:
• MOOCs
• OERs
• Open...
Landmarks in the history of open
education
• 1870: compulsory state-funded
public education up to 13 years
old (England)
•...
Landmarks in the history of open
education
• 1963: Robbins Report: ‘all
qualified by ability and
attainment’ (UK)
• 1969: ...
Landmarks in the history of open
education
• 1998: open educational
resources: ‘resources that
reside in the public domain...
Landmarks in the history of open
education
• 2007: iTunes U (1 billion
downloads, 2013; UK OU: 50
million)
• 2008: MOOCs (...
How do we evaluate open digital
resources?
What do we mean by ‘open’? Free?
Accessible? Flexible?
Is being ‘open’ enough? ...
How do we evaluate open digital
resources?
A C T I O N S model for evaluating
educational media (Bates, 1995)
A ccess
C os...
Models of innovative open education
Four models:
• MOOCs: Coursera (Duke University)
• Open educational resources: MERLOT
...
Model 1: MOOC
Duke University (Coursera):
bioelectricity
• open access (no formal qualifications)
• fully online (lecture ...
Model 2: OER
MERLOT: collection of peer-reviewed
OERs
• Roman architecture, Yale
• open access
• fully online (multi-media...
Model 3: Open university
Open University, UK: B.Sc. (Hons.) Science
• Open access (no formal qualifications)
• Multimedia ...
Model 4: Online credit program
UBC online credit program: Master in
Educational Technology (fully online)
• Single courses...
Applying the criteria:
Access (developed world)
14
Technology
(max 10 = good)
Admission
(max 10 = open)
MOOC 8 10
OER 9 10...
Applying the criteria:
Cost
15
Institutional
(max 10 =low)
Student
(max 10 = low)
MOOC 4 10
OER 7 10
Open university 6 4
C...
Applying the criteria:
Teaching
16
Content
(max 10 = good)
Pedagogy/completi
on rates
(max 10 = good)
MOOC 10 2
OER 7 5
Op...
Applying the criteria:
Interaction
17
With instructor
(max 10 = good)
Social (inc. tutor)
(max 10 = good)
MOOC 1 3
OER 1 5...
Applying the criteria:
Organization as a barrier to open
18
(max 10 = good)
MOOC 8
OER 5
Open university 8
Credit online 5
Applying the criteria:
Novelty/hype/PR
19
(max 10 = high)
MOOC 10
OER 8
Open university 6
Credit online 4
Applying the criteria:
Security/quality assessment
20
(max 10 = high)
MOOC 1
OER 3
Open university 8
Credit online 9
Making comparisons
21
MOOC OER Open U Credit
Access 18 19 19 16
Cost 14 17 10 11
Teaching 12 12 13 17
Interaction 4 6 10 1...
Conclusions
• ‘open’ is a multi-faceted concept
• access alone is not enough; quality
matters
• need a clear set of criter...
Questions
Evaluating the quality of open digital resources
1. Are these the right criteria? If not, what other criteria sh...
Upcoming SlideShare
Loading in...5
×

Linq 2013 plenary_keynote_bates

354

Published on

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
354
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Linq 2013 plenary_keynote_bates

  1. 1. LINQ 2013 Innovations and Quality: The Future of Digital Resources Rome, May 16-17, 2013 EVALUATING THE QUALITY OF DIGITAL RESOURCES Dr. Tony Bates, Tony Bates Associates Ltd, Vancouver, BC, Canada 1
  2. 2. Overview 1. Open education: a brief history 2. Criteria: the ACTIONS model 3. Applying the model to: • MOOCs • OERs • Open Universities • Online credit programs 4. Making comparisons 5. Conclusions
  3. 3. Landmarks in the history of open education • 1870: compulsory state-funded public education up to 13 years old (England) • 1944: GI Bill: access to higher education: 20 million (USA)
  4. 4. Landmarks in the history of open education • 1963: Robbins Report: ‘all qualified by ability and attainment’ (UK) • 1969: The Open University (UK): ‘open to people, places, methods, ideas’: 1.7 million University of East Anglia Walton Hall
  5. 5. Landmarks in the history of open education • 1998: open educational resources: ‘resources that reside in the public domain’ • 2001:Wikipedia • 2002: MIT OpenCourseWare
  6. 6. Landmarks in the history of open education • 2007: iTunes U (1 billion downloads, 2013; UK OU: 50 million) • 2008: MOOCs (Coursera: 3.4 million) Dave Cormier’s YouTube description of cMOOCs
  7. 7. How do we evaluate open digital resources? What do we mean by ‘open’? Free? Accessible? Flexible? Is being ‘open’ enough? Don’t forget the ‘education’ part ‘Open’ is an emotionally charged word Need for criteria that are VALID and RELIABLE 7
  8. 8. How do we evaluate open digital resources? A C T I O N S model for evaluating educational media (Bates, 1995) A ccess C ost T eaching I nteraction O rganization N ovelty S ecurity 8
  9. 9. Models of innovative open education Four models: • MOOCs: Coursera (Duke University) • Open educational resources: MERLOT • Open university: UK • Online credit program: UBC, Canada ‘quality assessment’ using the A C T I O N S model 9
  10. 10. Model 1: MOOC Duke University (Coursera): bioelectricity • open access (no formal qualifications) • fully online (lecture capture, CMAs) • free • top private research university in the USA • statement of accomplishment 10
  11. 11. Model 2: OER MERLOT: collection of peer-reviewed OERs • Roman architecture, Yale • open access • fully online (multi-media) • free • ‘open’ use 11
  12. 12. Model 3: Open university Open University, UK: B.Sc. (Hons.) Science • Open access (no formal qualifications) • Multimedia materials, tutors and online forums • Cost: 17,000 euros = 5,700 per year; (international students pay more) • Top 20% in research and teaching in UK 12
  13. 13. Model 4: Online credit program UBC online credit program: Master in Educational Technology (fully online) • Single courses/certificate: open access • Masters: graduate entry qualifications (bachelor’s degree) • From anywhere in world • Cost: 11,000 euros =5,500 per year or 1,100 per course • Tier 1 research university 13
  14. 14. Applying the criteria: Access (developed world) 14 Technology (max 10 = good) Admission (max 10 = open) MOOC 8 10 OER 9 10 Open university 9 10 Credit online 9 7
  15. 15. Applying the criteria: Cost 15 Institutional (max 10 =low) Student (max 10 = low) MOOC 4 10 OER 7 10 Open university 6 4 Credit online 6 5
  16. 16. Applying the criteria: Teaching 16 Content (max 10 = good) Pedagogy/completi on rates (max 10 = good) MOOC 10 2 OER 7 5 Open university 7 6 Credit online 9 8
  17. 17. Applying the criteria: Interaction 17 With instructor (max 10 = good) Social (inc. tutor) (max 10 = good) MOOC 1 3 OER 1 5 Open university 3 7 Credit online 8 7
  18. 18. Applying the criteria: Organization as a barrier to open 18 (max 10 = good) MOOC 8 OER 5 Open university 8 Credit online 5
  19. 19. Applying the criteria: Novelty/hype/PR 19 (max 10 = high) MOOC 10 OER 8 Open university 6 Credit online 4
  20. 20. Applying the criteria: Security/quality assessment 20 (max 10 = high) MOOC 1 OER 3 Open university 8 Credit online 9
  21. 21. Making comparisons 21 MOOC OER Open U Credit Access 18 19 19 16 Cost 14 17 10 11 Teaching 12 12 13 17 Interaction 4 6 10 15 Organiz. 8 5 8 5 Novelty 10 8 6 4 Security/as sessment 1 3 8 9 Total 67 70 74 77
  22. 22. Conclusions • ‘open’ is a multi-faceted concept • access alone is not enough; quality matters • need a clear set of criteria; alternative models possible • more innovation needed; and possible • But: well-funded public education system best guarantee of open access 22
  23. 23. Questions Evaluating the quality of open digital resources 1. Are these the right criteria? If not, what other criteria should be considered? 2. Do the numerical values make sense? 3. Is there a better/more ‘scientific’ way to evaluate quality? 23
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×