Grappling with "openness" and "impact in OER research. Paper presented at Open Education Global Conference, Banff, April 2015

1. Cheryl Hodgkinson-Williams
University of Cape Town
Open Education Global Conference, Banff, Canada
22-24 April 2015
Grappling with the concepts of “impact” and
“openness” in relation to OER: Current
developments in the ROER4D Project

2. Rising numbers of students in
the education sector
Education institutions under
political & financial pressure
Expensive, limited in number,
often outdated textbooks are
not entirely relevant to the
context
Employability of graduates
Reduction of educational
funding by governments
Key challenges facing education in developing countries
http://commons.wikimedia.org/wiki/File%3APCoE_students_at_AICTE_Regional_Office_in_Mumbai.jpg

3. OER as a response to some educational challenges
facing education in developing countries
http://www.elearning-africa.com/eLA_Newsportal/finding-the-sweet-spot-open-educational-resources-in-the-developing-world/
http://www.unesco.org/iiep/virtualuniversity/forumsfiche.php?queryforumspages_id=23 x

4. OER as a response to some educational challenges
facing education in general – NMC Horizon Report 2015
https://net.educause.edu/ir/library/pdf/HR2015.pdf

5. Slow uptake of OER in South America & Africa,
better in South East Asia
http://ocw.mit.edu/about/site-statistics/

6. https://stateof.creativecommons.org/report/

7. http://oermap.org/oer-evidence-map/
Most OER research taking place in Global North

8. ROER4D Research: OER Adoption & Impact

9. ROER4D Funding
International Development Research
Centre (IDRC)
Open Society Foundations (OSF)
UK Department for International
Development (DFID)
3 year project (27 Aug 2013 - 27 Aug
2016 with an extension to Feb 2017)
Grant 1 - IDRC CAD 2 million & OSF
Grant 2 - DFID CAD 500,000
3 Regions
 South America
 Sub-Saharan Africa
 Central, South & South-East Asia
18 research projects in 7 clusters
86 researchers & associates
26 countries
16 time zones
IDRC
OSF DFID

10. In what ways, and under what circumstances, can the adoption of
OER impact upon the increasing demand for accessible, relevant,
high-quality, and affordable education in the Global South?
Research on Open Educational Resources for Development (ROER4D)

11. In what ways, and under what circumstances, can the adoption of
OER impact upon the increasing demand for accessible, relevant,
high-quality, and affordable education in the Global South?
Research on Open Educational Resources for Development (ROER4D)
ADOPTION STUDIES
1. In what ways, and under what
circumstances are OER being
adopted in the Global South?

12. In what ways, and under what circumstances, can the adoption of
OER impact upon the increasing demand for accessible, relevant,
high-quality, and affordable education in the Global South?
Research on Open Educational Resources for Development (ROER4D)
IMPACT STUDIES
2. In what ways, and under what
circumstances can OER adoption impact
upon the increasing demand for accessible,
relevant, high-quality, and affordable
education in the Global South?
ADOPTION STUDIES
1. In what ways, and under what
circumstances are OER being
adopted in the Global South?

13. ROER4D
Network hub
OER Desktop
overview (1) Survey of OER
adoption by
academics & students
(1)
Academics’
adoption of OER
(2)
Teacher educators’
adoption of OER (3)
OER adoption in
one country (1)
OER impact
studies (7+1)
Baseline educational
expenditure (2)
Overview of ROER4D’s 7 Project Clusters

14. ROER4D’s Project Clusters & Coordination

15. Guiding questions to tease out “openness” &
“impact”
1. What is the
problem?
2. What is the
claim about
OER?
3. What are the
specific
objectives?
4. What would
count as
evidence?
5. From whom
could evidence
be obtained?
6. How would
you gather
evidence?
7. What is the
theory of
change?
1. What is the
problem?

16. 1. What is the problem?
 The problem is NOT the lack of OER uptake in the Global South per se
 The challenge is to identify the original / implicit / underlying educational problem that OER was
intended to address, e.g.
 Educational materials
 Increasing cost of textbooks, learning and teaching resources
 Out datedness of curricula, textbooks, learning and teaching resources
 Inaccessibility of books, journals, multimedia resources behind pay walls
 Generic texts and/or texts in a language other than the local language
 Learner-related problems
 Increasing numbers of students requiring education
 Lack of participation in informal and/or formal education
 Poor student retention
 Life-long learning required for 20th century work
 Educator-related problems
 Transmissionist pedagogy or teacher-centred approaches
 Lack of time to develop and re-develop good quality materials
Why it is important to identify the problem?
• To be able to identify the impact of the use of OER on the problem
OER for what?

17. 1. What is the
problem?
2. What is the
claim about
OER?
3. What are the
specific
objectives?
4. What would
count as
evidence?
5. From whom
could evidence
be obtained?
6. How would
you gather
evidence?
7. What is the
theory of
change?
Guiding questions
2. What is the
claim about
OER?

18. 2. What is the claim about OER adoption?
 The “promises” made about OER include:
 Educational materials
 Decreasing the cost of textbooks, learning and teaching resources
 Increasing the currency of textbooks, learning and teaching resources
 Allowing access to books, journals, multimedia resources previously
copyrighted and/or behind pay walls
 Localising and/or translating texts
 Learner-related problems
 Increasing numbers of students requiring education
 Widening participation in informal and/or formal education
 Improving student retention
 Providing opportunities for life-long learning
 Educator-related problems
 Improving teacher pedagogy to more constructivist / learner-centred
 Reducing the time to develop and re-develop good quality materials
Why it is important to identify the OER claims?
• To be able to formulate the implicit or explicit hypothesis
Use of
OER will
do what?

19. Guiding questions
1. What is the
problem?
2. What is the
claim about
OER?
3. What are the
specific
objectives?
4. What would
count as
evidence?
5. From whom
could evidence
be obtained?
6. How would
you gather
evidence?
7. What is the
theory of
change?
3. What are the
specific
objectives?

20. 3. What are the specific objectives?
 The challenge is to identify and define the key concepts underpinning the
practices being investigated. Are the objectives to establish:
 How aware people are of the concept of sharing materials freely and
legally
 How they find OER, in contrast to other materials on the Internet
 How and why they create OER
 Why they only reuse OER “as is” (i.e. make a copy)
 In what ways they revise OER (e.g. translate, re-sequence)
 In what ways the remix OER (e.g. combine with other OER)
And how these practices might influence the cost, relevance, accessibility of
educational materials
Why it is important to specify objectives?
• To be able to narrow down the research focus and formulate specific research
questions
• Tease out the concepts – e.g. what is OER; does it include OA articles?
What
‘open’
practices?

21. Hodgkinson-Williams & Gray (2009:110) & Hodgkinson-Williams (2014)
Copy
Customise (e.g. translate, add local content)
Combine (e.g. select and mix content)
Contribute (e.g. share locally & with the world)
OER: Degrees of openness: Practices & licences

22. Hodgkinson-Williams & Gray (2009:110) & Hodgkinson-Williams (2014)
Copy
Customise (e.g. translate, add local content)
Combine (e.g. select and mix content)
Contribute (e.g. share locally & with the world)
OER: Degrees of openness: Practices & licencesPRACTICELEGALTECHNICAL

24. Guiding questions
1. What is the
problem?
2. What is the
claim about
OER?
3. What are the
specific
objectives?
4. What would
count as
evidence?
5. From whom
could evidence
be obtained?
6. How would
you gather
evidence?
7. What is the
theory of
change?
4. What would
count as
evidence?

25. 4. What would count as evidence?
 The key challenge here is to map the problem, the OER claim (ie. the
underlying hypothesis) to the potential evidence, e.g.
Why it is important to establish what would count as evidence?
• To be able to establish the legality, ethics and feasibility of the study
• To establish what evidence would be most compelling (ie. “gold”), moderately
convincing (ie. “silver”) or marginally supportive (ie. “bronze”)
• To preempt the type of data reseachers can put forward as impact measures
before actually conducting the study
Problem Claim Evidence
Increasing cost of
textbooks, learning and
teaching resources
Decreasing the cost of
textbooks, learning and
teaching resources
Calculation of savings
based on number of OER
users who would have
spent funds on traditional
textbooks/educational
materials

26. Guiding questions
1. What is the
problem?
2. What is the
claim about
OER?
3. What are the
specific
objectives?
4. What would
count as
evidence?
5. From whom
could you
obtain
evidence?
6. How would
you gather
evidence?
7. What is the
theory of
change?
5. From whom
could evidence
be obtained?

27. 5. From whom can evidence be obtained?
 Who will gain from the benefit of the use of OER for xxx problem?
E.g. Cost reduction for …
 governments for some K-12 materials
 parents, bursary providers and/or students in college or university
 Who will suffer some consequence of the use of OER for xxx problem?
E.g. Reduction in income for …
 textbook and/or journal publishers
 educators who write their own books (that may have copyrighted
materials) and sell these to the students
Why it is important to identify from whom evidence can be obtained?
• To be able to establish from whom you can elicit responses directly and whose
perceptions might need to be ascertained indirectly

28. Guiding questions
1. What is the
problem?
2. What is the
claim about
OER?
3. What are the
specific
objectives?
4. What would
count as
evidence?
5. From whom
could evidence
be obtained?
6. How would
you gather
evidence?
7. What is the
theory of
change?
6. How can
evidence be
gathered?

29. 6. How can evidence be gathered?
Direct Indirect
Observation of performance Interview about perception of
performance
Artefact analysis Interview or questionnaire about
perceptions of quality
Interview of beliefs Survey of practices
Achievement scores Interview or questionnaire of perceptions
of value of OER in improved performance
Why it is important to identify how evidence can be gathered?
• To isolate the ‘open’ practices as carefully as possible
• To obtain the most accurate and comprehensive evidence to support your ‘impact’
claim

30. Guiding questions
1. What is the
problem?
2. What is the
claim about
OER?
3. What are the
specific
objectives?
4. What would
count as
evidence?
5. From whom
could you
obtain
evidence?
6. How would
you gather
evidence?
7. What is you
theory of
change?
7. What is the
theory of
change?

31. Open practices
enabled by CC &
open formats
“Openness”
Impact on
underlying
problem
“Impact”
Teasing out “openness” and “impact”

32. Guiding questions
1. What is the
problem?
2. What is the
claim about
OER?
3. What are the
specific
objectives?
4. What would
count as
evidence?
5. From whom
can evidence
be obtained?
6. How can
evidence be
gathered?
7. What is you
theory of
change?

33. Links
Website: www.roer4d.org
Contact Principal Invesigator:
cheryl.hodgkinson-williams@uct.ac.za
Follow us: http://twitter.com/roer4D
Presentations: www.slideshare.com/roer4D

34. Acknowledgments & Attribution
This work is licensed under a Creative Commons Attribution
4.0 International License.
Written by Cheryl Hodgkinson-Williams,
Henry Trotter, Tess Cartmill, Sukaina Walji,
Sarah Goodier & Thomas King
Contact:
cheryl.hodgkinson-williams@uct.ac.za
Graphics by Rondine Carstens
rondine.carstens@uct.ac.za, Cheryl
Hodgkinson-Williams & Henry Trotter
henry.trotter@uct.ac.za