The document outlines a 2-year research project funded by the William & Flora Hewlett Foundation, aimed at analyzing the impact of Open Educational Resources (OER) on student performance and satisfaction. Findings from surveys revealed that over 60% of students and educators experienced increased satisfaction and interest in subjects due to the use of OER, with significant financial savings reported. The research also highlighted the collaboration model employed, along with an impact map hosted on the OER Research Hub, to further disseminate findings and encourage contributions to a comprehensive study on OER impact.

1. Student Efficacy: Are They
Learning?
Rapid Fire Panel, CNX 2014
1 April 2014
Dr. Beck Pitt
The Open University

2. Background

3. • 2-year research project based at The Open University (UK)
• Funded by William & Flora Hewlett Foundation
• Aiming to build the most comprehensive picture of OER impact
• Eleven research hypotheses
• Collaboration model across different educational sectors
• Fellowship Scheme
• Global reach but with a US focus
• Practicing openness: CC-BY licensed research instruments / SOO Course /
Impact Map
OER Research Hub
oerresearchhub.org
#oerrhub @OER_Hub

4. Keyword Hypothesis
Performance OER improve student performance/satisfaction
Openness People use OER differently from other online materials
Access OER widen participation in education
Retention OER can help at-risk learners to finish their studies
Reflection OER use leads educators to reflect on their practice
Finance OER adoption brings financial benefits for students/institutions
Indicators Informal learners use a variety of indicators when selecting OER
Support Informal learners develop their own forms of study support
Transition OER support informal learners in moving to formal study
Policy OER use encourages institutions to change their policies
Assessment Informal assessments motivate learners using OER

5. Collaboration Model
Collaboration Model

6. Collaboration Model
Collaboration Model

8. OER Impact Map

9. OER Impact Map
http://chaos.open.ac.uk

10. OER Impact Map
http://chaos.open.ac.uk

11. Methodology

12. Educator and Student Surveys

13. Overview
OER improve student performance/satisfaction
People use OER differently from other online materials
OER use leads educators to reflect on their practice
OER adoption brings financial benefits for students/institutions
Spring-Summer 2013
Developing
Questionnaires with
OpenStax College
IRB Process
Autumn 2013-Christmas 2013
Surveys deployed
Ongoing
Interviews with
educators and students
Comparative impact
data / Stories of Impact
Adding evidence to the
Impact Map
Jan-Apr 2014
Data analysis and
Dissemination of findings
Interviews with Educators

14. Student Survey
In total 63 usable survey responses were recorded:
• OpenStax Newsletter Student User Survey (50) Incentivised
• College One (3)
• College Two (10)
44 out of the 63 respondents have used, or currently use, OpenStax
College textbooks:
• OpenStax Newsletter Student User Survey (39)
• College One (2)
• College Two (3)

15. Educator Survey
In total 97 usable survey responses were recorded:
• OpenStax Adopter Email list (52)
• OpenStax Newsletter Educator Survey (41)* Incentivised
• Individual Educators from direct introductions (4)
82 out of the 97 respondents have used, or currently use, OpenStax
College textbooks:
• OpenStax Adopter Email list (47)
• OpenStax Newsletter Educator Survey (31)
• Individual Educators from direct introductions (4)

16. Student Sample
• 44 Respondents in total (have used/use OSC textbooks)
• 56.8% Male (n=25) and 43.2% Female (n=19)
• 86.0% of respondents live in the United States (n=37)
• Other respondents live in Germany, Canada, Hong Kong, UK & India (14.0%, n=6)
• 65.9% of respondents have English as their first spoken language (n=29)
• Just under 5% of respondents consider themselves to have a disability (4.5%, n=2)
• Just under 70% of respondents were between 15-18 years old (68.3%, n=28)
• Nearly 60% of respondents told us they are studying at School (59.1%, n=26).
• Over 60% of respondents reported having either a High School Diploma or No Formal
Qualifications (n=10, 28.6% and 34.3%, n=12 respectively)
• Just under 10% of respondents who use OSC textbooks receive any form of financial
aid (9.5%, n=4)

17. Educator Sample
• 82 Respondents in total (have used/use OSC textbooks)
• 75.3% Male (n=61) and 24.7% Female (n=20)
• Over 85% of respondents live in the United States (86.4%, n=70)
• Other respondents live in Canada, Singapore, Malaysia & Australia (13.6%, n=11)
• 90.0% of respondents have English as their first spoken language (n=72)
• Over 90% of respondents have a Masters degree and/or PhD (38.3%, n=31, 54.3%,
n=44 respectively)
• Over 50% of respondents work in a HE/University context (57.3%, n=47)
• 78.0% of respondents do F/T face-to-face teaching (n=64)
• Nearly 70% of respondents have been teaching for more than 10 years (67.5%, n=54)

18. OER Behaviours

19. Student Overview
• Over a quarter of respondents studied their subject via OER before joining
their course (26.8%, n=11).
• 17.5% of respondents told us that their use of OER influenced their decision
to register for their current course of study (n=7)
• Just under a third of respondents told us that they have adapted OER to fit
their needs (31.0%, n=13)
• The biggest challenge for respondents when using OER was knowing where
to find resources (61.0%, n=25)

20. Student Overview
• 75.0% of respondents first became aware of OpenStax College Textbooks
via their instructor/course (n=30).
• 54.5% of respondents currently use the Biology textbook for their studies
(n=24).
• 55.0% of students told us OpenStax College textbooks were required
reading and they did not purchase any other textbooks for their course
(n=22)

21. Educator Overview
• 92.6% of respondents have adapted OER to fit their needs
• Just under 50% of respondents have created OER for study or teaching
(48.1%, n=39)
• The biggest challenge for respondents when using OER was finding
resources of a sufficiently high quality (67.5%, n=54)*
• Top three types of OER used for teaching/training by respondents: open
textbooks (98.9%, n=81), videos (78.0%, n=64) and images (72.0%, n=59)
• Almost 60% of respondents think their institution benefits financially by using
OER (59.3%, n=48)

22. Educator Overview
• 34.6% of respondents first became aware of OpenStax College Textbooks
via an internet search (n=28).
• Over a quarter of respondents became aware of OSC via a colleague or
other recommendation (25.9%, n=21)
• 61.0% of respondents currently use the College Physics textbook with their
students (n=50).
• Over 80% of respondents teach Science (82.7%, n=67) with 77.8% using
OER in this area (n=63) and over a third creating OER in this subject
(34.6%, n=28)

23. Student Perspectives

24. In which of these ways (if any) has your use of Open
Educational Resources made an impact on your formal
studies? (Check all that apply)
"Using Open Educational Resources (e.g. OpenStax College textbooks)
has led to my...”
Increased interest in the subjects taught (69.0%, n=29)
Increased satisfaction with the learning experience (66.7%, n=28)
Grades improving (47.6%, n=20)
Increased participation in classroom discussions (45.2%, n=19)
Having increased independence and self-reliance (45.2%, n=19)
Increased engagement with lesson content (45.2%, n=19)
Increased experimentation with new ways of learning (45.2%, n=19)
Gaining confidence (38.1%, n=16)
Increased enthusiasm for future study (38.1%, n=16)
Increased collaboration with my peers (35.7%, n=15)
Being more likely to complete my course of study (33.3%, n=14)
Becoming interested in a wider range of subjects than before I used these
resources (31.0%, n=13)

25. In what ways, if any, has using OpenStax College textbooks
impacted on your studies?
Range of responses, including comments on:
40.6% Better Study Experience (n=13)
31.3% Accessibility of Resources (n=10)
12.5% Cost savings (n=4)
9.4% Improved Grades (n=3)
9.4% Less Weight to Carry (n=3)

26. In what ways, if any, has using OpenStax College
textbooks impacted on your studies?
“Saved my back, I already have so many
other books in my backpack and using my
computer for more info saved additional
stress that would have been on my back.”

27. Educator Perspectives

28. Based on your experiences as a teacher, to what extent do
you agree with the following statements?
% of educators who strongly agree or agree with the following statements:
"Use of Open Educational Resources (e.g. OpenStax College textbooks) in the
classroom...”
Allows me to better accommodate diverse learners’ needs (67.9%, n=53)
Increases learners’ satisfaction with the learning experience (65.8%, n=52)
Increases learners’ experimentation with new ways of learning (64.1%, n=50)
Increases learners’ engagement with lesson content (60.3%, n=47)
Increases learners’ interest in the subjects taught (55.7%, n=44)
Develops learners’ increased independence and self-reliance (55.7%, n=44)
Increases learners’ participation in class discussions (49.4%, n=39)
Leads to improved students’ grades (47.4%, n=37)
Increases learners’ enthusiasm for future study (45.6%, n=36)
Builds learners’ confidence (45.5%, n=35)
Increases collaboration and/or peer-support among learners (44.2%, n=34)
Leads to learners becoming interested in a wider range of subjects than before
they used OER (41.0%, n=32)

29. In what ways, if any, has using OpenStax College textbooks
impacted on your own teaching practice?

30. In what ways, if any, has using OpenStax College
textbooks impacted on your students?
“The resource gave them a sound
understanding of the topic and they
were able to do self-directed study
from it. The result is that seniors
have excelled in their exams.”

31. In what ways, if any, has using OpenStax College
textbooks impacted on your students?
“Students seem
somewhat more
engaged.”
“more student engagement
in lab activities and lecture
presentation”

32. As a result of using OpenStax College textbooks…
• 96.3% of educators who use OSC textbooks are more likely to
recommend OpenStax College textbooks to fellow
educators/teachers (n=77)
• 81.0% of respondents who use OSC textbooks are more likely
to discuss using OpenStax College textbooks with their
institution’s administrators (n=64)
• Over three quarters of respondents who use OSC textbooks are
more likely to use OER for teaching (77.9%, n=60)

33. Financial Savings

34. Student Financial Savings
Do you think you have saved money
by using OpenStax College
textbooks?
“Yes” 77.3%
“No” 15.9%
“Don’t Know”
6.8%
Do you believe that your students
have saved money by using OER?
“Yes”
89.0%
“No”
8.5%
“Don’t Know”
2.4%
By using OpenStax College textbooks each student
surveyed saved on average ca. $155 (n=19)

35. Summary

36. Summary of Results
Efficacy: Are they Learning?
• Over 60% of both students and educators report increased satisfaction with their
own or their students’ learning experience as a result of using OER/OSC
textbooks
• Almost 70% of students surveyed told us that they thought use of OER/OSC
textbooks had led to an increased interest in the subjects taught
• Over 45% of students thought that their grades had improved due to using
OER/OSC textbooks
• Emerging evidence for different ways in which “open” is making a specific
difference: students can both access materials earlier (e.g. pre-course/pre-payment
of financial aid) but can also continue to access/use resources post-course
(don’t have to sell textbook/limited access). Positive impact on learning?

37. Next Steps

38. Next Steps
• Further analyses of data: e.g. comparison of adopters with newsletter
respondents (educators), non-users/users of OSC textbooks, students studying
in different contexts, different OSC textbooks etc.
• Comparison of data with other open textbook surveys
• Disseminating results (Impact Map inc. anonymised data)
• Future/current work focused on building case studies and comparative impact
data
• We need you! Looking for YOUR best examples of impact...
• Help us build the most comprehensive picture of the impact of OER by
contributing your evidence to the Impact Map

39. Thanks for listening!
oerresearchhub.org
chaos.open.ac.uk
Twitter: @OER_Hub @BeckPitt

40. in service of The Open University