The document summarizes a presentation about research being conducted on the impact of open educational resource (OER) degrees. The research is examining student outcomes and costs across multiple colleges participating in an OER degree initiative. The evaluation includes quasi-experimental studies comparing academic results of students in OER degree programs versus traditional programs, as well as analysis of cost savings and sustainability. Preliminary findings suggest OER degrees improve student progress and reduce financial burden.

1. Student Impact and Cost Analysis
Research for OER Degrees
Jessica Mislevy, SRI Education
Rick Staisloff, rpk GROUP
April 12, 2017, 10:00 am PST
Unless otherwise indicated, this presentation is licensed CC-BY 4.0

2. Collaborate Window Overview
Audio & Video
Participants
Chat
Tech Support available at:
1-760-744-1150 ext. 1537, 1554

3. Agenda
• Introductions
• CCCOER Overview
• OER Degrees
• Research on Student & Faculty Impact
• Cost Analysis for Students and Institutions
• Stay in the Loop
• Q & A
Image Front Page Attribution:
“DNA Origami by Alex Bateman @sangerinsti” by dullhunk, on Flickr, cc-by license

4. Jessica Mislevy, PhD,
Senior researcher
Welcome
Moderator:
Una Daly, Director CCCOER
Open Education Consortium
Rick Staisloff,
Founder & Principal

5. • Expand access to high-
quality open materials
• Support faculty choice
and development
• Improve student success
CCCOER Mission
http://cccoer.org
Come In, We're Open gary simmons
cc-by-nc-sa flickr

6. 250+ Colleges
in 21 States & Provinces

7. OER Degrees
• Faculty redesign a pathway of existing
courses, leading to a degree or a CTE
certificate, to use OER
Image license: creative beauty cc-by-nc-nd by
calexanderson @ flickr

8. Research Highlights
• OER efficacy and students savings in
multi-state, multi-institutional
project over 30-month grant period.
• Analyze institutional costs and
benefits.
• Perceptions by all stakeholders

9. Research on Student Impact and
Cost Analysis for OER Degrees
Jessica Mislevy & Rick Staisloff
CCCOER Webinar
April 12, 2017

10. © 2015 SRI International© 2015 SRI International
Overview of OER Degree Initiative
• Multifunder initiative led by
Achieving the Dream
− Funding from the William and Flora
Hewlett Foundation, the Bill &
Melinda Gates Foundation, the
Great Lakes Higher Education
Guaranty Corporation, the Shelter
Hill Foundation, and the Speedwell
Foundation
• Scaling up model with 38
colleges across 13 states

11. © 2015 SRI International© 2015 SRI International
OER Degree Initiative Partners

12. © 2015 SRI International
Who is participating in the OER Degree
Initiative so far?
12

13. © 2015 SRI International
• West Hills College Lemoore
• Lake Washington Institute of
Technology
• Odessa College
• Distance Minnesota Consortium (3)
• Montgomery College Foundation
• City University of New York (CUNY) (3)
• Broward College
• Santa Ana College
• Housatonic Community College
• Monroe Community College (SUNY) (5)
• Austin Community College District (4)
• Virginia Community College Consortium (6)
• Pierce College
• Florida State College at Jacksonville
• Forsyth Technical Community College
• Pima Community College
• Bunker Hill Community College
• Bay College
OER Degree Initiative Grantees

14. © 2015 SRI International
Degrees and Certificates
0 5 10 15 20 25 30
Gen. Studies/Lib. Arts/AA/AS
Business
Health & Health Sciences
Social Science
STEM
Other
Across the colleges, OER pathways are planned for 52
degrees or certificates.

15. © 2015 SRI International
Faculty Engagement
As of fall 2016, 410 instructors
were involved in developing and/or
teaching OER courses.
Over half of the instructors are or
will be teaching with OER for the
first time.

16. © 2015 SRI International
What does the research arm of the initiative
seek to understand?
16

17. © 2015 SRI International
Hypotheses to Test
• Some students do not purchase required textbooks due to high cost and do
poorly or withdraw from courses as a result
• Some students take fewer courses than they otherwise would due to cost of
textbooks
• If this cost is removed, students will progress more quickly and successfully
towards degree attainment
• OER Degrees are a cost effective way to improve student outcomes and
reduce financial burden on students
• Redesigning programs around OER will trigger additional improvements in
pedagogy and institutional culture

18. © 2015 SRI International
Research Questions for the Evaluation –
Academic impacts and implementation
1) Academic outcomes: Do students who take multiple OER degree
classes make more progress to degree than similar students who take
traditional classes? Are OER degrees more or less beneficial to particular
subgroups of students? What are the key mediators of effects on student
outcomes?
2) What are best practices, facilitators, and barriers associated with
implementation of OER degree programs?
3) What impacts do OER degrees have on key stakeholders’
experiences and on institutional culture?
(We’ll get to economic impacts later)

19. © 2015 SRI International
Our Approach to the Evaluation
 Quasi-experimental studies will enable us to estimate
the impact of OER Degrees on key student outcomes
 Combine with cost analysis to measure cost
effectiveness and sustainability
 Implementation research will enable us to provide
formative feedback to grantees and broader community
 Aggregate pass/completion data will enable us to
evaluate the overall impact of the OER Degree Initiative -
how many OER course sections are offered, how many
students are involved

20. © 2015 SRI International
To put this another way…
Enrollment/completion/cost reporting;
instructor survey – all 34 colleges
Quasi-experimental
studies –
11 Research
Partners
Implementation data
collection (phone
interviews, student
surveys, site visits)Cost deep
dives –
5 colleges

21. © 2015 SRI International
How did the colleges arrive at an OER degree?
21

22. © 2015 SRI International
How did
colleges
arrive at
this point?
Administrator led
Grassroots led
Service unit led

23. © 2015 SRI International
Administrator Led
• One or more senior administrators were the first
to learn about OER degrees and saw this model
as a strong fit with their institution’s strategic
goals and mission. They took the lead in
recruiting faculty, building support among key
constituents, and shepherding resources to
launch their OER degree initiatives.

24. © 2015 SRI International
Grassroots Led
• Numerous faculty members had independently
taken the initiative to experiment with OER and
convert their courses. These pioneers
established proof of concept on their campuses
and spread the word among their colleagues. In a
few cases where departments achieved strong
consensus around the decision to use OER,
entire gateway courses have been converted to
OER.

25. © 2015 SRI International
Service Unit Led
• Librarians or units responsible for innovation in
teaching and learning (e.g. instructional
designers and technologists) were the first to
introduce OER to their campuses. These
proponents began by offering OER workshops to
faculty, attracting a few early adopters and
providing support for their course development.

26. © 2015 SRI International
Where
can this
lead?
OER as
core
strategic
initiative

27. © 2015 SRI International
How is launching an OER degree different than
converting individual courses?
27

28. © 2015 SRI International
From Courses to Degrees
• Need to give more attention
to coherence of student
experience across all touch
points (advising, registration
systems, etc.), articulation
to four year institutions
• Instructors need more
supports, including
instructional design and
copyright checking.
Librarians may be able to
help with this.

29. © 2015 SRI International
How does CCCOER fit in?
29

30. © 2015 SRI International
Collaboration & Support
• Instructor desire to
collaborate with peers in
their field
• Community of practice for
OER leaders and
practitioners to share
experience & knowledge of
successful strategies.
• Opportunities to increase
awareness of and
participation in CCCOER
For Internal Project Participants Only - Not for Distribution

31. © 2015 SRI International
How do we measure the impact of OER
degrees on student outcomes?
31

32. © 2015 SRI International
Research Partner Focus
• Most research on OER effectiveness has
focused on course-level outcomes. This
project will focus primarily on student progress
to degree.
• Working with the colleges to identify
comparison population of similar students,
ideally with no OER course exposure
• Collecting data for all students who enroll in
OER degrees and comparison groups
32
Why so
important?
What does this
entail?

33. © 2015 SRI International
Concurrent vs. Historical Comparison
F’13 S’14 F’14 S’15 F’15 S’16 F’16 S’17 F’17 S’18 F’18
Treatment
Concurrent Comparison
Historical Comparison
OER Degree Initiative
Study Period

34. © 2015 SRI International
Outcome Measures
Progress towards degrees will be measured by the following
leading indicators:
1. Degree completion
2. Retention from prior term
3. Enrollment status (full-time or part-time)
4. Credits attempted
5. Credits attained
6. Credits attained within major
7. Cumulative credits
8. Term GPA
9. Cumulative GPA
10. Transfer with award
11. Transfer without award

35. © 2015 SRI International
Combining Results Across Impact Studies
Favors
OER Degree
Favors
Comparison
SRI will conduct a meta-
analysis of results from the
individual impact studies (by
college and degree program)
to estimate the impact of
OER degrees on key
outcome measures.

36. © 2015 SRI International
What are the opportunities and challenges for
sustainability?
36

37. © 2015 SRI International
Research Questions: Economic Outcomes
37
How does enrolling in OER degree courses impact costs for
students pursuing degrees?
What are the start-up and ongoing costs for an institution to
make the transition to OER degrees, including lost revenue
streams? Is this model self-sustainable?
What is the cost effectiveness of this model? What is the
“return” on investment (ROI) for students, colleges, and other
stakeholders?

38. © 2015 SRI International
Economic Analysis Framework
Student Impacts
Textbook savings
Course & materials fees
Cost impact from changing
enrollment patterns
Institution Impacts
Bookstore revenue
Institution revenue
Financial and non-financial
“return” (research partners)
AllGranteesCostPartners
Start-up Costs
Course development
Infrastructure support
Planning & professional
development
Ongoing Costs &
Revenues
Shifting faculty activity
Course revisions & scaling
Net revenue impact (research
partners)

39. © 2015 SRI International
Creating Sustainability for OER Initiatives
Sustainable innovation requires three components:

40. © 2015 SRI International
Achieving Sustainability
1. Ensure that stakeholders understand that OER is not a
grant, but another tool to support student success
2. Communicate timeframes for achieving success
3. Begin planning now to fund ongoing cost of supporting
OER
4. Identify opportunities to reallocate resources
5. Capture potential ROI for students, the institution and
other stakeholders
6. Decide how you will intentionally impact culture – address
turnover, hire for OER openness, continue training,
communicate across stakeholders

41. © 2015 SRI International
What’s next?
41

42. © 2015 SRI International
Stay tuned for more!
• Grant period continues through Dec. 2018
• Public implementation report in ~1-2 months
− Instructor survey and phone interview findings
• Final research report in fall 2019
− Impact and cost analysis
For Internal Project Participants Only - Not for Distribution

43. © 2015 SRI International
Silicon Valley, CA
(SRI International headquarters)
333 Ravenswood Avenue
Menlo Park, CA 94025
+1.650.859.2000
Washington, D.C.
1100 Wilson Boulevard
Arlington, VA 22209
+1.703.524.2053
www.sri.com/education
43
Thank You
 Questions or comments?
 Let’s continue the conversation
Jessica Mislevy
Email: jessica.mislevy@sri.com
Richard Staisloff
Email: rstaisloff@rpkgroup.com

44. Stay in the Loop
• Upcoming Conferences
– April 28-30, Creative Commons Summit, Toronto
– May 24-25, Open Textbook Summit, Vancouver
– June 13-14, Northeast Regional OER Summit
– See more on our website under “Get Involved”
• OER Vetting webinar – May 10
– Quality, licensing, accessibility
• Year of Open (all year long)
– http://yearofopen.org
Image: pixabay.com

45. Thank you for joining us!
Contact Info:
Una Daly: unatdaly@oeconsortium.org
Questions?
cccoer.org