Measuring the Impact of OER Initiatives: An Assessment Framework
•Download as PPTX, PDF•
0 likes•79 views
Presentation on measuring OER efficacy through an assessment framework for the MI OER Summit on September 21, 2018.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Measuring the Impact of OER Initiatives: An Assessment Framework
Similar to Measuring the Impact of OER Initiatives: An Assessment Framework (20)
More from Regina Gong
More from Regina Gong (20)
Recently uploaded
Recently uploaded (20)
Measuring the Impact of OER Initiatives: An Assessment Framework
- 1. Measuring the Impact of OER Initiatives: An Assessment Framework Regina Gong, OER Project Manager Dr. Karen Hicks, Director of Assessment Lansing Community College MI-OER Summit, September 21, 2018 This work is licensed under the Creative Commons Attribution 4.0 International License.
- 2. Here’s what we’re going to talk about: • Context of our OER project at LCC. • Importance of assessment. • Development of our assessment framework. • Elements and design of each assessment framework. • Snapshot of our data. • Questions and discussions.
- 3. Textbook affordability Allow faculty exploration and innovation in finding new, better, and less costly ways to deliver quality learning materials to students in addition to improved pedagogy. Goals of the OER Initiative at LCC
- 4. OER and Open Learning are two of the ongoing competitiveness and innovation projects in the 2017-2020 LCC Strategic Plan.
- 5. Because assessment makes the world go round Photo by John Jackson on Unsplash
- 6. Why is assessment important? Planned outcomes inform planned work. In turn, planned work drives planned outcomes.
- 7. How did we develop our assessment framework?
- 10. COUP framework Cost Outcomes Use Perceptions Slide from OER Research Brief Presentation (2016) by Rajiv Jhangiani.
- 11. ◉ Costs of textbooks previously assigned ◉ OER support fee models ◉ Changes in campus bookstore revenue ◉ Changes in tuition revenue due to changes in drop rates ◉ Changes in tuition revenue due to changes in enrollment intensity ◉ Changes in tuition revenue due to changes in persistence Financial Impact Slide from OER Research Brief Presentation (2016) by Rajiv Jhangiani.
- 12. Educational Outcomes ◉ Changes in the percentage of students receiving a C or better ◉ Changes in rates of completion ◉ Changes in drop rates ◉ Changes in enrollment intensity ◉ Changes in persistence ◉ Changes in attainment of progress milestones (e.g., first 15 credits) ◉ Changes in graduation rates Slide from OER Research Brief Presentation (2016) by Rajiv Jhangiani
- 13. How are they using it? Adoption? Adaptation? • Deleting material from the OER • Inserting other open material inside the OER • Moving material around within the OER • Editing material in the OER Creation? Are there patterns of use? Why are they using it? Why are they not using it? UseofOER
- 14. ◉ How do faculty and students perceive OER? ◉ How do they judge its quality? ○ Does it align with the curriculum? ○ Is it error free? ○ Is it up-to-date? ○ Does it come with ancillary resources? Perceptions of Faculty andStudents Slide from OER Research Brief Presentation (2016) by Rajiv Jhangiani
- 15. Designing an Assessment Framework • How to go about it (approach) • Collaborative • Reverse engineer/systemically • Aligned • Systematic • Key Elements • Logic model • Assessment plan • Monitoring & tracking plan
- 16. Elements of an Assessment Plan • Your data collection strategy must include these elements: • Performance indicators (i.e. data) • Sources of data • Methods of data collection • Methods of data analysis • Implementation of assessment plan • Management of data • Monitoring of data • Communication of data
- 17. Data collection strategies: Student surveys Faculty surveys Mixed methods • quantitative • qualitative CDS supplied data from Banner, Argos Photo by Fancycrave on Unsplash
- 18. OER Assessment Framework at LCC
- 19. OER Assessment Framework at LCC
- 20. OER Assessment Framework at LCC
- 21. OER Assessment Framework at LCC
- 23. Let’s look at the student and faculty surveys
- 24. Let’s look at our OER adoption file http://libguides.lcc.edu/oer/adopt
- 25. OER Adoptions at LCC 5 10 4 14 16 12 27 26 22 49 11 23 6 101 100 33 150 154 60 298 0 50 100 150 200 250 300 350 Fall 2015 Spring 2016 Summer 2016 Fall 2016 Spring 2017 Summer 2017 Fall 2017 Spring 2018 Summer 2018 Fall 2018 Courses and Sections Using OER Courses Using OER Sections Using OER
- 26. OER Adoptions at LCC 5 12 4 46 48 26 74 75 41 136 0 20 40 60 80 100 120 140 160 Fall 2015 Spring 2016 Summer 2016 Fall 2016 Spring 2017 Summer 2017 Fall 2017 Spring 2018 Summer 2018 Fall 2018 Faculty Using OER
- 27. Students Impacted by OER 317 540 129 2,825 2,558 724 3,724 3,711 1,177 6,033 - 1,000 2,000 3,000 4,000 5,000 6,000 7,000 Fall 2015 Spring 2016 Summer 2016 Fall 2016 Spring 2017 Summer 2017 Fall 2017 Spring 2018 Summer 2018 Fall 2018 Students Enrolled
- 28. Textbook Costs Savings $31,700 $54,000 $12,900 $282,500 $255,800 $72,400 $293,200 $371,100 $117,700 $603,300 $- $100,000 $200,000 $300,000 $400,000 $500,000 $600,000 $700,000 Fall 2015 Spring 2016 Summer 2016 Fall 2016 Spring 2017 Summer 2017 Fall 2017 Spring 2018 Summer 2018 Fall 2018 Total Textbook Savings ($100/student)
- 29. Forthcoming paper An Analysis of Cost, Outcomes, Use, and Perceptions in a Multi-Section Adoption of Open Textbooks in Introductory Psychology and Economics Courses Regina Gong and Karen Hicks Lansing Community College
- 31. Image source: https://unsplash.com/collections/167234/questions?photo=i--IN3cvEjg Regina Gong, gongr1@lcc.edu
Editor's Notes
- We already know that OERs can save students money. We are tracking that so we can have a benchmark figure for which we can compare ourselves with state and national OER initiative. But cost is just one metric. More importantly, and one we strive hard to do at LCC is to assess the impact of OER on student learning and faculty teaching. There must be more to it than just saving students money. We’re all in higher education and as you know, there has been an increased emphasis in assessment to measure the effect of various institutional projects (Guided Pathways, Co-requisite models, assessibility, Dev ed redesign, etc.) The goal is to inform leaders about the importance of these projects and how might we improve them.
- Alignment Plan for outcomes, then plan for work (not the other way around) Line of sight
- How aware are faculty and students of OER?
- Collaboratively identify research questions (multiple audiences, different metrics communicate values and the weights of those values, often dictated by role in the college) Think systemically with these research questions (beyond the short-term) Gather expectations of what everyone thinks OER should be accomplishing. Develop hypothesis together. Then, test your hypotheses at various project stages to see where it needs some tinkering to get closer to your planned consequences (i.e. outcomes).
- One of the ways in which we look at the OER project’s impact is by calculating the percentage of students enrolled in OER courses for a particular semester. So in this case, if we have 6,033 students enrolled in OER courses for fall 2018 and we look at our total campus headcounts, 36% of our students are exposed to OER through their enrollment in courses using an OER. We expect that by spring 2019 semester, we will be closer to our goal of having 50% of our students at LCC enrolled in a course using OER.
- Show Surveymonkey data for student and faculty feedback spring 2018
- Show Surveymonkey data for student and faculty feedback spring 2018
- I’m going to show you in the next slides our progress so far since we launched in fall 2015.
- 10,598 total students impacted (including fall 2017)
- We have already saved our students almost $1.1M in textbook cost since we started in fall 2015.