1. Using Open Textbooks in Community Colleges John Hilton III Jared Robinsonjohnhiltoniii.org tjaredrobinson.com
2. A Problem Textbook costs are a significant part of overall college expenses. High costs of education may deter some students from pursuing or continuing their education. While free resources are sometimes available, some question whether books that are distributed for free online will have the educational impact of full- color textbooks that have been vetted by traditional publishers.
3. What is happening in this situation?
4. First ImpressionsMay Not be Correct
5. Can Open Textbooks Make a Difference? Comparing Open and Traditional Textbooks  Houston Community College’s use of Flat World Knowledge’s Introduction to Psychology textbook.  Virginia State University’s adoption of several Flat World Knowledge business textbooks  The Kaleidoscope Project
6. Houston Community College Houston Community College (HCC) is a large community college with more than 70,000 students. 33% of its students are Hispanic, 33% are African American, 17% are white, 14% Asian, and 3% are classified as ‘other.’ Fifty-nine per cent of the students are female, and 41% are male. HCC’s Psychology department chose to adopt Flat World Knowledge’s (FWK) textbook Introduction to Psychology.
7. Why a FWK Textbook? They could modify it to meet the needs of their students (lowered the reading level, added in additional learning videos). It was a lot cheaper (free) when compared to other textbooks! Came with a variety of other resources, packaged along with it. During a pilot study 690 students used the FWK text. So what happened?
8. HCC – Student Feedback Eighty-four per cent of students surveyed agreed with the statement that ‘Having a free online book helps me go to college’. Of the 108 students who responded to a question regarding the difficulty of using an online text, 45 (42%) said that it was easy, 28 (26%) said that it was moderately easy, 26 (24%) said that it was neither hard nor easy, and nine (8%) said that it was moderately difficult. Zero students reported that it was difficult to use an online text.
9. HCC – Results Comparing Students fromtwo instructors who taught in Spring 2011 and Fall 2011
10. Virginia State University Virginia State University (VSU) has 5,300 students. They offer 55 baccalaureate and masters degree programs, two doctoral degree programs and three certificates. VSU’s Business Department adopted nine FWK text and paid $30.00 per student for an unlimited seat license for all FWK content.
11. VSU – Student Feedback from 148 students
12. VSU – Results 95% of students downloaded a digital textbook (in contrast with 47% of students purchasing textbooks one year previously). Students downloaded a variety of resources
13. VSU – Results In total there were seven courses that used FWK textbooks (COBU101, COBU200, COBU201, COBU210, COBU300, COBU301, COBU302, COBU310, MISY350), and ten courses that did not use FWK textbooks (COBU110, COBU111, COBU155, COBU170, COBU202, COBU260, COBU304, COBU342, COBU343, COBU400). While these were clearly different courses, they were roughly equivalent in terms of course difficulty. ABC_ DFW_ Total DFW ABC_ DFW_ Total DFW in in % in in % Core core with core core not Courses courses FWK courses courses using using using w/o w/o FWK FWK FWK FWK FWKFall 611 151 762 24.7% 830 270 1100 32.5%2010Spring 531 100 631 18.8% 834 242 1076 29.0%2011
14. Project KaleidoscopeThe study context is an open education initiativecalled Project Kaleidoscope (http://www.project-kaleidoscope.org/). This initiative is comprised ofeight community colleges serving predominantly at-risk students. These colleges work together to createcourses that replace traditional, expensive textbookswith OER. During the 2011-2012 academic year,Project Kaleidoscope (PK) impacted over 4,000students across 80 teachers in a controlled pilot, withanother 5,000 students using PK course designs andmaterials outside the controlled pilot.
15. Project Kaleidoscope – Teacher Perceptions 58 teachers from 8 PK institutions completed some items on the questionnaire. Student preparedness. Most teachers (63%) believed that their students were equally prepared in the course with PK texts compared to students in the same course before implementation of PK texts (Figure 3). At the same time, more than a quarter of teachers (26%) felt that students were more prepared than students in the same course in the past. Only 11 percent perceived students as being less prepared.
16. Project Kaleidoscope – Teacher Perceptions Teacher perceptions of quality. Twenty out of 57 (35%) teachers indicated that they thought the OER textbook was better than texts they had used previously to teach the course. Only 6 (11%) felt the OER texts were worse than traditional texts. The remaining 31(54%) teachers indicated that their OER texts were of the nearly the same quality as other texts they had used in the past.
17. Project Kaleidoscope – Student Perceptions 490 students from all eight PK institutions completed the questionnaire. 60% were female. 39% of students indicated that they thought the OER textbook was better than texts they had used in other courses. 6% felt the OER texts were worse and 55% indicated that their OER texts were of the nearly the same quality as other texts they had used in the past.
18. Project Kaleidoscope – Student Perceptions 160 students provided a description of what they thought made the OER texts better. Their responses clustered in six major categories: technical advantages (8%), learning aides (9%), customization (10%), cost (20%), access (26%), and quality/presentation (27%). Conversely, 20 students provided a description of what they thought made the OER texts worse than other texts they had used in the past. Their responses clustered in 2 major categories; six students described technology issues and 15 students took issue with text quality.
19. Project Kaleidoscope—Learning Outcomes Does open textbook adoption have an impact on student learning as manifest by grades? 649 students in intermediate algebra (developmental math) courses at four PK schools. What kind of analysis is appropriate?
20. The Problem
21. Propensity Score Matching
22. Covariate Balance
24. Strengths The data approximates random sample to the extent that we have captured key covariates. We have acceptable balance across all measured covariates. Statistical analyses are much more robust because the assumptions are met Have a stronger basis to engage in causal induction (Reuben Causal Model)
25. Limitations Grades are contextual—not universal markers of meaning We had a limited number of observed covariates. Some key covariates are unobservable. We lose statistical power by cutting our sample from 649 to 128.
26. Questions? John Hilton III Jared Robinsonjohnhiltoniii.org tjaredrobinson.com