California LearningResource Network (CLRN) http://clrn.org
What is CLRN?California Learning Resource Network http://clrn.org State-funded Education Technology Service Established as part of larger ed. Technology legislation in 1999Initial charter: Review supplemental electronic learning resources to CA content standardsExpansion to online courses, OER and data assessment tools
What is CLRN?California eLearning Census http://www.clrn.org/census/eLearning Strategies Symposium http://elearns.org @elearns
Why CLRN ReviewsCoursesDisrupting Class (2008)Full courses of studyDigital Textbook Initiative (2009/10)Authorization (2010)
Review Process: How &WhatLiterature review & initial criteria SREB, iNACOL, TxVSN, QM, and WA Digital LearningCLRN Stakeholder meetingsCriteria rewrite: CaliforniaCollaboration with TexasCriteria rewrite: national
How CLRN ReviewsUtilized existing CLRN review sites Five physical locations around CA Subject specific Established reviewer cohorts (15-25 educators)Three-person review teamsEdited & normed at CLRN Centralhttp://brianbridges.org
CLRN Review ProcessDatabase drivenPublishers complete submission onlineAttach standards correlation document Noting where each standard may be foundBoth teacher and student accounts/passwords are required
CLRN Review Process5. Online Standards Met, Partially Met, or Not Met Comments for all Partially Met standards Reviewers utilize CLRN comment sheet6. Features Course Teacher Instructional Design Universal Access Assessment Technology Support Materials
CLRN Review Process7. Profile Publisher Info Course length and type Accreditation (UC, NCAA, AP, CLRN-Certified) Languages, etc8. Abstract Full course summary Course objectives and outcomes Relevant Research
CLRN Central ReviewValidation & Norming Work the courseReview/update all 52 course standardsStandardize review commentsNotes inform reviewer retraining
CLRN Central ReviewEditing & Proofreading Review, modify, and standardize content standards rating & commentsPublisher Feedback Seven-day window New evidence required for Re-Review One Re-Review permitted per course.Publishing Reviews valid for three years
CLRN-CertifiedUniversity of California Partnership A-G Approved CoursesCLRN-Certified 80% content standards 80% online course standards 15 power standards
What’s Next?World Language ReviewsMath Common Core StandardsNext Generation Science StandardsAdding Grade 6-8 ReviewsCareer Technical Education StandardsProfessional Development
ContentA2. The course content and assignments are aligned with the state’s content standards… Is the course teaching, providing practice, and assessing each standard?
A Textbook is not a courseIf it were, you could just throw thekid a book and tell them to read it.
ContentA3: The course content and assignments are of sufficient rigor, depth and breadth to teach the standards being addressed. * Develop, Practice, Assess
EngagingB3: The course instruction and activities engage students in active learning.*Reading and watching are not active.
B3 ConsiderationThe course provides multiple opportunities for students to be actively engaged in the content that includes meaningful and authentic learning experiences such as collaborative learning groups, student‐led review sessions, games, analysis or reactions to videos, discussions, concept mapping, analyzing case studies, etc
It’s about PedagogyHow will you engage students?What authentic projects will you use for practice and assessment?How will students collaborate and participate in discussions?
Formative AssessmentsShould Inform InstructionB4. The course and course instructor provide students with multiple learning paths, based on student needs that engage students in a variety of ways.*
Lower Order ThinkingSkillsB5. The course provides opportunities for students to engage in higher-order thinking, critical reasoning activities and thinking in increasingly complex ways. *
B5 Consideration Assignments, activities, and assessments provide opportunities for student to elevate their thinking beyond knowledge and comprehension into the realm of analyzing situations, synthesizing information, or evaluating an argument. Activities should include open‐ended questions, and encourage students to categorize and classify information.
Media RichB11. Students have access to resources that enrich the course content.D4. Rich media are provided in multiple formats for ease of use and access in order to address diverse student needs.*
AssessmentsC2. The course structure includes adequate and appropriate methods and procedures to assess students’ mastery of content. *
C2 ConsiderationC2. Assessment types are matched to the level of knowledge being tested. Both formative assessments (that inform and support learning) and summative assessments (that demonstrate mastery) are a part of the course structure
AccessibleD10. Course materials and activities are designed to provide appropriate access to all students. *
VarietyC3. Ongoing, varied, and frequent assessments are conducted throughout the course to inform instruction. *
Student ProgressC4. Assessment strategies and tools make the student continuously aware of his/her progress in class and mastery of the content. *
ProfessionalDevelopmentE7. Course instructors… have been provided professional development in the behavioral, social, and when necessary, emotional, aspects of the learning environment.E8. Course instructors…receive instructor professional development, which includes the support and use of a variety of communication modes to stimulate student engagement online.
The Good NewsCLRN Reviews inform course improvement
CoursesContent Standards: 80%iNACOL Course Standards: 80% 15 Power StandardsCommercial Courses Only
AnalysisFirst 114 course reviews31% CLRN-Certified (36 total) 32 courses only missing D10 (28%)Most common problem Content standards alignment47 courses (41%)< 80% content standards Range from 29% met to 78% met