California Learning Resource Network
clrn.org
Inputs Matter
What is CLRN?
California Learning Resource Network
 State-funded Education Technology Service

Initial charter: Review ...
Open Educational Resources

6000+ free and commercial-free
resources
Aligned to CCSS and other CA
standards
Governor Schwarzenegger’s
Digital Textbook Initiative
California
Digital Textbook Initiative

Three Phases: Free or Open Source
High School Math, Science, & History
Review f...
California Legislation

AB 1398, relating to the use of textbook
funds
Redefines “technology-based materials”
to include...
California Legislation

SB 247 relating to high school textbook
purchases
Textbook funds may be used to
purchase electro...
Recent CA Legislation

SB 185 / AB 133
Requires publishers to provide
textbooks in digital format
Three Digital Textbook Paths

Self-contained digital
 Flat: Epub, Kindle, PDF
 Digital with interactive components

On...
What is CLRN?
Online Course Reviews
 Six subject areas

Standards Alignment
 Common Core State Standards
 California’...
Publisher Eight Step Entry
Step 2: Content Standards
Standards Correlation
Algebra I CCSS
Step 3: Course Standards
Step 4: Instruction
Type
Length
Accreditation
Step 5: Technology
Step 6: Features
Step 7: Abstract
CLRN Review Sites

Six, subject-specific sites
County office based
Managed by COE content expert
20-25 active teachers...
Reviewer Training & Norming

CLRN Review Process
Social Content Criteria
Online Course Standards
CLRN OCR Training Videos
OCR Retraining & Norming
CLRN Course Review Process
Nine-step process

1. Team assignment
2. Content (CCSS) Standards
review
 Develop, Practice,...
CLRN Review Process
5. Online Standards
6. Features

7. Profile
8. Abstract
Step 4: Social Content
Male/Female Roles

Ethnic/Cultural Groups
Older Persons/Aging Process

Disabled Persons
Religi...
iNACOL Standards for
Quality Online Courses
Online courses participate in the
teaching/learning process
iNACOL Course
Standards
Content

Instructional Design
Student Assessment

Technology
Course Evaluation and Support
Step 5: Course Standards
Reviewers vet 24 of the 52 OCR standards
Content
Content depth and breadth
Information literacy skills
Learning resources and materials
Communication process b...
Instructional Design
Course design and organization
Meaningful and authentic learning
experiences
Multiple learning pat...
OCR Standards/Considerations
B5. The course
provides
opportunities for
students to engage in
higher-order
thinking, critic...
Student Assessment

Alignment between the course goals
and activities and its assessment
strategies
Insure that there ar...
Technology
Course
architecture

User interface
Accessibility

Interoperability
Course Evaluation and
Support
Evaluating course effectiveness
Accreditation
Teacher and Student preparation and
support
CLRN Central Review
Validation & Norming
 Work the course

Review/update all 52 course
standards
Standardize review co...
CLRN Central Review
Editing & Proofreading
 Review, modify, and standardize content standards rating
& comments

Publis...
Texas H.B 4294

eTextbook publishers may submit
updated content for review
Districts/schools may select a
subscription-b...
Re-Review Policy
For Updated Courses
 Updated, not new (New=>30% new content)

 New content and/or course standards.
 ...
CLRN HOme
List View
Not Met standards
CLRNCertified

University of California Partnership
A-G Approved Courses

CLRN-Certified
 80% content standards
 80% o...
OCR Standards & Considerations
CLRN Certified Information
U.C. Online Learning Home
What steps does your school/district take
to validate that the rigor and quality of
online or blended learning courses mee...
Answers and Questions

We examine the course outlines with
teachers and administration.
Matching up the state standards
...
More Answers and Questions

Data supplied by curriculum
provider.
Student input on rigor compared to
last course or clas...
What factors did you consider
when selecting courseware?
Price

77%

Compared to content standards

60%

U.C. A-G list

60...
The Good News

CLRN Reviews inform course
improvement
Re-Reviews
Content Re-reviews

Additional content
New activities

Additional components to meet
course standards
 Syllabi, polici...
Analysis of the 460 published
 63% certified
 Up from 46%

 20% teach < 80% content standards
 Range from 4% met to 78...
Certification by Subject
ELA:52%
History: 64%
 Up from 27%

Math: 55%
 Up from 47%

Science: 58%
 Up from 28%

Wor...
Accessibility

You’re only a lawsuit away from
making your courses accessible
DOJ Story

D10: Course materials and activities are
designed to provide appropriate access
to all students…
Closed-Captions/Transcripts

Edgenuity (e2020)
BYU

Compass Learning
K12, Inc.

(Word has gotten out)
Why Inputs Matter
Resubmit an Assignment Story
Ed Code

Ed Code 49011 states that school
districts and schools shall not
establish a two-tier educational
system …throug...
What is not a great online
course
 Class 1: Watch lecture. Complete worksheets. Take a
multiple choice test. Repeat

 Cl...
What is not a great online lesson.

“Writing an introduction” slide show
Vocabulary exercises. Students then:

READ 12 ...
Not a great online
lesson, continued
READ 10 pages of textbook reading
about verb tense
READ 6 textbook pages about “ver...
What should we
expect from great
online courses?
High-quality course
Stimulating lectures

Writing

Just-in-time reading

Students
create, evaluate, an
d analyze.

Ac...
Make use of the medium

Rich media
 Lecture clips, video demonstrations and
clips, variety of multi-media, simulations, ...
California Learning Resource Network
clrn.org
Inputs Matter
Ccsso clrn
Ccsso clrn
Ccsso clrn
Ccsso clrn
Ccsso clrn
Upcoming SlideShare
Loading in...5
×

Ccsso clrn

96

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
96
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Inputs matter. How can you tell if an online course is any good, whether it engages students in active learning, or challenges them with authentic, higher-level work? We’ll detail the California Learning Resource Network (CLRN) course review process, from publisher entry to publication, outlining criteria, including iNACOL’s course standards, for receiving our certification as part of our University of California partnership. We’ll also share data from the California eLearning Census.
  • What’s missing is that
  • As the census drew to a close, we sent a supplemental survey to those districts that were purchasing courses, asking them about their selection process. When asked what factors they considered when selecting courseware, the top four choices were price, comparing courses to content standards, the U.C. A-G approved list, and examined course outlines. Sadly, few districts realize that by selecting from the UC A-G Approved courses, they’re also depending on CLRN’s certified course reviews.None10%Selected only CLRN Certified courses.3 6%Colleague recommendation17 32%Vendor demonstration2242%Data supplied by curriculum provider22 42%Examined course outlines2955%U.C. A-G list3260%Compared the course to the content standards32Price41 77%
  • Short video clip of a boring teacher?If you walked into a f2f class and saw this type of teaching/learning, would you be impressed? Would you recommend this class to other students? Would you send a struggling student there? The lesson begins with “Writing an introduction” slide show, which would be stronger if narrated, followed by some vocabulary exercises. Students then read:12 textbook pages about verbs13 pages of textbook reading about verb agreement10 pages of textbook reading about verb tensesix textbook pages about “verbs made easy”a one-page textbook page about writing a first draft, andtwo textbook pages about simple narratives.
  • Inputs matter. How can you tell if an online course is any good, whether it engages students in active learning, or challenges them with authentic, higher-level work? We’ll detail the California Learning Resource Network (CLRN) course review process, from publisher entry to publication, outlining criteria, including iNACOL’s course standards, for receiving our certification as part of our University of California partnership. We’ll also share data from the California eLearning Census.
  • Ccsso clrn

    1. 1. California Learning Resource Network clrn.org Inputs Matter
    2. 2. What is CLRN? California Learning Resource Network  State-funded Education Technology Service Initial charter: Review supplemental electronic learning resources
    3. 3. Open Educational Resources 6000+ free and commercial-free resources Aligned to CCSS and other CA standards
    4. 4. Governor Schwarzenegger’s Digital Textbook Initiative
    5. 5. California Digital Textbook Initiative Three Phases: Free or Open Source High School Math, Science, & History Review for Content Standards Only
    6. 6. California Legislation AB 1398, relating to the use of textbook funds Redefines “technology-based materials” to include electronic equipment required to use them
    7. 7. California Legislation SB 247 relating to high school textbook purchases Textbook funds may be used to purchase electronic versions Districts must ensure all students have access at home & school
    8. 8. Recent CA Legislation SB 185 / AB 133 Requires publishers to provide textbooks in digital format
    9. 9. Three Digital Textbook Paths Self-contained digital  Flat: Epub, Kindle, PDF  Digital with interactive components Online, subscription-based  Some digital or interactive components Online Courses
    10. 10. What is CLRN? Online Course Reviews  Six subject areas Standards Alignment  Common Core State Standards  California’s other standards  Next Generation Science Standards iNACOL Standards for Quality Online Courses
    11. 11. Publisher Eight Step Entry
    12. 12. Step 2: Content Standards
    13. 13. Standards Correlation Algebra I CCSS
    14. 14. Step 3: Course Standards
    15. 15. Step 4: Instruction Type Length Accreditation
    16. 16. Step 5: Technology
    17. 17. Step 6: Features
    18. 18. Step 7: Abstract
    19. 19. CLRN Review Sites Six, subject-specific sites County office based Managed by COE content expert 20-25 active teachers Meet monthly during school year
    20. 20. Reviewer Training & Norming CLRN Review Process Social Content Criteria Online Course Standards
    21. 21. CLRN OCR Training Videos
    22. 22. OCR Retraining & Norming
    23. 23. CLRN Course Review Process Nine-step process 1. Team assignment 2. Content (CCSS) Standards review  Develop, Practice, Assessment 3. Minimum Requirements  Reading levels, spelling/grammar, content accuracy, etc. 4. Social Content Review  13 areas, including male/female roles, ethnic/cultural groups, & advertising.
    24. 24. CLRN Review Process 5. Online Standards 6. Features 7. Profile 8. Abstract
    25. 25. Step 4: Social Content Male/Female Roles Ethnic/Cultural Groups Older Persons/Aging Process Disabled Persons Religion Brand Names/Advertising
    26. 26. iNACOL Standards for Quality Online Courses Online courses participate in the teaching/learning process
    27. 27. iNACOL Course Standards Content Instructional Design Student Assessment Technology Course Evaluation and Support
    28. 28. Step 5: Course Standards Reviewers vet 24 of the 52 OCR standards
    29. 29. Content Content depth and breadth Information literacy skills Learning resources and materials Communication process between teachers, parents, and students Content accuracy and bias
    30. 30. Instructional Design Course design and organization Meaningful and authentic learning experiences Multiple learning paths for students to master the content Higher-order thinking skills Instructor-student and student-student
    31. 31. OCR Standards/Considerations B5. The course provides opportunities for students to engage in higher-order thinking, critical reasoning activities and thinking in increasingly complex ways. ★ Assignments, activities, and assessments provide opportunities for student to elevate their thinking beyond knowledge and comprehension into the realm of analyzing situations, synthesizing information, or evaluating an argument. …Opportunities for group work, decision‐making, and finding patterns should also be
    32. 32. Student Assessment Alignment between the course goals and activities and its assessment strategies Insure that there are adequate and appropriate methods to assess students Assure that students are constantly
    33. 33. Technology Course architecture User interface Accessibility Interoperability
    34. 34. Course Evaluation and Support Evaluating course effectiveness Accreditation Teacher and Student preparation and support
    35. 35. CLRN Central Review Validation & Norming  Work the course Review/update all 52 course standards Standardize review comments Notes inform reviewer retraining
    36. 36. CLRN Central Review Editing & Proofreading  Review, modify, and standardize content standards rating & comments Publisher Feedback  Seven-day window  New evidence required for Re-Review  One Re-Review permitted per course. Publishing  Reviews valid for three years
    37. 37. Texas H.B 4294 eTextbook publishers may submit updated content for review Districts/schools may select a subscription-based electronic textbook
    38. 38. Re-Review Policy For Updated Courses  Updated, not new (New=>30% new content)  New content and/or course standards.  Strengthen alignment to the content standards and/or the course standards.  Publishers provide specific evidence. Once per year
    39. 39. CLRN HOme
    40. 40. List View
    41. 41. Not Met standards
    42. 42. CLRNCertified University of California Partnership A-G Approved Courses CLRN-Certified  80% content standards  80% online course standards  15 power standards
    43. 43. OCR Standards & Considerations
    44. 44. CLRN Certified Information
    45. 45. U.C. Online Learning Home
    46. 46. What steps does your school/district take to validate that the rigor and quality of online or blended learning courses meets or exceeds that of “traditional” courses taught in a brick and mortar classroom?
    47. 47. Answers and Questions We examine the course outlines with teachers and administration. Matching up the state standards with course, requiring minimums for passing scores None required
    48. 48. More Answers and Questions Data supplied by curriculum provider. Student input on rigor compared to last course or class taken. Ensure alignment to state standards – we need to be doing more
    49. 49. What factors did you consider when selecting courseware? Price 77% Compared to content standards 60% U.C. A-G list 60% Examined course outlines 55% Data supplied by provider 42% Vendor demonstration 42% Colleague recommendation Selected CLRN Certified courses. None 0 10 20 30 40 50
    50. 50. The Good News CLRN Reviews inform course improvement
    51. 51. Re-Reviews
    52. 52. Content Re-reviews Additional content New activities Additional components to meet course standards  Syllabi, policies, resources, etc.
    53. 53. Analysis of the 460 published  63% certified  Up from 46%  20% teach < 80% content standards  Range from 4% met to 78% met  12% only missing captions or transcripts
    54. 54. Certification by Subject ELA:52% History: 64%  Up from 27% Math: 55%  Up from 47% Science: 58%  Up from 28% World Languages: 89%
    55. 55. Accessibility You’re only a lawsuit away from making your courses accessible
    56. 56. DOJ Story D10: Course materials and activities are designed to provide appropriate access to all students…
    57. 57. Closed-Captions/Transcripts Edgenuity (e2020) BYU Compass Learning K12, Inc. (Word has gotten out)
    58. 58. Why Inputs Matter Resubmit an Assignment Story
    59. 59. Ed Code Ed Code 49011 states that school districts and schools shall not establish a two-tier educational system …through payment of a fee or purchase of additional supplies that the school district or school does not provide; and
    60. 60. What is not a great online course  Class 1: Watch lecture. Complete worksheets. Take a multiple choice test. Repeat  Class 2: Read. Take a multiple choice test. Repeat  Class 3: Read. Print and take quiz. Grade your quiz. Take multiple choice test. Repeat  Class 4: Read a physical book. Take quiz from a physical book. Take part in online discussion. Repeat  Moving a textbook online is not a foundation for a great course.
    61. 61. What is not a great online lesson. “Writing an introduction” slide show Vocabulary exercises. Students then: READ 12 textbook pages about verbs READ 13 pages of textbook reading about verb agreement
    62. 62. Not a great online lesson, continued READ 10 pages of textbook reading about verb tense READ 6 textbook pages about “verbs made easy” READ a one-page textbook page about writing a first draft, and READ two textbook pages about simple narratives.
    63. 63. What should we expect from great online courses?
    64. 64. High-quality course Stimulating lectures Writing Just-in-time reading Students create, evaluate, an d analyze. Active learning Engaging activities that go beyond knowledge and comprehension Variety of assessment types
    65. 65. Make use of the medium Rich media  Lecture clips, video demonstrations and clips, variety of multi-media, simulations, … Ease of use  LMS helps inform instruction. Students/teachers always know where they are.
    66. 66. California Learning Resource Network clrn.org Inputs Matter
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×