This document outlines OTC Online's updated process for reviewing online courses for development and delivery. The new process separates development and delivery reviews, is based on research and best practices from organizations like Quality Matters and Blackboard, and takes a collaborative approach involving departments, disability support services, and an Online Review Task Force team. The goal is to provide constructive feedback to support continuous improvement of online courses and ensure accessibility and compliance standards are met. Reviewers will use a new document organized by navigation that includes pre-review comments and feedback sections. The process was piloted in fall 2011 and will continue to involve online instructors and solicit feedback to guide further improvements.

1. Online Course Development and Delivery Reviews: An Interactive Approach Carla R. BradleyOTC OnlineOzarks Technical Community Collegebradleyc@otc.edu

2. Words

3. Words Evaluation Evaluator Evaluator Best practices Problems Rubric Requirements Peer review Coach Mentor Effective practices Challenges Guide Consistency standards

4. Goals Explain how OTC Online arrived at updated online review process. Apply hints from other reviewers to online reviews. Evaluate an online course for development and delivery. * * Practice review set needs to be completed by new online reviewers.

7. Chasen, Michael. Blackboard World 2010.

8. Chasen, Michael. Blackboard World 2010.

9. 3 Resources Quality Matters Blackboard Exemplary Course Chico State

10. Quality Matters “The course review process is a collegial discussion between faculty peers committed to Continuous quality improvement. It is not an evaluation. Centered in national standards of best practice, the research literature and instructional design principles designed to promote student learning. . . . And it’s not a win/lost or pass/fail test…… the rubric in the review process functions as a diagnostic tool to facilitate the continuous improvement of online courses.”

11. Quality Matters Course overview and introduction Learning objectives Assessment and measurement Resources and materials Learner engagement Course technology Learner support Accessibility

12. QM Example Striving for Quality in Online Courses-J.J. Johnson-Pierce College in Washington State Quality assurance team (of six): Sensitive about academic freedom Student feels comfortable- course layout Anticipate questions Student comfort (home page)- consistent interface Measure of course design Composed of instructors in diverse disciplines Provide mentoring and coaching Goal- Go to the next level and tie to PD sessions

13. Blackboard Exemplary Course “The Blackboard Exemplary Course Program began in 2000 with the goal of identifying and disseminating best practices for designing engaging online courses.” Course design Interaction and collaboration Assessment Learner support

14. Bb ECP Example Excellence within our Grasp: Using Blackboard’s Exemplary Course Rubric- Mark Burris- University of Arkansas Little Rock and Sue Burris- National Park Community College Goal 1: Promote instructional excellence across Arkansas through online student engagement principles, moving beyond “buttonology” Goal 2: Promote academic collaboration through local and statewide community of practice

15. Chico State Learner support and resources Online organization and design Instructional design and delivery Assessment and evaluation of student learning Innovative teaching with technology Faculty use of student feedback

16. Chico State Example Online Course Assessment (James West)- Missouri Baptist University Peer evaluations results from spring 2005 to spring 2010 High score has been consistent Low score has had an upward trend Mean has been upward Consistent improvement over 5-year period

18. Previous Online Review Document Checklist and comments Review document with “yes” or “no” that combined development and delivery Lacked extensive information on instructional content, technology, academic integrity, and accessibility.

19. Unique to College Navigation Online syllabus template (current) Instructor response time Assessment data

20. Pilot Introduced in OTL 101 in fall 2011 Fall 2011 online instructors complete Spring and summer 2011 online instructors complete (some courses reviewed) Fall 2011 only online instructors reviewed complete

21. Approach to Online PD

22. Review Document Engage department and begin research Separate development and delivery Add pre-review comment section Organize by navigation Pilot review documents and solicit feedback Create team approach Involve departments Check with DSS Invite feedback from OTC Online Committee Meet online with Online Review Task Force Team

23. Accessibility and Compliance “Creates duplicate channels of information to benefit all learners.” Planning will have a cost-savings rather than retrofitting. Builds awareness. Georgia Tech Research on Accessible Distance Education (GRADE). CATEA Access E Learning. http://www.accesselearning.net/mod1/1_01.php

25. Reminders Research-based Objective Not a judgment of instructor (but of course development and delivery)

26. bradleyc@otc.edu