Quality? Who Says?
  WCET Annual Conference
    November 9, 2007
Objectives
   Walk away with multiple views on rubrics for
    assessing quality of online instruction

   Take ideas fr...
Our Panel
Kay Kane, Quality Matters, MarylandOnline

Allison Peterson, Texas Woman’s University

David Curtis, Park Univer...
Agenda
Introduce QA programs
Address Key Issues:
  Impact on participation
  Study outcomes
  Value of the process
  Lesso...
Quality Assurance
   Programs
    What’s it all about?
    How do we do it?
Texas Woman’s
  University
 A Brief Introduction to the Quality
         Enhancement Plan
Texas Woman’s University
   Texas Higher Education Coordinating Board
    (THECB)
       Principles of Good Practice (PG...
Quality Matters
Course




   Course Meets
Quality Expectations              Peer Review




Course                 Feedback
Revision
The Rubric

   Eight standards:
       Course Overview and Introduction
       Learning Objectives
       Assessment a...
QM Strengths

Grounded in:
 research literature

 national standards of best practice

 instructional design principles...
Park University
                      By the Numbers!

   Online Students = 52,000+
       This counts all students taki...
Quality Starts…
                        …and doesn’t stop


   Training

   Developing

   Reviewing
Key Issues
   Impact on participation
   Study outcomes
   Value of the process
   Lessons Learned
Impact of
Participation on
 Faculty Peer
  Reviewers
Year One – Spring 2007
   January 2007 – August 2007
   10 Faculty
   6 Staff
   QM Reviewer Training
   Revise 1 Cou...
Year One – Results
   3 Course Reviews
     1 Review and Recognized
     1 in Review

     1 Preparing for Review

  ...
Year Two – Fall 2007
   September 2007 – August 2008
   25 Faculty
   1 Staff
   QM Reviewer Training
   Revise 1 Cou...
Year Three – Fall 2008
   September 2008 – August 2009
   35 Faculty
   QM Reviewer Training
   Revise 1 Course – All ...
Training Changes
   Online Educator Symposium
   Online Course
   Presentations
   One-to-one consultations
Quality Matters
  Study Outcomes
FY07 Course Reviews

85 courses reviewed FY07
   23 different institutions

   45 QM Managed

   40 Independent
FY07 Course Reviews

Upon initial review:
   38% met expectations
   62% do not yet meet expectations
       30% of the...
Common Themes

   Common areas for course improvement

   These are potential targets for:
     faculty training
     ...
Common Areas for Improvement

Module objectives measurable/consistent with course objectives    45%

"Self-check“/practice...
Impact on Faculty
     and Reviewers during Grant
Faculty said as a result of the review:
  91% made changes in the course...
Student Learning & Interaction
                College of Southern MD

   General education IT course (100+ students)

 ...
Impact of Navigation Directions
            Prince George’s CC, MD

   Standard I.1 – course navigation

   After revisi...
Student Perceptions & Priorities
                     SUNY Canton

   3 standards most often noted as not being met:
    ...
Park University

 Value of the Process
Value of the Process

   The Administration has always been concerned
    with maintaining academic integrity in all
    ...
Value of the Process

   The adoption of the Quality Matters program and
    ideology assures that the design and develop...
Value of the Process

   SO 141 – Introduction to Sociology example
Quality (at) Training
                            …get off on the right foot

Training workshop required one year out
   ...
Quality Development
                                    …from the first word

Developing in stages
   Checkpoint #1
    ...
Quality Reviews
                        …before the course runs

Reviewing from multiple perspectives
   Technical review...
Quality Evolves
                               …or becomes extinct

Critical factors
   Faculty buy-in
       Student sa...
Technical calls &
                    student evaluations
200
14.00%

180                                               Te...
Quality Looks Ahead
                                     …or falls behind

Planned changes
   Training
       Best pract...
Wrap Up / Q&A
Final Thoughts…
Thank you!
Upcoming SlideShare
Loading in...5
×

Quality, Who Says

554

Published on

WCET 2008 Conference presentation

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
554
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Quality, Who Says

  1. 1. Quality? Who Says? WCET Annual Conference November 9, 2007
  2. 2. Objectives  Walk away with multiple views on rubrics for assessing quality of online instruction  Take ideas from three different quality assurance methods  Recognize the impact and value of a quality assurance program
  3. 3. Our Panel Kay Kane, Quality Matters, MarylandOnline Allison Peterson, Texas Woman’s University David Curtis, Park University Michael Anderson, UT TeleCampus
  4. 4. Agenda Introduce QA programs Address Key Issues: Impact on participation Study outcomes Value of the process Lessons Learned Q&A Final Thoughts
  5. 5. Quality Assurance Programs What’s it all about? How do we do it?
  6. 6. Texas Woman’s University A Brief Introduction to the Quality Enhancement Plan
  7. 7. Texas Woman’s University  Texas Higher Education Coordinating Board (THECB)  Principles of Good Practice (PGP)  Institutional Context & Commitment  Quality Enhancement Plan (QEP)  Quality Matters (QM)
  8. 8. Quality Matters
  9. 9. Course Course Meets Quality Expectations Peer Review Course Feedback Revision
  10. 10. The Rubric  Eight standards:  Course Overview and Introduction  Learning Objectives  Assessment and Measurement Key components  Resources and Materials must align.  Learner Interaction  Course Technology  Learner Support  ADA Compliance
  11. 11. QM Strengths Grounded in:  research literature  national standards of best practice  instructional design principles www.QualityMatters.org
  12. 12. Park University By the Numbers!  Online Students = 52,000+  This counts all students taking all classes, so a student taking four online class is counted as four (4) students.  Online Courses = 322 Discrete Courses  A total of 524 Sections  Online Faculty = 455 +
  13. 13. Quality Starts… …and doesn’t stop  Training  Developing  Reviewing
  14. 14. Key Issues  Impact on participation  Study outcomes  Value of the process  Lessons Learned
  15. 15. Impact of Participation on Faculty Peer Reviewers
  16. 16. Year One – Spring 2007  January 2007 – August 2007  10 Faculty  6 Staff  QM Reviewer Training  Revise 1 Course – 14 Essential Standards  $400 Stipend
  17. 17. Year One – Results  3 Course Reviews  1 Review and Recognized  1 in Review  1 Preparing for Review  2 Course Reviewers  1 Summer  1 Fall  3 Potential Reviewers
  18. 18. Year Two – Fall 2007  September 2007 – August 2008  25 Faculty  1 Staff  QM Reviewer Training  Revise 1 Course – All 40 Standards  Review 1 Course  $400 Stipend
  19. 19. Year Three – Fall 2008  September 2008 – August 2009  35 Faculty  QM Reviewer Training  Revise 1 Course – All 40 Standards  Review 1 Course  $400 Stipend
  20. 20. Training Changes  Online Educator Symposium  Online Course  Presentations  One-to-one consultations
  21. 21. Quality Matters Study Outcomes
  22. 22. FY07 Course Reviews 85 courses reviewed FY07  23 different institutions  45 QM Managed  40 Independent
  23. 23. FY07 Course Reviews Upon initial review:  38% met expectations  62% do not yet meet expectations  30% of these were revised and met expectations Currently Recognized by QM: 56%
  24. 24. Common Themes  Common areas for course improvement  These are potential targets for:  faculty training  special attention in initial course development
  25. 25. Common Areas for Improvement Module objectives measurable/consistent with course objectives 45% "Self-check“/practice assignments for timely feedback 42% Instructions on how to meet the learning objectives 40% Equivalent alternatives to auditory and visual content 39% Instructor response and availability 38% Requirements/skills/prerequisites clearly stated 35% Navigational instructions 32% Criteria for the evaluation of students' work and participation 31%
  26. 26. Impact on Faculty and Reviewers during Grant Faculty said as a result of the review: 91% made changes in the course 89% felt quality of course design improved Reviewers said as a result of the review: 73% made changes to own online course 100% said valuable professional development activity
  27. 27. Student Learning & Interaction College of Southern MD  General education IT course (100+ students)  Revisions made in all learning modules:  created Learning Guides (explicit roadmap)  reorganized presentation and design  added classroom assessment techniques (CATs)  Increase in:  the frequency of content access by students  % of A grades awarded in the course
  28. 28. Impact of Navigation Directions Prince George’s CC, MD  Standard I.1 – course navigation  After revision of navigation bar:  Students asked fewer questions: course navigation, locating information, course requirements  Less student concern about what they needed to do to succeed
  29. 29. Student Perceptions & Priorities SUNY Canton  3 standards most often noted as not being met:  Course navigation directions  Assessments & measurement provide feedback to students  Clear standards for instructor response & availability
  30. 30. Park University Value of the Process
  31. 31. Value of the Process  The Administration has always been concerned with maintaining academic integrity in all programs, on-ground and online.  The goal was never to be “the biggest”, just the best. And by being the best, we might be the biggest 
  32. 32. Value of the Process  The adoption of the Quality Matters program and ideology assures that the design and development of online classes will meet the highest quality standards of academic excellence.  The “master course” structure for online courses, combined with using the QM rubric for online reviews, assures that a course AND all sections of that course will met QM Standards and Expectations.
  33. 33. Value of the Process  SO 141 – Introduction to Sociology example
  34. 34. Quality (at) Training …get off on the right foot Training workshop required one year out  Course examples  Content design  Interactivity (communication)  Assessment  Multiple online courses  Principles of Good Practice Self-Study  Instructional Design Tutorial
  35. 35. Quality Development …from the first word Developing in stages  Checkpoint #1  Syllabus, sample lesson  Checkpoint #2  ½ of the course, Mac  Content, interaction, assessment  Copy editing
  36. 36. Quality Reviews …before the course runs Reviewing from multiple perspectives  Technical review (external)  Content  Communication  Assessments  Services  Links  Final Check (internal)
  37. 37. Quality Evolves …or becomes extinct Critical factors  Faculty buy-in  Student satisfaction  Objective  Pre-tech reviews  Tech reviews  Subjective  Checkpoints  Rubrics
  38. 38. Technical calls & student evaluations 200 14.00% 180 Technical issues Courses 12.00% 1 160 in courses 10.00% 140 120 8.00% 100 6.00% 80 60 4.00% 40 Student design 2.00% 20 complaints2 0.00% 0 SP00 FA00 SP01FA01 SP02 FA02 SP03 FA03 SP04 FA04 SP05 FA05 SP06 SP00 FA00 SP01 FA01 SP02 FA02 SP03 FA03 SP04 FA04 SP05 FA05 SP06 FA06 FA06 SP07 SP07 1 Help Desk tickets as 2 Studentnegative a percent of courses comments on design as a percent of evaluations
  39. 39. Quality Looks Ahead …or falls behind Planned changes  Training  Best practices  Advanced workshops  Development  Community of Practice  Template/technique sharing  Reviewing  cQual application
  40. 40. Wrap Up / Q&A
  41. 41. Final Thoughts…
  42. 42. Thank you!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×