New Directions for the Quality Matters Program

2,130
-1

Published on

From its beginnings as a quality assurance tool for online course design, the Quality Matters Program is evolving to address a broader range of online learning quality assurance and faculty development issues.

Published in: Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,130
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Marker: Display this on the screen as participants enter the meeting room. Add the institution's name, your name and title if you wish.
  • Marker : This section introduces the audience to basic features of the Quality Matters Program and Process.
  • Quality Matters provides inter-institutional quality assurance in online learning. This is achieved through a not-for-profit subscription service providing tools and training for quality assurance of online courses.
  • The underlying principles of QM are a primary reason for this wide-spread adoption. Quality Matters provides a faculty-driven, peer review process that is…. Collaborative : QM was designed by and for faculty to share expertise and experience relative to the design of a course. Collegial : The course review process is a collegial discussion between faculty peers committed to Continuous quality improvement. It is not an evaluation. Centered in national standards of best practice, the research literature and instructional design principles designed to promote student learning. The underlying principles of QM are a primary reason for this wide-spread adoption. Quality Matters provides a faculty-driven, peer review process that is…. Collaborative : QM was designed by and for faculty to share expertise and experience relative to the design of a course. Collegial : The course review process is a collegial discussion between faculty peers committed to Continuous quality improvement. It is not an evaluation. Centered in national standards of best practice, the research literature and instructional design principles designed to promote student learning.
  • The map provides another view of the wide-spread adoption of QM. The color red/rust indicates states where there are QM subscribers and yellow indicates statewide subscriptions.
  • This table provides a comparison of the QM program during the FIPSE grant period (2003-2006) and today. A half-million dollar grant from FIPSE funded the initial development of the tools and provided resources to offer free trainings and course reviews. Over the 3-year grant period, individuals from 160 institutions across 28 states had participated. In the Fall of 2006, at the conclusion of the grant, MarylandOnline launched QM as a self-supporting program with only the 19 MOL institutions as charter members. In the first program year after the grant, QM had 86 subscribers with Louisiana becoming the first statewide system to adopt Quality Matters. By the end of year two, in June 2008, the number of subscribers had more than doubled to 186. We now have over 370 institutions that subscribe.
  • Marker : This section introduces the audience to basic features of the Quality Matters Program and Process.
  • Marker : This section introduces the audience to basic features of the Quality Matters Program and Process.
  • The development of the QM rubric was (and continues to be) research supported. It was developed after review of best practices and existing DE research literature. Moving forward, QM’s research agenda intends to contribute to the body of original research and the collection of best practice data by this year’s focus on IMPACT. That is, QM is pursuing a research agenda focused on quality improvement (IMP) and action (ACT) to implement those improvements at the individual, course, program, and institution level. This year’s efforts will culminate in IMPACT data presentation at the Quality Matters conference June 11-13 in Chicago. Kay Shattuck, D.Ed., Director of Research, Quality Matters, 2009 QM Conference
  • Faculty developers (those who have had their own courses formally QM reviewed) indicate that they are making more changes to their courses during training and prior to the review than was the case in 2006. Fewer report changes needed to meet standards. 97% of the respondents in 2009 reported they would recommend the QM process to other faculty. Those serving on a QM review team as peer reviewers reported making changes in their own courses as result of participating in the review. Data collected from the end-of-review survey in 2006 and 2008-2009. Outcomes studied: Effect learner-content interaction on engagement, grades, course completion. Effect of course navigation on student satisfaction Impact on student satisfaction and success between QM school and non-QM schools
  • New Directions for the Quality Matters Program

    1. 1. New Directions for the Quality Matters Program Ron Legon, Executive Director Deb Adair, Director Sloan-C International Conference Oct. 29, 2009 Inter-Institutional Quality Assurance in Online Learning © MarylandOnline, Inc., 2009. All rights reserved.
    2. 2. Presentation Agenda <ul><ul><li>Quality Matters Today </li></ul></ul><ul><ul><li>New Challenges </li></ul></ul><ul><ul><li>QM Research Agenda </li></ul></ul><ul><ul><li>QM Implementation Plan Recognition </li></ul></ul><ul><ul><li>K-12 Rubric </li></ul></ul><ul><ul><li>QM Program Audits/ Reviews </li></ul></ul><ul><ul><li>Q & A </li></ul></ul>
    3. 3. Quality Matters Today
    4. 4. “ Quality Matters: Inter-Institutional Quality Assurance in Online Learning” <ul><li>Quality Matters is a not-for-profit subscription service providing tools and training for quality assurance of online courses </li></ul><ul><li>Developed by MarylandOnline with funding from FIPSE, it was designed by faculty for faculty </li></ul><ul><li>Adopted by a large and broad user base, QM represents a shared understanding of quality in online course design </li></ul>
    5. 5. The QM Approach <ul><li>The QM toolset and process </li></ul><ul><li>A faculty-driven, peer review process that is… </li></ul><ul><ul><li>C ollaborative </li></ul></ul><ul><ul><li>C ollegial </li></ul></ul><ul><ul><li>C ontinuous </li></ul></ul><ul><ul><ul><li>C entered - in academic foundation </li></ul></ul></ul><ul><ul><ul><li>- around student learning </li></ul></ul></ul>
    6. 6. Current QM Applications <ul><li>A set of standards (rubric) for the design of online and hybrid courses </li></ul><ul><li>A peer review process (faculty to faculty) for reviewing and improving online and hybrid courses </li></ul><ul><li>A faculty support tool used by instructional development staff </li></ul><ul><li>A professional development opportunity </li></ul>
    7. 7. QM: A National Standard <ul><ul><li>370 + current subscribers </li></ul></ul><ul><ul><li>41 states represented </li></ul></ul><ul><ul><li>12 statewide systems; 5 large consortium </li></ul></ul><ul><ul><li>International presence (Canada and Bermuda) </li></ul></ul><ul><ul><li>5000+ faculty and instructional design staff have been formally trained </li></ul></ul><ul><ul><li>Award winning program </li></ul></ul>
    8. 8. Quality Matters Subscribers MONTANA WYOMING IDAHO WASHINGTON OREGON NEVADA UTAH CALIFORNIA ARIZONA NORTH DAKOTA SOUTH DAKOTA NEBRASKA COLORADO NEW MEXICO TEXAS OKLAHOMA KANSAS ARKANSAS LOUISIANA MISSOURI IOWA MINNESOTA WISCONSIN ILLINOIS INDIANA KENTUCKY TENNESSEE MISS ALABAMA GEORGIA FLORIDA SOUTH CAROLINA NORTH CAROLINA VIRGINIA WV OHIO MICHIGAN NEW YORK PENN MARYLAND DELAWARE NEW JERSEY CONN RI MASS MAINE VT NH ALASKA HAWAII Current Subscribers Statewide Subscribers PUERTO RICO VIRGIN ISLANDS
    9. 9. QM under FIPSE v QM Today <ul><li>FIPSE GRANT 03-06 </li></ul><ul><li>QM 3 YEARS LATER </li></ul>Funding Grant Subscriptions & fees Members 19 MOL schools at launch 370+ across US & abroad Tool Development Lit Review & COP Lit review, COP, users, QMAAC & research Rubric 2004 & 2005 editions 06/07, 08-10, K-12 editions Trainings 3 for implementation 10 including faculty development trainings Trainees: Reviews: 694 111 >5000 434 (official) Focus Course review Courses, faculty devel. & institutional improvement
    10. 10. New Challenges
    11. 11. Meeting the Needs of an Expanding Subscriber Base <ul><li>Demand for greater variety of QM-related experiences </li></ul><ul><ul><li>Greater Flexibility </li></ul></ul><ul><ul><li>Empowerment of Subscribers </li></ul></ul><ul><ul><li>Long Range Planning </li></ul></ul><ul><ul><li>Improved Tools </li></ul></ul><ul><ul><li>More Workshops </li></ul></ul><ul><ul><li>Networking Opportunities </li></ul></ul><ul><ul><li>Conferences </li></ul></ul><ul><ul><li>Publications </li></ul></ul>
    12. 12. Heightened Concern with Quality Assurance <ul><li>Legitimate Programs Jostling with Diploma Mills </li></ul><ul><li>Regionals Paying More Attention to Online Quality </li></ul><ul><ul><li>Middle States requires distance learning expertise on site visiting teams (2009) </li></ul></ul><ul><li>Federal Higher Education Opportunity Act (2008) </li></ul><ul><ul><li>Requires national accrediting agencies to </li></ul></ul><ul><ul><ul><li>Demonstrate their standards for evaluating program quality </li></ul></ul></ul><ul><ul><ul><li>Create review teams that are well trained and knowledgeable regarding distance ed. </li></ul></ul></ul><ul><ul><li>Requires National Research Council to compare quality of distance education with campus-based courses (unfunded) </li></ul></ul><ul><li>Department of Education Meta-Study (2009) </li></ul><ul><ul><li>More studies to come! </li></ul></ul>
    13. 13. Rising Interest from Other Sectors of Online Learning Community <ul><li>Rapid growth of online learning in other sectors increases interest in and demand for quality assurance: </li></ul><ul><ul><li>K-12 Education </li></ul></ul><ul><ul><li>Continuing Education (credit and non-credit) </li></ul></ul><ul><ul><li>Training Industry </li></ul></ul><ul><ul><li>ASP Providers </li></ul></ul><ul><ul><li>Learning Management System Providers </li></ul></ul>
    14. 14. New Tools and Paradigms <ul><li>Continued development of new technology increases challenges to design standards </li></ul><ul><ul><li>New ways to access online learning </li></ul></ul><ul><ul><ul><li>Ipods, Smartphones, Netbooks, Ebooks, etc. </li></ul></ul></ul><ul><ul><li>New software to build online learning </li></ul></ul><ul><ul><ul><li>Enhancements to existing LMS’s </li></ul></ul></ul><ul><ul><ul><li>Open Source Tools </li></ul></ul></ul><ul><ul><ul><li>Blogs </li></ul></ul></ul><ul><ul><ul><li>Web 2.0 Social Sites </li></ul></ul></ul><ul><ul><ul><li>What’s Next… Google Wave? </li></ul></ul></ul><ul><li>Quality Matters must be creative to keep pace </li></ul>
    15. 15. QM Research Agenda
    16. 16. QM sponsored research Sharing of user impact data Research Agenda: IMPACT
    17. 17. Research To Date <ul><li>Impact on Faculty Developers </li></ul><ul><ul><li>Ongoing data collection by QM </li></ul></ul><ul><li>Impact on Peer Course Reviewers </li></ul><ul><ul><li>Ongoing data collection by QM </li></ul></ul><ul><li>Impact on Student Learning Outcomes </li></ul><ul><ul><li>Independent research projects focused on specific rubric standards. (Exception: Aman, 2009, found potential effect of QM adoption on student satisfaction) </li></ul></ul>
    18. 18. QM Sponsored Research FY10 <ul><li>QM Standard 8 (Accessibility) Pilot Project (Bowen, Dallas TeleCollege TX) [Student voice re accessibility] </li></ul><ul><li>Impact of 67 QM-Trained Faculty on Large, Multi-Campus Organization (Diaz, Maricopa Community Colleges, Tempe AZ) </li></ul><ul><li>The Impact of QM Standards on Learning Experiences in Online Courses. (Knowles, Park University, MO) </li></ul><ul><li>QM Rubric as ‘Teaching Presence’: Application of Community of Inquiry Framework to Analysis of the QM Rubric’s Effects of Student Learning (Hall, Delgado Community College, Metairie LA) </li></ul><ul><li>Linking Course Design to Learning Processes Using the Quality Matters and Community of Inquiry Frameworks (Swan, University of Illinois/Springfield) </li></ul>
    19. 19. IMPACT: Research & Best Practice <ul><li>2 nd Annual QM Conference : “Improving Outcomes through Quality Standards” (Chicago, June 11-13 th ) </li></ul><ul><ul><li>Presentation of QM-sponsored Research projects </li></ul></ul><ul><ul><li>Sessions focused on presentation of positive outcomes by QM subscribers and participants </li></ul></ul><ul><ul><li>Conference serves as forum for the aggregate of best practices on quality IMProvements and ACTions to implement those improvements. </li></ul></ul>
    20. 20. QM Implementation Plans
    21. 21. QM Subscribers Plan Ahead <ul><li>Engagement with QM is a long-term commitment </li></ul><ul><li>QM’s spread through a school can be a slow process: </li></ul><ul><li>Resource availability for full implementation of QM may not be available in the short run </li></ul><ul><li>QM should be integrated into the school’s strategic planning process </li></ul>
    22. 22. Schools Seek to Assure Stakeholders of Commitment to Quality <ul><li>QM’s seal may be displayed on a reviewed course </li></ul><ul><ul><li>But this may not be visible to the public, or even to non-enrolled students </li></ul></ul><ul><li>QM subscribers asked for ways to publicly announce their commitment to QM standards for all their courses </li></ul><ul><li>QM’s response: Recognition of Impelementation Plans </li></ul>
    23. 23. QM Implementation Plan Process <ul><li>QM staff work with a school to develop a plan </li></ul><ul><ul><li>Minimum 3 year timeframe </li></ul></ul><ul><ul><li>May be institution-wide, or limited by program </li></ul></ul><ul><ul><li>Conditioned by school’s goals and resources </li></ul></ul><ul><ul><li>Endorsed by institutional leadership </li></ul></ul><ul><li>Plan submitted to the QM Academic Advisory Council for review and approval </li></ul><ul><li>Once approved, the school must report progress annually </li></ul><ul><li>School may display special seal: and statement on their commitment to QM standards </li></ul>
    24. 24. K-12 Rubric
    25. 25. “ Why can’t we use QM in K-12?” <ul><li>We heard this question for years </li></ul><ul><li>Many elements of the QM Rubric are relevant to K-12 </li></ul><ul><ul><li>But there are also many differences </li></ul></ul><ul><li>QM needed a partner in K-12 to tackle this challenge </li></ul><ul><ul><li>The Florida Virtual School </li></ul></ul><ul><ul><ul><li>Leading state virtual school </li></ul></ul></ul><ul><ul><ul><li>Exports courses to many other states </li></ul></ul></ul>
    26. 26. Process to Develop a Rubric for K-12 Course Design <ul><li>Joint steering committee established </li></ul><ul><li>Fit / Gap analysis of current QM Rubric </li></ul><ul><ul><li>3 FLVS courses reviewed by two joint teams </li></ul></ul><ul><ul><ul><li>Majority of standards fit middle and high school </li></ul></ul></ul><ul><ul><ul><li>Some critical issues in K-12 were not addressed </li></ul></ul></ul><ul><ul><ul><li>Annotations did not speak to K-12 </li></ul></ul></ul><ul><li>Decision to move ahead with Grades 6-12 Rubric </li></ul><ul><ul><li>Joint working group appointed </li></ul></ul><ul><ul><ul><li>7 month process to perfect the new rubric </li></ul></ul></ul><ul><ul><li>Beta testing in November/December 2009 </li></ul></ul><ul><li>Release of Grades 6-12 Rubric in early 2010 </li></ul>
    27. 27. Key Elements in Grades 6-12 Rubric <ul><li>Compliance with state standards </li></ul><ul><li>Protection of student privacy </li></ul><ul><li>Incorporation of 21st Century Skills </li></ul><ul><li>Attention to student reading level </li></ul><ul><li>Increased emphasis on </li></ul><ul><ul><li>Netiquette </li></ul></ul><ul><ul><li>Integrity </li></ul></ul><ul><ul><li>Self-check activities </li></ul></ul><ul><li>Resulted in 100 point standard </li></ul>
    28. 28. Online Learner’s Bill of Right s ©
    29. 29. Quality Matters’ Online Learner Bill of Rights© <ul><li>Students value the QM standards </li></ul><ul><ul><li>Studies by </li></ul></ul><ul><ul><ul><li>Rick Aman (College of Western Idaho) </li></ul></ul></ul><ul><ul><ul><li>Penny Ralston-Berg (Penn State) </li></ul></ul></ul><ul><li>Ralston-Berg study translates the language of the rubric into statements students would understand: </li></ul><ul><ul><li>“ The self-introduction by the instructor is appropriate” --> </li></ul></ul><ul><ul><li>“ The instructor introduces her or himself” </li></ul></ul><ul><li>Major Conclusion: </li></ul><ul><ul><li>High Correlation between Quality Matters standards and student’ concerns about online learning </li></ul></ul>
    30. 30. Next Steps in Development of Online Learner’s Bill of Rights <ul><li>Further testing of student attitudes </li></ul><ul><li>Seek institutional pledges to honor these practices </li></ul><ul><li>E.g., </li></ul><ul><ul><li>“ Online learners have a right to feedback on their graded assignments” </li></ul></ul><ul><ul><li>“ Online learners have a right to know the purpose of each major assignment” </li></ul></ul>
    31. 31. QM Program Audits/Reviews
    32. 32. Schools Seek External Validation of their Online Quality <ul><li>Many schools look for ways to validate their online programs </li></ul><ul><ul><li>To distinguish themselves from diploma mills </li></ul></ul><ul><ul><li>To answer skeptical stakeholders - dwindling, but still powerful </li></ul></ul><ul><ul><li>To improve mission-critical online programs </li></ul></ul><ul><ul><li>To recruit increasingly sophisticated students </li></ul></ul><ul><li>QM’s course reviews and implementation plan recognition are partial responses, but schools seek more… </li></ul>
    33. 33. Sources of Standards <ul><li>The Quality Matters Rubric </li></ul><ul><ul><li>Implicit in the rubric are expectations of the institution that go far beyond course design, e.g., </li></ul></ul><ul><ul><ul><li>Technology infrastructure </li></ul></ul></ul><ul><ul><ul><li>Faculty training </li></ul></ul></ul><ul><ul><ul><li>Student support services </li></ul></ul></ul><ul><li>CHEA and regional accreditors’ best practices </li></ul><ul><li>Association recommendations - Sloan-C, ITC, WCET, iNACOL, etc. </li></ul><ul><li>Research on effectiveness of distance learning </li></ul><ul><li>QM Academic Advisory Council </li></ul>
    34. 34. QM Audit/Review Standards & Process <ul><li>Standards cover </li></ul><ul><ul><li>Institutional Commitment </li></ul></ul><ul><ul><li>Teaching and Learning Standards </li></ul></ul><ul><ul><li>Faculty Support </li></ul></ul><ul><ul><li>Learner Support </li></ul></ul><ul><ul><li>Ongoing Evaluation </li></ul></ul><ul><li>Process includes </li></ul><ul><ul><li>Self study </li></ul></ul><ul><ul><li>Committee Review </li></ul></ul><ul><ul><li>One-day site visit </li></ul></ul><ul><ul><li>Report - Institutional Response - Final Report & Recommendations </li></ul></ul>
    35. 35. Next Steps <ul><li>Beta testing on several volunteer schools </li></ul><ul><li>Survey of QM subscribers to determine </li></ul><ul><ul><li>Level of interest </li></ul></ul><ul><ul><li>Pricing </li></ul></ul>
    36. 36. Looking Beyond
    37. 37. Many Challenges Ahead <ul><li>Keeping our rubrics current and relevant </li></ul><ul><li>Supplying the changing needs of subscribers </li></ul><ul><li>Looking at needs elsewhere in distance learning </li></ul><ul><li>Forming strategic alliances with other organizations devoted to the advance of distance learning </li></ul><ul><li>Exporting the Quality Matters approach </li></ul><ul><ul><li>Beyond the U.S. </li></ul></ul><ul><ul><li>Beyond distance learning </li></ul></ul><ul><li>Others we can’t forsee…. </li></ul>
    38. 38. Questions and Comments
    39. 39. Thanks to YOU… Quality Matters! Sloan-C International Conference Oct. 29, 2009 © MarylandOnline, Inc., 2009. All rights reserved.

    ×