How Do You Get There From Here


Published on

Overcoming Barriers in Implementing a Quality Assurance Process

WCET 21st Annual Conference
Deb Adair, Quality Matters
Evelyn Everett Knowles, Park University
Li Wang, University of Northern Colorado
Heidi Ashbaugh, Texas Woman’s University

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Quality Matters provides inter-institutional quality assurance in online learning. This is achieved through a not-for-profit subscription service providing tools and training for quality assurance of online courses.
  • Quality Matters has become a widely adopted program of quality assurance. It currently has more than 370 subscribers across 41 states and Canada. Statewide systems also subscribe to provide coordination and more economical access to QM tools for their member institutions. Please see for a current list of subscribers. Quality Matters has trained well over 3000 faculty and instructional design staff and has been recently recognized by both the Sloan Consortium and USDLA for excellence.
  • The map provides another view of the wide-spread adoption of QM. The color red/rust indicates states where there are QM subscribers and yellow indicates statewide subscriptions.
  • The underlying principles of QM are a primary reason for this wide-spread adoption. Quality Matters provides a faculty-driven, peer review process that is…. Collaborative : QM was designed by and for faculty to share expertise and experience relative to the design of a course. Collegial : The course review process is a collegial discussion between faculty peers committed to Continuous quality improvement. It is not an evaluation. Centered in national standards of best practice, the research literature and instructional design principles designed to promote student learning. The underlying principles of QM are a primary reason for this wide-spread adoption. Quality Matters provides a faculty-driven, peer review process that is…. Collaborative : QM was designed by and for faculty to share expertise and experience relative to the design of a course. Collegial : The course review process is a collegial discussion between faculty peers committed to Continuous quality improvement. It is not an evaluation. Centered in national standards of best practice, the research literature and instructional design principles designed to promote student learning.
  • The goal of Quality Matters is something all faculty and stakeholders can support – to make online instruction as good as it can be to facilitate student learning. While a QM peer course review quantifies quality as meet QM standards at a minimum of 85%, there are many positive outcomes of implementing QM beyond an improved course design.
  • Reaching that goal of quality requires schools to have an effective course development process, faculty and instructional design staff trained on the same standards and tools, and a planned process to build institutional capacity to become self-sufficient in the implementation of the process, training, and tools.
  • This slide introduces the next presenter who will address challenges with implementing a course development process. The presenter is Dr. Evelyn Knowles, Coordinator of Program Quality Development for Park University.
  • Welcome
  • Started in 1996 Courses going online Rapid Growth – by 2003 Platform change to eCollege 2004 culture of assessment began because of HLC 2005 Members of KCREACH which introduced us to Quality Matters in Fall 2005 Quality Matters Although we had a good internal course review based on best practices, we decided to adopt Quality Matters rubric because it had national recognition and had more literature research to back it up. Collaboration We hired more instructional designed to collaborate with faculty developers to make the development process smoother.
  • When we develop courses we have to be aware of many University policies and make sure that the courses are compliant with these policies. For example, the instructional designer verifies that the textbook used in the course is BOTH approved by the department and is ordered by the bookstore. This summer we are beginning to upgrade our courses . . . to the most recent accessibility standards . . . so that students can use screen reader programs such as Jaws to participate in online classes. This will provide Park the ability to serve a wider population of students.
  • This is a visual representation of the Continuous Quality Course Development Model. As you can see, both the University policies and department requirements, and the Quality Matters standards drive the course development process. We encourage faculty to work with the academic department program coordinator to identify their specific course development needs at the beginning of the course development process: 1) For example, some departments such as Psychology require the home page to contain a link to the PsychBytes web site which contains tutorials on APA formatting. Instructional designers inserted the links into the course home page for all of the online psychology courses. The courses must comply with University policies : 2) For example we support the Assessment initiative by verifying the approved CLOs are implemented in the course and that learning activities and assessments align to the CLOs. The capabilities within the eCollege and technology resources available can also impact course development: 3) For example the platform does not support audio PowerPoint presentations. The instructional designers will convert audio PowerPoints to Flash presentations. We use a checklist as a guide to identify ways to implement these standards and policies when we develop the courses. At the end of course development, the instructional design quality is reviewed using the Quality Matters Rubric and the content is approved by the academic department
  • To help faculty understand how to apply the standards and implement the University policies, we developed a common course structure and a course development checklist. The common course structure is a basic framework to help students identify where to locate content such as the course textbook information. For example, all courses have a link to the Online Student Help and Resource web site. The checklist is mapped to the standards and policies . For example, one OIES evaluation criteria requires adjuncts to participate in weekly discussions. During course development, the instructional designer verifies that the course design has implemented weekly discussion threads.
  • Marker : This slide introduces the next presenter who will address challenges with faculty training. The presenter is Dr. Li Wang, an instructional designer from the Center for the Enhancement of Teaching and Learning at the University of Northern Colorado.
  • Marker : This slide introduces the challenges identified regarding implementing QM and training at UNC. Awareness: Only a small percentage of instructors had heard about QM at the needs assessment meeting. Time: Online training, face to face training , and continuous dedication are time consuming. Incentives: Currently there is no incentives for instructors who participate except for the Office of Extended Studies Request For Proposal. Retention: Instructors who demonstrate interest at the beginning but drop out later. Collaboration: Only a few instructors are collaborating during the review process.
  • Marker : This slide introduces the next presenter who will address challenges with implementing a multi-year QM plan. The presenter is Heidi Ashbaugh, senior instructional designer from Texas Woman’s University.
  • ID philosophical stance is that mandatory participation will limit impact and diffusion with our faculty (many have spent entire career here; little or no exposure to how other institutions conduct business)
  • The program began through Distance Education with a pilot project to gauge interest in the program. DE has made a deliberate choice to manage the project rather than govern. At conclusion, this will be a faculty driven process facilitated by DE. We currently see no benefit to governing a project we fully intend to give away to faculty. Funding - $12 per credit hour, per student, per course that is designated as DE. This comes directly to the DE office, pays our salaries and is our operating budget. That is on top of tuition and course fees.
  • Marker :
  • How Do You Get There From Here

    1. 1. How do you get there from here? <ul><li>Overcoming Barriers in Implementing a Quality Assurance Process </li></ul><ul><li>WCET 21 st Annual Conference </li></ul><ul><li>Deb Adair, Quality Matters </li></ul><ul><li>Evelyn Everett Knowles, Park University </li></ul><ul><li>Li Wang, University of Northern Colorado </li></ul><ul><li>Heidi Ashbaugh , Texas Woman’s University </li></ul>10/23/2009 Denver, Colorado
    2. 2. Presentation Agenda <ul><ul><li>QM Program Introduction </li></ul></ul><ul><ul><ul><ul><li>Deb Adair </li></ul></ul></ul></ul><ul><ul><li>Challenges/Solutions in Implementation: </li></ul></ul><ul><ul><ul><li>Course Development & Review Process </li></ul></ul></ul><ul><ul><ul><ul><li>Evelyn Knowles </li></ul></ul></ul></ul><ul><ul><ul><li>Faculty Development Training </li></ul></ul></ul><ul><ul><ul><ul><li>Li Wang </li></ul></ul></ul></ul><ul><ul><ul><li>A Multi-Year Plan for Implementation </li></ul></ul></ul><ul><ul><ul><ul><li>Heidi Ashbaugh </li></ul></ul></ul></ul><ul><ul><li>Sharing Your Questions, Issues and Solutions </li></ul></ul>
    3. 3. <ul><li>Deb Adair, Ph.D. </li></ul><ul><li>Director of Quality Matters </li></ul><ul><li>Adjunct Associate Professor, University of Maryland University College </li></ul>What is Quality Matters?
    4. 4. “ Quality Matters: Inter-Institutional Quality Assurance in Online Learning” <ul><li>Quality Matters is a not-for-profit subscription service providing tools and training for quality assurance of online courses </li></ul><ul><li>Initially developed by MarylandOnline with funding from FIPSE, it was designed by faculty for faculty </li></ul><ul><li>Adopted by a large and broad user base, QM represents a shared understanding of quality in online course design </li></ul>
    5. 5. Quality Matters is … <ul><li>A set of standards (rubric) for the design of online and hybrid courses </li></ul><ul><li>A peer review process (faculty to faculty) for reviewing and improving online and hybrid courses </li></ul><ul><li>A faculty support tool used by instructional development staff </li></ul><ul><li>A professional development opportunity </li></ul>
    6. 6. QM as a National Standard <ul><ul><li>370 + current subscribers </li></ul></ul><ul><ul><ul><li>( ) </li></ul></ul></ul><ul><ul><li>41 states represented </li></ul></ul><ul><ul><li>QM has trained 5000+ faculty and instructional design staff </li></ul></ul><ul><ul><li>Recognized by </li></ul></ul><ul><ul><ul><li>- Sloan C Excellence in Online Teaching and Learning Award </li></ul></ul></ul><ul><ul><ul><li>-2008 USDLA Outstanding Leadership in the field of Distance Learning </li></ul></ul></ul>
    8. 8. The QM Approach <ul><li>The QM toolset and process </li></ul><ul><li>A faculty-driven, peer review process that is… </li></ul><ul><ul><li>C ollaborative </li></ul></ul><ul><ul><li>C ollegial </li></ul></ul><ul><ul><li>C ontinuous </li></ul></ul><ul><ul><ul><li>C entered - in academic foundation </li></ul></ul></ul><ul><ul><ul><li>- around student learning </li></ul></ul></ul>
    9. 9. A shared goal: Quality online courses <ul><li>Improved courses </li></ul><ul><li>Engaged faculty </li></ul><ul><li>Reduced course development time </li></ul><ul><li>Ongoing faculty development </li></ul><ul><li>Quality benchmarking </li></ul><ul><li>Awareness & support for online learning </li></ul><ul><li>Institutional improvement </li></ul>85 %
    10. 10. Getting there with QM can include… <ul><li>A course development process leading to QM-recognized courses </li></ul><ul><li>Professional development (training) for faculty and staff </li></ul><ul><li>Institutional self-sufficiency in conducting official QM peer course reviews </li></ul>
    11. 11. <ul><li>Evelyn Everett Knowles, Ph. D. </li></ul><ul><li>Coordinator of Program Quality Development </li></ul><ul><li>Park University </li></ul>Course Development Challenges
    12. 12. A Model for Enhancing Online Course Development : The Park University Continuous Quality Development Model
    13. 13. Continuous Quality Development Model Rapid Growth in Online Learning Culture of Assessment Platform Change / Content Manager Continuous Quality Development Model Collaboration between Faculty and Instructional Designer Quality Matters
    14. 14. Development Contract Course Developer Requirements Payment for Development New Online Course Development Initiate Course Development Maintenance Contract Course Maintenance Requirements Payment for Maintenance Online Course Development Policies Limits on the Timing and Number of Courses Developments per Developer Policies Directly Impacting Online Course Development Accessibility and ADA standards and requirements Academic Freedom Full-Time Faculty Contract Online Adjunct Faculty Contract Faculty Course Load Faculty Course Preparation Faculty Evaluation Student Academic Issues Online Attendance and Grades Proctoring for Online Programs Online Course Student Training Academic Policies Operational Policies Information Security Bookstore and Library Institutional Policies
    15. 15. Continuous Quality Development Model Literature Research Research Standards Best Practices Course Development Quality Matters Standards Online Course Development CD Requirements & Checklist Quality Matters Rubric Park Online Quality Standards Review QSR University and CDL Policies Academic Department Requirements Park University Requirements Content Approval eCollege capabilities Best Practices
    16. 16. <ul><li>Common Course Structure : </li></ul><ul><ul><li>Where things are located </li></ul></ul><ul><ul><li>What content to develop </li></ul></ul><ul><ul><li>Accountability </li></ul></ul><ul><li>Checklist mapped to : </li></ul><ul><ul><li>Quality Matters standards </li></ul></ul><ul><ul><li>University requirements </li></ul></ul><ul><ul><li>Best Practices </li></ul></ul><ul><ul><li>OIES </li></ul></ul>
    17. 17. Quality Review <ul><li>Quality Standards Review </li></ul><ul><ul><li>Quality Matters Rubric </li></ul></ul><ul><ul><ul><li> </li></ul></ul></ul><ul><ul><ul><li>Report returned to Instructional Designer </li></ul></ul></ul><ul><ul><ul><li>Recommendations and changes are made before the courses are duplicated for the next term </li></ul></ul></ul><ul><ul><li>Academic Department Review </li></ul></ul><ul><ul><ul><li>Required changes are made by the instructional designer with the Course Developer </li></ul></ul></ul>
    18. 18. Recognized Courses <ul><li>2009 Quality Matters Recognized Courses </li></ul><ul><li>The following courses were recently recognized by Quality Matters in 2009. CourseTitleFaculty DeveloperInstructional Designer AC435Ethics for AccountantsJolene LamptonChris Slupianek BI210 The Human Body Carol Sanders Jim Whelan CJ332 Institutional, Industrial and Commercial Security Joe Angelo </li></ul><ul><li>Jackie Artmayer, David Curtis EN234Intro to FictionJane WoodJackie Artmayer FI363Financial Institutions and MarketsJoe KubecCasey Johnson GO205Intro to MeteorologyLeAnn AllisonChris SlupianekPS424Industrial and Organizational PsychologyWilliam NastJim Whelan </li></ul><ul><li>2008 Quality Matters Recognized Courses </li></ul><ul><li>The following courses were recently recognized by Quality Matters in 2008. CourseTitleFaculty DeveloperInstructional Designer CJ232Introduction to CorrectionsMichael Eskey  CJ400 Constitutional Law in Criminal JusticeSamantha BurkeJackie Artmayer CJ430 Criminal Justice Research MethodsMike EskeyJim Whelan CS322Web Programming IIKathleen Kalata EC300 Intermediate MicroeconomicsGuerman KornilovTeri Wright EC301Intermediate MacroeconomicsGuerman Kornilov Teri Wright EN201 Introduction to Literature Jane WoodJackie Artmayer EN221 Afro-American Literature Cynthia WilliamsJackie Artmayer EN306A Scientific and Technical Writing Dees Stallings EN306B Business Communications Dees Stallings EN306CProfessional Writing in the Disciplines Emily Donnelli HR434 Compensation Management Gina HellerKristy Sailors IS205 Managing Information Systems Frank UzinskyKathleen Kalata LG201 Systems Engineering and Analysis Ernest ReidJackie Artmayer LG415 Quality Control Scott SowderJackie Artmayer MG261 Business Law II Cathy TaylorDavid Curtis MG401 Senior Seminar in Business Paul FordKristy Sailors MG495 Business Policy Dennis HermerdingJim Whelan PS221Adolescent PsychologyTeresa MasonKristy Sailors PS361 Cross Cultural Psychology Dennis KerkmanJim Whelan PS410 Social Influence and Persuasion Jean MandernachJim Whelan SO208 Social Inequalities David Curtis SO210 Social Institutions Pat McGintyTeri Wright SO306Writing for Social SciencesJean MandernachJim Whelan </li></ul><ul><li>2007 Quality Matters Recognized Courses </li></ul><ul><li>The following courses were recognized by Quality Matters in 2007. </li></ul><ul><li>CourseTitleFaculty DeveloperInstructional Designer BI101Biological ConceptsCarol SandersJim Whelan CS321 Web Programming 1Katie KalataPS315 Theories of PersonalityJean Mandernach SO141 Introduction to SociologyDavid Curtis  </li></ul>
    19. 19. <ul><li>Li Wang, Ph.D. </li></ul><ul><li>Instructional Designer </li></ul><ul><li>Center for the Enhancement of Teaching and Learning </li></ul><ul><li>University of Northern Colorado </li></ul>Faculty Training Challenges
    20. 20. Implementing Introducing Planning Goals CETL OES Future Ultimate faculty
    21. 21. <ul><li>Center for the Enhancement of Teaching & Learning (CETL) </li></ul><ul><ul><ul><li>Models </li></ul></ul></ul><ul><ul><ul><li>Conversations </li></ul></ul></ul><ul><ul><ul><li>Funds </li></ul></ul></ul><ul><li>Office of Extended Studies (OES) </li></ul>Implementing Introducing Planning Goals
    22. 22. <ul><li>Needs Assessment (meetings in fall 2008) </li></ul><ul><li>Faculty Training (December 2008: onsite and three-day training) </li></ul><ul><li>QM project key staff training and training customization (ongoing) </li></ul><ul><li>Offering Workshops: group and one on one </li></ul>Implementing Introducing Planning Goals
    23. 23. Implementing Introducing Planning Goals CETL OES faculty Next : CETL
    24. 24. <ul><li>Implementing: CETL </li></ul><ul><li>Goals : 4 C Principles </li></ul><ul><li>Incentives : Travel award, training fees, & recognition </li></ul><ul><li>Timeline : 1. Training (Spring 09) </li></ul><ul><li> 2. Self review (summer 09) </li></ul><ul><li> 3. Peer review (fall 09) </li></ul>Implementing CETL OES faculty Next : CETL Cont.
    25. 25. <ul><li>Implementing: CETL Cont. </li></ul><ul><li>Focus : Improve the quality of course design </li></ul><ul><li>Collaboration : Support, Share ideas across disciplines </li></ul><ul><li>Important Considerations : Time, Organization, collaboration </li></ul>Implementing CETL OES faculty Next : Faculty Back: CETL
    26. 26. Implementing CETL OES faculty Next : OES <ul><li>Cohort Spring 2009 </li></ul><ul><li>Non Cohort Summer 2009 </li></ul><ul><li>Individuals </li></ul>Comments: “ This workshop was highly beneficial in the following ways:  encourages quality in online teaching, fosters faculty collaboration, gives support for faculty” Back: CETL Cont. <ul><li>Spanish </li></ul><ul><li>Philosophy </li></ul><ul><li>Nursing </li></ul><ul><li>Criminal Justice </li></ul><ul><li>Special Education </li></ul><ul><li>Sociology </li></ul><ul><li>Library </li></ul><ul><li>Bilingual ESL </li></ul><ul><li>Educational Technology. </li></ul>
    27. 27. <ul><li>OES: New Programs and Courses </li></ul><ul><li>Goals: Quality improvement, teaching, collaboration All new programs developed using QM </li></ul><ul><li>Incentives: RFP - $1000 to $2000 stipend for faculty developing new courses with QM </li></ul><ul><li>Progress: 3 new programs, 4 major revisions </li></ul><ul><li>2 sets professional development modules </li></ul><ul><li>1 program - QM Program Level Certification </li></ul><ul><li>New: MA Criminal Justice, MA Spec. Ed. Gifted/Talented, Interventionist Spec. Ed. Professional Development Series, ASL Interpreting Series </li></ul><ul><li>Revised: MS and Ph.D. in Nursing Ed, RN to BSN, Nursing Ed. Cert </li></ul>Implementing CETL OES faculty Next : Main Presentation Back: Faculty Cont.
    28. 28. Implementing Introducing Planning Goals Future Ultimate Next : Future
    29. 29. <ul><li>Future Goals </li></ul><ul><li>CETL: Cohort and Non Cohort </li></ul><ul><ul><ul><li>Conducting Peer Review </li></ul></ul></ul><ul><ul><ul><li>Recruiting participants </li></ul></ul></ul><ul><li>OES: </li></ul><ul><ul><ul><ul><ul><li>Continuing use of RFP </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Courses informally reviewed prior to delivery </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>One certified faculty in each participating program </li></ul></ul></ul></ul></ul><ul><li>Others: </li></ul><ul><ul><ul><li>Continuing collaboration between CETL & OES </li></ul></ul></ul><ul><ul><ul><li>Continuing support for QM related scholarship of teaching and learning </li></ul></ul></ul><ul><ul><ul><li>Involving more faculty as non cohort (offering workshops) </li></ul></ul></ul><ul><ul><ul><li>Conducting research </li></ul></ul></ul>Goals Future Ultimate Next : Ultimate
    30. 30. Goals Future Ultimate <ul><li>Ultimate Goals </li></ul><ul><li>QM certified Representatives from each Program/department/college </li></ul><ul><li>Ongoing Official Course Reviews </li></ul><ul><li>Ongoing Official Program Reviews </li></ul><ul><li>Research dissemination </li></ul><ul><li>4 C Principles </li></ul>Back: Future End
    31. 31. <ul><li>Awareness </li></ul><ul><li>Time </li></ul><ul><li>Incentives </li></ul><ul><li>Retention </li></ul><ul><li>Collaboration </li></ul>
    32. 32. <ul><li>Heidi Ashbaugh </li></ul><ul><li>Senior Instructional Design Specialist </li></ul><ul><li>Texas Woman’s University </li></ul>Multi-Year Plan Challenges
    33. 33. Implementing the QM Process at Texas Woman’s University Presented by - Heidi Ashbaugh Sr. Instructional Design Specialist Texas Woman’s University with much input from - Allison Peterson Sr. Instructional Design Specialist Texas Woman’s University
    34. 34. Implementation <ul><li>Phase 1 (Started Spring 2007) </li></ul><ul><ul><li>Goal: To train 70 faculty as QM Peer Reviewers </li></ul></ul><ul><ul><li>Completed on schedule (2 ½ years) </li></ul></ul><ul><li>Phase II (In process) </li></ul><ul><ul><li>Goal: To initiate a Peer Review process at TWU </li></ul></ul><ul><ul><li>Approximately 3 years </li></ul></ul><ul><ul><ul><li>First 2 years focus on set up and trial of an internal process </li></ul></ul></ul><ul><ul><ul><li>Third year goal is to be able to manage the formal QM process internally </li></ul></ul></ul>
    35. 35. Concerns <ul><li>Misunderstandings by faculty about instructors being “required” to participate </li></ul><ul><ul><li>We have never considered making this a “required” process. </li></ul></ul><ul><ul><li>Both phases of the project are voluntary. </li></ul></ul><ul><li>Concerns by faculty about “requiring” courses to meet QM standards </li></ul><ul><ul><li>We do not have any control or reason to “require” that courses meet QM. </li></ul></ul>
    36. 36. Governance & Resources <ul><li>Currently Distance Education/Lifelong Learning are the governing authorities for QM </li></ul><ul><ul><li>This is more managing the logistics of the program than really “governing”. </li></ul></ul><ul><li>Funding </li></ul><ul><ul><li>Fortunately DE has a budget that supports our continuation and expansion of this project. </li></ul></ul><ul><ul><li>Faculty participation in QM training is considered faculty development. </li></ul></ul><ul><ul><li>Some of our annual budget is dedicated to training and a limited number of national reviews. </li></ul></ul>
    37. 37. Instructional Design Concerns <ul><li>Time commitments - QM is not a full time job for anyone </li></ul><ul><ul><li>The IR has to devote time from an already busy schedule to manage training and reviews on both the national and campus level. </li></ul></ul><ul><ul><li>The Instructional Designers have to make time for multiple informal reviews. </li></ul></ul><ul><ul><li>Our Administrative Assistant must make time to organize finances for the annual subscription, as well as for purchasing materials, additional trainings, and official reviews. </li></ul></ul>
    38. 38. Current Status <ul><li>Working to move forward and introduce a Peer Review process at TWU </li></ul><ul><ul><li>We have enough faculty who have been through peer reviewer training to do this. </li></ul></ul><ul><li>Continuing concerns with having the program getting too big too fast </li></ul><ul><ul><li>Time constraints are the main concern here. </li></ul></ul><ul><li>Looking at new challenges - considering putting up some complete programs for national review </li></ul><ul><ul><li>There are many details that need to be addressed before this becomes a reality. </li></ul></ul><ul><ul><li>Time constraints will be a concern with this, as well. </li></ul></ul>
    39. 39. The TWU Team <ul><li>Keith Restine, Associate Director of Distance Education </li></ul><ul><li>Alli Peterson, Senior Instructional Design Specialist, Denton & </li></ul><ul><li>QM Institutional Representative </li></ul><ul><li>Valerie Shapko, Senior Instructional Design Specialist, Houston </li></ul><ul><li>Heidi Ashbaugh, Senior Instructional Design Specialist, Dallas </li></ul><ul><li>Tracey Mac Gowan, Instructional Design Specialist </li></ul><ul><li>Jake McBee, Instructional Design Specialist </li></ul>
    40. 40. <ul><li>What are your quality assurance initiatives? </li></ul><ul><li>What challenges do you face? </li></ul><ul><li>How are you addressing them? </li></ul>What is are your Challenges and Solutions in QA Implementation?