About The Content: There are six learning objectives to review with the participants. Be sure to point out that the purpose of this training is to prepare the participants to apply the QM rubric to their own or another’s online course (NOT to teach them how to teach online or to review their own online courses). NOTE: The day’s evaluation is based on how well the participants believe that these objectives were met. This training is specifically designed for two audiences: 1. Participants who want to learn about and apply the QM rubric and process WHO: Faculty members, instructional designers, administrators who want to know more about QM and apply the process but who are not eligible to become peer reviewers OR who do not plan to serve on an official QM peer review team. What recognition will you receive?: Once your facilitator submits the completion data to QM, you will be able to print your certificate from MyQM. 2. Participants who want to serve as certified QM Peer Reviewers WHO: Online instructors who meet the following criteria will be eligible to serve as QM Peer Reviewers upon successful completion of the online Peer Reviewer Course: You must complete the Applying the QM Rubric workshop. You must have taught online within the last 18 months. You must complete the two week online Peer Reviewer Course and complete the application and MOU to be registered in the QM Database. What should you do?: Upon completion of this Applying training, the Trainer will forward your name to QM. You should then register for the online Peer Reviewer Course at the QM web site (www.qualitymatters.org). The fee for this week-long session is $150 per person for subscribers/$300 per person for non-subscribers . You should not apply to be a Peer Reviewer until you have successfully completed the QM Peer Reviewer Course.
Marker : The following slides address the “key” QM concepts.
About The Content: The underlying principles of QM are a primary reason for this wide-spread adoption. Quality Matters provides a faculty-driven, peer review process that is…. Collaborative : QM was designed by and for faculty to share expertise and experience relative to the design of a course. Collegial : The course review process is a collegial discussion between faculty peers committed to Continuous quality improvement. It is not an evaluation. Centered in national standards of best practice, the research literature and instructional design principles designed to promote student learning.
About The Content: This slide presents a QM KEY concept: Emphasize that QM is not looking for “just good enough.” QM reviewers are looking for above average (approximately 85% or B+). Point out that although this is somewhat subjective, the basis for the decision is based in the standards and the annotations which are grounded in the research literature and widely accepted standards about effective distance learning. Peer Reviewers are all experienced online faculty who have attended QM training and learned to apply the rubric. QM relies on the experience, expertise and common sense of its faculty reviewers to conduct reviews fairly and consistently and to judge whether the course meets expectations at the “85%” standard. The emphasis on above average is also the reason that QM primarily reviews “mature” courses (taught at least two semesters). If the course has been taught over several semesters, the faculty developer has had time to “fine tune” the course and to make management and content improvements. NOTE FOR FACILITATORS: Many new Peer Reviewers find the concept of 85% confusing. There are actually TWO 85 % thresholds: 1. The first is that a course must earn at least 81 out of 95 points (or about 85%). The points from all MET or NOT YET decisions on specific standards are added and must be at the 85% or greater percentage - AND meet all 21 essential standards - to be QM recognized. 2. The second is that when peer reviewers should use the 85% rule to determine whether or not a specific standard is met. The standard does not have to be 100% to be a “met". The "85% rule" is a guide for reviewers to gauge whether they will decide “met" or "not yet" for that particular standard. The assumption is that experienced online faculty members will have a "gut level" recognition of what that means.
About the Content : There are many factors that affect the quality of an online course. Among these factors are: the course design (the forethought and planning that goes into an online course,) the course delivery (i.e. the way the course is taught, also known as faculty performance), the course content, the learning management system and its functionality; technical support the institutional infrastructure (help desk, online library access, online tutoring access, etc), a faculty member ’ s training and readiness for online teaching, and the students ’ role with respect to engagement and readiness for an online course. QM reviews just one aspect of online course quality – Course Design .
About The Content: This slide reinforces the concept that QM is about Design, not Delivery. QM recognizes that this is a fine line, but course design is the primary emphasis during a review. Design is usually about what the faculty member plans and prepares for BEFORE the students arrive in the course; delivery is what happens after the students login.
About The Content: This slide presents a concrete example of how you might distinguish Course Design from Course Delivery. Feel free to use an example from your own course if you have one. How To Present It: Ask the participants if they can think of their own examples (from “real” life or from their teaching experience) and share them with the group.
**** Handout: Participants must have a copy of the QM Rubric to use during the training.**** About The Content: There are 8 main sections of the rubric (these are the 8 General Review Standards and are noted in red letters in the QM Workbook. The rubric consists of 41 Specific Review Standards which are distributed over these 8 general categories. Take the time to point out the organization of the QM Workbook (if you are using them) and how the rubric is organized: Eight General Review Standards Forty-one Specific Review Standards In Table Format: 1 st column is Specific Review Standards, 2 nd column is point value, and the last column is the annotation. Talk about QM’s intention to be “holistic” and that a QM review is intended to ensure that all parts of the course work together. You can use the metaphor of a cake recipe. Baking a cake is fairly simple if you follow the recipe and correctly measure and add the right ingredients (and the result is also fairly simple and straightforward: if you do it correctly, you end up with a cake). Reviewing an online course is much more complex: not only must you include all the “ingredients” but they must all work together to support the learning objectives. For example, you could have strong, measurable learning objectives but if they don’t align with the assessments, you still don’t have a quality online course. NOTE: Your discussion of alignment should be brief and introductory; you will spend more time explaining “Alignment” in another section of the presentation.
About The Content: Again, highlights the five General Review Standards that must work together. Note that although there are only four boxes here, there are actually FIVE general standards that align (and six specific review standards).
About The Content: The purpose of this slide is to focus on what QM is and is not: about course design, not about the instructor about course quality, not about faculty evaluation score that comes out of a QM course review is Diagnostic rather than Judgmental. It tells us how much revision a course may need. about continuous improvement, no about pass or fail You can acknowledge that QM is walking a fine line because it is difficult to separate the course design and the instructor. However, the intention of a QM review is to focus on the course design. Rather than reading the negatives, you might want to focus primarily on what QM is about.
Marker : The following slides deal with the “key” QM concepts.
About The Content: This slide emphasizes that QM is both a Process and a Rubric (tool) for course reviews. Participants (and institutions) should understand that there are significant benefits of using QM informally, but that official QM recognition comes only if the “official” process is followed.
About The Content: QM Circle Process : This visual highlights the QM review process. Key points: QM is designed for continuous improvement; the goal is that ALL courses will eventually meet QM expectations. How To Present It: Ask participants to look at the diagram and make inferences about it. Typically, they will respond with “it’s a circular process;” “it’s continuous:” etc). Begin with the COURSE and then continue through PEER COURSE REVIEW, FEEDBACK, COURSE REVISION and MEETS QUALITY EXPECTATIONS. Speak briefly about how each item contributes to the process. Participants will learn more about each item throughout the workshop. This diagram illustrates the focus on continuous improvement and summarizes the Quality Matters course review process: COURSE : Beginning at the top of the cycle, institutions decide to examine an online or hybrid course as part of a peer review. The institution may submit its course to QM for a formal review or, if the institution is a QM subscriber, it may conduct the formal course review in-house (both a formal QM review and a formal in-house process lead to QM recognition). The institution may also decide to use the rubric informally and do a review using its own process (this informal process does not lead to QM recognition). PEER COURSE REVIEW: The course is then reviewed by a team of three peer reviewers using the QM Rubric. The QM rubric is based in national standards of best practice, the research literature, and instructional design principles. Peer reviewers must have online teaching experience and complete the Peer Reviewer Training to be eligible to serve on a formal QM review. The peer review team consists of at least one member from an institution other than the course’s home institution. The team also consists of one member from a discipline that matches that of the course. This combination of reviewers ensures a diverse set of perspectives. FEEDBACK: Following the course review, the review team’s feedback is provided to the faculty member or team that developed the course. The feedback consists of two components: Scoring - indicates which QM standards were and were not met by the course. Feedback - a rich set of comments from the reviewers indicating the strengths of the course, areas for improvement and specific recommendations and suggestions for improving the course. COURSE REVISION : Upon initial review, the course may or may not have met Quality Expectations. In either case, the QM review provides support for course revisions and improvement. If a course did not initially meet Quality Expectations, the team chair will re-review the course after revisions. MEETS QUALITY EXPECTATIONS: QM expects that all courses will work toward and achieve quality expectations. The QM review process is not meant to be a test in which a course passes or fails. The overall goal is to provide a system for the improvement of course quality, rather than the simple assignment of a grade or quality level to the initial course.
About The Content: The QM rubric and tools can, of course, be used to review new online courses (or even courses in the development state). However, institutions make a significant investment in time, resources, and funds when they choose to review a course. The QM team suggests reviewing “mature” courses to maximize this investment. Other triggers could also indicate a re-review; it’s really up to the institution to decide how often each course will be reviewed or re-reviewed.
About The Content: Be sure to point out that there a 3 faculty peer reviewers who actually complete the Course Review Management System (the web-based review form). One of these reviewers must be a Subject Matter Expert (SME) and at least one must be external to the course’s home institution (although a single person could play multiple roles, could be both the SME and the master reviewer). Emphasize that the Faculty Course Developer (the Instructor) should be consulted by the team as questions or issues arise.
About The Content: Actual review time varies widely. In general the first review takes longer than subsequent reviews (in other words, reviewers become more efficient as they gain experience during reviews). Master Reviewers are advised to include new reviewers on their teams.
About The Content: Emphasize here that ALL courses will meet expectations, either at the time of the initial review or after a period of revision. A typical question at this point is: After the course has been revised, does the entire team need to see it again? The answer is that for most courses, only the Team Chair/Master Reviewer looks at the revised course and makes decisions about whether/not it meets expectations. The Faculty Developer completes a Course Amendment form that explains the changes that were made. The MR checks the course for any changes made and can determine if the course now meets the standard or may write more recommendations and note that the standard is not yet met.
Marker: The following slides highlight the origins and organization of the QM rubric.
About The Content: If you haven’t already done so, point out the breakdown of the 41 Specific Review Standards by points: 21 Standards are ESSENTIAL. Without all essential standards being met at the 85% level, a course cannot be considered a quality online course. 12 Standards are Very Important to an online course 8 Standards are Important to an online course. This results in a total of 41 Specific Review Standards and a total of 95 points .
About The Content: Emphasize again that each reviewer individually decides Met or Not Met on a particular standard and enters this decision in the course review management system (web-based Rubric tool). The web form then automatically totals the points (majority rules) and determines whether/not the course meets the standard. Tell participants that the peer reviewers don’t have to reach consensus. Since the instructor will see all decisions and all feedback, it will be very helpful to know that there was a “split decision” on a particular standard and to read the specific recommendations for how to improve.
About The Content: This information is important because new Peer Reviewers frequently think they are assigning points (and sometimes think they can assign 2 points for a 3-point standard if the standard is “sort of” met). In practice they decide whether a Specific Review standard is met at the 85% standard, indicate MET or NOT MET in the course review management system (the web-form), and then the web form actually assigns the points.
About The Content: There are TWO thresholds for meeting expectations and a course must meet BOTH thresholds to meet expectations. Mathematically, this works out to about an 85% standard.
About The Content: There are TWO thresholds for meeting expectations and a course must meet both thresholds to meet expectations. These thresholds are highlighted on slide 31: “Yes” on all 21 of the 3-point “essential” standards and a minimum of 81 out of 95 points. Mathematically, this works out to about an 85% standard (81/95 = 85%). New QM reviewers often have many questions about the 85% standard and often find it confusing. There are actually two 85% issues to consider during a QM review: The first is that a course must earn at least 81 out of 95 points (or about 85%). Each of the 41 specific review standards receives a "yes" or "no" vote regardless of point value. The points are added up and must be at the 85% or greater percentage (AND meet all 21 essential standards) to be QM recognized. The second is that when you are conducting your independent review of the course and making your determination if the specific standard is met, use the 85% rule for yourself in making that judgment. The standard does not have to be 100% to be a "yes". The "85% rule" is a guide for you as a reviewer to gauge whether you will choose "yes" or "no" for that particular standard. Here's an example : A Chemistry course might have 15 course-level learning objectives. Of these, two are not measurable (the other 13 are measurable). Most reviewers would decide YES, this meets Standard 2.1. They would then write strong recommendations for how to improve the two weak objectives. QM is admittedly a subjective process and making decisions about whether a standard is or is not met can be difficult. QM provides extensive annotations and training for its reviewers, but there are likely to be many instances of differing opinions. We know that everyone wants a "right/wrong" answer, but frequently, that's just not possible. Online courses are very complex and likely to be more shades of gray than absolute. And that's the reason QM "hires" three experienced online faculty members to serve on the peer review team. QM relies on the collective experience, knowledge and common sense of the team to arrive at the best possible feedback for the course instructor.
About The Content: Again, highlights the General Review Standards that must work together. Note that although there are only four boxes here, there are actually FIVE general standards that align (and six specific review standards). This is the second time your participants will see this slide…emphasize the importance of alignment to the QM rubric.
About The Content: This slide lists a learning objective from an Intro to Psych course and five possible examples of activities that might support this objective. Discuss whether/not and HOW each of the examples flows from or supports the objective. Questions to consider: If information is delivered in the first assignment, why is the second assignment important? (it serves to reinforce the first assignment and provides visual and auditory alternative delivery) How does the graded assignment support the learning objective? (it serves as “scaffolding to ensure that students understand the basic content) How does the graded discussion forum support the learning objective? (this is the major evidence that students have the content and understanding to meet the learning objective) Is the exam necessary to verify that this learning objective is met? (No, the exam is an objective test that doesn’t align with the learning objective since there’s no way to measure “summarize” on such an exam. It could be eliminated since feedback is provided in the graded assignment and the graded discussion.) What would happen if we eliminated the graded assignment and the graded discussion forum? (the learning objective is no longer measured since the mid-term does not provide an opportunity for students to demonstrate that they can “summarize” the relationships. What else might change if the instructor decided that perhaps the course-level objective should be “identify” instead of “summarize” since this is, after all an introductory course? (You could eliminate the graded discussion forum,)
About The Content: Direct participants to look at: 8 Specific Review Standards support the general standard: 2 Essential Standards 2 Very Important Standards 4 Important Standards Introducing General Standard 1: Let's put standard 1 into perspective: You've just finished designing your online or hybrid course, creating rich content, connected activities and aligned assessments. The course is rigorous and meets your pedagogical standards. You're done, right? What you might not recognize is that you're at a pedagogical challenging point of online course design. As the designer you know how the course is organized and where/how students are to begin. But, you haven't shared this with your students! What becomes the easiest thing for a designer to forget at the end of the process - that is, clear instructions to the students about how to begin - becomes the first thing that students are faced with when they click into an online course. Imagine yourself in your f2f classroom. What would happen if you didn’t show up on the first day of the course, stand in the front of the room, welcome your students, provide information and give directions? Right…nothing! The students would at first be confused, then angry…and then leave (perhaps to go visit your department chair’s office).The same is true for your online classroom…you can’t expect your online students to know what to do to get started. As the course designer, you definitely have the inside track on this one. After all, you put the course content, activities, and assessments in the most logical place - you can locate everything in a heartbeat! You know exactly where to click once you login to the course. You might catch yourself thinking, "anyone would intuitively know to click on the correct place to get to where they need to be." But, of course, it doesn't work that way. You have to tell them how to get started in and be successful in YOUR course. The acronym ONE can be useful when you’re getting ready to make sure you’re addressing the important issue of making sure your design includes ways to welcome students to your online course. O rganizing to give your students a good start by presenting the whats, whens, whys, and hows of the course design. N avigating and leading students to locate key course components and make connections that will promote a positive feeling for students. E stablishing and e xplaining relationships to help your students recognize learning relationships in your course. How things “fit” together in your course.
About The Content: Direct participants to the workbook to read the specific review standards: Standard 1.1 of the Quality Matters Rubric states: Instructions make clear how to get started and where to find various course components. This is seen as an essential standard in the QM Rubric. A word about Standard 1.1: As you’ll note in the annotation, there are many ways to meet this standard. Instructors often prefer using a Start Here button since it’s so clear. As you review a course (or apply standard 1.1 to your own course), keep in mind that just having a Start Here button is not enough. You also need to evaluate it: does it provide clear directions for how students are to get started in this course? How To Present It: Apply standard 1.1 to the ANT 103 course (this is a small group activity): 1. Login to ANT 103 To login to Dr. Habermacher's Anthropology course, do the following: * URL: http://pgcconline.blackboard.com * Enter your username: fipse * Enter your password: fipse * Click "Login" * On the next screen, choose "ANT103 Intro Cultural Anthropology" as that is the course you will use to complete this activity. 2. Imagine that you are a new student (new to the course, the discipline, and/or the course management system) 3. Answer the questions posed on the slide.
About The Content: Direct participants to look at: 5 Specific Review Standards support the general standard: 5 Essential Standards Introducing General Standard 2 General Standard 2 is the foundation of the QM rubric and the key standard for alignment; it is the only general standard in which all specific review standards are essential. It acknowledges the use and importance of course learning objectives and learning objectives at the module/unit level of a course. The learning objectives for the course set the standards and conditions for student learning and are critical to a successful outcome. The learning objectives are also the first element of "alignment," an instructional design concept that ensures that the core elements of a course reinforce each other and work together to promote student learning.
About the Content: In a succinct sentence, Sasha Thackaberry from Cuyahoga Community College explains all of Standard 2.
About The Content: This language is taken directly from standards 2.1 and 2.2 – refer participants to the QM workbook so they will be able to find this information after the session.
About The Content: This definition may be new to some participants and review for others. Refer participants to standards 2.1 and 2.2 for more information.
About The Content: Many participants in this workshop may not be familiar with the anatomy of a measurable learning objective. Take this opportunity to present (or review) what a measurable learning objective looks like.
About The Content: As noted in Standard 2.5, taxonomies "describe levels of learning can be helpful in categorizing learning objectives by level" and are very helpful in constructing and analyzing your learning objectives. In the mid 1950s, Benjamin Bloom and a group of educational psychologists were tasked with classifying levels of intellectual behaviors that were integral to the learning process. Bloom is credited with the development of a hierarchical representation designed to categorize cognitive functioning into distinct levels. Bloom's Taxonomy of Intellectual Behaviors is considered to be one of the most important pedagogical endeavors in the history of education. According to Bloom, learners demonstrate mastery of concepts through the six categories of cognitive behavior listed and described below: Knowledge: Ability to recall learned concepts. Comprehension: Ability to understand the concepts in order to be able to explain concepts. Application: Ability to use material that has been learned in new and defined situations and problems Analysis: Ability to break material down into its component parts in order to compare/contrast, categorize, or differentiate between concepts and materials. Synthesis: Ability to combine parts to form a new whole to be able to formulate, design, or devise new materials. Evaluation: Ability to evaluate or judge the appropriateness or worth of concepts against stated criteria. An initiative in the 1990s updated the taxonomy. Bloom's six major categories were changed from noun to verb forms, and some of the categories were renamed. For example, knowledge --the lowest level of the original taxonomy—was changed to remembering .
About The Content: University of Victoria Counseling Services - an excellent overview of Bloom's Taxonomy written from the student's perspective that provides a list of demonstrated skills and question cues for each level. The site suggests that students who know more about the types of questions that are likely to be asked can better prepare for exams. http://www.coun.uvic.ca/learning/exams/blooms-taxonomy.html How To Present It: Ask participants to take a look at Bloom’s taxonomy and decide whether or not the ANT course level objectives are written at a fairly low level or a fairly high level. This will be useful information as we progress through the essential standards.
***** Handout: Analyze Course and Module-level Learning Objectives ***** How To Present It: Participants should work with a partner to complete the table. Take the time to relate the Standard 2 specific review standards to the worksheet…at the end of this exercise, participants should be familiar with the content of these specific review standards.
How To Present It: Go through each learning objective and ask participants if it is measurable and/or precise? And if they have suggestions for how to improve the learning objective? Answers: Realize the significance of ethical behavior in the business environment. (Not measurable; an instructor cannot observe “realize.”) Document the critical events leading to the beginning of World War I. (Measurable; precise) Identify the seven most serious risks associated with cigarette smoking. (Measurable; precise) Understand the continuing impact of World War II on the European Union. (Not measurable; an instructor cannot observe “understanding.”) Demonstrate the effects of improper blood handling in a critical care environment. (Measurable; not precise since it’s unlikely the instructor really wants the risk associated with asking students to “demonstrate” improper blood handling) Describe how to create a financial statement using MS Excel. (Measurable. Precise if the instructor wants a written or oral description. Not precise if the instructor wants the students to actually “create” and submit an Excel table)
About The Content: Direct participants to look at: 6 Specific Review Standards support the general standard: 2 Essential Standards 2 Very Important Standards 2 Important Standards Introducing General Standard 4 General Standard 4 prompts you to take a close look at the course’s instructional materials. In the early days of online teaching (way back in the '90s), many faculty thought that building an online course simply involved selecting a good textbook and/or writing out their lectures and posting them within the online class. As our understanding of online learning has evolved, it's become clear that this approach in itself is not sufficient to engage our online students and allow them to meet the learning objectives. Instructors must choose appropriate materials from a wide variety of options ranging from adopting a standard textbook to providing a list of resources from the campus’ online library to selecting learning objects from MERLOT (see the QM Resources List) to having students create their own materials (most likely in a graduate course. Faculty may be confused or over whelmed by the many choices available to them…hence, the importance of using the stated learning objectives as the basis for selecting appropriate instructional resources. General Standard 4 also considers the depth and breadth of your materials, prompts you to appropriately cite them and ensure that students can distinguish required materials from those that are optional.
How To Present It: Login to the ANT course and apply Standard 4.1. Locate the course level objectives and make a list of the required and optional course materials In your opinion, does the ANT course meet Standard 4.1? What suggestions for improvement would you give to the ANT instructor? Use the table in this slide to help you write a recommendation (participants can use their own paper) to this instructor that cites evidence from the course and appropriate language from the standard and its annotation to support your decision. Make sure that your comment meets the criteria for a helpful comment (constructive, specific, measurable, sensitive and balanced).
About The Content: Direct participants to look at: 4 Specific Review Standards support the general standard: 3 Essential Standards 1 Very Important Standard Introducing General Standard 5 In more traditional face-to-face classrooms, instructors often planned courses around four main elements: lectures, readings, exams and a written paper or project. Students interacted with the instructor and each other by asking questions or participating in discussions within the time allotted in the classroom. Students with more initiative could visit their professors during office hours or perhaps arrange to meet fellow students in a study group. While this approach might work in a f2f classroom on a college campus, it cannot succeed in the online environment. Students need consistent guidance and support from their instructors and meaningful opportunities to engage with the content and their fellow students. Engage the learner with the course content by providing your students with the opportunity to "practice" their learning as preparation for meeting the stated learning objectives. This concept of "practice learning" is particularly important in an online course. You cannot expect your students to simply read the text and then be successful on your assessments. It's your responsibility as the instructor to design opportunities for the students to practice their learning at the appropriate levels to ensure that they can meet your learning objectives.. Interaction between the instructor and students, among students, and between students and course materials is employed to motivate students and foster intellectual commitment and personal development. And it’s critical that instructors also be “present” and engaged in the online classroom…and that the instructor clearly tells his students how and when he will communicate with them, participate in discussions, and provide feedback. Assessments include all activities, assignments, exams, discussions, etc, for which feedback is provided and/or grades assigned.
***** Handout: The Verb Wheel ***** How To Present It: Use the Verb Wheel (provided as a handout), to identify at least three activities that would be appropriate for this objective (try to be creative – after all, you don’t have to actually devise a rubric for and/or grade the activities): A word about Standard 5.1: Specific Review Standard 5.1 is part of the "alignment" concept embedded in the QM Rubric. To meet this standard, you must align your learning activities with your stated course and module level objectives. You learned about alignment in General Standards 2, 3, and 4 (General Standard 6 follows); Standard 5 is a continuation of this concept. As you apply this standard, look for matches and mis-matches between the stated learning objectives and the learning activities (examples in the annotation help to make this clear). If you are reviewing a course in a discipline that is not your own, you may wish to consult your review team’s Subject Matter Expert (SME) for guidance. Don’t be afraid to rely on your own expertise and experience as an online instructor. If the objective and activities don’t seem to match, record the specific example on the rubric web form and then talk with your team about your concerns.
How To Present It: Login to the ANT 103 course and look for evidence of the three types of interaction. Ask participants to record specific course examples in the chart (they can use their own paper) and comment on whether these seem appropriate to this course and meet standard 5.2. What suggestions for improvement do they have for the ANT instructor? A word about Standard 5.2: Specific Review Standard 5.2 is an essential standard that ensures that the course design encourages students to practice their learning through appropriate activities. Please note that this standard does not require that all three types of interaction be present in a single course. Reviewers must consider the nature of the course, the information provided by the instructor through the Instructor Worksheet, and, of course, the stated learning objectives. For example, in a Business Communication course, you would expect to see all three types of interaction; the nature of this course demands significant written, electronic and oral communication between the instructor and students. In a Statistics course, the instructor might prefer to emphasize student-instructor and student-content and minimize student-student interaction for fear that students might confuse each other by writing misinformation.
About The Content: Direct participants to look at: 5 Specific Review Standards support the general standard: 3 Essential Standards 2 Very Important Standards Introducing General Standard 3 Quality Matters stresses the importance of "measurable" objectives so that these learning objectives can guide instructors to develop assessments that accurately assess student accomplishment. For example, a learning objective that states students will "understand modern art" cannot be measured as there is no way to adequately define and then assess a student's understanding. However, a learning objective that states students will "describe the five characteristics of paintings classed as modern art" is something that can be measured through a variety of assessment techniques. This is a simplistic example, but it illustrates what QM is looking for. The terms "assessment" and "measurement" used in General Standard 3 mean more than just quizzes and exams. "Assessment" refers to any student activity for which the instructor provides feedback that guides the student to improved learning. These assessment activities might range from a brief, automatically-graded self-check to a fully annotated paper to the final essay exam…but each is important because it helps the instructor form an accurate, current picture of student learning and provides crucial, regular feedback to help students assess their own progress.
How To Present It: Login to the ANT course and apply Standard 3.1 to the assessments (be sure to read the annotation carefully). Locate the course level objectives and make a list of the types of assessments used in the course. Does the ANT course meet Standard 3.1? Use the table as a model to help you write a recommendation to this instructor that cites evidence from the course and appropriate language from the standard and its annotation to support your decision (participants can use own paper). Make sure that your comment meets the criteria for a helpful comment (constructive, specific, measurable, sensitive and balanced).
About The Content: Direct participants to look at: 5 Specific Review Standards support the general standard: 3 Essential Standards 1 Very Important Standard 1 Important Standard Introducing General Standard 6 General Standard 6: Course Technology deals with the use of instructional media and the endless number of technology tools that are well-suited to online delivery and may be used in online and blended courses. It seems like every week there's a new technology on the market, particularly in the social media area, and it is a challenge to stay abreast of these technological advancements and how they can enhance student learning. The specific function of technology in an online course is to support your learning objectives. In addition, General Standard 6: Course Technology focuses on navigation and the ease of finding elements within the course. The use of tools and media is not required for online or blended courses to meet Quality Matters standards. Many exemplary courses consist of text-only study guides or web pages. Instructors and course designers may not have the equipment, skills, or institution support to develop advanced course components. Online students may still use dial-up internet service; these students will not have the connectivity to receive media requiring significant bandwidth. General Standard 6: Course Technology seeks to assure that those tools that are used have a clear purpose, support the learning objectives and are accessible to students who enroll. Assessments include all activities, assignments, exams, discussions, etc, for which feedback is provided and/or grades assigned.
About The Content: Many participants are confused about the distinction between “tools” and “media.” “Tools” are functional software and contribute to interaction and engagement with the content, with other students and with the instructor. “Media” are technologies used to deliver content via one-way communication (students “receive” the content). Examples might include a course-related documentary video, a podcast of a lecture, a You-Tube video (the Professor Dancealot activity used in slide 84 is an example of “media”). Take the time to review the annotation for standard 6.1. Point out the definition in the actual QM Workbook since you want participants to know where to look for this information after the APP session; don’t just speak from the slide.
About The Content: Activity: Login to the ANT 103 course and look for evidence of the use of technology. Record your specific course examples (at least three different types of examples) in the chart below and comment on whether these meet standard 6.1 (include both strengths and weaknesses). A word about Standard 6.1: Specific Review Standard 6.1 is part of the "alignment" concept embedded in the QM Rubric. To meet this standard, selected technologies must align with the stated course and module level objectives. Standard 6 is a continuation of the alignment concept which includes General Standards 2, 3, 4, 5 and 6. To apply this standard, look for matches and mis-matches between the stated learning objectives and the chosen tools and media (examples in the annotation help to make this clear). If you are reviewing a course in a discipline that is not your own, you may wish to consult your review team’s Subject Matter Expert (SME) for guidance. For example, it’s common practice to use lab simulations in a Chemistry course; if you are reviewing a Chemistry course that uses only a textbook, your SME is likely to raise this issue. Don’t be afraid to rely on your own expertise and experience as an online instructor. If the objectives and selected technologies don’t seem to match, record the specific example on the rubric web form and then talk with your team about your concerns.
About The Content: Direct participants to look at: 4 Specific Review Standards support the general standard: 2 Essential Standards 1 Very Important Standard Important Standard Introducing General Standard 7 General Standard 7: Learner Support deals with the importance of providing our online learners with comparable services to those found on campus. In the very early days of online courses, institutions invested in a course management system and technical support (and almost always online registration) but often had difficulty re-tooling other support services. It took much longer for most institutions to recognize that their online students also needed academic support (testing, tutoring, library) and student support services (advising and counseling, in particular). And, of course, for those online students who need special services, it’s critical to offer disabilities support services. Since providing these services is an institutional responsibility, rather than an individual faculty member responsibility, General Standard 7 is designed to ensure that instructors are not penalized if their institutions don’t offer a full array of online services. Note that each of the four specific review standards require links to institutional services, making the assumption that offering these services is not the responsibility of a single instructor within a specific course. Clearly, however, it is the instructor’s responsibility to direct his students to those services which are available and critical to a particular course. For example, in an online History class, you would expect to find significant use made of online library services and resources and, perhaps, links to the Online Writing Lab for help with preparing papers.
About The Content: Direct participants to look at: 4 Specific Review Standards support the general standard: 1 Essential Standard Very Important Standards Introducing General Standard 8 General Standard 8: Accessibility deals with the importance of designing online courses that are accessible for all students, including those with special needs such as the visual or aural disabilities. Institutional support for accessibility varies widely among institutions. Some institutions are committed to accessibility issues and provide both training and design support for their faculty. Other institutions are more minimal in their approach to accessibility and offer little guidance or support for faculty. The Quality Matters rubric acknowledges this disparity by including only one essential standard (8.1) that requires faculty to assess accessibility in their own courses and provide directions for how their students should access it.
About The Content: Standard 8 references accessibility issues and cites these two websites as references for accessibility guidelines (“ The accessibility standard incorporates the principles of Universal Design for Learning (UDL) and is consistent with Web Content Accessibility Guidelines (WCAG)”. About How To Present It: Break participants into small groups and ask them to explore both sites. You can use the following prompts to spur discussion: How is the website organized? Is it easy to find information? Is the website written in technical terms or for the non-technical user? What was the most helpful idea or suggestion your group found? What other questions do you have about using this website or applying Standard 8?
A word about Standard 8.1: As you’ll note, Standard 8.1 is the only essential standard included in General Standard 8. This standard requires instructors to review the technologies used in their courses and document their accessibility for students with disabilities. Instructors who are just designing courses or adding technology components are in an enviable position: since there are so many technology choices, it’s best to carefully select technologies that are accessible and/or also select a comparable alternative. Note that this standard has two conditions that must be met: Condition 1. The course must be offered in a course management system that is accessible. If you look at item 9 on the ANT 103 Instructor Worksheet, you will see a statement and link documenting that Blackboard is accessible to students with special needs. In this scenario, refer the participants to the annotations for Standard 8.1. What would reviewers look for when making the determination as to if the standard is met? They would refer to the Instructor Worksheet to look for a statement about Desire2Learn's accessibility, or a link from the course to such a statement. Condition 2. If the instructor requires students to use any additional technology tools or software, he/she must list these and document their accessibility. If any tools or software are not accessible, the instructor must indicate how to get the necessary accommodation. For example, if the instructor chooses to include a video, he/she should look to see if it is accessible (includes closed captioning and/or other visual descriptors). The instructor must provide documentation regarding whether or not the videos are accessible. If they are not, then the instructor must provide instructions on how to obtain accommodation.
How To Present It: Now that you’ve thought about each General Standard and how alignment works, perhaps you can help a colleague understand how to use the concepts. Watch this brief You Tube video (You Tube: http://www.youtube.com/watch?v=1k8aeDUC9XQ) in which Professor Dancealot attempts to teach his students how to dance. Use your understanding of alignment to convince the Professor that he needs to “practice alignment” if he wants to help his students meet the stated learning objectives of this course. Remind participants of the elements of a helpful recommendation (cite the QM standard and its annotation and provide evidence from the course to support your decision) AND the five characteristics of a helpful recommendation (constructive, specific, balanced, measurable, sensitive). You may want to put these characteristics on the screen as reminders. Advance to the next slide for a suggested approach to this assignment.
*****Handout: Print this handout prior to the session. ***** How To Present It: Since this is a workshop, we suggest you do this as a small group activity (2 or 3 participants for each group). Ask participants to use this handout to record their thoughts on the advice they will give to Professor Dancealot. Remind them of the elements of a helpful recommendation (cite the QM standard and its annotation and provide evidence from the course to support your decision) AND the five characteristics of a helpful recommendation (constructive, specific, balanced, measurable, sensitive). You may want to put these characteristics on the screen as reminders.
About The Content: As a trainer, please encourage your participants to “self-select” into the PR training. If they don’t meet the eligibility requirements AND/OR they don’t plan to actually serve on a QM review, there’s no real value in completing the assessments now. If eligibility or desire changes, they can always enroll in the online Peer Reviewer Course for a fee. The current procedures for registering and accessing the online Peer Reviewer Course are available at the QM web site (www.qmprogram.org).
How To Present It: Remind participants to visit MyQM to print their certificates. If time permits and the printers work, you can do this before they leave the session. This is the ending slide of the presentation and can be left up while participants are exiting the room.
In early iterations of the Peer Reviewer process, many participants focused on HOW to do the review, but missed the WHY. This slide emphasizes the key concepts behind QM and how they contribute to a quality course. It’s not necessary to read these aloud in order… Briefly talk about each item that you may not have raised in earlier slides.
Emphasize the first bullet: If the QM process is followed, ALL courses will meet expectations. This is an important concept that new Peer Reviewers often miss.
Be sure to point out that there a 3 faculty peer reviewers who actually complete the web-based review form. One of these reviewers must be a Subject Matter Expert and at least one must be external to the course’s home institution. Emphasize that the Faculty Course Developer (the Instructor) is a part of the review team and should be consulted by the team. You might want to ask participants if any of them are Faculty Course Developers…since the Faculty Course Developer is always invited to attend training prior to the review.
There are TWO thresholds for meeting expectations and a course must meet both thresholds to meet expectations. Mathematically, this works out to about an 85% standard.
The 11-13 version of the rubric includes the necessary elements so that it can be used to review a hybrid course. A course can be reviewed using the blended rubric if it meets the following criteria: The course is organized around a Learning Management System and the major course elements (including what will be covered at the f2f meetings) are available through the LMS .
About The Content: Emphasize that reviewers should assume the STUDENT’s point of view. This is an opportunity to see the course from the student’s point of view (not the faculty member’s). This point of view often reveals aspects of the course that could be improved to make navigation easier and improve the learning environment for the students...aspects that may not be obvious to the faculty developer because he/she is just too close to the course. Encourage participants to support their decisions with citations from the rubric AND evidence from the course. If they can’t find the evidence they need, they should not assume it is or isn’t there. If the review team needs help, they should consult with the instructor to determine if the required evidence is somewhere in the course. This might be a good place to ask if any of today’s participants have been students in an online course. Ask them if being a student changed the way they think about online teaching.
Marker: You may want to add your own contact information so participants can contact you if they have questions.
Applying the QM Rubric 2011-13 Rubric
After this workshop, you will be able to:• Identify the underlying principles of QM• Use the QM Rubric• Apply standards and make decisions• Access QM Website www.qmprogram.org• Login to MyQM
Underlying Principles of QMQM is a faculty-driven, peer review process that is… Collaborative Collegial Continuous Centered - in academic foundation - around student learning
For Our Purposes, Quality Is…• More than average; more than “good enough”• Attempt to capture what’s expected in an effective online course at about an 85% level• Based on research and widely accepted standards
Factors Affecting Course Quality QM Reviews Course Design ONLY
Design vs. Delivery The faculty member is integral to both design and delivery.Course Design … Course Delivery …is the forethought and is the actual teaching of theplanning that a faculty course, the implementation ofmember puts into the course. the design.QM is about DESIGN - not delivery or faculty performance
Distinguish between design and delivery… Example: Discussion BoardDesign: Delivery:Discussion board How often the facultyplanned in course; member actuallystudents told how they participates in theshould participate and discussion; what thehow they can expect the faculty member actuallyfaculty to participate. says to students.
About The QM Rubric 2011-2013Eight General Standards: 1. Course Overview and Introduction 2. Learning Objectives (Competencies) 3. Assessment and Measurement 4. Instructional Materials Key components must align. 5. Learner Interaction & Engagement 6. Course Technology 7. Learner Support 8. Accessibility Alignment: Critical course elements work together to ensure that students achieve the desired learning outcomes.
Course Learning Objectives (2) Learner Interactions & Activities (5) Resources, Materials (4) & Technology (6) Assessment and Measurement (3)Key Sections that Must Align
What QM is NOT About…• Not about an individual instructor (it’s about the course)• Not about faculty evaluation (it’s about course quality)• Not about judgment (it’s about diagnosis and improvement)• Not about “win/lose” or “pass/fail” (it’s about continuous improvement in a supportive environment)
QM = Process and Rubric Process RubricOFFICIAL Outcome: Earn QM recognition Outcome: Earn QM recognition Must follow official QM guidelines Tool to assess online courses during and procedures formal QM review Outcome: Improve courses, Outcome: Improve courses, meet meet institutional goals institutional goals, demonstrate commitment to quality Guide to develop new online courses AND review and update online coursesINFORMAL Outcome: Improve courses, meet Outcome: Improve courses, meet institutional goals, demonstrate institutional goals, demonstrate commitment to quality commitment to quality Institutions determine use and Guide to develop new online procedures courses AND review and update online courses
About the Course• QM is designed to review “mature” courses (taught at least two semesters)• QM logo indicates year course met expectations• Triggers for subsequent reviews: – Faculty request – More than 3 years since original review – New textbook or instructor – Professional or accreditation review pending
The Peer Review Team• 3 faculty peer reviewers: • must be experienced online instructors • must attend QM training • one MUST be external to the course’s originating institution • there must be a subject matter expert (SME) on the team. NOTE: The SME could also be the external reviewer. AND• Faculty course developer: • access to rubric prior to review • involved in pre-review discussions • consulted during review
About the Review• On average, a course review takes 7-10 hours• Factors affecting review time include • Reviewer familiarity with the discipline • Reviewer familiarity with the LMS • Reviewer familiarity with the QM review process • Organization of the course
Review Outcome• If meets expectations and is recognized: – Recognized by Quality Matters – Notifications distributed – ID support provided if requested and available – Recognized courses on QM website• If does not yet meet expectations: – Instructor (and/or ID) make changes – Team Chair/Master Reviewer approve revisions – Course meets expectations and is recognized
Rubric Scoring# Standards Points Relative Value Total Points 21 3 Essential 63 12 2 Very Important 24 8 1 Important 8 41 95
In an Official QM Review• Team of 3 reviewers initially score individually• One score per standard based on team majority• Pre-assigned point value• Met/Not Met decision; All/None points• Consensus is NOT required
Points are NOT on assigned on a sliding scale…Standards If the standard If the standard is met … is not met …Essential 3 points 0 pointsVery 2 points 0 pointsImportantImportant 1 point 0 points
To Meet Expectations…Two ThresholdsA course must achieve: “Met” on all 21 of the 3-point “essential” standards.1. A minimum of 81 out of 95 points (81/95 = 85%)2.
The 85% RuleTwo 85% Uses:A minimum of 81 out of 95 points(81/95 = 85%)Reviewers use their own gauge of85% as to if a standard is met or not.Standards do not have to be 100% tobe marked “Met.”
Key Sections That Must Align Course Learning Objectives (2) Learner Interactions & Activities (5) Resources, Materials (4) & Technology (6) Assessment and Measurement (3)
Think About “Alignment”Objectives From Intro Assignment: Read Chapter 4 (assume it deals withPsych relationships between biology and human behavior) in text; review study guide, objective 2, page 4.Course : Summarize Assignment: View Chapter 4 PPT; listen to Chapter 4the relationships that podcastexist between biology Graded Assignment:and human behavior Prepare table that lists the 7 major biological areas; ask students to define/describe each area and list 3 specificModule ways this area affects human behavior.1.Identify and define 7 Graded Discussion Forum Question: Psychology is themajor biological areas. study of behavior and mental processes. Why, then, are2.Recognize examples psychologists interested in biological matters such as theof how each area nervous system, the endocrine system and heredity?affects behavior. Read and respond to classmates’ postings. Exam: 50-item multiple choice exam taken in campus Testing Center.
General Standard 1 About the Brief Description To meet the General Standard from the Rubric General StandardGeneral Standard 1, The course introduction You must share your deepCourse Overview and sets the tone for the knowledge of courseIntroduction: course, tells students what 1.Organization to expect, and provides 2.NavigationThe overall design of the guidance to ensure they get 3.Expectations (Policies)course is made clear to the off to a good start.student at the beginning ofthe course.
Standard 1Answer the following questions:1.Do you know what to do first?2.Do you know how to start the course?3.Note your initial reaction to the course:4.Do you feel comfortable with the navigation?5.What are your first impressions about how this course works?
General Standard 2 About the Brief Description To meet the General Standard from the Rubric General StandardGeneral Standard 2, The learning objectives You must provideLearning Objectives: establish a foundation upon measurable, precise which the rest of the course learning objectives at theLearning objectives are is based. course- and module-levelmeasurable and are stated objectives so yourclearly. students will know what is expected of them.
Standard 2 as a SentenceQuality ismeasurable objectives (2.1)and consistent module-level objectives (2.2),that are written from the student perspective (2.3),with instructions on how to meet them (2.4),that are appropriate for the level of the course (2.5). Submitted by Sasha Thackaberry, Cuyahoga Community College
Standards 2.1 & 2.2Write measurable course and module learningobjectives that•precisely describe what students will gain frominstruction•guide instructors to accurately assess studentaccomplishment.•are consistent
Measurable Learning Objectives• Begin with an action verb• Precisely describe behavior that can be observed or evaluated
Test of “Measurability”A Measurable Learning Objective should completethis sentence:Upon completion of this course/module,students will be able to (DO SOMETHING).Helpful Hundred….
Standard 2: Learning Objectives Bloom In Practice University of Victoria Counseling Serviceshttp://www.coun.uvic.ca/learning/exams/blooms- taxonomy.html
Analyze Learning Objectives• Complete the table (handout): Look at ANT 103 learning objectives, Apply Standards 2.1 through 2.5 Quality is measurable objectives (2.1) and consistent module-level objectives (2.2), that are written from the student perspective (2.3), with instructions on how to meet them (2.4), that are appropriate for the level of the course (2.5).
General Standard 4 About the Brief Description To meet the General Standard from the Rubric General StandardGeneral Standard 4,Instructional Materials: The instructional materials You must provide rich, form the core of the course, robust and appropriateInstructional materials are and these standards respect resources and materialssufficiently comprehensive the instructor’s prerogative that support your statedto achieve stated course in selecting them. The focus learning objectives.objectives and learning of this standard is onoutcomes. supporting the course objectives and competencies, rather than on qualitative judgments about the materials.
Apply Standard 4.1Standard 4.1 Met Not Met Helpful RecommendationThe instructionalmaterialscontribute to theachievement of thestated course andmodule/unitlearning objectives.
General Standard 5 About the Brief Description To meet the General Standard from the Rubric General StandardGeneral Standard 5,Learner Interaction & Engaging students to You must provideEngagement: become active learners meaningful and contributes to the learningForms of interaction process and to student productive activities thatincorporated in the course persistence. prompt your students tomotivate students and actively practice theirpromote learning. learning.
Standard 5.1Learning Objective Possible Aligned Activities 1.Analyze thecauses of theCivil War and 2.how theypersist in our 3.modernsociety.
Standard 5.2Type of InteractionStudent-InstructorStudent-ContentStudent-Student
General Standard 3 About the Brief Description To meet the General Standard from the Rubric General StandardGeneral Standard 3,Assessment and Assessment is implemented Your assessments mustMeasurement: in a manner that not only align with your stated allows the instructor a broad learning objectives andAssessment strategies are perspective on the students’ guide students todesigned to evaluate mastery of the content, but measure their ownstudent progress by also allows students to learning progress.reference to stated learning measure their own learningobjectives; to measure the throughout the course.effectiveness of studentlearning; and to be integralto the learning process.
Apply Standard 3.1 using Learning Objective Standard 3.1 Met Not Met Helpful RecommendationThe types ofassessmentsselected measurethe stated learningobjectives and areconsistent withcourse activitiesand resources.
General Standard 6 About the Brief Description To meet the General Standard from the Rubric General StandardGeneral Standard 6,Course Tools and The technology enabling the You must select the toolsTechnology: various course components and media that best facilitates the student’sCourse navigation and learning experience and is support your learningtechnology support student easy to use, rather than objectives and provideengagement and ensure impeding the student’s opportunities for youraccess to course progress. students to activelycomponents. practice their learning.
Standard 6 Includes…Course Tools –•functional software that provide areas for interaction inthe course•Examples include forums, quizzes, surveys, blogs, socialmedia, wikis, etc.Media –•one-way delivery modalities that enhance learning•Examples include YouTube videos, podcasts, audiorecordings, pictures, graphics, etc.
General Standard 7 About the Brief Description To meet the General Standard from the Rubric General StandardGeneral Standard 7,Learner Support: In the learner support You must direct your standard, four different online students to theThe course facilitates kinds of support services are support services theystudent access to addressed: technical need to be successful andinstitutional support support, accessibility would be available toservices essential to student support, academic services them if they attendedsuccess. support, and student campus-based courses. services support.
General Standard 8 About the Brief Description To meet the General Standard from the Rubric General StandardGeneral Standard 8,Accessibility: The accessibility standard You must design your incorporates the principles course so students withThe course demonstrates a of Universal Design for special needs can accesscommitment to accessibilityfor all students. Learning (UDL) and is course materials and be consistent with Web successful in your course. Content Accessibility Guidelines (WCAG).
Sites to ExploreUniversal Design for Learning WCAGSite provides “a framework for •Web Content•designing educational Accessibility Guidelinesenvironments that enable all (WCAG)learners to gain knowledge, •Site providesskills, and enthusiasm for documents that “explainlearning how to make Web•reducing barriers to the content more accessiblecurriculum to people with•providing rich supports for disabilities.”learning.”
Standard 8.1Consider the following scenario:An online Nursing course is offered through Desire2Learn andrequires that students watch six videos that illustrate keyconcepts in critical care. These videos are provided by thetextbook publisher and are not closed-captioned.If you were reviewing this course, how you would applyStandard 8.1 and what helpful comments would you give theinstructor?
Professor Dancealot Needs Help!• Watch this brief You Tube video (You Tube: http://www.youtube.com/watch?v=1k8aeDUC9XQ)• Apply your understanding of the General Standards and alignment.• Convince the Professor that he needs to “practice alignment” if he wants to help his students meet the stated learning objectives of this course.
Spur Your ThinkingCourse Objectives Standard 3.1Upon completion ofthis course, you willbe able to : Standard 4.1:1. Perform the basicsteps of foxtrot, waltz,swing, chacha, rumbaand tango in proper Standard 5.1:dance position.2. Demonstrateconfidence on the Standard 6.1:dance floor.3. Find your eternaldance partner.
Peer Reviewer EligibilityHow Do I You must be ELIGIBLE:Become a •Must complete this F2F Applying the QM Rubric workshop.QM Peer •Must complete all assessments in theReviewer? online Peer Reviewer Course – Two weeks online – Includes review material – Objective and written assessments •Must have recent (within last 18 months) online teaching experience.
Register for Peer Reviewer Course Sign up for a Peer Review Course at http://qmprogram.orgPrint your Certificate of Completion at MyQM!
Underlying Principles of QM1. Based on national standards of best practice, the research literature and instructional design principles2. The course under review does not have to be “perfect” but better than just “good enough” (Standards met at about 85% level or better.)3. Integral to a continuous quality improvement process4. Valued as part of a faculty-driven, peer review process5. Designed to promote student learning
Underlying Principles of QM cont.6. Designed to ensure all reviewed courses will eventually meet expectations7. QM is a collegial review process, not an evaluation process8. A review team must include an external peer reviewer9. Set up so that the course faculty or instructor considered part of the review team
Who’s On the Team? 3 faculty peer reviewersFormal – Must attend QM Training – Must be experienced online instructorsPeer – One MUST be external to the course’s originating institutionReview – Must include one subject matter expert (SME) on theTeam team NOTE: The SME could also be the external reviewer – Chaired by Master Reviewer AND Faculty course developer (Instructor) – Access to rubric prior to review – Involved in pre-review discussions – Consulted during review
To Meet Expectations: Two ThresholdsA course must achieve: “Yes” on all 21 of the 3-point “essential” standards.1. A minimum of 81 out of 95 points2.
Online & Hybrid Courses• Rubric designed for both• Same set of standards apply to both• How standards achieved may differ• For blended courses, focus on pedagogical integration of online and onsite components
Your Point of View …As a QM Course Reviewer, you should:• Take the students’ point of view• Advocate for the student• Support your decisions with – Citations from the standards and annotations – Evidence from the course
Thanks to YOU…Quality Matters! More Information at: www.qmprogram.org