K-12: Growing Quality Online Courses
•
1 like•100 views
Slides Christine Voelker's presentation at the iNACOL Symposium in 2017. Explore trends in online course reviews using data from Quality Matters Official K-12 Course Reviews. Topics include which areas of course design schools, districts and providers have worked to continuously improve, which areas have shown the most growth, and which areas always meet Standards.
Recommended
Recommended
More Related Content
Similar to K-12: Growing Quality Online Courses
Similar to K-12: Growing Quality Online Courses (20)
Recently uploaded
Recently uploaded (20)
K-12: Growing Quality Online Courses
- 1. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Growing Quality Online Courses Christine Voelker K-12 Program Director, Quality Matters @voelkerc | cvoelker@qualitymatters.org Joining Forces to Create Better Blooms: Findings from the Field #iNACOL17
- 2. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Growing Quality Online Courses Joining Forces to Create Better Blooms: Findings from the Field
- 3. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Growing Quality Online Courses Joining Forces to Create Better Blooms: Findings from the Field
- 4. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc.
- 5. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. https://i.ytimg.com/vi/nGEeApmpXO0/maxresdefault.jpg
- 6. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc.
- 7. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc.
- 8. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. 1. Higher Education Rubric 2. Higher Education Publisher Rubric 3. K-12 Secondary Rubric 4. K-12 Publisher Rubric 5. Continuing & Professional Education Rubric 5 Rubrics for Online/Blended Courses
- 9. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Fourth Edition Revised periodically Supported by research and best-practices The QM K-12 Secondary & Publisher Rubrics
- 10. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. LibraryResearch
- 11. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Membership
- 12. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc.
- 13. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Growing Quality Online Courses Joining Forces to Create Better Blooms: Findings from the Field
- 14. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Course Reviews
- 15. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Course Reviews
- 16. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Course Reviews
- 17. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Course Reviews
- 18. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Course Reviews
- 19. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Course Reviews
- 20. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Course Reviews
- 21. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Course Reviews
- 22. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Think Pair Share K-12 Secondary Course Reviews MET most often NOT MET most often
- 23. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc.
- 24. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. K-12 Secondary Course Reviews MET Most Often NOT MET Most Often Alignment Standards 4.3 6.4 8.4
- 25. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc.
- 26. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. K-12 Secondary Course Reviews MET Most Often NOT MET Most Often 4.5 8.3 1.4 7.3 Learning Objectives 4.3 6.4 8.4
- 27. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Think Pair Share K-12 Publisher Course Reviews MET most often NOT MET most often
- 28. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc.
- 29. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. K-12 Publisher Course Reviews MET Most Often NOT MET Most Often 5.2 C 6.5 T 6.6 T 6.8 T
- 30. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc.
- 31. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. K-12 Publisher Course Reviews MET Most Often NOT MET Most Often 5.2 C 6.5 T 6.6 T 6.8 T 1.3 T 4.5 C 2.5 C 8.3 C
- 32. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. What are we doing well?
- 33. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Schools & Districts Reading Level4.3
- 34. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Schools & Districts Current tools & technology6.4
- 35. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Schools & Districts Facilitating readability & Minimizing distractions 8.4
- 36. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Publishers Opportunities for Learner-Content interaction 5.2 C
- 37. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Publishers Technical supports for grade calculation and gradebook integration 6.5 T
- 38. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Publishers Instructor control over assessment release and sequencing 6.6 T
- 39. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Publishers Instructor control over release of content6.8 T
- 40. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Which areas have shown the most growth?
- 41. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Schools & Districts - Rubric Editions
- 42. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Growth within Organizations over time
- 43. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Growth Within Organizations over time 1. Template Reviews 2. Learning Objectives/Alignment 3. Change in Course Representative
- 44. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Growth during the Review
- 45. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. K-12 Secondary Course Reviews MET Most Often AFTER AMENDMENT Standard % MET Initial After Amend Learning Objectives: 2.1, 2.2, 2.3 1.4 68 41 27 8.3 91 64 27 7.3 68 50 18 Growth during the Review
- 46. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. K-12 Publisher Course Reviews MET Most Often AFTER AMENDMENT Standard % MET Initial After Amend Learning Objectives: 2.1, 2.2, 2.3 1.3 T 100 50 50 5.3 T 100 75 25 7.3 T 100 75 25 Growth during the Review
- 47. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Where are we continuing to improve?
- 48. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Schools/Districts Etiquette expectations1.4
- 49. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Schools & Districts Academic support services7.3
- 50. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Publishers Student technical skills1.3
- 51. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Publishers Instructor support resources7.3
- 52. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Schools/Districts & Publishers Learning Objectives 2.1 2.2 2.3
- 53. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Schools/Districts & Publishers Citations4.5
- 54. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Schools/Districts & Publishers Accessibility8.3
- 55. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. What is the impact?
- 56. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. 6,500+ What is the impact?
- 57. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. 94% What is the impact? Submitting Organizations Course Reviewers 81%
- 58. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. 80% 65% What is the impact? Submitting Organizations Course Reviewers
- 59. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. What is the impact?
- 60. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. What is the impact? student by Wilson Joseph from the Noun Project education laptop by Rockicon from the Noun Project
- 61. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc.
- 62. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. How can you help?
- 63. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. How can you help?
- 64. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. How can you help?
- 65. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. How can you help? Image: Dinosoft Labs from Noun Project
- 66. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc.
- 67. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc.
- 68. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Growing Quality Online Courses Joining Forces to Create Better Blooms: Findings from the Field
- 69. Helping you deliver on your online promise qualitymatters.org ©2017 MarylandOnline, Inc. Questions? Christine Voelker K-12 Program Director, Quality Matters @voelkerc | cvoelker@qualitymatters.org
Editor's Notes
- Good afternoon! Thank you so much for coming to my session. My name is Christine Voelker, and I am the K-12 Program Director for Quality Matters. You know, we were just at lunch, and seeing those beautiful bouquets of flowers on the tables reminded me of an old Kabbalah story that I had once heard. As a former teacher librarian, I have heard and told a lot of stories!
- The story is called “The Best Flower in the Garden.” It’s about different flowers all growing in a garden. It takes place as summer is coming to an end, and the flowers are starting to wonder which of them the garden visitors thought was the best. The small yellow flowers said, “We are the best flowers, because we are the first to bloom in the spring.” The purple ones said, “Oh no, we are the best because we have pretty flowers all summer long.” The big red flowers said, “Don’t be silly, we are the best because we are the tallest.” And it went on like that throughout the summer with each flower arguing that they were the best. When people would come to the garden, they would all quiet down and look proud so that the visitors would think they were the best. One day the gardener came into the garden. The small yellow flowers fluffed their petals so that they would look the best. All of the purple flowers stood up tall so that they would look the best. The red flowers beamed in the sunlight so that they would look the best. All of the flowers were sure that the gardener would say they were the best. But the gardener just smiled and said, “look at all of my pretty flowers.” We’ll come back to the little garden in a bit, but this story reminds me of the courses that are growing in our collective online learning garden.
- Whether those courses are developed by a vendor or a local school district, they all stand proud. And they have good reason to do so. Designing an online course is no easy task, as may of you out there know. There are a lot of components that need to go into creating a good bloom. However, not everyone has a green thumb when it comes to online course design.
- Ruth in the garden with the kids. This is my mother-in-law. She has a green thumb. She was also a pre-school teacher for over 30 years. Ruth would tell you that students deserve to be around beautiful blooms. It’s why she takes the time to share her love of gardening with her grandchildren. What Ruth may not tell you is that she actually has a secret weapon. She swears by a product called Miracle Grow and feeds it regularly to her plants. Her other secret weapon is homemade apple sauce, but that’s a story for another day.
- Magic Elixir for online courses. I’d like to say that there is a magical elixir for growing perfectly designed online courses, but there isn’t one. Our students have differing needs, our states have different curriculum, and our schools and districts have differing processes and procedures when it comes to online courses. However, there are some basic guidelines and standards sets that we can follow when designing online courses so that our students can have an exceptional learning experience.
- Many of you here in this room know that iNACOL has their National Standards for Quality Online Courses.
- QM also has Standards for Course Design. Honestly, as long as you find one standard set that works for you- you are making a difference!
- But today, I am going to focus on the Quality Matters Rubric Standards, of which there are actually 5 sets that you can see on your screen. Higher Education Rubric Higher Education Publisher Rubric K-12 Secondary Rubric K-12 Publisher Rubric Continuing & Professional Education Rubric, which is helpful if you are creating online PD offerings
- The findings that I am going to share today come from the QM K-12 Secondary and Publisher Rubrics. The K-12 Secondary Standards are used for schools and districts who are developing courses locally, while the Publisher Rubric is used for courses that are developed by vendors or districts for use in other schools or districts. These are dynamic documents which changes as our collective understanding of online and blended learning grows. The Rubrics represents the current research regarding online and blended learning, the current thinking of the QM management team, the Rubric Revision Committee- whose members recommended the new changes, and the many teachers and instructional designers who have provided insight and feedback. We revise the Rubric roughly every three years, with a team of 12 people from organizations throughout the country dedicating their time to help. In doing the Rubric revision, we first conduct a comprehensive literature review, which is used to inform the committee as they decide to recommend changes, and then look at current best practices. This last Rubric Revision Culminated in the fourth editions of the K-12 Rubrics. The QM Rubrics are actually aligned with the iNACOL National Standards for Course Design, and we provide a derived iNACOL Standards report for organizations that can be generated once a QM review is complete.
- Over 1,000 studies have been Sorted, Sifted, Scrutinized. Solid research is the foundation – and catalyst for improvement – of the QM Quality Assurance System. [Click to bring in Research Library images} We have a research library on our website which allows you to search by standard to see what research is out there to support each one.
- We are a small, but mighty non-profit empowering educators to make a difference in online learning. And we are mighty because of our community. We have over 1,000 member organizations from all over the world- and all of our members have heard of us through word of mouth, folks presenting on their work at conferences, or just sharing their quality assurance processes with others. It’s because of our membership that I am able to share these findings today.
- So, if you aren’t familiar with Quality Matters, our mission is to promote and improve the quality of online education and student learning nationally and internationally through: Development of current, research-supported, and practice-based quality standards and appropriate evaluation tools and procedures. Recognition of expertise in online education quality assurance and evaluation. Fostering a culture of continuous improvement by integrating QM standards and processes into organizational plans to improve the quality of online education. Providing professional development in the use of rubrics, tools and practices to improve the quality of online education. Peer review and certification of quality in online education.
- The data from “peer review and certification of quality in online education,” is what we will be taking a look at today. So, we'll examine trends from three years’ worth of Quality Matters K-12 Official Course Reviews. Looking at data from both home-grown district developed courses and third-party courses, we'll see which areas of course design schools, districts, and providers have worked to continuously improve, which areas have shown the most growth, and which areas most always meet standards.
- So how does the QM course review process work? While any member of our community can practice quality assurance by using our tools, like our self-review tool, it is our official course reviews that lead to QM-Certification. [Click to bring in the picture of the mark.] These types of reviews are available to anyone, member of not, looking for feedback, validation, or just another way to market their online courses.
- So, I’ll tell you a little about the Course Review process, and then we’ll dive into the data.
- So what happens during a QM Course Review? Each Review team is made up of 2 or 3 QM-Certified Course Reviewers depending on the type of the review. They must be experienced online instructors or course developers who become officially QM-Certified by attending QM PD. 2 Subject Matter Experts (SME) and 1 Master Reviewer who has completed additional PD. This person serves as the chair and guides the team. [CLICK to bring in last flower] AND the Course Representative who has access to the Rubric prior to review, is involved in pre-review discussions, and is consulted during the review
- During the Review, each member of the team, except of course, The Course Representative fills in their form individually. Ultimately the results get compiled and the course receives one score per standard based on team majority. Each specific review standard has a pre-assigned point value. A standard gets either all of none of the points based on whether or not it was determined to be Met or NOT MET by the review team.
- Each reviewer looks for evidence in the course that a particular standard is MET at the 85% level. They know what to look for, because each Standard is fully annotated with detailed explanations, look-fors and best practices.
- Each reviewer writes helpful feedback based on their decision which culminates in a final report.
- Reviewers learn how to write a helpful recommendation in their PD experiences- and the Master Reviewer on each team is tasked with making sure this is done.
- The feedback written is specific and measurable. A teacher, school, district, or vendor can tell what they are doing well, but also where they need to make improvements. If they have NOT MET a certain standard, they can then go back and make the changes. All essential standards must be MET in order for a course to meet standards overall. If a course DOES NOT meet standards, it goes into an amendment process – a time when those changes can be made. Remember, this is a continuous improvement process, and we believe that all courses can eventually MEET standards. K-12 Secondary: 87/102 pts. + all essential standards. K-12 Publisher: 90/105 pts. + all essential standards.
- Think, Pair, Share. You each have received a handout that lists each of the Standards from the K-12 Secondary Rubric. Take a look at those Standards, and think about which ones you think may be harder for individual teachers, schools and districts to meet than others. Discuss with a partner. On a separate sheet of paper, or your sticky note, together list the 3 Standards you feel are met most often in Official Course Reviews, and which ones you think schools and districts struggle with.
- Which QM K-12 SECONDARY Standard do you think was MET most often? https://www.polleverywhere.com/free_text_polls/AWivAC3CFUzavEU
- So, for K-12 Secondary Course Reviews, remember these are courses developed by individual teachers, schools and districts, what we find is that the alignment Standards are MET most often, however, there is a reason for this. Since the Learning Objectives are an integral piece in connecting the alignment standards, and the Learning Objectives are the foundation upon which each course is built, a course review cannot be continued or completed if those alignment standards are NOT MET. Therefore, we always find them to be MET, even though there may be revisions needed before they are determined to be MET. Standard 4.3 is one that is almost always determined to be MET. The course content is appropriate to the reading level of the intended students. 6.4- Course tools and technologies are current, and 8.4- The course design facilitates readability. How many of you guess right? ------------- Top 5 MET: Alignment Standards Fourth Edition: 1.5, 4.3, 4.7, 5.3, 6.4, 6.5, 8.4 Second Edition: 4.3, 6.2, 6.3 (8.1), 6.5 (6.4), 8.3 (8.4) Better over time: 1.2, 3.4, 4.2, 4.4, 4.5,
- Which QM K-12 SECONDARY Standard do you think was NOT MET most often? https://www.polleverywhere.com/free_text_polls/UN48hPeruUHU2cU
- So, which standards are most often NOT MET in K-12 Secondary Course Reviews? 8.3 – Accessibility 1.4 – Netiquette or behavior expectations 7.3 – Academic Support Services And 4.5 – Citations. How many of you guessed right? ------------ Top 5 NOT MET: 8.3* 1.4 7.3 4.5 2.5*
- Now let’s consider the K-12 Publisher Standards. You each have received a handout that lists each of the Standards from the K-12 Publisher Rubric. Take a look at those Standards, and think about which ones you think may be harder for vendors, and schools and districts developing course to market to other districts to meet than others. Discuss with a partner. On a separate sheet of paper, or your sticky note, together list the 3 Standards you feel are met most often in Official Course Reviews, and which ones you think schools and districts struggle with.
- Which QM K-12 PUBLISHER Standard to you think was MET most often? https://www.polleverywhere.com/free_text_polls/uzTO5r4vQHsIB4H
- So, for K-12 Publisher Course Reviews, remember these are courses developed by vendors or schools and districts looking to have their course adopted by others, what we find is that in addition to the alignment Standards being MET most often, Standard 5.2 is one that is almost always determined to be MET. Learning activities provide opportunities for learner-content interaction. 6.5- student privacy and confidentiality of student information 6.6- selecting and sequencing assessments. And 6.8 – controlling release of content. How many of you guess right? ------------- Top 5 MET: Alignment Standards 2.4 C Consider removing since it is contingent upon LOs 5.2 C 6.5 T 6.6 T 6.8 T Alignment Standards over time…. & % they need to make amendments. Learning Objectives: Halt Review. Top 5 NOT MET: 1.3 T 50% 4.5 C 31% 2.5 C 28% 8.3 C 24% 3.2 C 18% Another one: 1.3 T. The thing about our template standards is that because they populate into the content standards in order to give a subsequent score, we give organizations an opportunity to get these all MET before we close a Template Review.
- Which QM K-12 PUBLISHER Standard do you think was NOT MET most often? https://www.polleverywhere.com/free_text_polls/fpiaimeK8MPORla
- Standards that are routinely determined to be NOT MET include: 1.3 T – Listing student technical skills 4.5 – Citing sources 2.5 – Showing the relationship between the learning objectives or competencies and course activities, AND 8.3 Accessibility How did you do? --------------- Top 5 MET: Alignment Standards 2.4 C 5.2 C 6.5 T 6.6 T 6.8 T Alignment Standards over time…. & % they need to make amendments. Learning Objectives: Halt Review. Top 5 NOT MET: 1.3 T 50% 4.5 C 31% 2.5 C 28% 8.3 C 24% 3.2 C 18% Another one: 1.3 T. The thing about our template standards is that because they populate into the content standards in order to give a subsequent score, we give organizations an opportunity to get these all MET before we close a Template Review.
- Let’s bring the blooms into focus. What have we learned from course reviews? First, let’s look at what everyone is doing well.
- Our reviewers look at the reading level Quantitatively, Qualitatively, and Matching Reader to Text.
- Keeping in mind district limitations.
- The course design facilitates readability
- Which areas have shown the most growth?
- Growth from Rubric – Rubric over time We have seen an overall improvement amongst the K-12 Secondary Reviews from Rubric Editions. For example, we note that for Standard 4.5, citations, schools and districts are getting better about this. 29% of the time, this was found to be NOT MET in the second edition, while in the fourth edition, it was NOT MET only 19% of the time. Same thing with Standard 7.3, Academic services, Second edition met only 17% of the time, but only 5% of the time was it NOT MET in the fourth edition. Standard 8,3, dealing with accessibility, was found to be NOT MET 10% of the time in the second edition, after amendments, but 5% of the time in the fourth edition. This to me, shows a greater effort on the part of our schools and districts to create accessible content.
- Growth within organizations over time This is a data point that is a bit harder to gather, and we really need to do a deeper dive here. To take a look at how an organization has done over time. (first review/last review). It is proving to be a more difficult undertaking, since our Rubric standards are revised over time. So while we can get a broader view of how organizations do, the standards do change, some more significantly than others.
- But, I can deliver a little bit of anecdotal evidence here. When we look at Publishers who put large numbers of courses through for review, we notice trend there. Template Reviews. With each Rubric revision, organizations are required to have a new Template Review on file. That’s because the template standards get populated into each subsequent content review. Template reviews are done by a team of Master Reviewers, and since the findings from that review are populated into the content standards, and aggregated into the score, we encourage the publishers to try their best to meet each of the template standards. We implemented Template Reviews in our second edition of that particular Rubric, and now it has been through 2 revision cycles. For our Publishers who have had a Template review conducted each time, we find that they MEET more of the Template Standards the second and third time around. But. we continue to see those publishers who have a first time template making revisions (mostly to the Standards I mentioned earlier) before they move forward. That is consistent across the board. Learning Objectives. We tend to see this as more of a rough spot for both schools, districts, AND publishers. However, this also seems to change over time. Not only are the Learning Objectives measurable from the start, but they are consistent with the activities within the course. If the objective says that they are going to describe something, the activities and the assessments are true to that. So the alignment is there. The more course reviews that an organization puts through for official review, the time our review teams spend in providing guidance for creating and including measurable objectives within their course goes down significantly. We are starting to see organizations, like Michigan Virtual, who have used the feedback on their course reviews to inform their course development teams, where they create templates for the design process. So, for those of you just starting out, you may consider having one or two of your courses reviewed as a benchmark, so that you can apply that feedback going forward to the rest of your courses, even incorporating it into the design process from the start. Remember there is a course representative from each organization on each team. They serve as the contact for each review. As the course representative becomes more familiar with the review process, we see their courses meeting standards without amendment more often. Not only do they get better at the process, but they get better at knowing what the review teams are going to be looking for – they learn the standards. When a new course representative comes on board we see the growth process happen over again.
- Remember that during a review process, a course has a chance to make amendments and ultimately MEET all of the standards- thereby showing growth during the review. So, what areas have we seen the most improvement during a review- from start to finish?
- Where are we continuing to improve?
- Schools and districts struggle with 1.4 Etiquette expectations. You all know that this is also referred to as Netiquette. We seem to struggle with clearly stating the expectations for student conduct in our online courses. We post this in our brick and mortar classrooms. This needs to be posted in our online classrooms, as well.
- Also, academic support services. These are things like library media resources, tutoring, support staff, etc. We need to make sure that these are all available from the course navigation or student dashboard.
- Third party courses struggle with listing technical skills that a student will need to be successful in their course.
- They also struggle with providing instructor support- like assignment keys and grading guides.
- In particular, schools and districts and publishers struggle with Specific Standard 2.1 which are standards at the course level, Specific Standard 2.2, which are standards at the module or unit level, and Specific Standard 2.3, which ensures that those leaning objectives or competencies are aligned with state standards and/or other accepted content standards. Sometimes these standards are not written in measurable terms. Sometimes they aren’t present in the course at all. Since the learning objectives are the foundation on which any course is built, f any of these three standards are found to be NOT MET, the review is halted until the school, district, or publisher can rectify the issue.
- Sources for materials used in the course are clearly identified and stated. We continue to struggle in this area.
- Standard 8.3 continues to be a challenge for both school districts and publisher. The purpose of this standard is to ensure that course materials are provided in alternate formats or mediums for all students, allowing for alternative means of access through, for example, assistive technologies.
- Impact So what do we know about the impact that QM Course Reviews have?
- Impact Since 2006, we have officially reviewed and certified over 6,500 courses.
- Impact Official QM Course Reviews - both QM- and subscriber-managed - result in all participants (Course Representatives and Review Team members) taking the recommendations and experience from the review back to their own online and face-to-face courses and improving them. For Course Reviews completed between February 2015 and October 2016: 94% of Course Representatives and 81% of Peer Reviewers indicated they either had or planned to make changes in their online courses as a result of their Course Review experience
- Impact Official QM Course Reviews - As you can imagine, this has implications for F2F courses as well. 80% and 65% of them, respectively, indicated they either had or planned to make changes in their face-to-face courses
- Impact of Official QM Course Reviews The impact is why schools and districts choose to have their courses reviewed. Why do schools and districts Review? The data obtained from course reviews can ultimately inform the review and certification of quality for entire programs. These certifications demonstrate your commitment, not only to creating an environment of excellence and quality, but of maintaining that commitment. Reviews Provide Benchmarks At their base, QM Reviews serve to determine how your courses and programs stack up against our rigorous rubrics to measure quality. But more than that, they offer a basis for comparison for courses from differing sources. A way to demonstrate that your offerings stack up against the competition and compare favorably to the expectations of learners, peers, stakeholders. Reviews Chart a Path for Improvement Once your courses or programs have been benchmarked, a clear path for improvement is set. From small to large, specific areas are targeted for improvement, as each review culminates with a final report full of measurable feedback. Reviews Lead to Certifications Quality, especially as it pertains to educational courses and programs, can be a nebulous word. But with Quality Matters Certification, learners, parents, and stakeholders can be confident that you not only take the quality of your courses and programs seriously, but that you meet nationally and internationally recognized standards. For you, Quality is more than just a claim, it’s a commitment. And the QM Certification mark proves it.
- Impact Official QM Course Reviews - So you can do the math. The number of students touched by a course that has gone through a course review is enormous.
- Bottom Line Is that Growth Creates Blossoms. Everyone involved is making a difference- improving online learning for students.
- So how can you help to make a difference for Students?
- So how can you help to make a difference for Students? Ask vendors if their courses are QM-Certified. When buying online courses write QM into your RFP. Ask to see their final report. Educate yourself and know if the course is a good match for your school or district.
- Become a QM-Certified Reviewer. QM course reviewers say that serving on a review was one of the best PD experiences they have had. They get to work others who are passionate about making a difference. You get to see other courses, get ideas for your own, pass along your best practice examples, and make a difference for students.
- Have your own courses reviewed. [Show Shoreline Video]
- [Show Shoreline Video]
- So maybe my mother-in-law IS onto something. Maybe there is a magical elixir to grow online course blooms. And that magical elixir is our fabulous community. And because of everyone working together, we are collectively making a difference for our students.
- Let’s go back to our little garden story. Remember how all of the flowers were sure that the gardener would say they were the best. But the gardener just smiled and said, “look at all of my pretty flowers.” What the gardener did next shocked all of the flowers. One day the gardener came into the garden. The small yellow flowers fluffed their petals so that they would look the best. All of the purple flowers stood up tall so that they would look the best. The red flowers beamed in the sunlight so that they would look the best. He proceeded to take a basket and began to put the small yellow flowers into it. The yellow flowers were sure they were special because they were picked first. But the purple flowers just laughed at the roses, "Ha-ha! You are not pretty enough to stay in the garden." Next the gardener put some purple flowers in the basket and the red flowers began to laugh, "Told you! We are the best because we are the only flowers left in the garden." Finally the gardener put the mums into the basket. Again, all the flowers began to fight over who was the best. When the gardener got to the house, he began to put all of the beautiful flowers into a vase. First he put the small yellow flowers into the vase, remembering that they were the first flowers to bloom in the spring. Next he put the purple flowers into the vase and thought how wonderful to see them every day when he walked in the garden. Finally he put the red flowers into the vase; he was very excited to finally see the autumn flowers. He had been waiting all summer to see them. The gardener put the vase on the table and said, "I have the prettiest bouquet of flowers. Alone each flower is the best, but together they look perfect!" Suddenly the flowers realized that all along each one was the best that they could be. But it wasn't until the gardener made a bouquet of all of the flowers together that they became something special. At last all the flowers were happy. Each one of our online courses is a real achievement. Some of them require a bit more love and caring. And we know that each one can continue to improve. By working together and sharing our collective understanding of what works in online learning, we can truly make a difference for students. A beautiful bouquet of student success.
- Questions? Anything else I can bring into focus today?