What Makes a Quality Online Course: The Student Perspective

5,920 views

Published on

What Makes a Quality Online Course: The Student Perspective presented at Distance Teaching and Learning 2009, Madison, WI

Published in: Sports, Business
0 Comments
12 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,920
On SlideShare
0
From Embeds
0
Number of Embeds
767
Actions
Shares
0
Downloads
0
Comments
0
Likes
12
Embeds 0
No embeds

No notes for slide

What Makes a Quality Online Course: The Student Perspective

  1. 1. Penny Ralston-Berg August 7, 2009 Distance Teaching and Learning Madison, WI
  2. 2. Outline <ul><li>Project background </li></ul><ul><li>Findings </li></ul><ul><li>Implications for course design </li></ul><ul><li>Questions </li></ul>
  3. 3. Project Background <ul><li>Benchmarks in quality </li></ul><ul><ul><li>Institute for Higher Education Policy (2000) </li></ul></ul><ul><ul><li>Sloan-C Quality Framework (2002) </li></ul></ul><ul><ul><li>AECT Best Practices in Instructional Design (2003) </li></ul></ul><ul><ul><li>Quality Matters (2005-06) </li></ul></ul><ul><ul><li>E-Learning Maturity Model (2006) </li></ul></ul><ul><ul><li>NACOL National Standards for Quality Online Teaching (2008) </li></ul></ul><ul><ul><li>Quality Matters (2008-10) </li></ul></ul>
  4. 4. Project Background <ul><li>Benchmarks in quality </li></ul><ul><ul><li>Institute for Higher Education Policy (2000) </li></ul></ul><ul><ul><li>Sloan-C Quality Framework (2002) </li></ul></ul><ul><ul><li>AECT Best Practices in Instructional Design (2003) </li></ul></ul><ul><ul><li>Quality Matters (2005-06) </li></ul></ul><ul><ul><li>E-Learning Maturity Model (2006) </li></ul></ul><ul><ul><li>NACOL National Standards for Quality Online Teaching (2008) </li></ul></ul><ul><ul><li>Quality Matters (2008-10) </li></ul></ul>
  5. 5. Project Background <ul><li>Quality Matters (QM) stands out </li></ul><ul><ul><li>Research refined over time </li></ul></ul><ul><ul><li>Comprehensive literature review </li></ul></ul><ul><ul><li>Distilled to one-page rubric </li></ul></ul><ul><ul><ul><li>Eight categories </li></ul></ul></ul><ul><ul><ul><li>Forty specific elements ranked as essential, very important, important </li></ul></ul></ul><ul><ul><li>Subscription service : http://www.qualitymatters.org </li></ul></ul>Reference: © 2008. MarylandOnline, Inc. All rights reserved.
  6. 6. Additional Resources <ul><li>Previous research (Ralston-Berg & Nath) </li></ul><ul><li>Garrison & Arbaugh (2007) Community of Inquiry Framework </li></ul><ul><li>Stodel, Thompson, & McDonald (2006)  Learners' perspectives on what is missing from online learning </li></ul><ul><li>Young & Norgard (2006) Quality from the students’ perspective </li></ul>
  7. 7. Quality: A Definition <ul><li>Quality can be defined as an educational system in which the learner has a reasonable opportunity for success. (Ragan & Sax, 2005) </li></ul><ul><li>Ask students about quality in terms of which items contribute to their success in an online environment. </li></ul>
  8. 8. Key Questions <ul><li>Do students agree with published benchmarks? </li></ul><ul><li>Are published characteristics of quality perceived as contributing to student success? </li></ul><ul><li>Is there anything missing? </li></ul><ul><li>How does this impact our design practices for online learners? </li></ul>
  9. 9. Method <ul><li>Descriptive research </li></ul><ul><li>Quantitative online survey design </li></ul><ul><li>Respondents will be at least 18 years of age </li></ul><ul><li>Completed course or currently enrolled </li></ul><ul><li>Participation is voluntary, anonymous and confidential. </li></ul>
  10. 10. Method <ul><li>Examine quality elements listed in: Quality Matters rubric standards 2008-2010 edition with assigned point values © 2006 MarylandOnline, Inc. http://www.qualitymatters.org Other sources </li></ul><ul><li>Convert quality elements to student-centered questions </li></ul>
  11. 11. Method <ul><li>Example Original: Instructions make clear how to get started and where to find various course components. Revised: Clear instructions tell me how to get started and how to find various course components. </li></ul>
  12. 12. Survey Design <ul><li>Delivery in online format – Survey Monkey </li></ul><ul><li>89 items </li></ul><ul><ul><li>69 quality indicators </li></ul></ul><ul><ul><li>3 open response </li></ul></ul><ul><ul><li>17 demographic </li></ul></ul>
  13. 13. Survey Design <ul><li>Rank value on scale of 0 to 3 </li></ul><ul><ul><li>0=Not at all important - does not contribute to my success </li></ul></ul><ul><ul><li>1=Important </li></ul></ul><ul><ul><li>2=Very important </li></ul></ul><ul><ul><li>3=Essential – could not succeed without it </li></ul></ul>
  14. 14. Survey Delivery <ul><li>Four-year universities </li></ul><ul><li>Two-year institutions </li></ul><ul><li>Online student communities </li></ul><ul><ul><li>Facebook </li></ul></ul><ul><ul><li>Student.com </li></ul></ul><ul><ul><li>eLearners.com </li></ul></ul><ul><ul><li>DegreeInfo Distance Learning Discussion Forums </li></ul></ul><ul><ul><li>OOEN.net </li></ul></ul>
  15. 15. Demographics <ul><li>202 responses </li></ul><ul><li>68% Female / 32% Male </li></ul><ul><li>Age </li></ul>
  16. 16. Demographics <ul><li>Courses taken </li></ul>
  17. 17. Demographics <ul><li>Cohort-based courses </li></ul><ul><li>Comfortable or very comfortable with technology </li></ul><ul><li>1/2 are full time students </li></ul><ul><li>1/3 spend more than 16 hours per week on courses </li></ul><ul><li>2/3 employed full time </li></ul>
  18. 18. Demographics
  19. 19. Do Students Agree? <ul><li>YES! </li></ul><ul><li>All QM items scored >1 </li></ul><ul><li>39 items scored >1.5 </li></ul><ul><li>QM 2 just as important as QM 3 </li></ul><ul><li>QM 1 related to instructions, usability just as important as QM 2 </li></ul>
  20. 20. Top 10 Student Countdown <ul><li>Assessments are appropriately timed, varied and appropriate </li></ul><ul><li>Instructions on how to access resources are sufficient and easy to understand </li></ul><ul><li>Course components are web-based or easily downloaded for use offline </li></ul><ul><li>Requirements for interaction are clearly explained </li></ul><ul><li>Opportunity to speak with instructor, classmates through email, bulletin boards, any time / any place </li></ul>
  21. 21. Top 10 Student Countdown <ul><li>Technologies required are readily available – provided or easily downloaded </li></ul><ul><li>Clear instructions tell me how to get started and to find course components </li></ul><ul><li>Criteria for how I will be evaluated is descriptive and specific </li></ul><ul><li>Navigation is logical, consistent, and efficient </li></ul><ul><li>The grading policy is clearly stated </li></ul>
  22. 22. Themes <ul><li>Technology function / support </li></ul><ul><li>Clear expectations </li></ul><ul><li>Navigation </li></ul><ul><li>Assessments appropriate / varied </li></ul><ul><li>More traditional communication </li></ul>
  23. 23. Do Students Agree? <ul><li>But… </li></ul><ul><li>Interaction scores lower than expected </li></ul><ul><ul><li>Build a community </li></ul></ul><ul><ul><li>Synchronous communication </li></ul></ul><ul><ul><li>Students as contributors of content </li></ul></ul><ul><ul><li>Multimedia </li></ul></ul><ul><ul><li>Games </li></ul></ul>
  24. 24. Bottom 10 <ul><li>I find course-related content and share it with the instructor and classmates </li></ul><ul><li>I use wikis, shared documents, or other online collaborative tools to complete group work </li></ul><ul><li>I am asked to introduce myself to the class </li></ul><ul><li>I coach other students and help answer their questions </li></ul><ul><li>Opportunity to speak with my instructor and classmates through… same time, same place technology </li></ul>
  25. 25. Bottom 10 <ul><li>Instructor does not participate in class discussions </li></ul><ul><li>Course contains interactive games or simulations </li></ul><ul><li>I author / create course-related content and share it with instructor and classmates </li></ul><ul><li>I work in groups with other students </li></ul><ul><li>Course contains audio and video content </li></ul>
  26. 26. Types of Interaction Students’ Rank Learning activities encourage me to interact with content in the course. 1.90 Learning activities encourage me to interact with my instructor. 1.58 Learning activities encourage me to interact with other students. 1.33
  27. 27. Content Interaction Students’ Rank The course provides practice exercises that I may complete on my own to master difficult content. 1.82 Course contains instructional graphics that help explain complex concepts. 1.67 Ungraded practice quizzes are provided so that I may check my own knowledge and understanding of a topic. 1.63 I find course-related content and share it with the instructor and classmates. 1.02 Course contains audio and video content. 0.96 I author / create course-related content and share it with instructor and classmates. 0.90 Course contains interactive games or simulations. 0.86
  28. 28. Instructor Interaction Students’ Rank Clear standards are set for instructor response (turn-around time for email, grade posting, etc.). 2.32 Clear standards are set for instructor availability (office hours, etc.) 2.17 Instructor actively participates in the class discussions. 2.01 Instructor summarizes and guides the class discussions. 1.96 Instructor shares personal insights and experiences above and beyond the reading assignments. 1.74 Instructor does not participate in class discussions. 0.42
  29. 29. Student-Student Interaction Students’ Rank Requirements for my interaction with the instructor, content, and other students are clearly explained. 2.40 Opportunity to ask questions of other students and get help from sources other than the instructor. 1.64 I share my ideas, opinions, and findings with my classmates in a public forum. 1.40 I coach other students and help answer their questions. 1.09 I am asked to introduce myself to the class. 1.06 I use wikis, shared documents, or other online collaborative tools to complete group work. 1.04 I work in groups with other students. 0.93
  30. 30. Themes <ul><li>Faculty presence </li></ul><ul><li>Same time, same place </li></ul><ul><li>Collaborative content creation </li></ul><ul><li>Media / games </li></ul><ul><li>Student as source of content / mentor </li></ul><ul><li>Essential vs. supportive </li></ul>
  31. 31. Design Implications <ul><li>Is <name your technology> essential? </li></ul><ul><ul><li>Maybe not </li></ul></ul><ul><li>Is <name your technology> useful? </li></ul><ul><ul><li>Maybe / maybe not </li></ul></ul><ul><li>Focus </li></ul><ul><ul><li>Expectations / outcomes </li></ul></ul><ul><ul><li>Teaching and learning </li></ul></ul><ul><ul><li>Less on specific technology / media </li></ul></ul>
  32. 32. Design Implications <ul><li>Clear expectations </li></ul><ul><li>Relevance </li></ul><ul><li>Usability </li></ul><ul><li>Availability </li></ul><ul><li>Functionality </li></ul><ul><li>Tied to learning / assessment </li></ul>
  33. 33. What’s missing? <ul><li>Get info prior to course start </li></ul><ul><li>All-inclusive syllabus </li></ul><ul><li>Flexible schedule / deadlines </li></ul><ul><li>Reminders about due dates / keeping on task </li></ul><ul><li>Affordability </li></ul><ul><li>Fairness </li></ul><ul><li>Learning from colleagues / professionals </li></ul><ul><li>Computer / Internet are all I need </li></ul>
  34. 34. What’s missing? <ul><li>Leadership of the instructor </li></ul><ul><li>Feedback on every graded item / fast turnaround </li></ul><ul><li>Use of outside resources to increase comprehension </li></ul><ul><li>Labs (for science courses) </li></ul><ul><li>Context (how lessons relate to one another) </li></ul><ul><li>Cohesive course materials (materials from different sources flow together) </li></ul><ul><li>Quizzes that auto-save answers </li></ul>
  35. 35. Design Implications <ul><li>Technology is “make or break” (functionality) </li></ul><ul><li>Content and instructor / student interaction are core </li></ul><ul><li>Media, Web 2.0 supplemental </li></ul><ul><li>Layered design </li></ul><ul><li>Facilitation skills / availability </li></ul><ul><li>Flexibility </li></ul><ul><li>Convenience </li></ul><ul><li>Students see the total package </li></ul>
  36. 36. Invitation <ul><li>Student perspectives in quality ongoing study </li></ul><ul><li>Replicate study with your institution / school? </li></ul><ul><ul><li>Students have completed at least one online course </li></ul></ul><ul><ul><li>Students complete the survey online </li></ul></ul>
  37. 37. Contact Info <ul><li>Penny Ralston-Berg Instructional Designer Penn State World Campus [email_address] (papa – lima – romeo – one – five at psu – dot – edu) Slideshare </li></ul>
  38. 38. Course Overview and Introduction Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank Clear instructions tell me how to get started and how to find various course components. 2.56 Minimum preparation or prerequisite knowledge I need to succeed in the course is clearly stated. 2.06 Minimum technical skills expected of me are clearly stated. 1.92 The instructor introduces her- or himself. 1.84 A statement introduces me to the purpose of the course and its components. 1.82 Etiquette (or “netiquette”) guidelines for how to behave online are clearly stated. 1.46 I am asked to introduce myself to the class. 1.06
  39. 39. Learning Objectives Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank Instructions on how to meet the learning objectives are adequate and stated clearly. 2.28 The learning objectives are appropriate for the level of the course. 2.12 The course learning objectives describe outcomes that I am able to achieve. 1.81 The course contains learning objectives listing what I am expected to learn. 1.80 The course contains learning objectives listing what I am expected to learn. 1.80 The module/unit learning objectives describe outcomes that I am able to achieve and are consistent with the course-level objectives. 1.76 All learning objectives are clearly stated and written from my perspective. 1.74
  40. 40. Assessment and Measurement Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank Assessments measure the stated learning objectives and are consistent with course activities and resources. 2.38 The grading policy is stated clearly. 2.48 Criteria for how my work and participation will be evaluated are descriptive and specific. 2.55 Assessments are appropriately timed within the length of the course, varied, and appropriate to the content being assessed. 2.44 “ Self-check” or practice assignments are provided, and I am provided with timely feedback. 2.08 The course provides practice exercises that I may complete on my own to master difficult content. 1.82 Ungraded practice quizzes are provided so that I may check my own knowledge and understanding of a topic. 1.63
  41. 41. Resources and Materials Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank Instructional materials have sufficient breadth, depth, and currency for me to learn the subject. 2.34 Instructional materials contribute to the achievement of the course and module/unit learning objectives. 2.28 Instructional materials contribute to the achievement of the course and module/unit learning objectives. 2.28 The relationship between the instructional materials and the learning activities in clearly explained to me. 2.09 All resources and materials used in the course are appropriately cited. 1.72 Activities are included with each lesson. 1.58
  42. 42. Learner Engagement Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank Requirements for course interaction clearly articulated 5.28 Clear standards set for instructor response and availability 5.20 Activities promote achievement of stated outcomes 5.12 Activities foster content-student interaction 5.00 Course design prompts instructor to be present, active, engaged 4.94 Activities foster instructor-student interaction 4.64 Activities foster student-student interaction 4.24
  43. 43. Course Technology Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank Required technologies are provided or easily downloaded 5.40 Instructions to access online resources are sufficient and easy to understand 5.24 Tools and media support outcomes and are integrated with text and assignments 4.96
  44. 44. Learner Support Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank Clear description of technical support offered 4.78 Tutorials and resources to answer basic questions 4.60 Clear description of academic support offered 4.44
  45. 45. Accessibility Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank QM Rank The course incorporates ADA standards and reflect conformance with institutional policy regarding accessibility in online and hybrid courses. 5.02 3 Course pages and course materials provide equivalent alternatives to auditory and visual content. 4.86 2 Course pages have links that are self-describing and meaningful. 4.56 2 The course ensures screen readability. 4.26 1

×