What Makes a Quality Online Course: The Student Perspective

Penny Ralston-Berg August 7, 2009 Distance Teaching and Learning Madison, WI

Outline
Project background
Findings
Implications for course design
Questions

Project Background
Benchmarks in quality
Institute for Higher Education Policy (2000)
Sloan-C Quality Framework (2002)
AECT Best Practices in Instructional Design (2003)
Quality Matters (2005-06)
E-Learning Maturity Model (2006)
NACOL National Standards for Quality Online Teaching (2008)
Quality Matters (2008-10)

Project Background
Quality Matters (QM) stands out
Research refined over time
Comprehensive literature review
Distilled to one-page rubric
Eight categories
Forty specific elements ranked as essential, very important, important
Subscription service : http://www.qualitymatters.org
Reference: © 2008. MarylandOnline, Inc. All rights reserved.

Additional Resources
Previous research (Ralston-Berg & Nath)
Garrison & Arbaugh (2007) Community of Inquiry Framework
Stodel, Thompson, & McDonald (2006) Learners' perspectives on what is missing from online learning
Young & Norgard (2006) Quality from the students’ perspective

Quality: A Definition
Quality can be defined as an educational system in which the learner has a reasonable opportunity for success. (Ragan & Sax, 2005)
Ask students about quality in terms of which items contribute to their success in an online environment.

Key Questions
Do students agree with published benchmarks?
Are published characteristics of quality perceived as contributing to student success?
Is there anything missing?
How does this impact our design practices for online learners?

Method
Descriptive research
Quantitative online survey design
Respondents will be at least 18 years of age
Completed course or currently enrolled
Participation is voluntary, anonymous and confidential.

Method
Examine quality elements listed in: Quality Matters rubric standards 2008-2010 edition with assigned point values © 2006 MarylandOnline, Inc. http://www.qualitymatters.org Other sources
Convert quality elements to student-centered questions

Method
Example Original: Instructions make clear how to get started and where to find various course components. Revised: Clear instructions tell me how to get started and how to find various course components.

Survey Design
Delivery in online format – Survey Monkey
89 items
69 quality indicators
3 open response
17 demographic

Survey Design
Rank value on scale of 0 to 3
0=Not at all important - does not contribute to my success
1=Important
2=Very important
3=Essential – could not succeed without it

Survey Delivery
Four-year universities
Two-year institutions
Online student communities
Facebook
Student.com
eLearners.com
DegreeInfo Distance Learning Discussion Forums
OOEN.net

Demographics
202 responses
68% Female / 32% Male
Age

Demographics
Courses taken

Demographics
Cohort-based courses
Comfortable or very comfortable with technology
1/2 are full time students
1/3 spend more than 16 hours per week on courses
2/3 employed full time

Demographics

Do Students Agree?
YES!
All QM items scored >1
39 items scored >1.5
QM 2 just as important as QM 3
QM 1 related to instructions, usability just as important as QM 2

Top 10 Student Countdown
Assessments are appropriately timed, varied and appropriate
Instructions on how to access resources are sufficient and easy to understand
Course components are web-based or easily downloaded for use offline
Requirements for interaction are clearly explained
Opportunity to speak with instructor, classmates through email, bulletin boards, any time / any place

Top 10 Student Countdown
Technologies required are readily available – provided or easily downloaded
Clear instructions tell me how to get started and to find course components
Criteria for how I will be evaluated is descriptive and specific
Navigation is logical, consistent, and efficient
The grading policy is clearly stated

Themes
Technology function / support
Clear expectations
Navigation
Assessments appropriate / varied
More traditional communication

Do Students Agree?
But…
Interaction scores lower than expected
Build a community
Synchronous communication
Students as contributors of content
Multimedia
Games

Bottom 10
I find course-related content and share it with the instructor and classmates
I use wikis, shared documents, or other online collaborative tools to complete group work
I am asked to introduce myself to the class
I coach other students and help answer their questions
Opportunity to speak with my instructor and classmates through… same time, same place technology

Bottom 10
Instructor does not participate in class discussions
Course contains interactive games or simulations
I author / create course-related content and share it with instructor and classmates
I work in groups with other students
Course contains audio and video content

Types of Interaction Students’ Rank Learning activities encourage me to interact with content in the course. 1.90 Learning activities encourage me to interact with my instructor. 1.58 Learning activities encourage me to interact with other students. 1.33

Content Interaction Students’ Rank The course provides practice exercises that I may complete on my own to master difficult content. 1.82 Course contains instructional graphics that help explain complex concepts. 1.67 Ungraded practice quizzes are provided so that I may check my own knowledge and understanding of a topic. 1.63 I find course-related content and share it with the instructor and classmates. 1.02 Course contains audio and video content. 0.96 I author / create course-related content and share it with instructor and classmates. 0.90 Course contains interactive games or simulations. 0.86

Instructor Interaction Students’ Rank Clear standards are set for instructor response (turn-around time for email, grade posting, etc.). 2.32 Clear standards are set for instructor availability (office hours, etc.) 2.17 Instructor actively participates in the class discussions. 2.01 Instructor summarizes and guides the class discussions. 1.96 Instructor shares personal insights and experiences above and beyond the reading assignments. 1.74 Instructor does not participate in class discussions. 0.42

Student-Student Interaction Students’ Rank Requirements for my interaction with the instructor, content, and other students are clearly explained. 2.40 Opportunity to ask questions of other students and get help from sources other than the instructor. 1.64 I share my ideas, opinions, and findings with my classmates in a public forum. 1.40 I coach other students and help answer their questions. 1.09 I am asked to introduce myself to the class. 1.06 I use wikis, shared documents, or other online collaborative tools to complete group work. 1.04 I work in groups with other students. 0.93

Themes
Faculty presence
Same time, same place
Collaborative content creation
Media / games
Student as source of content / mentor
Essential vs. supportive

Design Implications
Is <name your technology> essential?
Maybe not
Is <name your technology> useful?
Maybe / maybe not
Focus
Expectations / outcomes
Teaching and learning
Less on specific technology / media

Design Implications
Clear expectations
Relevance
Usability
Availability
Functionality
Tied to learning / assessment

What’s missing?
Get info prior to course start
All-inclusive syllabus
Flexible schedule / deadlines
Reminders about due dates / keeping on task
Affordability
Fairness
Learning from colleagues / professionals
Computer / Internet are all I need

What’s missing?
Leadership of the instructor
Feedback on every graded item / fast turnaround
Use of outside resources to increase comprehension
Labs (for science courses)
Context (how lessons relate to one another)
Cohesive course materials (materials from different sources flow together)
Quizzes that auto-save answers

Design Implications
Technology is “make or break” (functionality)
Content and instructor / student interaction are core
Media, Web 2.0 supplemental
Layered design
Facilitation skills / availability
Flexibility
Convenience
Students see the total package

Invitation
Student perspectives in quality ongoing study
Replicate study with your institution / school?
Students have completed at least one online course
Students complete the survey online

Contact Info
Penny Ralston-Berg Instructional Designer Penn State World Campus [email_address] (papa – lima – romeo – one – five at psu – dot – edu) Slideshare

Course Overview and Introduction Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank Clear instructions tell me how to get started and how to find various course components. 2.56 Minimum preparation or prerequisite knowledge I need to succeed in the course is clearly stated. 2.06 Minimum technical skills expected of me are clearly stated. 1.92 The instructor introduces her- or himself. 1.84 A statement introduces me to the purpose of the course and its components. 1.82 Etiquette (or “netiquette”) guidelines for how to behave online are clearly stated. 1.46 I am asked to introduce myself to the class. 1.06

Learning Objectives Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank Instructions on how to meet the learning objectives are adequate and stated clearly. 2.28 The learning objectives are appropriate for the level of the course. 2.12 The course learning objectives describe outcomes that I am able to achieve. 1.81 The course contains learning objectives listing what I am expected to learn. 1.80 The course contains learning objectives listing what I am expected to learn. 1.80 The module/unit learning objectives describe outcomes that I am able to achieve and are consistent with the course-level objectives. 1.76 All learning objectives are clearly stated and written from my perspective. 1.74

Assessment and Measurement Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank Assessments measure the stated learning objectives and are consistent with course activities and resources. 2.38 The grading policy is stated clearly. 2.48 Criteria for how my work and participation will be evaluated are descriptive and specific. 2.55 Assessments are appropriately timed within the length of the course, varied, and appropriate to the content being assessed. 2.44 “ Self-check” or practice assignments are provided, and I am provided with timely feedback. 2.08 The course provides practice exercises that I may complete on my own to master difficult content. 1.82 Ungraded practice quizzes are provided so that I may check my own knowledge and understanding of a topic. 1.63

Resources and Materials Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank Instructional materials have sufficient breadth, depth, and currency for me to learn the subject. 2.34 Instructional materials contribute to the achievement of the course and module/unit learning objectives. 2.28 Instructional materials contribute to the achievement of the course and module/unit learning objectives. 2.28 The relationship between the instructional materials and the learning activities in clearly explained to me. 2.09 All resources and materials used in the course are appropriately cited. 1.72 Activities are included with each lesson. 1.58

Learner Engagement Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank Requirements for course interaction clearly articulated 5.28 Clear standards set for instructor response and availability 5.20 Activities promote achievement of stated outcomes 5.12 Activities foster content-student interaction 5.00 Course design prompts instructor to be present, active, engaged 4.94 Activities foster instructor-student interaction 4.64 Activities foster student-student interaction 4.24

Course Technology Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank Required technologies are provided or easily downloaded 5.40 Instructions to access online resources are sufficient and easy to understand 5.24 Tools and media support outcomes and are integrated with text and assignments 4.96

Learner Support Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank Clear description of technical support offered 4.78 Tutorials and resources to answer basic questions 4.60 Clear description of academic support offered 4.44

Accessibility Quality Matters Rubric Standards 2008-2010 edition with Assigned Point Values © 2006 MarylandOnline, Inc. Students’ Rank QM Rank The course incorporates ADA standards and reflect conformance with institutional policy regarding accessibility in online and hybrid courses. 5.02 3 Course pages and course materials provide equivalent alternatives to auditory and visual content. 4.86 2 Course pages have links that are self-describing and meaningful. 4.56 2 The course ensures screen readability. 4.26 1

http://www.slideshare.net	7
http://gallium.pd.local	1
http://blackboard.owens.edu	1
http://paper.li	1

What Makes a Quality Online Course: The Student Perspective

by Penny Ralston-Berg on Aug 07, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

4 Embeds 10

Statistics

What Makes a Quality Online Course: The Student Perspective — Presentation Transcript