Do quality standards matter to students?Penny Ralston-BergThe Quality Matters Program 2nd Annual ConferenceJune 11, 2010http://www.slideshare.net/plr15
From their fingertips to our ears2300+, 31 institutions, 22 statesQualitative, preliminary surveyThrow light on how students perceive qualitySnapshot of trendsNot trying to predict behaviorLay groundwork for future quantitative studies
InstrumentConverted existing quality elements to student-centered languageHow important are these features to success in an online course?0 - Not at all important - does not contribute to my success1 - Important	2 - Very important	3 - Essential – could not succeed without it(Ragan & Sax, 2005)
DemographicsRange in age from 18 to 65+Range from 0 to 9+ courses completedMost comfortable or very comfortable with technologyMost have experienced only cohort-based or combo of cohort and some self-paced57% female59% employed full or part time64% taking at least 2 courses
25 Areas of studyAgriculture Architecture, Engineering, and Drafting Arts, Visual and PerformingBusiness and Finance Community and Social Services Computers and Math Construction and Extraction Education, Museum Work, and Library Science Environment Health Care Support, Diagnosis, Treatment, or Technology Humanities Installation, Maintenance, and Repair Law and Government Management Media and Communications Life and Physical Science Office and Administrative Support Personal Care and Culinary Services Production Protective Services Sales Social Science Sports and Fitness Technical Transportation
AgendaHow do students see quality?Are QM standards important to students?What do students say about quality?What else should we consider?Topics for more study
Quality online course?
Quality online course?
Holistic View of QualityOutside Course
Holistic View of Quality
Holistic View of Quality
Holistic View of Quality
Holistic View of Quality
Holistic View of Quality
Yes, quality mattersAgree – QM rubric items are importantMany QM 2,1 items => QM 3 itemsKnown effective course design practicesAdd valueContribute to qualityRelevance and alignment more important than specific media, tool, delivery mode
QM: Course Overview and Introduction
QM: Learning Objectives
Clear expectationsWhat’s due when -- and how to do it; “no surprises”Don’t assume we know how to do itConcise instructions with links to more detailed instructionsRealistic, reasonable workloadExpectations for instructor
QM: Assessment and Measurement
Opportunities to enhance learning
Opportunities to enhance learningDon’t repeat text / readings
Supplement with audio / video
Share expertise
Share alternate perspectives
Supplement with experts other than instructor
Explain and provide examplesOpportunities to enhance learningAutomated comprehension check (Do I “get” it?)
Access to tutor prior to assignment due date
Instructor answers questions about materials
Study group reviews materials
Feedback on draft assignments
Able to review example / exemplary assignments
FYI resourcesOpportunities to enhance learningGraded activities between exams (I’m on track / learning the material.)
Time to think between posts
Discuss work with instructor and peers
Study group debrief of graded activitiesOpportunities to enhance learningStudy guide provided / study group
Feedback on graded activities
Practice online exam (for comprehension and technical practice)
Resubmit assignments to improve grade based on detailed feedback
Feedback on activities received prior to assessmentOpportunities to enhance learningGrade as well as detailed feedback
See results of online exams
Chance to re-learn what I didn’t get the first time (Why did I lose points? What do I study to “get it?”)QM: Resources and Materials
Learning materialsMaterials accessible prior to course startMaterials designed specifically for courseDirect links to online resources – not generic site / list of articles / page of reserves Too many resources can dilute focus / clarify required vs. FYI
Learning materialsTutorials, examples, explanations; multiple ways to grasp the conceptExpert opinions other than instructor / multiple sources of research Relevant, thorough, important, up-to-dateDraws in current events; related to real world
QM: Resources and Materials / Media
QM: Learner Engagement - Activities
Course design activitiesMore than just reading and writing Activities:ChallengingPromote deeper understandingAnalysisCritical thinkingApplication to real world activities Don't assume student is already working in field
QM: Learner Engagement - Instructor
InstructorAlters teaching style for online courseAs active as students – “don't phone it in” Willing to answer questions immediately; relates materials to real worldStrong computer / technical skillsNew instructors receive mentoring from more experienced instructors Brings out the best in students
Instructor is not the authorStudents notice when an instructor:Doesn’t appear in videos, etc.Doesn’t have handle on grades / assignmentsNot familiar with materialsNot familiar with CMSDon’t appreciate, “Don’t blame me – I didn’t write this course.”
FeedbackTimelyRelevantDetailed, meaningful, helpfulJustifies the grade Tells me “where I'm at” or “how I'm doing” Constructive, positive“Good or bad - am I on the right track?”Given prior to assessments, future graded assignments
QM: Learner Engagement - Students
Student interactionOff-topic / open discussionNot simple post / post reply Meaningful, thought provoking questions Discuss what you've learned with someone other an professor “Wouldn't have opportunity to talk to as many people” if the class was face-to-faceDon’t like it – other students hold up my progress
Learner Engagement - Technology
QM: Course Technology
Convenient Technology Available 24/7 Works with all computer platforms / browsers / media players One password for everythingEmail works inside and outside the courseTools are established (not experimental)Portability; apps for handheld devices; import to calendarReal time text / IM / chat with instructor (to answer questions)Reach the professor at any convenience
QM: Learner Support
Support servicesComplete research online Contact proxy if instructor isn't available Access tutoring Prepare a job searchAccess the library Contact counselor / advisor / admissions Check status of financial aid Buy booksGet training on how to use course system
QM: Accessibility
Accessibility good for allWatch/listen, read online, or printUse handouts or transcripts to take notesReplay or print explanations / directionsFind specific points in video or audioCaptions: better understanding is speaker talks too fast or is difficult to understand
How do students see quality?Beyond course qualityLearning is an organic experienceTied with items inside and outside coursesTied to program and institution

QM2010Keynote

  • 1.
    Do quality standardsmatter to students?Penny Ralston-BergThe Quality Matters Program 2nd Annual ConferenceJune 11, 2010http://www.slideshare.net/plr15
  • 2.
    From their fingertipsto our ears2300+, 31 institutions, 22 statesQualitative, preliminary surveyThrow light on how students perceive qualitySnapshot of trendsNot trying to predict behaviorLay groundwork for future quantitative studies
  • 3.
    InstrumentConverted existing qualityelements to student-centered languageHow important are these features to success in an online course?0 - Not at all important - does not contribute to my success1 - Important 2 - Very important 3 - Essential – could not succeed without it(Ragan & Sax, 2005)
  • 4.
    DemographicsRange in agefrom 18 to 65+Range from 0 to 9+ courses completedMost comfortable or very comfortable with technologyMost have experienced only cohort-based or combo of cohort and some self-paced57% female59% employed full or part time64% taking at least 2 courses
  • 5.
    25 Areas ofstudyAgriculture Architecture, Engineering, and Drafting Arts, Visual and PerformingBusiness and Finance Community and Social Services Computers and Math Construction and Extraction Education, Museum Work, and Library Science Environment Health Care Support, Diagnosis, Treatment, or Technology Humanities Installation, Maintenance, and Repair Law and Government Management Media and Communications Life and Physical Science Office and Administrative Support Personal Care and Culinary Services Production Protective Services Sales Social Science Sports and Fitness Technical Transportation
  • 6.
    AgendaHow do studentssee quality?Are QM standards important to students?What do students say about quality?What else should we consider?Topics for more study
  • 7.
  • 8.
  • 9.
    Holistic View ofQualityOutside Course
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
    Yes, quality mattersAgree– QM rubric items are importantMany QM 2,1 items => QM 3 itemsKnown effective course design practicesAdd valueContribute to qualityRelevance and alignment more important than specific media, tool, delivery mode
  • 16.
    QM: Course Overviewand Introduction
  • 17.
  • 18.
    Clear expectationsWhat’s duewhen -- and how to do it; “no surprises”Don’t assume we know how to do itConcise instructions with links to more detailed instructionsRealistic, reasonable workloadExpectations for instructor
  • 19.
  • 20.
  • 21.
    Opportunities to enhancelearningDon’t repeat text / readings
  • 22.
  • 23.
  • 24.
  • 25.
    Supplement with expertsother than instructor
  • 26.
    Explain and provideexamplesOpportunities to enhance learningAutomated comprehension check (Do I “get” it?)
  • 27.
    Access to tutorprior to assignment due date
  • 28.
  • 29.
  • 30.
  • 31.
    Able to reviewexample / exemplary assignments
  • 32.
    FYI resourcesOpportunities toenhance learningGraded activities between exams (I’m on track / learning the material.)
  • 33.
    Time to thinkbetween posts
  • 34.
    Discuss work withinstructor and peers
  • 35.
    Study group debriefof graded activitiesOpportunities to enhance learningStudy guide provided / study group
  • 36.
  • 37.
    Practice online exam(for comprehension and technical practice)
  • 38.
    Resubmit assignments toimprove grade based on detailed feedback
  • 39.
    Feedback on activitiesreceived prior to assessmentOpportunities to enhance learningGrade as well as detailed feedback
  • 40.
    See results ofonline exams
  • 41.
    Chance to re-learnwhat I didn’t get the first time (Why did I lose points? What do I study to “get it?”)QM: Resources and Materials
  • 42.
    Learning materialsMaterials accessibleprior to course startMaterials designed specifically for courseDirect links to online resources – not generic site / list of articles / page of reserves Too many resources can dilute focus / clarify required vs. FYI
  • 43.
    Learning materialsTutorials, examples,explanations; multiple ways to grasp the conceptExpert opinions other than instructor / multiple sources of research Relevant, thorough, important, up-to-dateDraws in current events; related to real world
  • 44.
    QM: Resources andMaterials / Media
  • 45.
  • 46.
    Course design activitiesMorethan just reading and writing Activities:ChallengingPromote deeper understandingAnalysisCritical thinkingApplication to real world activities Don't assume student is already working in field
  • 47.
  • 48.
    InstructorAlters teaching stylefor online courseAs active as students – “don't phone it in” Willing to answer questions immediately; relates materials to real worldStrong computer / technical skillsNew instructors receive mentoring from more experienced instructors Brings out the best in students
  • 49.
    Instructor is notthe authorStudents notice when an instructor:Doesn’t appear in videos, etc.Doesn’t have handle on grades / assignmentsNot familiar with materialsNot familiar with CMSDon’t appreciate, “Don’t blame me – I didn’t write this course.”
  • 50.
    FeedbackTimelyRelevantDetailed, meaningful, helpfulJustifiesthe grade Tells me “where I'm at” or “how I'm doing” Constructive, positive“Good or bad - am I on the right track?”Given prior to assessments, future graded assignments
  • 51.
  • 52.
    Student interactionOff-topic /open discussionNot simple post / post reply Meaningful, thought provoking questions Discuss what you've learned with someone other an professor “Wouldn't have opportunity to talk to as many people” if the class was face-to-faceDon’t like it – other students hold up my progress
  • 53.
  • 54.
  • 55.
    Convenient Technology Available24/7 Works with all computer platforms / browsers / media players One password for everythingEmail works inside and outside the courseTools are established (not experimental)Portability; apps for handheld devices; import to calendarReal time text / IM / chat with instructor (to answer questions)Reach the professor at any convenience
  • 56.
  • 57.
    Support servicesComplete researchonline Contact proxy if instructor isn't available Access tutoring Prepare a job searchAccess the library Contact counselor / advisor / admissions Check status of financial aid Buy booksGet training on how to use course system
  • 58.
  • 59.
    Accessibility good forallWatch/listen, read online, or printUse handouts or transcripts to take notesReplay or print explanations / directionsFind specific points in video or audioCaptions: better understanding is speaker talks too fast or is difficult to understand
  • 60.
    How do studentssee quality?Beyond course qualityLearning is an organic experienceTied with items inside and outside coursesTied to program and institution

Editor's Notes

  • #10 Outside: 3rd party tools not included with cms, guest experts, research, resources, colleagues who have taken the course – anything outside the core design, materials, and instructor experience
  • #11 Outside: 3rd party tools not included with cms, guest experts, research, resources, colleagues who have taken the course – anything outside the core design, materials, and instructor experience
  • #12 Outside: 3rd party tools not included with cms, guest experts, research, resources, colleagues who have taken the course – anything outside the core design, materials, and instructor experience
  • #13 Outside: 3rd party tools not included with cms, guest experts, research, resources, colleagues who have taken the course – anything outside the core design, materials, and instructor experience
  • #14 Outside: 3rd party tools not included with cms, guest experts, research, resources, colleagues who have taken the course – anything outside the core design, materials, and instructor experienceOutside program: student communities, LinkedIn, job search resources, professional colleagues
  • #15 Outside: 3rd party tools not included with cms, guest experts, research, resources, colleagues who have taken the course – anything outside the core design, materials, and instructor experienceOutside program: student communities, LinkedIn, job search resources, professional colleagues
  • #17 QM 1 just as important as QM 3. Introducing myself doesn’t have as much importance.
  • #18 Instructions on how to meet the objectives rank higher than the learning objectives themselves.