Qualityin Manufacturing Definition?What does ISO say? InternationalOrganization for Standardization
ISO 8402:1986• This ISO standard defines quality as “the totality of features and characteristics of a product or service that bears its ability to satisfy stated or implied needs.” Does that actually mean anything?
Another Definition• In manufacturing, a measure of excellence or a state of being free from defects, deficiencies, and significant variations, brought about by the strict and consistent adherence to measurable and verifiable standards to achieve uniformity of output that satisfies specific customer or user requirements. Just like education, right?
Quality in Education Definition?High standards, consistently applied,with efficiency or “value for money”
Quality in Education Definition?Or, what D. Green said in 1994What is quality in higher education?Buckingham, U.K.: SRHE and The Open University Press.
What is quality in higher education?• …quality is a relative concept, ....different interest groups or ‘stakeholders’ in higher education have different priorities and their focus of attention may be different. The best that can be achieved is to define as clearly as possible the criteria that each stakeholder uses when judging quality, and for these competing views to be taken into account when assessments of quality are undertaken (p. 17).
Through the use of rubrics and standards related to the quality of online courses (i.e. Quality Matters™), we are sufficiently addressing the questions about e-learning quality60% 1.Real ity40% 2.Myth
Quality Matters is Sufficient Learning• Um, no, it isn’t!!• Quality Matters looks at the quality of course design. Quality• That’s good, but it’s Concerns only one leg holding up the stool. Teaching Design
3 Major Components of e-Quality Learning Level Is High Learning Assessment Teaching Level Course Design Is High Meets Standards
Possible Reasons? Learning Level Is Low Learning AssessmentTeaching Level Course Design Is High Meets Standards
Is This a Problem? Learning Level Is High Learning AssessmentTeaching Level Course Design Is High Below Standard
Independent Study, Perhaps? Learning Level Is High Learning Assessment Teaching Level Course Design Is Low Meets Standards
Beautiful, Just Beautiful Learning Level Is Low Learning AssessmentTeaching Level Course Design Is Low Meets Standards
Maybe, but unlikely Learning Level Is Low Learning AssessmentTeaching Level Course Design Is High Below Standard
Triple Ick – Just Start Over? Learning Level Is Low Learning Assessment Teaching Level Course Design Is Low Below Standard
CCRC Reports: Washington & Virginiahttp://ccrc.tc.columbia.edu/Publication.asp?UID=872
CCRC Reports: Washington & VirginiaWashington State community and technical colleges in the fall of 2004.Students were tracked for nearly five years, until the spring of 2009 “students were more likely to fail or withdraw from online courses than from face-to-face courses”“were slightly but significantly less likely to attain aneducational award or transfer to a four-year institution” http://ccrc.tc.columbia.edu/Publication.asp?UID=872
Say What?• “slightly but significantly less likely”• Translation … “I’m really, really sure that there’s a very small difference.”
CCRC Makes Big Headlineshttp://californiawatch.org/dailyreport/online-community-college-students-more-likely-fail-withdraw-11581
CCRC Makes Big Headlineshttp://www.insidehighered.com/news/2011/07/19/study_finds_higher_dropout_rates_for_community_college_students_who_take_online_courses
What are ReasonableExpectations for Completion Rates for Online Learners? And how do you know? Is completion a measure of quality?
Online Student Satisfaction Is satisfaction a measure of quality?
PSOL BasicsThere are 72 questions that comprise the PSOL.Average completion time is 15 minutes.26 Priorities statements (can add 10)7 Information sources about school/program11 Factors to enroll in the program3 Overall satisfaction questions14 Demographics questions (can add 1)NOTE: questions are answered on a 7-point Likert scale, where 7 is high. 34
How Well Do You Know Your Students? What do They Expect from Online Learning?
2009 PSOL – Summary StatementSo far, how has your college experience met your expectations? 7=Much better than expected Overall score6=Quite a bit better than I expected 5.0 5=Better than I expected 4=About what I expected 3=Worse than I expected2=Quite a bit worse than I expected 1=Much worse than expected 0% 5% 10% 15% 20% 25% 30%
You Already Have Expectations• The questions are … – How clearly have they been communicated? – How easy are they for the following to find? •Students? •Faculty? <= We’ll focus here •Staff and Administration?
A Common Scenario• “What do you mean you expect interaction in online classes?”• “I’ve never heard that before.” This came from a 10-year veteran of online teaching at the college.
Course Design Rubrics Can HelpV. LEARNER INTERACTIONGeneral Review Standard: The effective design of instructor-learner interaction andmeaningful learner cooperation is essential to learner motivation, intellectual commitment,and personal development. (From LSC Course Design Rubric)Specific Review Standards PointsV.1 The course design provides learning activities tofoster instructor-student, content-student, and if 3appropriate, student-student interaction.V.2 The student requirements for course interactionare clearly articulated. 3V.3 Clear standards are set for instructor responseand availability (turn-around time for email, grades 2posted, etc.)
Course Design Rubrics Can Help• But they only go so far.• Did the interactions actually occur?• Did the feedback actually happen in a timely manner?• The design rubric is only part of the answer.
Do You Evaluate Teaching Quality?• If not, why not?• If so, what do you base the evaluation on?• How clear are the expectations of the factors upon which they will be evaluated?
Expectations for (of) Faculty Links to these pages at: http://xlents.com
St. Petersburg College• Examples of posted expectations for faculty:• Respond to email within 24 hours, 5 of the 7 days a week at the instructor’s discretion.• Exam marked within 7 days of due date.• Assignment marked within 7 days of due date• Routine updates to all students every 7 days.
Lawrence Tech University• Examples of posted expectations for faculty:• Check the discussion forum daily and be sure to post responses to student contributions at least four times weekly.
Penn State• Examples of posted expectations for faculty:• The instructor is asked to grade and submit to students all digitally formatted assignments and exams within two business days of receipt.
Colorado CC Online• Examples of posted expectations for faculty:• As mandated by our NCA accreditation, CCCOnline courses cannot be self-paced. Therefore, CCCOnline requires faculty to create a Course Schedule that provides unit, week, and specific activity dates.• Instructor responds individually to all student introductory posts.
Lake Superior• Examples of posted expectations for faculty:• Course Outlines: There is only one official course outline for each course title offered at LSC. This means the following: – Delivery method is not generally considered during the curriculum approval process. – Each class syllabus should include the official course outcomes regardless of the delivery method employed. – Each class syllabus should include the official course description regardless of the delivery method employed.
College expectations of online faculty • Example - Final Exams:• Sample of possible topics: – Online faculty are expected to give final – Course design exams (or due dates for other types of final – Interaction w/students projects) during the final – Online office hours exam period as established and posted by – Feedback/response time the college. – Final exams – The final exam period typically consists of four – Proctored exams weekdays and may or may – Due dates/times not include a weekend. This schedule is posted on – Sick/Personal leave the college website and otherwise makes the information available to all faculty.
Questions without Answers• Optional or required?• Formative or Summative?• Anonymous?• Incentives?• Short or long? (Brief or comprehensive?)• Comparable to on-campus evals?
Optional or Required?• Better practice: Required to opt-out if not interested/willing.• Through conditional release – force students to view the evaluation form – with ability to opt-out if desired.• Good practice: Purely optional (expect low response rates)
Formative or Summative?• Formative eval – mid-term assessment – Students provide feedback to instructor• Summative eval – end-of-term – Students provide feedback to faculty & admin• Better practice: Use both types
Anonymous?• Difficult for formative evals to be anonymous.• Important for students to believe that summative evals are anonymous – but many still won’t believe it.• Good practice: Do not use the course shell to give the eval if it is intended to be anonymous.
Incentives?• Good practice: impress upon students that their participation is an important part of the quality improvement process• Major issue: any kind of tangible incentive eliminates the possibility of true anonymity, or even the appearance thereof.
Short or Long?• An evaluation form that is longer than 10-15 questions will usually result in low response rates.• Good practice: 10 important questions.• If students are allowed to opt-out, they are more likely to opt-in if they see that the survey is very short and their time invested will be very small.
Comparable to On-Campus Evals?• How is this valuable?• What is your purpose for asking the same questions?• Good practice: gather data that are valuable to the online teaching and learning experience – whether or not they are comparable to on-campus data.
A GPA-type Rating Scale• 4 – Strongly agree• 3 – Agree• 2 – Neutral• 1 – Disagree• 0 – Strongly disagree• N/A – Choose not to answer, Don’t know, or Not applicable
Sample Statements• The instructor has created a course layout that is easy to navigate.• This course challenged me intellectually.• The instructor provided feedback in a timely manner.• The feedback received on my coursework was helpful.
More Sample Statements• There was agreement between the posted course objectives (in syllabus) and what was taught in the course.• Policies for determining grades in this course were clearly explained.• Overall, this instructor has created a valuable learning experience for students.
Eval Statements Tied to Rubric?• Instructor presence?• Expectations for interaction?• Effective use of media?• Positive learning environment?• Clear written communication?
Eval Statements Tied to Other?• PSOL Data? –Student assignments are clearly defined in the syllabus –The frequency of student and instructor interactions is adequate. –Faculty are responsive to student needs.• NSSE or CCSSE?
Classroom Observation An Administrator’s Right?