Quality Standards and  Organizational Self-Assessment for  Distance Learning and Continuing       Education OrganizationsK...
SLOAN-C Quality Scorecard• Quality of online education is always in question.• How do we determine online program quality?...
SLOAN-C Quality Scorecard  Institute for Higher Education Policy National Education  Association report, Quality On the Li...
Quality Scorecard DevelopmentProcess• Delphi Research method was used.• Panel of experts was identified by Sloan-C and con...
Quality Scorecard DevelopmentProcess
Quality Scorecard DevelopmentProcess
Quality Scorecard DevelopmentProcessResults of process - 9 areas of quality/70 indicators•   Quality Institutional Support...
Implementing the SLOAN-C QualityScorecard• Online interactive scorecard: http://sloanconsortium.org/  node/add/scorecard-o...
Quality Scorecard: Guidelines forScoring9 areas of quality/70 indicators• 0 points = Not Observed. The administrator does ...
Quality Scorecard: Guidelines forScoring9 areas of quality/70 indicators• A perfect score = 210 points.• 90-99% = 189-209 ...
Quality Scorecard: Guidelines forScoring
Category 1: Institutional Support1   Quality Scorecard Indicators of Institutional Support:    4. Governance structure ena...
Implementing the SLOAN-C QualityScorecard Example Justification Comments in the Category of Institutional Support:
Category 2: Technology Support2   Quality Indicators of Technology Support:    2. There is a technology plan with security...
Category 3: Course Dev & ID3   Quality Indicators of Course Development & Instructional    Design:    2. Minimum standards...
Category 3: Course Dev & ID - continued3   Quality Indicators of Course Development & Instructional    Design:    1. Cours...
Category 4: Course Structure4   Quality Indicators of Course Structure:    1. Students have access to an online course syl...
Category 5: Teaching & Learning5   Quality Indicators of Teaching and Learning:    1. Student-student and faculty-student ...
Category 6: Social & Student Engagement 6   Quality Indicators of Social and Student Engagement:     1. Students are provi...
Category 7: Faculty Support7   Quality Indicators of Faculty Support:    1. Technical assistance is provided for both onli...
Category 8: Student Support84.      Quality Indicators of Student Support:     Students can determine if they will be succ...
Category 8: Student Support - continued8   Quality Indicators of Student Support: 4. Minimum requirements for tech skills ...
Category 8: Student Support - continued8   Quality Indicators of Student Support:    1.   Students are instructed in appro...
Category 9: Student Support9   Quality Indicators of Evaluation and Assessment: 5. There are specific standards for online...
Category 9: Student Support                       - continued9   Quality Indicators of Evaluation and Assessment: 5. Recru...
Questions about the SLOAN-CQuality Scorecard1.   How is this different from Quality Matters? QM is a course design rubric ...
Questions about the SLOAN-CQuality Scorecard3.   Can I use the scorecard freely without constraints?     The scorecard is...
Questions about the SLOAN-CQuality Scorecard5.   Can I give the scorecard to others?      Yes, share the link http://tinyu...
Thank You!
Upcoming SlideShare
Loading in …5
×

Sloan-C Quality Score Card

3,231 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,231
On SlideShare
0
From Embeds
0
Number of Embeds
551
Actions
Shares
0
Downloads
45
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • There are rubrics for online courses such as QM, but not a rubric or scorecard for an online education program that clearly defines standards. She saw a need to assist administrators identify elements of quality within an online program and measure: standards, process, guidance, tools. need tools and guidance for assessing their programs. So she set about developing a quality scorecard to evaluate the adminisstration of online education programs.
  • Institute for Higher Education Policy published a report commissioned by the National Education Association in 2000 According to IHEP these benchmarks may assist policymakers, faculty, students, and others in making reasonable and informed judgments with regard to the quality of Internet-based distance education.
  • To determine what quality standards are relevant to online education programs today and would be necessary for creating a quality scorecard. It is a structured flow of information using a series of systematic surveys and reciprocal feedback, used to gain consensus from a panel of experts on the selected topic with the results being fed back to the panel of experts and iterations occur until there is consensus. The study took 18 weeks with 6 survey rounds to gain consensus on what elements indicate quality Panelists were identified by Sloan-C – I was one. Broad representation from a variety of institutions with at least 5 years of experience. 76 experts were identified 43 completed the first round More than 83% of the panel members had 9 or more years of experience in the administration of online education programs 69% had 10 or more years of experience Level of experience of the panel strengthened the validity of the study.
  • This process resulted an instrument or rubric with 9 different areas of quality for a total of 70 indicators.
  • The scorecard is for measuring and quantifying elements of quality within online programs in higher education. It is an easy-to-use tool for online administrators for program evaluation. Hand out the paper version of the scorecard: By evaluating each of the respective quality indicators within the established categories, an online administrator can determine strengths and weaknesses of their program. The identification of the weaknesses can be used to support program improvement and strategic planning. The scorecard may also be used to demonstrate elements of quality within the program as well as an overall level of quality to accrediting bodies. Sloan C has developed an interactive version to use. Your institution must be a member of Sloan-C to access this version of the scorecard. The site allows you to select your score for each quality indicator and provide justification notes and upload artifacts for support such as org charts, faculty training materials, and survey results. Anything you have to substantiate your score. However, the paper version is just fine because more importantly, it is about the justifications and artifacts you can produce to justify your score. The justification supports your self-assessed score. For example, to support a score of 3 - meeting the criteria completely on Institutional Support one could use a flow chart for decision making for #1 and one ’ s org chart for #4 Go to the interactive scorecard to show: You type the title of the program and then choose the name of your institution from the drop down menu. You see the 9 categories to the left, when you click on each one, it brings up the corresponding quality indicators for that category. The first is Institutional Support.
  • The panel agreed on guidelines for scoring.
  • Here is an example of scorecard for one category showing the three points available for each indicator.
  • The institutional support category addresses 4 areas. Review recommendations on Pgs 8-12
  • The technology support category addresses 6 areas. Review recommendations on Pgs 13-20
  • The course development and Instructional design Category addresses 12 areas Review recommendations on (pp. 21-35)
  • The course development and Instructional design Category addresses 12 areas Review recommendations on (pp. 21-35)
  • The course structure Category addresses 8 areas Review recommendations on (pp. 36-44))
  • The teaching and learning category addresses 5 areas Review recommendations on (pp. 45-52;)
  • The social and student engagement Category addresses 1 area Review recommendations on (pp. 53-64)
  • The Faculty Support Category addresses 6 areas Review recommendations on (pp. 55-62)
  • The student Support Category addresses 17 areas Review recommendations on (pp.63-81)
  • The student Support Category addresses 17 areas Review recommendations on (pp.63-81)
  • The student Support Category addresses 17 areas Review recommendations on (pp.63-81)
  • The student Support Category addresses 11 areas Review recommendations on (pp.82-95)
  • The student Support Category addresses 11 areas Review recommendations on (pp.82-95)
  • The interactive tool lets you view your score card by the pillars.
  • Sloan-C Quality Score Card

    1. 1. Quality Standards and Organizational Self-Assessment for Distance Learning and Continuing Education OrganizationsKim A. Scalzo, DirectorSUNY Center for Professional DevelopmentAlexandra M. Pickett, Associate DirectorSUNY Learning Network
    2. 2. SLOAN-C Quality Scorecard• Quality of online education is always in question.• How do we determine online program quality? Kaye Shelton, Ph.D. former Dean of Online Education for Dallas Baptist University and currently Associate Professor, Educational Leadership, Lamar University.
    3. 3. SLOAN-C Quality Scorecard Institute for Higher Education Policy National Education Association report, Quality On the Line: Benchmarks for Success in Internet-Based Distance Education (2000) http://www.ihep.org/Publications/publications-detail.cfm?id=69• Identified 24 benchmarks that are essential to ensure quality in Internet-based distance education.• These 24 IHEP standards were used as the starting point for the study that resulted in the SLOAN-C award- winning and endorsed Quality Scorecard.
    4. 4. Quality Scorecard DevelopmentProcess• Delphi Research method was used.• Panel of experts was identified by Sloan-C and consisted of online education administrators in higher education.
    5. 5. Quality Scorecard DevelopmentProcess
    6. 6. Quality Scorecard DevelopmentProcess
    7. 7. Quality Scorecard DevelopmentProcessResults of process - 9 areas of quality/70 indicators• Quality Institutional Support• Technology Support• Course Development and Instructional Design• Course Structure• Teaching and Learning• Faculty Support• Student Support• Evaluation and Assessment
    8. 8. Implementing the SLOAN-C QualityScorecard• Online interactive scorecard: http://sloanconsortium.org/ node/add/scorecard-online-program • Your institution must be a member of Sloan-C. • Available to SLN education members. http://wiki.sln.suny.edu/display/SLNED/Sloan- C+Institutional+Membership+Benefits• Paper version: http://sloanconsortium.org/quality_scorecard_online_pro gram • Free available to anyone.
    9. 9. Quality Scorecard: Guidelines forScoring9 areas of quality/70 indicators• 0 points = Not Observed. The administrator does not observe any indications of the quality standard in place.• 1 point = Insufficiently Observed. The administrator has found a slight existence of the quality standard in place. Much improvement is still needed in this area.• 2 points = Moderate Use. The administrator has found there to be moderate use of the quality standard. Some improvement is still needed in this area.• 3 points = Meets Criteria Completely. The administrator has found that the quality standard is being fully implemented and there is no need for improvement in this area.
    10. 10. Quality Scorecard: Guidelines forScoring9 areas of quality/70 indicators• A perfect score = 210 points.• 90-99% = 189-209 - Exemplary (little improvement is needed)• 80-89% = 168-188 - Acceptable (some improvement is recommended)• 70-79% = 147-167 - Marginal (significant improvement is needed in multiple areas)• 60-69% = 126-146 - Inadequate (many areas of improvement are needed throughout the program)• 59% and below = 125 pts and below - Unacceptable
    11. 11. Quality Scorecard: Guidelines forScoring
    12. 12. Category 1: Institutional Support1 Quality Scorecard Indicators of Institutional Support: 4. Governance structure enables decision making. 5. Student authentication policies. 6. IP policy - Copyright ownership policy for course materials. 7. Communication of strategic value of online education.
    13. 13. Implementing the SLOAN-C QualityScorecard Example Justification Comments in the Category of Institutional Support:
    14. 14. Category 2: Technology Support2 Quality Indicators of Technology Support: 2. There is a technology plan with security measures in place. 3. Technology delivery is reliable and there is a process system task benchmarking for the servers. 4. There is a central support system for infrastructure. 5. Online course delivery is considered mission critical – such as a commitment to 24X7 hour uptime. 6. A backups system is in place. 7. The development and the use of new technologies is supported.
    15. 15. Category 3: Course Dev & ID3 Quality Indicators of Course Development & Instructional Design: 2. Minimum standards for design and development are in place. 3. Technology used as a tool to support learning outcomes – not just there for bells and whistles. 4. There is a systematic process for the review of course materials. 5. Courses designed to meet learning outcomes. 6. The course learning outcomes are measurable. 7. Appropriate assessments are designed for the course and the audience. For example, it would be difficult for an 18 year old to complete an assignment based on work experience the corporate world but appropriate for an adult learner.
    16. 16. Category 3: Course Dev & ID - continued3 Quality Indicators of Course Development & Instructional Design: 1. Courses are student-centered. 2. Courses across the program have a consistent design/navigation. 3. Courses are designed for faculty-student engagement. 4. The program has a process to evaluate and recommend emerging technologies. 5. There is instructional design support for effective online pedagogy provided to the instructor. 6. Faculty have responsibility for curriculum.
    17. 17. Category 4: Course Structure4 Quality Indicators of Course Structure: 1. Students have access to an online course syllabus that details all aspects of the course & requirements prior to registration. 2. Online students have access to learning resources such as library, writing center, labs, etc. 3. Grading expectations and assignment instructions are provided. 4. Access to technical support for students is clearly provided in the course. 5. Course materials are easy to access and to use. 6. The course adequately addresses special needs students. 7. Student to student collaboration is encouraged, supported, and technically facilitated in course activities, assignments, etc. 8. Consistent document formats used to present course content/share content.
    18. 18. Category 5: Teaching & Learning5 Quality Indicators of Teaching and Learning: 1. Student-student and faculty-student interaction is facilitated. 2. Faculty provide effective, engaging and timely feedback. 3. Effective research methods are taught. 4. Library support is provided. 5. Instructor presence is demonstrated in specific, deliberate, and effective ways.
    19. 19. Category 6: Social & Student Engagement 6 Quality Indicators of Social and Student Engagement: 1. Students are provided a common space for student interaction outside the course.
    20. 20. Category 7: Faculty Support7 Quality Indicators of Faculty Support: 1. Technical assistance is provided for both online course development and delivery. 2. Faculty development and support is provided. 3. Faculty understand Fair Use, ethical issues, and guidelines for plagiarism. 4. Ongoing professional development is provided to continuously improve online teaching and learning. 5. Standards are established for faculty regarding online teaching expectations. 6. Workshops are provided for faculty in the use of emerging technologies.
    21. 21. Category 8: Student Support84. Quality Indicators of Student Support: Students can determine if they will be successful online. 5. Minimum technology and skills standards are provided. 6. Clear program information is provided. 7. Student receive training and information on how to access and use library materials and resources. 8. Technical assistance is provided. 9. There is a process in place to collect and address student questions, problems, bugs, feedback, etc. 10. Students are provided academic, personal, & career counseling.
    22. 22. Category 8: Student Support - continued8 Quality Indicators of Student Support: 4. Minimum requirements for tech skills and equipment are established and made available to students. 5. Support services such as advising, financial aid help, payment services, peer support are provided. 6. Services are provided for students with disabilities. 7. Course material information including (ISBN) numbers are provided at the time of registration. 8. Support services are tailored and provided specifically for the online student. 9. engagement w/ prog./institution is facilitated at a distance.
    23. 23. Category 8: Student Support - continued8 Quality Indicators of Student Support: 1. Students are instructed in appropriate netiquette. 2. Technology changes are supported. 3. Tutoring is available. 4. Students are instructed on how to get help when needed.
    24. 24. Category 9: Student Support9 Quality Indicators of Evaluation and Assessment: 5. There are specific standards for online program evaluation. 6. A variety of data is captured for continuous programmatic improvement. 7. There is a clear regular and systematic process for the evaluation of learning outcomes at both course and program levels. 8. Faculty and student support services are systematically assessed. 9. Course and program retention is assessed.
    25. 25. Category 9: Student Support - continued9 Quality Indicators of Evaluation and Assessment: 5. Recruitment and retention are reviewed. 6. The program demonstrates ADA compliance. 7. Course evaluations are taken into account in faculty performance evaluations. 8. Faculty performance is regularly assessed. 9. Learning outcomes align from course to course. 10. Student feedback is collected and reviewed.
    26. 26. Questions about the SLOAN-CQuality Scorecard1. How is this different from Quality Matters? QM is a course design rubric that focuses only on the instructional design of an online course, the scorecard examines the entire online program.2. How does it fit in with the Sloan-C 5 pillars? The quality standards for the scorecard clearly fall into those same 5 pillars in the Sloan-C Quality Framework of learning effectiveness, access, faculty and student satisfaction, and scale.
    27. 27. Questions about the SLOAN-CQuality Scorecard3. Can I use the scorecard freely without constraints?  The scorecard is free to use.  To use the interactive scorecard on the website, you must be a Sloan-C institutional member.  SLN education members have an affiliated membership that gives them access to Sloan-C resources as a benefit of membership in SLN.4. What if I already have a list of standards that I use?  The scorecard should mesh with other standards easily.  The Sloan-C pillars of quality aligned easily.
    28. 28. Questions about the SLOAN-CQuality Scorecard5. Can I give the scorecard to others? Yes, share the link http://tinyurl.com/qualitysc6. Will my information in the interactive scorecard public? Private, the information you provide will not be shared.7. Why is the scorecard input-based and not output? The scorecard offers broad goals for outputs, but focuses on things administrators can influence.
    29. 29. Thank You!

    ×