Student Evaluation of Teaching: Creating Reports with Class Climate


Published on

This course is designed to provide the advanced knowledge, procedures, and hands-on experience required for the successful implementation and operation of online course evaluations with the Class Climate software. Topics for this workshop include: creating faculty and departmental reports. This course is recommended for personnel responsible for creating, managing and data collection for course evaluation surveys and already have basic skills with Class Climate.

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Student Evaluation of Teaching: Creating Reports with Class Climate

  1. 1. Center for Excellence in Learning and Technology Student Evaluation ofTeaching: Creating Reports with Class Climate Laura Bestler, Instructional Support Specialist 294-5357 • eLearning: Class Climate:
  2. 2. Objectives • Introductions • Demonstrating different reports • Preparing reports • Creating reports • Sending results
  3. 3. Establishing Indicators• Effects Every Report in Class Climate• Indicators May be Added/Changed with Running Surveys• Allows for 2 – 11 Options With or without Abstention
  4. 4. Establishing Indicators• Question Group(s) = Question Categories • Indicators are an Attribute of the Group • Indicators May be Added while Questionnaire in use• Indicators Calculate the Mean Value of all Scaled Questions within the Group (Subtotal)• Any Group Containing a Scaled Question Should have the Indicator Defined• All Questions to be Considered in the Sub- Total Calculation MUST Match the Number of Options Set in the Indicator Value for that GroupQuestions with a Different Scale (number ofoptions) will be individually calculated butexcluded from the Sub-Total Calculation.
  5. 5. Indicators in Questionnaires
  6. 6. Weighted Computation of Indicators• Total Number of Answers for Each Question used for Computation of the Indicator Value• Example: Question Group with Two (2) Questions, each with a 5-Point Scale, 4 respondents • Q1=4 Respondents / 5, 5, 4, 3 (Question Average = 4.25) • Q2=3 Respondents / 5, 5, 5 (Question Average = 5.0) • Indicator = 5 + 5 + 4 + 3 + 5 + 5 + 5 = 32 Divided by # of Responses = 32 / 7 Average of Question Group = 4.57 • Not 4.25 + 5 = 9.25 / 2 = 4.62• Question Group Averages are Averaged into Global Indicator
  7. 7. Calculation of the Average vs. Median• Total Number of Answers for Each Question used for Computation of the Median• All question responses are sorted• Example of a Question with 5 respondents (odd #) • Q1=5 Respondents / 5, 5, 4, 3, 1 • Average of Question = 3.6 (5+5+4+3+1=18/5=3.6) • The Median is 4 (the absolute middle number)• Example of a Question with 4 respondents (even #) • Q2=4 Respondents / 5, 4, 2, 2 • Average of Question = 3.75 (5+4+2+2=13/4= 3.75) • The Median is 3 (4+2=6/2=3)
  8. 8. Calculation of the Average vs. Median• Always specify when a Group contains scaled questions• Question Scales within a Group should have Identical Options • 2 – 11 Options • With or Without Abstention • Mirrored Scales Required if Positive on Left • Should be set at Question Level if „Traffic Lights„ will be used
  9. 9. Report Creator• Change your role to [Report Creator]• Click on [Folders] Think: File Cabinet• Click on [Create new Folder] • Name folder [Example: F2011]
  10. 10. Report Creator: Single Category• Include / Exclude questions as desired• Produce Reports by Single Category • First 3 options in pull-down • Only ONE Choice per Selection • SubUnit: Pick: 1 Subunit / 1 Period • Program of Study (if Activated and populated): Pick: 1 POS / 1 Period • Instructor Profile – Pick: 1 Instructor / 1 Period
  11. 11. Report Creator: Any CompilationAny Compilation (Combination)Last 2 options in pull-down • Most Flexible • Combine Any Surveys (except General Surveys) • Choose multiple criteria from a standard choice box • Allows Multiple Subunits / Periods / Types / Questionnaires
  12. 12. Report Creator: Profile Line• Create profile line emails (may be printed or saved) • Compares scaled questions across selected surveys • 2-3 optimal (more becomes difficult to read) Compare Instructor to their Up to 8 (eight) Up to 8 (eight) DEPT Peers courses taught by Instructors who single Instructor taught same/similar Course(s) Line 1 Entire Faculty Course 1 Instructor A Line 2 DEPT Averages Course 2 Instructor B Line 3 Instructor/Course Course 3 Instructor C Averages Line 4 Total Average of Total Average of Courses 1,2,3 Instructor A,B,C
  13. 13. Report Creator: Profile Line
  14. 14. Mapping Questions Across Surveys• Used to Combine Surveys with Similar Questions • Automatic • Use When: • Questionnaires are Identical • Manual • Use When: • Questionnaires are Somewhat Similar • Worded Differently, and Re-Sequenced • You decide which questions are similar enough to average together • Ability to define the number of character differences
  15. 15. Mapping Questions Across Surveys• Include Answers to Open Questions • Not Extremely Useful when Combining Different Instructors / Courses• Weighted Averages • Weighted is Default • Number of Responses will be used to Calculate the Averages• Un-Weighted Averages • Number of Surveys will be used to Calculate the Averages • Not Eligible for Export to CSV nor SPSS • Raw Score Data is not Provided, only the Average • Not Widely Used
  16. 16. Weighted vs. Unweighted• By default, the reports are summarized weighted, meaning that the calculation is effected according to the number of returns.• Example: If you summarize a survey with 100 return questionnaires and one with 10, the survey with 100 questionnaires counts for more than the one with 10.• Mathematically expressed: All individual results are summed for each question (each single value that was selected) and divided by the total number of returns (in this example: 110).
  17. 17. Weighted vs. Unweighted• You must explicitly select unweighted compilation when creating a report. Unweighted calculation precludes the possibility to adequately undertake further processing in statistics programs.• With unweighted compilations, each survey counts as 1, regardless of how many questionnaires have been processed for a given survey.• Mathematically expressed: You add the respective total average of each question and divide by the number of the surveys that you are summarizing (in the example above: by 2).• Individual response data is lost.
  18. 18. Evaluate Subgroup Data • Used with SINGLE-CHOICE questionsExample: If your survey asked: “Reason fortaking this course” and the responses included:“Required” and “Elective”, you could produce areport which included ONLY the responses fromthose who attended as a required part of theirmajor.
  19. 19. Opens a 2nd screen toEvaluate Subgroup Data allow multiple criteria Name report as desired
  20. 20. Evaluate Subgroup: Multiple Criteria Name report as desired
  21. 21. Merging Survey Data• Combine Reports Created by Evaluating a Subgroup Example: Combine any reports, including those reports created using: Evaluate Subgroup
  22. 22. Single Profile Line Comparison• Combine any 2 Subgroup Reports Example: Compare the responses from the 2 reports: “Required” and “Elective”, you could see which group evaluated the course higher / lower.
  23. 23. Report for Comparison onQuestionnaire Report for Comparison
  24. 24. Report for Comparison onQuestionnaire
  25. 25. Report for Comparison onQuestionnaire The “Hook” Profile line into standard Report
  26. 26. Administrator Reports• Change your role to [Administrator]• Note: You are unable to do anything with surveys/questionnaires as a Report Creator
  27. 27. “Printing” Results • Click on [Batch Events] (Left Menu) Creates PDFs toFor surveys that have already been save on your localevaluated, the evaluations of a selection CPU or networkof courses can be requested. There aretwo possibilities:• The PDF reports of the selected courses are created together in a common PDF file. This is advantageous for example, when the PDF reports for multiple courses are to be printed at the same time.• The PDF reports requested are summarized as single PDF files in a ZIP file. This is advantageous for example, when the single PDF files are to be passed on in electronic format, deposited in a public folder or, when one wants to print them individually.
  28. 28. IMPORTANT: Discuss with your “Sending” Results Department Chair how the reports should be sent to Instructors • Click on [Batch Events]Send PDF ReportsYou can determine that an instructor receives allevaluations of his courses accumulated in one email.To do this, please activate the option “Reports of allcourses of an instructor collected in one email”. Shouldthis not be the case, the instructor will receive aseparate email for each course. Please observe thedefined maximum size for email attachments (“SystemSettings/Class Climate Settings/Network Settings/Maximum document size”). In “System Settings/TextTemplates” the relevant email text may be edited(“EMAIL: Results of a survey” and “E-MAIL: Results ofa survey (Summary)”).
  29. 29. IMPORTANT: Discuss with your “Sending” Results Department Chair how the reports should be sent to Instructors • Click on [Batch Events]Sending PDF Reports as Instructor Profiles• You can send your instructor profile for surveys that have already been evaluated as a batch email. By activating the option “Include courses with secondary instructors in instructor profiles”, courses with additional report recipients are also included.• You can determine whether or not the answers to open questions are to be included in your instructor profile. To do this, please activate the option “Include answers to open questions”.• If the respective instructor, in addition to his profile, is to receive a detailed report for each of his courses, simply activate the option “Also send single PDF reports”.• Furthermore, you can determine whether profiles and PDF reports are sent solely to the respective instructor (“Send to instructors”) or only to a specified email address (“Send all to the following email address”).
  30. 30. CELT Training Adapted from: • Oliver, M. (2011, October 25). Advanced topics in Class Climate: The Report Creator(s) Version 4 & 5 [Scantron webinar]. • Scantron Corporation. (2011). Class Climate Course Evaluation System User Guide v5.0. Eagan, MN: Scantron Corporation.