EGLS3EGLS3 - Evaluation for Greater Learning Student Survey System        Implementation at Houston Community CollegeDr. A...
EGLS3                         Houston Community CollegeA comprehensive community college serving the Houston areaOver 75,0...
EGLS3            Student Evaluation of Instruction – History  Paper-Based Student Evaluation of Instruction (SEOI) survey ...
EGLS3               HCC’s SEOI Survey Paper Distribution System  IR manually prepared and routed more than 5,000 packets a...
EGLS3Student Evaluation of Instruction (SEOI)– Paper Base Instrument                                                      ...
EGLS3EGLS3 Student Evaluation Survey Instrument  Houston Community College                                   EGLS3 – Evalu...
EGLS3                 Important EGLS3 Features from UNT’s SETE  These are the factors or elements of teaching effectivenes...
EGLS3                SEOI-EGLS3 Survey Instrument Comparison     Student Evaluation of       Instruction(SEOI)            ...
EGLS3                                      Early Stages  Vice Chancellor of Planning and Institutional Effectiveness has F...
EGLS3                Faculty Governance and Endorsement  Faculty Senate President requests the evaluation of SETE for repl...
EGLS3                    Seeking Additional EndorsementApproval of Pilot from Vice Chancellor of Instruction, Deans’ Counc...
EGLS3        Student’s evaluation process, the transition from paper to onlineIR Partnership with Faculty SenateIR Respons...
EGLS3       Solutions for Log-In Process found with PS IntegrationHCC - SmarterServices People Soft login Integration of E...
EGLS3 Student’s evaluation process, the transition from paper to onlineHCC - SmarterServices People Soft login Integration...
EGLS3    Integration and Responsibility of IR in EGLS3 Implementation  Institutional Research liaison between SmarterServi...
EGLS3                         Fall 2011 - Full Implementation Facts Only Regular Term and Second Start Classes surveyed, i...
EGLS3                                FALL 2011, ImplementationFull Implementation Fall 2011. District-Wide  Steering Commi...
EGLS3                              Spring 2012-Next Steps  Spring 2012 Next Steps  Immediate: Increase response rate  As s...
EGLS3                                     About SmarterServices  Incorporated in 2002.  Served over 500 educational instit...
EGLS3                               About UNT’s SETE™ InstrumentBenefit from extensive research at the University of North...
EGLS3  Student’s Perspective within SmarterSurveys™
EGLS3                    Student’s Perspective within SmarterSurveys™18 Standard SETE Questions16 Likert scale questions (...
EGLS3                    Student’s Perspective within SmarterSurveys™Custom Question can be added to SETE.Custom questions...
EGLS3                    Student’s Perspective within SmarterSurveys™Confirmation pop-up is presented to Student’s for ver...
EGLS3                   Student’s Perspective within SmarterSurveys™After completing a survey, the course is marked as “Co...
EGLS3                     Faculty’s Perspective within SmarterSurveys™HCC choose to suppress results until the end of the ...
EGLS3                                 SETE™ Data Processing FlowFaculty and Student Attributes          + Grades          ...
EGLS3                   Faculty’s Perspective within SmarterSurveys™Once survey results open, faculty can view course resp...
EGLS3      Faculty Reports within SmarterSurveys™Individual Course Response Distribution Report – Per Course              ...
EGLS3      Faculty Reports within SmarterSurveys™Individual Course Response Distribution Report – Per Course              ...
EGLS3   Faculty Reports within SmarterSurveys™         EGLS3 Score Report – Per Term
EGLS3    Dean Reports within SmarterSurveys™
EGLS3    Dean Reports within SmarterSurveys™
EGLS3        34
EGLS3                Questions….           Contact Information:Alan Ainsworth 713.718.8709; alan.ainsworth@hccs.edu Mario ...
Upcoming SlideShare
Loading in …5
×

HCCS - Evaluation for Greater Learning Student System

489 views
306 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
489
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Introduction. New Instrument and onlineTheEGLS3 (Evaluation for Greater Learning Student Survey System) is Houston Community College’s version of the SETE (The Student Evaluation of Teaching Effectiveness) developed by the University of North Texas and implemented by SmarterServices.
  • (MH)
  • (MH)
  • -MH
  • No only Instructional questions(MH)
  • -MH
  • - MH
  • -MH
  • (MH)Develop the timeline and monitor progressInteract with SmarterServices and other HCC’s Department, Data Submission Faculty TrainingData Definition, Submissionbecause students has to login thought Smarter Services using their college emails account.
  • (MH)IT and SmarterServices develops the application that allows Faculty and students to access the survey from People Soft. They login only once.OIR interacted with Faculty Senate, Information TechnologyHelpDeskCommunications HCC Distance Learning OIR:Some task
  • (MH)IT and SmarterServices develops the application that allows Faculty and students to access the survey from People Soft. They login only once.OIR interacted with Faculty Senate, Information TechnologyHelpDeskCommunications HCC Distance Learning OIR:Some task
  • (MH)OIR interacted with Faculty Senate, Information TechnologyHelpDeskCommunications HCC Distance Learning OIR:Some task
  • (MH)
  • HCCS - Evaluation for Greater Learning Student System

    1. 1. EGLS3EGLS3 - Evaluation for Greater Learning Student Survey System Implementation at Houston Community CollegeDr. Alan Ainsworth Dr. Mac AdkinsFormer HCC Faculty President Senate President SmarterServices LLC Mario Heredia Jeff Worford Director Research & Lead Developer Support Services SmarterSurveys™ Product February 23, 2012 1
    2. 2. EGLS3 Houston Community CollegeA comprehensive community college serving the Houston areaOver 75,000 students credit, non credit in Fall 2011One of highest number of international students of all community colleges in the country 2
    3. 3. EGLS3 Student Evaluation of Instruction – History Paper-Based Student Evaluation of Instruction (SEOI) survey has been in place for 27years, revised incrementally. For Face to face classes, the SEOI was the HCC’s paper method to administer the student evaluation For online classes Distance Education used their own online survey Structure. Four part-time staff plus a supervisor working for two months. Large work room Given the current technology, Institutional Research sought -Online administration methodologies -New Instrument 3
    4. 4. EGLS3 HCC’s SEOI Survey Paper Distribution System IR manually prepared and routed more than 5,000 packets and 100,000 SEOI instruments eachsemester. Each packet contains the SEOI instrument and instructions for faculty and proctors. The packets were sent to their respective campuses administrators. Campuses administrators were responsible for distributing the materials to the Instructors and collecting from the proctors (who administer the survey to the students). Once IR received the packets, entered them manually in the system to track the return rate. IR cleaned the packets, scanned the surveys and generated data file. The report was run in SPPS. The final report was saved on disks and sent to the College Administrators for manual distribution.Students’ hand written comments were sent back to faculty. Process is cumbersome, labor intensive and costly. SEOI’s means report results were not used systematically in the evaluation process. Return rates have been an issue, but in Spring 2010 the rate of return increased to 95% and the response rate was of 73%. 4
    5. 5. EGLS3Student Evaluation of Instruction (SEOI)– Paper Base Instrument 5
    6. 6. EGLS3EGLS3 Student Evaluation Survey Instrument Houston Community College EGLS3 – Evaluation for Greater Learning Student Survey System Organization and Explanation of Materials Strongly Disagre Strongly Agree Agree e Disagree 1. My Instructor explains difficult material clearly.     1. My Instructor communicates at a level that I can     understand. 1. My Instructor makes requirements clear.     1. My Instructor identifies relationships between and     among topics. Learning Environment Strongly Disagre Strongly Agree Agree e Disagree 1. My Instructor establishes a climate of respect.     1. My Instructor is available to me on matters pertaining     to the course. 1. My Instructor respects diverse talents.     1. My Instructor creates an atmosphere in which ideas     can be exchanged freely. Self-Regulated Learning Strongly Disagre Strongly Agree Agree e Disagree 1. My Instructor gives assignments that are stimulating     to me. 1. My Instructor encourages me to develop new     viewpoints. 1. My instructor arouses my curiosity.     1. My Instructor stimulates my creativity.     Overall Opinions Strongly Disagre Strongly Agree Agree e Disagree 1. I like this instructor.     1. I am interested in this subject.     1. I think the classroom was appropriate for this class.     1. I would recommend a course taught by this     instructor. General Information 1. This class is:  A Required Course  An Elective  I am not sure 1. What grade do you expect to earn in this course? A B C D F 6
    7. 7. EGLS3 Important EGLS3 Features from UNT’s SETE These are the factors or elements of teaching effectiveness that are being measured:Factor 1: Organization and explanation of materialsFactor 2: Learning EnvironmentFactor 3: Self-regulated Learning Four questions are used in the EGLS3 validity analysis. Scale scores (the weighted responses) are reported for each of the three domains. An overall score is also computedThe following are sample scale ranges for EGLS3 7
    8. 8. EGLS3 SEOI-EGLS3 Survey Instrument Comparison Student Evaluation of Instruction(SEOI) EGLS3 - Survey Instrument• Paper-Based • Is implemented online to allow• Home Grown Survey flexibility, transparency, and• Included non-instructional Questions accountability• Library and course content questions • Provides a measure of teaching• No Validated effectiveness as perceived by students.• Means report • Allows college-wide comparison in key• IR cleaned the packets, scanned the areas surveys and generated data file. The • Facilitates student evaluations of their report was run in SPPS. instructors (not the course or the• The final report was saved in a disk and content) sent to the College Administrators. Hand- • The EGLS3 scale scores can be used to written comments returned to faculty. make assessments for continual• Process was cumbersome, labor intensive improvement of the institution and costly. • Results available on line for individual• The results were not used in a systematic faculty. evaluation process. • User built their own reports 8
    9. 9. EGLS3 Early Stages Vice Chancellor of Planning and Institutional Effectiveness has Faculty and OIR representativesvisit University of North Texas to discuss SETE. HCC Administration and Faculty Senate decide to move forward with assessing SETE for possible online evaluations. 9
    10. 10. EGLS3 Faculty Governance and Endorsement Faculty Senate President requests the evaluation of SETE for replacement of SEOI byEducational Affairs Committee Faculty Senate sees reports of SETE and votes unanimously to support pilot program for Spring 2011 10
    11. 11. EGLS3 Seeking Additional EndorsementApproval of Pilot from Vice Chancellor of Instruction, Deans’ Councils.Forums held for information and feedback 11
    12. 12. EGLS3 Student’s evaluation process, the transition from paper to onlineIR Partnership with Faculty SenateIR Responsibility: Develop and implement the Operational planFaculty Senate President: Navigating Faculty Governance and EndorsementPilot Implementation Spring 2011 . Two Locations were selectedOutcome: Response Rate 20%Follow up: Student Focus GroupsFindings: Login process cumbersome 12
    13. 13. EGLS3 Solutions for Log-In Process found with PS IntegrationHCC - SmarterServices People Soft login Integration of EGLS3 - Student Center 13
    14. 14. EGLS3 Student’s evaluation process, the transition from paper to onlineHCC - SmarterServices People Soft login Integration of EGLS3 - Faculty Center 14
    15. 15. EGLS3 Integration and Responsibility of IR in EGLS3 Implementation Institutional Research liaison between SmarterServices and the HCC departments participating in the EGLS3. Develop timeline and monitor progress Develop and review documentation with SmarterServices, IT/Costumer Support Develop and review communication plan with Communications Department. Several media channels used to notify the institution: meetings, email, paper, social media,web page, plasma screens, posters, etc. 15
    16. 16. EGLS3 Fall 2011 - Full Implementation Facts Only Regular Term and Second Start Classes surveyed, including Distance Education and Dual Credit Data submitted: 6,460 classes with 140,800 enrollment. System runs smoothly 23,005 responses with a Response Rate of 16%Technical Issues Pop –ups blocked in College Computer Labs and other computers. System Slow due to lack of services running on the web server.Reporting Available on our Web Site Faculty, Administrator access Administrators can Build their Own ReportsFollow up Faculty Focus Groups 16
    17. 17. EGLS3 FALL 2011, ImplementationFull Implementation Fall 2011. District-Wide Steering Committee in place Teams and strategy are developed Operations Strategy : How do we implement? What processes and procedures need to be developed Communication Strategy: How do we communicate? What and where? What are the benefits? How is it different? How does it relate to our vision? Feedback Strategy : How will this improve student learning? How will it improve teaching effectiveness? 17
    18. 18. EGLS3 Spring 2012-Next Steps Spring 2012 Next Steps Immediate: Increase response rate As survey reports became available, faculty and administrators will be more involved inthe process Develop committees to work to integrate this students evaluation with HCC Faculty/Staffevaluation system PEP. Redevelop PEP Research Project 18
    19. 19. EGLS3 About SmarterServices Incorporated in 2002. Served over 500 educational institutions including K-12, technical colleges, career colleges, communitycolleges, private colleges, universities, and corporations. Some of the other solutions provided by SmarterServices include SmarterMeasure, SmarterFaculty,and SmarterProctors.
    20. 20. EGLS3 About UNT’s SETE™ InstrumentBenefit from extensive research at the University of North Texas.Control for bias of 24-variables Student grades Time of course Delivery systemTrack improvement with a standardized effectiveness score (1-1000 scale)Useful for benchmarking and longitudinal analysis of faculty effectivenessMeet state reporting guidelines
    21. 21. EGLS3 Student’s Perspective within SmarterSurveys™
    22. 22. EGLS3 Student’s Perspective within SmarterSurveys™18 Standard SETE Questions16 Likert scale questions (4 per Category)2 General Information questions
    23. 23. EGLS3 Student’s Perspective within SmarterSurveys™Custom Question can be added to SETE.Custom questions can be Likert, open ended, multiple choice, etc.HCC choose to add 5 open ended questions to their evaluation
    24. 24. EGLS3 Student’s Perspective within SmarterSurveys™Confirmation pop-up is presented to Student’s for verification upon submission of a survey.
    25. 25. EGLS3 Student’s Perspective within SmarterSurveys™After completing a survey, the course is marked as “Completed”Any remaining surveys can be taken immediately or at another time
    26. 26. EGLS3 Faculty’s Perspective within SmarterSurveys™HCC choose to suppress results until the end of the survey period.Faculty can still see their completion percentages, survey status, and results release date.
    27. 27. EGLS3 SETE™ Data Processing FlowFaculty and Student Attributes + Grades EGLS3 Scores Generated Data Processed Faculty ID, Course ID & Scores Survey Responses
    28. 28. EGLS3 Faculty’s Perspective within SmarterSurveys™Once survey results open, faculty can view course responses and also pull aggregate reports for the term.
    29. 29. EGLS3 Faculty Reports within SmarterSurveys™Individual Course Response Distribution Report – Per Course Header and Likert Responses
    30. 30. EGLS3 Faculty Reports within SmarterSurveys™Individual Course Response Distribution Report – Per Course Open Ended Responses
    31. 31. EGLS3 Faculty Reports within SmarterSurveys™ EGLS3 Score Report – Per Term
    32. 32. EGLS3 Dean Reports within SmarterSurveys™
    33. 33. EGLS3 Dean Reports within SmarterSurveys™
    34. 34. EGLS3 34
    35. 35. EGLS3 Questions…. Contact Information:Alan Ainsworth 713.718.8709; alan.ainsworth@hccs.edu Mario Heredia 713.718.8627; mario.heredia@hccs.edu SmarterSurveys™ Info 877.499.SMARTER (7627); info@smarterservices.com 35

    ×