Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Symposium visual final


Published on

  • Be the first to comment

  • Be the first to like this

Symposium visual final

  1. 1. NEERO 45th Annual Confernece Symposium 2.2 | Thursday, 10:15-11:45am Symposium Title: Student Evaluations of Higher and Basic Education Instructors: Three Case Studies Presenters: Tom Neuschafer Nichol Murrray Diane Onorato
  2. 2. Objectives of Presentation Secondary Heading 1. To demonstrate that high school students can recognize good teaching and can collaborate to build an assessment tool which parallels professionally tested instruments 2. To demonstrate evidence that college students must have buy-in and confidence that their opinions are valued and to share strategies for increasing the response rates of evaluations. 3. To demonstrate a process by which an online or blended course can be improved by student evaluations using an instrument which correlates to the course design and management
  3. 3. Student-Designed Evaluations The Evaluation • The need for an evaluation • Wright’s article • The evaluation • What they were told / what they weren’t told • Questions on parenthesis • Grading system • Explanation sections
  4. 4. Student-Designed Evaluations The Evaluation Let’s take a look at the actual evaluation! Sections of the evaluation • About the instructor • About the course • About the program Now let’s hear some of their reasons, in their own words!
  5. 5. Student-Designed Evaluations Student Reasons
  6. 6. Top Complaints of Faculty General • Instructors arriving late or unprepared • Instructors that lecture for an entire class period • Instructors taking too long to respond to messages and provide feedback on work • Instructors who do not provide enough examples or case studies • Instructors who are boring • Instructors with poor speaking skills/Cannot understand instructor • When an instructor thinks that he or she is better and above the students
  7. 7. Top Complaints of Faculty Online Courses • Instructions that are not clear • Waiting too long for grades/feedback • Missing or disengaged instructors (lack of support) • Poor course design – navigation is confusing, hard to find assignments
  8. 8. Top Praises of Faculty • Professionalism – dress, punctuality, behavior • A detailed, organized syllabus that clearly conveys expectations • Flexibility – when an instructor is willing to make changes to the schedule and content when appropriate • Interesting and relevant course material • Varied methods of instruction • When an instructor shows a passion for teaching • Ample interaction with students • Respect and fairness Source: Butler County Community College Office of Academic Affairs
  9. 9. Student Evaluations of Higher Education Faculty Concerns Students are being selective in who they choose to evaluate. Evaluations are now administered online. Ineffective Feedback Students are not taking the evaluations seriously. Instructors do not receive results until after the semester is over.
  10. 10. Online Course Evaluation link Actions Taken • • • Discussed concerns with VP of Academic Affairs Looked at the current evaluation instrument for alignment with my teaching Added additional questions to the evaluation instrument: Please rate the following statement as it applies to you: 1. I feel that my instructor cares about my success in this course. 2. My instructor responds to my questions in a timely manner. 3. My instructor takes time to include lessons that are purposeful and can be applied to real-life situations. 4. My instructor values me as an individual.
  11. 11. Case Studies Fall 2012 Spring 2013 • 4 course sections (81 students) • 4 different approaches to encourage participation • Administered ongoing course feedback survey • 2 course sections (27 students) • Reflective writing activity regarding attitudes toward instructor evaluations • Class discussion about evaluations
  12. 12. Results – Fall 2012 Response Rate 6/25 = 24% Section L01 – Announced that the evaluation window has opened 9/22 = 41% Section L02 – Announced that the evaluation window opened; sent weekly e-mail reminders 9/17 = 53% Section C51– Announced that the evaluation window opened; sent students a link to the online evaluation site (through their personal e-mail) 13/17 = 76% Section B01– Class discussion on instructor evaluations; gave students opportunity to complete evaluations during class time in the computer lab
  13. 13. Respose Rates
  14. 14. Ongoing Feedback Survey Administered 2/3 into semester Important to share and discuss comments with students 1. My instructor is available to me in person, by phone, and via e-mail. 2. Participation in this class is valued. 3. I feel comfortable in this class. 4. Which of the following were the most interest and helpful methods of instruction for you? (methods listed) 5. Comment on any suggestions you may have to improve your experience in this course.
  15. 15. Spring 2013 – Reflective Writing Activity Survey on Faculty and Course Evaluations  How seriously do you approach faculty evaluations?  When the time comes to complete evaluations, do you complete them for all of your instructors, some of them, or none?  Do you think the instructors read and take your comments into consideration?  Think of any negative experiences you have had with a course/instructor. What made it a negative experience?  How would you describe your ideal instructor?  If you could add one question to a faculty evaluation, what would it be?  If you were the President of the college, what methods would you use to evaluate instructors?
  16. 16. Findings from Survey The majority stated that they complete evaluations for the instructors that they either had a very positive experience with or one that they thought was very bad. “I complete them if I had a bad experience with a teacher.” “If they don’t ask, I don’t care. If they do ask, then I’ll do it.” “Students do not learn or care about classes that the instructor does not care about.”
  17. 17. Findings from Survey When describing an ideal instructor: Respectful, on time, easy-to-reach, interesting, varies instruction, posts grades online “One who applies it to real-life rather than just making me memorize” “I like professors who make a lasting impression.” Spring 2013: Results will be available after May 7.
  18. 18. Strategies to Increase Awareness of Faculty Evaluations • Take evaluation questions into consideration when developing your course syllabus, objectives, content, and assessments. • Provide opportunities during class time to complete evaluations. • Reflective writing activities • In-class discussions • Mid-semester surveys • Share findings with students
  19. 19. Evaluation of Blended College Courses Using a Community of Inquiry Survey: Opportunity for Improvement Diane Z. Onorato Indiana University of PA Mercyhurst University, North East, PA Survey retrieved from
  20. 20. Babson Survey Research Group and the College Board’s 2011 report on the status of online learning in the US, 1. 6.1 million college students took an online course in Fall 2010 2. 31 percent of all college students were taking at least one online course in 2011 3. Online education had 10% growth rate; traditional education, a 2% growth rate 4. 65 percent of all reporting tertiary level schools include online education as part of their strategic plan. 5. According to the most recent distance and online learning report from Converge (2011), in five years, if the percentages of increase stay the same online students will outnumber traditional students. 6. 1.8 million online basic education student in 2009-10 (Converge, 2011, p. 5)
  21. 21. Theoretical framework of a Community of Inquiry (CoI) “collaborativeconstructivist… learning experience through the development of three interdependent elements – social, cognitive and teaching presence.”
  22. 22. Definition of Blended Learning Blended learning is the organic integration of thoughtfully selected and complementary face-to-face and online approaches and technologies. Blended learning is seen as an opportunity to fundamentally redesign how we approach teaching and learning in ways that higher education institutions may benefit from increased effectiveness, convenience and efficiency. (Garrison & Vaughan, 2008)
  23. 23. Teaching Presence is “the design, facilitation, and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes” (Anderson, Rourke, Garrison, & Archer, 2001, p. 5). Categories Examples of Indicators Facilitating Discourse Defining, focusing, motivating with knowledge of students’ efficacy; Clarifying ideas, shared experiences; Managing netiquette; Providing feedback and directions Direct Instruction Discussion; Questioning, Injection of new knowledge Instructional Design & Organization Setting curriculum, standards, targets, scaffolding; Calendar; Designing methods, activities; Use of medium; Offering direction for technical support
  24. 24. Teaching Presence: Survey Items Design & Organization 1. Clearly communicated important course goals 2. Clearly communicated important course topics 3. Provided clear instructions on how to participate 4. Clearly communicated important due dates/time frames Facilitation 5. Identifying areas of agreement and disagreement …to help me to learn 6. Guiding class towards understanding course topics …to clarify my thinking 7. Keeping participants engaged and participating in productive dialogue 8. Keeping the course participants on task 9. Encouraged course participants to explore new concepts 10. Reinforced the development of a sense of community Direct Instruction 11. Focus discussion on relevant issues in a way that helped me to learn 12. Feedback that helped me understand my strengths and weaknesses Retrieved from 13. Feedback in a timely fashion
  25. 25. Social presence is “the ability of participants to identify with the community (e.g., course of study), communicate purposefully in a trusting environment, and develop inter-personal relationships(Garrison, 2007; van Shie, 2008, retrieved from by way of projecting their http://communitiesofinqu individual personalities” pt-map.gif) (Garrison, 2009, p. 352). Categories Examples of Indicators—sense of self and audience Emotional Expression Using humor, self-disclosure, emoticons; Expressions of feelings; a sense of safety to speak without unfair treatment Open Communication Continuing threads; Quoting and referring to each others’ statements; Complimenting and agreeing; Risk-free Expression Group Cohesion Encouraging and including others; Using names and greetings
  26. 26. Social Presence: Survey Items Retrieved from /methodology Affective expression 14. Getting to know other[s] gave me a sense of belonging 15. Able to form distinct impressions of [others] 16. Online . . . is an excellent medium for social interaction Open communication 17. Comfortable conversing 18. Comfortable participating in . . . discussions 19. Comfortable interacting with other[s] Group cohesion 20. Comfortable disagreeing . . . while still maintaining sense of trust 21. My point of view was acknowledged by other[s] 22. Develop a sense of collaboration
  27. 27. Cognitive Presence is “the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse” (Garrison, Anderson, & Archer, 2001, p. 11). Category Examples of Indicator–reflection and interaction Triggering Event Sense of puzzlement about a topic; Recognizing the problem but not yet beginning an activity on the topic Exploration Information exchange through discussion; Recognition of divergent views; Offering of suggestions; Brainstorming Integration Connecting ideas; Finding points of convergence; Offering tentative conclusions or solutions Resolution Test and defend solutions; Apply new ideas
  28. 28. Cognitive Presence: Survey Items Triggering event 23. Problems posed increased my interest 24. Activities piqued my curiosity 25. Felt motivated to explore content related questions. Exploration 26. Utilized a variety of information sources 27. Brainstorming and finding relevant information helped me resolve content-related questions 28. Discussions help[ed] me appreciate different perspectives.
  29. 29. Cognitive Presence: Survey Items (continued) Integration 29. Combining new information helped me answer questions 30. Activities helped [with]. . . explanations/ solutions 31. Reflection . . . helped me understand fundamental concepts Resolution 32. Can describe ways to test and apply [course ideas] 33. Have developed [practical] solutions 34. Can apply the knowledge created in this course to my work or . . . other activities
  30. 30. Results from CoI Survey Indicator Class Class Class Class Class Class Class Class Class #1 #2 #3 #4 #5 #6 #7 #8 #9 TP design & organization 8% 0% 0% 4% 10% 11% 10% 10% 15% TP facilitation 5% 25% 4% 6% 17% 12% 8% 13% 17% TP direct instruction 13% 17% 7% 13% 13% 13% 9% 13% 20% SP affective expression 3% 17% 4% 8% 3% 11% 7% 10% 27% SP open communication 0% 0% 4% 8% 10% 11% 9% 13% 20% SP group cohesion 0% 0% 7% 9% 13% 13% 11% 10% 27% CP triggering event 17% 0% 7% 12% 10% 11% 11% 12% 13% CP exploration 17% 0% 4% 13% 20% 14% 9% 13% 20% CP integration 0% 0% 0% 11% 10% 9% 7% 8% 20% CP resolution 10% 0% 7% 10% 13% 15% 11% 13% 33%
  31. 31. Findings & Implications • • • • Scaffolding Time for reflection Clear expectations, models Active and collaborative learning • Manage the learning environment; summarize the conversation/new direction • Provide feedback beyond grades • Students’ self-efficacy
  32. 32. Findings & Implications Terry Ann Morris (2011) 65% completion for online CC and 72% for traditional. Survey says, 1. Enjoyed discussion activities (multiple days) 2. Instructor feedback was the high point (24 hrs.) 3. Don’t like chat rooms, social networking with no instructional purpose 4. Self-efficacy is determining factor in successful completion 5. Varied favorite activities underscore the diversity of the population
  33. 33. To enhance social & cognitive presence, use a Wiki using a scaffold to create a pro/con debate
  34. 34. Cognitive Presence—Arbaugh (2007) most difficult Elder and Paul’s Model of Critical Thinking (2010) Carefully design word tasks to promote critical thinking. They developed a model of questions based on standards that improve learner’s thinking skills and help them question ideas before they decide if they should accept them. Questions include clarification, accuracy, precision, relevance, logic, significance, fairness.
  35. 35. To keep learners engaged, Survey (Bonk)
  36. 36. Teaching presence Welcome email Scavenger hunt Wiki introductions Once upon a time Guest lectures Templates Scaffolding Multiple discussion forums Pride Wall
  37. 37. Importance of Teaching Presence “The body of evidence is growing rapidly attesting to the importance of teaching presence for successful online learning … “The consensus is that teaching presence is a significant determinate of student satisfaction, perceived learning, and sense of community.” Dr. Marti Cleveland-Innes, Athabasca University Dr. Randy Garrison, University of Calgary (2007) 13th Annual Sloan-C Conference
  38. 38. Students must feel a sense of participation • Students know and can articulate what good teaching, learning, and collaboration is at the high school level. • If there is no student evaluation procedure, a high school class can collaborate to produce a suitable instrument. • Students must have buy-in or an accessible, secure means of participation in course evaluation in order for significant numbers to provide usable feedback.
  39. 39. The evaluation must measure class procedures to allow for improvement The evaluation instrument should measure what happens in class and be a standard across similarly constructed classes so all faculty can be evaluated fairly. Results of a course evaluation by a student should provide data for teachers to improve their courses with respect to increased perceived learning and student satisfaction.
  40. 40. NEERO 45th Annual Confernece Symposium 2.2 | Thursday, 10:15-11:45am Symposium Title: Student Evaluations of Higher and Basic Education Instructors: Three Case Studies Thank you for your attention! Tom Neuschafer Nichol Murrray Diane Onorato