Establishing Quality Standards For Faculty Development In Teaching Online Courses


Published on

The University of Dubuque (UD) completed its second year of offering online courses to undergraduate students. In this time, UD has learned several valuable lessons in the delivery of quality online courses that include faculty development, support services, quality assurance checks, and 360-assessment. This presentation addresses several of these lessons.

Published in: Technology, Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Establishing Quality Standards For Faculty Development In Teaching Online Courses

  1. 1. ESTABLISHING QUALITY STANDARDS FOR FACULTY DEVELOPMENT IN TEACHING ONLINE COURSES: LESSONS FROM THE TRENCHES Gail Hodge Associate Dean for Academic Affairs Director of On-line Programming Jonathan Helmke Faculty Librarian
  2. 2. WHO WE ARE <ul><li>University of Dubuque </li></ul><ul><ul><li>Small, private, Christian college </li></ul></ul><ul><ul><ul><li>1400 undergraduate </li></ul></ul></ul><ul><ul><ul><li>123 graduates (MBA and MAC) </li></ul></ul></ul><ul><ul><ul><li>136 Seminary </li></ul></ul></ul><ul><ul><li>Mission driven </li></ul></ul><ul><ul><li>Began offering undergraduate online courses in Fall 2007 </li></ul></ul>
  3. 3. GOALS OF ONLINE PROGRAM <ul><li>Accessibility : Provide Students with alternative ways to earn course credits (avoid institutional time conflicts) </li></ul><ul><li>Vocation Preparation : Build upon career skills for working in online environments </li></ul><ul><li>Revenue Generating : Provide course opportunities to non-traditional students and non-degree seeking students who are new to the University </li></ul>Presented to Board of Trustees
  4. 4. The goal for online education is to provide mission-driven, competitively marketed, income generating, and clearly branded undergraduate and graduate courses, as well as selected graduate degree programs, to enhance and expand educational opportunities. Written by Online Advisory Committee
  5. 5. STANDARDS NEED TO REFLECT OUR MISSION: BRANDING <ul><li>What are we best known for? </li></ul><ul><li>Incorporate these same attributes into the online product </li></ul><ul><ul><li>Personal attention </li></ul></ul><ul><ul><ul><li>Two voice contacts </li></ul></ul></ul><ul><ul><li>Learner-centered pedagogy </li></ul></ul><ul><ul><ul><li>Assignments that help students explore and discover various concepts/theories (engage in active learning) </li></ul></ul></ul><ul><ul><ul><li>Students interact with peers, instructors, and others </li></ul></ul></ul><ul><ul><li>Serve first-generation students </li></ul></ul><ul><ul><ul><li>Diverse learning styles </li></ul></ul></ul><ul><ul><ul><li>Teach to different senses: Text, graphics, audio, interactive discussions </li></ul></ul></ul><ul><ul><ul><li>Academically under-prepared when entering college </li></ul></ul></ul>
  7. 7. FRAMEWORK FOR TODAY‘S PRESENTATION <ul><li>Sloan-C Pillars </li></ul><ul><li>Lessons Learned integrated w/ Sloan-C Pillars </li></ul><ul><li>Gap analysis </li></ul><ul><li>360 Assessment </li></ul><ul><li>Recommendations for next steps </li></ul>Learning Effectiveness Cost Effectiveness Access Faculty Satisfaction Student Satisfaction
  8. 8. 1: IDENTIFY CHAMPIONS – EVEN IF IT MEANS BUYING THEIR AFFECTION <ul><li>Recruitment of online Faculty </li></ul><ul><ul><li>Offer course developmental dollars on top of regular teaching compensation to encourage faculty to come out of the woodwork – our pioneers. </li></ul></ul><ul><li>Open Learning Management System (LMS) to face-to-face (F2F) classes </li></ul><ul><ul><li>Builds faculty skills in authoring content and knowledge of using LMS </li></ul></ul><ul><ul><li>Exposes students to LMS </li></ul></ul>Sloan-C Pillar: Faculty Satisfaction and Cost Effectiveness
  9. 9. 2: THE GREAT DIVIDE: DIGITAL IMMIGRANTS VERSUS DIGITAL NATIVES <ul><li>Faculty nervous about students being ahead of them technologically </li></ul><ul><ul><li>Unsure if they can field/trouble-shoot technical questions </li></ul></ul><ul><ul><li>Digital natives catch-on to the LMS fairly quickly </li></ul></ul><ul><ul><ul><li>No surprise as Digital Natives are the same audience who use social networking sites, like Facebook TM </li></ul></ul></ul><ul><ul><li>Digital Natives are also the drivers, pushing faculty into adopting the LMS </li></ul></ul><ul><li>Techno-phobia reinforces need for strong training program for faculty </li></ul><ul><ul><li>Beginning, middle, and end of academic year </li></ul></ul><ul><ul><li>LMS Support Group meetings throughout year (MUG) </li></ul></ul>Sloan-C Pillars: Cost Effectiveness, Faculty Satisfaction and Student Satisfaction
  10. 10. AFTER YEAR 1 <ul><li>We began developing standards for Faculty Development </li></ul><ul><ul><li>Established set training requirements </li></ul></ul><ul><ul><li>Instituted Peer and Self Review for assessment </li></ul></ul><ul><ul><li>Incorporated course and student support standards/requirements </li></ul></ul><ul><ul><ul><li>Common LMS interface </li></ul></ul></ul><ul><ul><ul><ul><li>“ Wall” consistency </li></ul></ul></ul></ul><ul><ul><ul><ul><li>First block of content above the fold line </li></ul></ul></ul></ul><ul><ul><ul><li>Welcome video to introduce students to course (makes instructor a “real” person) </li></ul></ul></ul><ul><ul><ul><li>Two voice contacts with each student </li></ul></ul></ul><ul><ul><ul><li>All content needed to be developed prior to start of class </li></ul></ul></ul>
  11. 11. FACULTY TRAINING ON LMS <ul><li>Online Faculty: Mandatory 9 hours of training </li></ul><ul><ul><li>1 hour – Quality standards overview/update </li></ul></ul><ul><ul><li>6 hours of LMS training, from intro to advanced applications </li></ul></ul><ul><ul><li>2 hours of resource training (e.g. MediaSite, Skype) </li></ul></ul><ul><li>LMS User Group </li></ul><ul><ul><li>Meets twice a semester </li></ul></ul><ul><ul><li>Members discuss how-to’s/issues they have with LMS </li></ul></ul><ul><li>F2F Faculty: Optional 2 – 6 hours of training </li></ul><ul><ul><li>August and May (before and after academic year) </li></ul></ul><ul><ul><li>2 hours – introductory </li></ul></ul><ul><ul><li>2 -4 hours – advanced applications </li></ul></ul>
  12. 12. TRAINING SCHEDULE FOR ONLINE INSTRUCTORS <ul><li>Offered Between Semesters </li></ul><ul><li>Week 1 </li></ul><ul><ul><li>1 hour of QA tips, morning (repeated in Spring for past instructors) </li></ul></ul><ul><ul><li>2 hours of Introduction to LMS, afternoon </li></ul></ul><ul><li>Week 3, morning </li></ul><ul><ul><li>2 hours of Intermediate LMS </li></ul></ul><ul><li>Week 3, afternoon </li></ul><ul><ul><li>2 hours of Advanced LMS </li></ul></ul><ul><li>Week 5, morning </li></ul><ul><ul><li>1 hours of training on instructional design tools (MediaSite and Skype) </li></ul></ul><ul><li>Week 5, afternoon </li></ul><ul><ul><li>1 hours of training on library resources (eReserve) </li></ul></ul>
  13. 13. 3: PROOF IS IN THE PUDDING: INSIST ON ADHERING TO MILESTONES IN THE COURSE DEVELOPMENT SCHEDULE! <ul><li>Courses needs to be developed and QA’d (reviewed) before they go public </li></ul><ul><ul><li>Adherence to standards </li></ul></ul><ul><li>Seems like a no-brainer; but was difficult to enforce </li></ul><ul><ul><li>Some faculty not confident in building whole course without testing initial modules </li></ul></ul><ul><ul><li>Procrastination </li></ul></ul><ul><ul><ul><li>Some faculty approached course development like their f2f class -- prepped day before, or they think they can wing it </li></ul></ul></ul><ul><li>SUGGESTION: Consider developing the online course for delivery in a F2F class before it is offered online </li></ul>Sloan-C Pillar: Learning Effectiveness; Access
  14. 14. 4: BE PREPARED TO THROW-OUT THOSE OLD NOTIONS ABOUT STUDENT LEARNING <ul><li>Behaviorist vs Constructivist </li></ul><ul><li>Move from “Sage on the Stage” to “Guide on the Side” </li></ul><ul><ul><li>Student-centered learning is putting students in charge of their learning </li></ul></ul><ul><ul><li>Role of instructor is to provide resources </li></ul></ul><ul><ul><ul><li>Develop content that engages students </li></ul></ul></ul><ul><ul><ul><li>Provide strong resources (e.g. library) </li></ul></ul></ul><ul><ul><li>Take advantage of Web 2.0 and interactive media </li></ul></ul><ul><li>Measure/assess student engagement </li></ul>Sloan-C Pillar: Learning Effectiveness
  15. 15. ONLINE TEACHING <ul><li>What direction is communication taking place in your online course? </li></ul><ul><ul><ul><li>Instructor Student </li></ul></ul></ul><ul><ul><ul><li>Student Instructor </li></ul></ul></ul><ul><ul><ul><li>Student Student </li></ul></ul></ul>
  16. 16. Applicable Sloan C- Pillars Learning Effectiveness Cost Effectiveness Access Faculty Satisfaction Student Satisfaction #3 Proof is in the Pudding: Insist on Adhering to Milestones in the Course Development Schedule! #4 Throwing out old notion about student learning #2 Training: Digital Immigrants versus Digital Natives #1 Identify Champions – Even if It Means Buying Their Affection #2 Training: Digital Immigrants versus Digital Natives #3 Adhering to Milestones #1 Identify Champions – Even if It Means Buying Their Affection #2 Training: Digital Immigrants versus Digital Natives #2 Training: Digital Immigrants versus Digital Natives #4 Throwing out old notion about student learning
  17. 17. GAP ANALYSIS
  18. 18. THE FIRST GAP <ul><li>Faculty training has been primarily focused on learning the LMS “tool” </li></ul><ul><li>Shift to assisting faculty on how to develop sound instructional design activities that promote student learning in the online environment that is motivating, engaging, and meaningful. </li></ul>
  19. 19. EXAMPLE – FORUM DISCUSSION <ul><li>Reading: pp. 451-459. Sometimes the most difficult part of an essay, when writing about an art work or an artist, is to describe it without using the word &quot;masterpiece&quot; or the idea of &quot;the insane&quot;. </li></ul><ul><li>So, assuming that Starry Night on p. 457 is NOT a masterpiece and is NOT by an insane artist, what is going on in the picture? The internet and resources available in our Library might help you with some alternative ideas, though the picture is discussed in your book. </li></ul><ul><li>About 400 words, please. </li></ul>Starry Night by Van Gogh Challenge faculty to think of ways to improve upon this activity What kind of feedback might the student expect after answering this question?
  20. 20. ASSESSMENT SHOULD CONNECT TO GOALS <ul><li>Reminder of our Goals: </li></ul><ul><ul><li>Accessibility : Provide Students with Alternative Ways to Enroll in classes (in order to avoid institutional conflicts with other activities/courses) </li></ul></ul><ul><ul><li>Vocation Preparation : Building upon career skills for working in online environments </li></ul></ul><ul><ul><li>Revenue : Increase new enrollment by attracting new markets </li></ul></ul><ul><ul><li>WHAT’S MISSING … </li></ul></ul><ul><ul><li>Sloan-C Pillars: Faculty and Student Satisfaction </li></ul></ul>
  21. 21. THE SECOND GAP <ul><li>What’s missing from our goals are the things we really want to know: </li></ul><ul><li>Have student learning outcomes (SLOs) been met, as much if not better, in the online environment as F2F classes? </li></ul><ul><ul><li>How can we get at direct assessment? </li></ul></ul><ul><li>Are students more or less satisfied with taking online courses than taking the F2F courses? </li></ul><ul><ul><li>How do students measure course satisfaction? </li></ul></ul><ul><ul><li>Are we asking the right questions of students? </li></ul></ul><ul><ul><li>Are students answering the questions we are asking or are they answering the questions we aren’t asking? </li></ul></ul><ul><ul><li>Sloan-C Pillars: Learning Effectiveness and Student Satisfaction </li></ul></ul>
  22. 22. ASSESSMENT 360 Peer, Self, and Student
  23. 23. 360 ASSESSMENT <ul><li>Peer review (COIL) </li></ul><ul><ul><li>Boiled down to 10 questions from 50 </li></ul></ul><ul><li>Self review </li></ul><ul><ul><li>Used same 10 questions from Peer Review </li></ul></ul><ul><li>Student evaluations </li></ul><ul><ul><li>Separate instrument </li></ul></ul><ul><ul><ul><li>Open-ended question on satisfaction </li></ul></ul></ul><ul><ul><ul><li>Quantitative items on accessibility </li></ul></ul></ul><ul><li>Open forum discussion of all Online Faculty </li></ul><ul><ul><li>Provide lunch </li></ul></ul><ul><ul><li>Discuss what went well; what didn’t work; recommended changes for next time </li></ul></ul>
  24. 24. COIL (PEER-SELF QUESTIONS) <ul><li>Presents course content in a manner that hierarchically structures the sequence of information </li></ul><ul><li>Provides activities that actively engage students in interactions with web-based course content. </li></ul><ul><li>Provides activities that can engage students in understanding and mastering different learning strategies. </li></ul><ul><li>Provides clear and adequate guidance on how to navigate the course, engage in course work, and turn in assignments. </li></ul><ul><li>Clearly delineates institutional policy on cheating and plagiarism at the start of course. </li></ul>
  25. 25. <ul><li>Provides rules/expectations for active participation in all online activities. </li></ul><ul><li>Personalizes communication by/with student-student and student-instructor (e.g., feedback on assignments, replies to discussion forum postings) </li></ul><ul><li>Provides students with timely, continuous, frequent support, and feedback. (e.g., feedback on assignments came within 1 week after deadline. Feedback was instructional/remedial) </li></ul><ul><li>Provide opportunities for student to question instructor to insure accuracy of understanding. (e.g., a chat room or discussion forum that encourages students to post questions. Syllabus and site provide instructor contact information and times of availability) </li></ul><ul><li>Conducts a teleconference (or other “voice” contact) at least twice with each student during the course. </li></ul>
  26. 26. RESULTS OF 360 ASSESSMENT <ul><li>Hypothesis 1: Do instructors rate themselves similarly to their peers? </li></ul><ul><li>T-test run </li></ul><ul><li>Findings: Instructors tended to rate themselves lower than their peers </li></ul><ul><ul><li>14 cases </li></ul></ul><ul><ul><li>12 were lower than peer reviews </li></ul></ul><ul><ul><li>2 were higher then peer reviews </li></ul></ul><ul><ul><ul><li>1 was within 1 STD below mean </li></ul></ul></ul><ul><ul><ul><li>1 was outside 2 STD below mean </li></ul></ul></ul><ul><ul><ul><ul><li>Second Peer Review reinforced first Peer Reviewers observation </li></ul></ul></ul></ul>
  27. 27. T-TEST
  28. 28. Difference Mean 0.08 Standard Error 0.035019618 Median 0.08 Mode 0.2 Standard Deviation 0.131031411 Sample Variance 0.017169231 Kurtosis -0.001776348 Skewness -0.788888516 Range 0.42 Minimum -0.2 Maximum 0.22 Sum 1.12 Count 14 Largest(1) 0.22 Smallest(1) -0.2 Confidence Level(95.0%) 0.075655284
  29. 29. 1 instructor fell below 2 STD of mean * *
  30. 30. STUDENT EVALUATION OF ONLINE EXPERIENCE <ul><li>Open-ended Qualitative Questions </li></ul><ul><ul><li>Was this class intellectually stimulating? Yes/No, </li></ul></ul><ul><ul><li>Did this class stretch your thinking? Yes/No </li></ul></ul><ul><ul><li>Please write a few statements supporting your answer as to why or why not you felt the class was intellectually stimulating, and or stretched your thinking. </li></ul></ul><ul><ul><li>What aspects of this class contributed most to your learning? </li></ul></ul><ul><ul><li>What aspects of this class detracted from your learning? </li></ul></ul><ul><ul><li>What suggestions do you have for improving the class? </li></ul></ul>
  31. 31. STUDENT EVALUATIONS … <ul><li>Quantitative Questions </li></ul><ul><li>Scale of 1 – 5 (strongly disagree – strongly agree) </li></ul><ul><ul><li>The online course was more challenging than a similar face-to-face course. </li></ul></ul><ul><ul><li>I think I learned more by taking this course online than by taking it face-to-face. </li></ul></ul><ul><ul><li>I felt “more connected” to my online instructors than to my face-to-face instructor(s). </li></ul></ul><ul><ul><li>Based on my experience with this online course, I would take another online course. </li></ul></ul><ul><ul><li>Knowing what I know now, I would NOT take this course ONLINE . </li></ul></ul><ul><ul><li>I interacted more with the other students in this online course then I did in my face-to-face classes. </li></ul></ul><ul><ul><li>I felt isolated in this online course. </li></ul></ul>
  32. 32. EXAMINING THE OUTLIER TO STUDENT FEEDBACK <ul><li>No real indication of problems </li></ul><ul><li>How much weight do we assign student evaluations? </li></ul><ul><li>What aspects of a course are most important to undergraduate students? </li></ul><ul><ul><li>Time demands </li></ul></ul><ul><ul><li>Subject Matter Interest </li></ul></ul><ul><ul><li>Rigor of assignments </li></ul></ul><ul><ul><li>Developing a connection to the professor </li></ul></ul><ul><li>Build in next student evaluation instrument question items that get to the heart of these attributes </li></ul>
  34. 34. WHAT DID WE LEARN? <ul><li>Peer evaluation tends to be higher than instructor-self evaluation </li></ul><ul><li>Student evaluations had no correlation to peer or instructor evaluation </li></ul><ul><li>Student evaluations clustered around the mean </li></ul><ul><li>We need to improve our student course evaluation instrument to get feedback on what it is that students are really evaluating. </li></ul><ul><li>We need to implement direct assessment of student learning outcomes </li></ul>
  35. 35. NEXT STEPS <ul><li>Re-write goals to include quality assurance of student learning outcomes, as well as faculty and student satisfaction </li></ul><ul><li>Adopt standards for faculty development </li></ul><ul><li>Adopt assessment strategies for faculty development. </li></ul><ul><ul><li>Are we asking the right questions that will help faculty in their course development and delivery? </li></ul></ul><ul><ul><li>Are we providing adequate support for faculty development? </li></ul></ul>
  36. 36. RECOMMENDED STANDARDS FOR FACULTY DEVELOPMENT <ul><li>Require training on functions of LMS </li></ul><ul><li>Require participation in faculty development workshops for developing strong pedagogically student-centered activities that match-up with student learning outcomes </li></ul><ul><li>Provide professional development funds so faculty can attend online education conferences/workshops, as well as use as incentives for developing online courses </li></ul><ul><li>Include online teaching as a hallmark in faculty promotion and tenure guidelines </li></ul><ul><li>Standardize on an assessment tool that provides meaningful feedback to faculty and students that will help shape and improve online course delivery </li></ul>