Brightspace Teaching and Learning Webinar: January 30, 2018. Presenter: Barry Dahl, D2L This two-part webinar addresses the development of a process that can be used by academic administrators to evaluate the quality of online teaching. The current evaluation processes at many colleges do not specifically address online instruction, and most academic administrators have limited personal experience with online course design and instruction. As part of this webinar, effective practices in online instruction and online faculty evaluation will be explored.​ Recording: https://www.d2l.com/resources/webinars/evaluating-quality-online-teaching-part-one/

1. Evaluating the Quality of Online Teaching
Part One

2. Customize Your Webinar Viewing:
You can resize your own windows

3. Customize Your Webinar Viewing:
Check bottom tray for more options
Open Twitter Widget – tweet
with your own account
Open Group
Chat - hidden
by default

4. Brightspace Connections
www.d2l.com/events/regional/
Alberta Connection
• Friday, March 9
• SAIT – Main Campus
• Calgary, AL
Ontario Connection
• Friday, March 2
• Sheridan College
• Oakville, ON

5. Upcoming Webinars
Tips and tricks for a
successful roll out of Daylight
• Wednesday, February 21
• Sandra Earl, Product Manager, and Jeff
Geurts, Senior Product Designer, D2L
www.d2l.com/resources/webinars/
Evaluating the Quality of
Online Teaching – Part Two
• Tuesday, February 13
• Barry Dahl, D2L
Adding Action to Your Online Course: Outside-The-Box Techniques
to Increase Student Engagement From a Distance
• Tuesday, March 6 Amanda Dills, Oklahoma City University

6. Houston, Texas!! July 25-27, 2018
www.d2l.com/fusion

7. www.d2l.com/resources/webinars/
Webinars One-stop Shop
Registration for Upcoming Webinars
and Recordings of Past Webinars

8. Evaluating the Quality of Online Teaching
Part One

9. “Quality”
in Online Education

10. What is a Good Definition for Quality
•Generally speaking…
• High Grade
• Superiority
• Excellence

11. Define Quality in Education
•Generally speaking…
• High standards
• Consistently applied
• With an eye on efficiency (or value for money)

12. Define Quality in Online/Digital Education
•Generally speaking…
• Passing the Quality Matters (QM) rubric review
But
It shouldn’t be

13. 3 Major Components of e-Quality
Learning
Teaching Design

14. Learning Level
Is High
Teaching Level
Is High
Course Design
Meets Standards
Learning Assessment
Measuring the Major Components

15. Comprehensive Faculty Evaluations
• Purpose
• Accreditation
• History of Online Faculty Evaluation
• Components
• A few examples
• What about classroom observations for online courses?

16. Purposes of Faculty Evaluation
1. Encourage excellence in teaching and learning.
2. Facilitate long-term quality improvement by continually
monitoring instructional performance.
3. Provide constructive feedback to faculty by identifying
areas of strength and areas for improvement in classroom
instruction.
4. Inspire professional growth and development.

17. What about Accreditation?
• Here’s an example from the Higher Learning
Commission
• Guidelines for the Evaluation of Distance Education
(On-line Learning)
• NOTE: most other regional accreditors use the
same or similar language.

18. Guidelines for the Evaluation of DE
Faculty responsible for delivering the on-line learning
curricula and evaluating the students’ success in
achieving the on-line learning goals are appropriately
qualified and effectively supported.
• Examples of evidence: a. On-line learning faculties
are carefully selected, appropriately trained,
frequently evaluated, and are marked by an
acceptable level of turnover.

19. A Little History…Maybe
• The first known (to me, anyway) online faculty
evaluation system was started in 2001 at
Park University.
• Initially based on the evaluation system used
for face-to-face classroom instructors.
• As such, they attempted to replicate the
traditional classroom evaluation in the online
classroom.

20. But Online Was (Is) Different
The unique features and functions in an online classroom
were not initially considered.
For example, the original evaluation system did not include
an examination
• of learning outcomes
• of overall classroom management
• facilitation and guidance
• the concept of faculty presence
• communication response rates
• accessibility of materials
• course-related administrative tasks, etc.

21. Adapting the Instrument
• To address the unique evaluation concerns for
online teaching, Park University College for
Distance Learning developed a formalized Online
Instructor Evaluation System (OIES).
• OIES launched as a pilot in Fall 2004, and was
used through 2008.
• OIES morphed into the Faculty Online Observation
(FOO) model and is still in use today.

22. Evaluation System Components
• Student Evaluation of Instruction
• Faculty Self-Evaluation
• Professional Development Plan
• Faculty Peer Review
• Faculty Portfolio
• Supervisor’s Evaluation of Faculty
Usually some combination of the above components

23. Student Evaluation of Online Instruction
• Not our main focus here.
• However, some questions or issues to focus on:
• How to get decent response rates
• Use incentives, or not?
• What effect does the length of survey have on responses?
• Does it need to be comparable to F2F evaluation surveys?
• How valuable are these overall?

24. Faculty Self-Evaluation
• You might love these, you might hate these. You also might
mostly ignore these.
• Let’s look at two examples:
• Open-ended questions
• Specific questions tied to student eval

25. Open-ended questions
Sample questions from faculty self-evals:
1. How has your understanding of your role as a faculty
member changed/developed since your last evaluation?
2. As you prepare for future semesters, what more could you
do to provide students with a successful learning
experience?
3. What more can the university do to support you in your
professional goals and development?

26. Questions tied to student evals
Wharton County Junior College
• Online Course Evaluation (completed by students)
4. Instructor requires me to be an active participant in class.
9. Instructor is present regularly in the course
(announcements, emails, discussion board, feedback, etc.).
• Online Faculty Self-Evaluation
4. I require students to be an active participant in class.
9. I am regularly present in my course (announcements,
emails, discussion board, feedback, etc.).

27. Faculty Peer Review of Online Teaching
Great example comes from Penn State University
• College of Earth and Mineral Sciences
• Based on the “Seven Principles for Good Practice in
Undergraduate Education”
• Two Parts:
• An Instructor Input Form to be completed for the reviewer by the
reviewee in advance of the peer review, and
• The actual Peer Review Guide for Online Teaching at Penn State,
which is to be completed by the reviewer during the peer review.

28. Screenshot of Guide
Last revision June 23, 2017 - Ann H. Taylor, Dutton e-Education Institute, College of Earth and Mineral Sciences, The Pennsylvania State University

29. Examples of Evidence to Look For
Last revision June 23, 2017 - Ann H. Taylor, Dutton e-Education Institute, College of Earth and Mineral Sciences, The Pennsylvania State University

30. Where to Look, and Resources Links
Last revision June 23, 2017 - Ann H. Taylor, Dutton e-Education Institute, College of Earth and Mineral Sciences, The Pennsylvania State University

31. Portfolio for Faculty (Austin CC)
The portfolio will include the following:
a. Syllabus for each course (not section) taught, up to four courses
b. Samples of major assignments, tests, projects
c. Statement of Teaching Philosophy
d. Course Commentary
e. Professional Development Plan

32. Supervisor’s Evaluation of Teaching
• Most schools have some sort of process where a Dean,
Department Head, Director, or similarly positioned person
evaluates faculty performance.
• One part of this process is typically the classroom
observation.
• In-class observations for online courses/instructors are
inherently different from F2F class observations.

33. Issues with Teaching Observations
Timeline:
• F2F – typically a single class period
• Online – as much or as little as you choose/allow
Familiarity:
• Most deans have taught F2F at one time or another
• Most deans have NOT taught online, although this is improving
Criteria:
• F2F – Most schools have established criteria for observations
• Online – Many do not have established criteria

34. Coming in Part 2 - February 13
Online Faculty Evaluation Rubric
• Sharable & Editable (Creative Commons BY-NC-SA 3.0)
Components included:
1. Pre-evaluation worksheet
2. Professional Development documentation
3. Expectations to be Evaluated
• Examples of Meeting Expectation
• Evaluator Comments

35. Q & A
Excellence in e -Education
Online Faculty Evaluation Rubric
Faculty name
Evaluator name
Course evaluated
Term/Semester
Start/End dates