This document outlines a backwards design approach to evaluating the effectiveness of faculty development programs for blended instruction. It discusses defining what constitutes a good blended course and appropriate pedagogical models. It also addresses designing faculty development programs, including opportunities and experiences to help instructors learn effective blended teaching practices. Finally, it discusses tools and methods for evaluating blended courses and communicating results, including collecting and analyzing student and course data and disseminating findings to relevant audiences.

1. Evaluating the Effectiveness of Faculty
Development Programs for Blended
Instruction
July 9th, 2014
Sloan-C Blended Conference
Tanya Joosten, @tjoosten, and Dylan Barth, @dylanbarth
University of Wisconsin – Milwaukee
tjoosten@uwm.edu | djbarth@uwm.edu

2. Tech
Enhanced
Blended
Online
Traditional
Self-paced
MOOCs
Flex
Faculty development
programs and pedagogical
consultation
Technology training and
support
Evaluation and research
About us

3. About
UWM Online

4. What is
blended?

5. A Backwards
Design Approach
to Evaluation

6. Evaluation planning
Input
• Medium (F2F
or Online)
• Instructor
demographics
• Discipline
• Course level
Process
• Defining good
blended and
online course
• Defining
pedagogical
model
• Designing
faculty
development
program
Output
• Course level
• Program and
Institutional
level

7. F2F is not the gold
standard
Focus on the
process…the
pedagogy
Inputs

8. Define a good
blended and
online course
Define
pedagogical
model
Design a faculty
development
program
Process

9. Step 1: Defining
effective

10. 1. What is a “good” blended course? What
pedagogical model facilitates a “good”
blended course?

11. Good course
Higher retention
Better grades on assessment
• Quizzes and exams
• Projects and papers
• Overall
Satisfied students
• Future enrollments
Learning
Engaging
Enrollments

12. Pedagogical model
Content
• Text
• Images
• Audio
• Video
Interactivity
• Discussions
• Groups
• Feedback
Assessment
• Written and oral examination
• Discursive
• Portfolio

13. Content

14. Interactivity

15. Assessment
Cognitive
Affective
Psychomotor
or Behavioral
Individual Group
Projects
Discussions
Writing
Quizzes

16. Step 2:
Facilitating
success

17. 2. What elements and format should be
considered in designing and developing a faculty
development program?
What opportunities and experiences should be
available to help instructors learn effective
practices in design, delivery, and teaching
blended courses?

18. UWM’s
faculty
development

19. • Ten questions
• Designing learning modules
• Online vs. F2F - Integration
• Decision rubric for
content choices
• Learning objects
Content
• Progressive/summative
• Before, during, and after
• Self evaluation
• Peer evaluation
• Student evaluation
Course Evaluation
• Rubrics
• CATs
• Templates
• Traditional formats
Assessment
• Synchronous/asynchronous
• Establishing voice
• Discussion forums
• Small groups
Interactivity
• Managing expectations
• Time management
• Technology support
Helping Your Students
• Staying organized
• Managing workload
• Avoiding course and a half
Course Management
Course Redesign
Transitioning to
online and
blended teaching
Pedagogy and more

20. Experience

21. F2F 1
3 hrs
Online 1
F2F 2
3 hrs
Online 2
F2F 3
3 hrs
Showcase
3 hrs
Post-
Program
Blended

22. Backwards design

23. Active learning

24. Practice-based

25. Participant
Self-
Reflection
Progressive
Participant
Evaluation
Facilitator
Self-
Reflection
Summative
Participant
Evaluation
Program evaluation

26. Step 3:
Documenting
success

27. 3. How will you know when faculty are
providing quality online and blended
courses? What tools or services could be
provided for evaluating the effectiveness of
online and blended courses? How will
quality be communicated to others?

28. Did it work?
CC Flickr katherine.a

29. Course evaluation

30. Course evaluation

31. Student Evaluation Data
Course evaluation

32. Course evaluation

33. Who is your audience?
What variables will be
examined?
How will the data be collected?
analyzed?
How (and where) will the data
be presented?
Who will be involved in the
evaluation process?
When will the evaluation be
completed?
Evaluation plan

34. Audience

35. Input Process Output
Students Interactivity Students/Faculty
Demographics Communication Learning
Age Engagement Satisfaction
Gender Social Presence Performance
Ethnicity Content
Student Status Assessment Course/Institutional
Data
Full time/Part time Retention
Employment Status Drop/Withdrawal Rate
Zip Code Grade
Course/Instructor
Evaluation
Course
Discipline
Course Level
Instructor
Mode of Delivery
Variables

36. Collecting

37. Analyzing

38. Dissemination
CC Flickr bengray

39. Human Resources
CC Flickr Vandy CFT

40. Timeline
CC Flickr zamboni.andrea

41. Challenges
CC Flickr EverExplore

42. Questions

43. Thank you!
Tanya Joosten, tjoosten@uwm.edu, @tjoosten
Dylan Barth, djbarth@uwm.edu, @dylanbarth