This document discusses the importance of assessing and demonstrating the impact of faculty development activities. It outlines three steps for documenting impact: developing goals, determining how to know when goals are achieved, and collecting appropriate evidence. The document then discusses common frameworks for measuring impact, including participation, satisfaction, learning, efficiency, teaching practices, research/scholarship, student learning, and institutional impact. It acknowledges challenges in documentation and provides recommendations and resources.

1. DEMONSTRATING THE IMPACT OF FACULTY
DEVELOPMENT ACTIVITIES
CHRISTINA M. SAX
ASSOCIATE PROVOST AND DEAN
ACADEMIC OUTREACH & INNOVATION
SHIPPENSBURG UNIVERSITY
ORIGINALLY PREPARED FOR
INNOVATIONS IN FACULTY
DEVELOPMENT SYMPOSIUM -
JUNE, 2015
TWITTER
CHRIS SAX - @CMSAX
SHIP - @SHIPPCDE

2. WHY ASSESS & DEMONSTRATE THE IMPACT
OF FACULTY DEVELOPMENT (FD) ?
• INCREASING CALLS FOR ACCOUNTABILITY
• INCREASING CALLS FOR FISCAL RESPONSIBILITY
• HELPS OTHERS UNDERSTAND & VALUE FD WORK & ACTIVITIES
• INFORMS OUR FD DECISION MAKING & RESOURCE ALLOCATIONS
• PROVIDES USEFUL DATA FOR IMPROVING FD ACTIVITIES

3. INSTITUTIONAL VARIATIONS IN FD
FD ACTIVITIES ARE OFTEN DRIVEN BY INSTITUTION-SPECIFIC FEATURES
• MISSION, FOCUS, GOALS
• FACULTY NEEDS
• RANGE OF ACTIVITIES
• ORGANIZATIONAL STRUCTURE
• STAKEHOLDERS
ONE SIZE FD DOES NOT FIT ALL WITH RESPECT TO …
• WHAT, WHY, WHEN, HOW OF FD
• ASSESSING AND DEMONSTRATING IMPACT OF FD

4. DIFFERING PERSPECTIVES ON
FD GOALS AND IMPACT
• INDIVIDUAL FACULTY
• ACADEMIC COLLEGE/DEPARTMENT/PROGRAM
• DEAN
• DEPARTMENT CHAIR
• INSTITUTION
• FACULTY DEVELOPMENT OFFICE/UNIT/COMMITTEE
• FACULTY DEVELOPMENT EVENT/ACTIVITY
• STUDENTS
Perspectives
can impact
priorities &
perceived
values

5. MANY TYPES OF FD ACTIVITIES TO ASSESS
• WORKSHOPS
• TRAININGS
• BROWN BAGS
• WEBINARS
• AUTOMATED SELF-
PACED TUTORIALS
• NEEDS ASSESSMENT
• PEER MENTORING
• CONSULTATIONS
• INTERNAL GRANTS
• EXTERNAL GRANTS
• CONFERENCES
• NEWSLETTERS
• LEARNING COMMUNITIES
• DISCUSSION GROUPS
• BOOK CLUBS
• FACULTY WRITING GROUPS
• SCHOLARSHIP – DISCIPLINARY
CONTENT & RESEARCH
• SCHOLARSHIP - TEACHING &
LEARNING
• COURSE REDESIGN
• HIGH IMPACT PRACTICES (HIPS)
• INSTITUTIONAL INVOLVEMENT
(COMMITTEES)
• TEACHING CERTIFICATIONS
• FACULTY FELLOWS
• NEW FACULTY SUPPORT
• VETERAN FACULTY &
ONGOING DEVELOPMENT
• TEACHING & LEARNING
INSTITUTES
• IN-HOUSE RESOURCES
• COMMERCIAL RESOURCES
• INTER-INSTITUTIONAL
COLLABORATIONS

6. COMMON FRAMEWORK:
STRATEGIC PLANNING AND FD
Institutional Strategic Plan:
Mission, Vision, Values, Goals (5-10 year)
Faculty Development Program Strategic Plan:
Mission, Vision, Values, Goals (3-5 year)
Faculty Development Events
and Activities (1-2 year)
Institutional plan drives FD plan
FD plan drives FD activities
FD activities
support
achievement of
FD plan
FD activities
support
achievement of
institutional
strategic plan

7. 3 STEPS TO DOCUMENTING IMPACT
STEP 1
• DEVELOP & IDENTIFY GOALS
STEP 2
• HOW WILL WE KNOW WHEN THE GOALS
HAVE BEEN ACHIEVED?
• WHAT DO WE WANT TO KNOW AND WHY?
STEP 3
• WHAT TYPE OF EVIDENCE WILL ACCURATELY
REFLECT THE ACHIEVEMENT OF THE GOALS?
• ASSESS, MEASURE, DOCUMENT WHAT MATTERS
AND IS VALUED
• COLLECT DATA THAT IS MOST IMPORTANT, NOT
JUST WHAT’S EASIEST TO COLLECT
• CONSIDER QUANTITATIVE AND QUALITATIVE
INFORMATION
• TAKE INTO ACCOUNT VARIATION OF ACTIVITIES
• TAKE INTO ACCOUNT MULTIPLE WAYS TO MEASURE
& DOCUMENT ACHIEVEMENT OF ANY ONE GOAL
• TELL THE STORY OF IMPACT

8. FD GOALS, PURPOSE, DESIRED OUTCOMES
•
•
•
•
•
•
•
•
•
•

9. MEASURE & DOCUMENT IMPACT
FD Event or
Activity
Latter
Observation
or Outcome
Immediate
Observation
or Outcome
Immediate
Direct
Relatively easy
Longitudinal
Often indirect
More difficult
Cursory Measures of
Impact:
1. Participation
2. Satisfaction
3. Efficiency
4. Learning
Deep Measures of
Impact on:
5. Teaching
6. Research &
Scholarship
7. Student Learning
8. Institution

10. PARTICIPATION
WHO PARTICIPATED IN FD ACTIVITIES?
• NUMBER OF ATTENDEES/PARTICIPANTS
• NUMBER OF UNIQUE PEOPLE
• NUMBER OF REPEAT ATTENDEES
• SCOPE OF PARTICIPANTS
• CHANGES IN METRICS OVER TIME
• USAGE OF IN-HOUSE AND CONTRACTED RESOURCES &
MATERIALS

11. IMPACT ON EFFICIENCY
WHAT IS THE COST-BENEFIT ANALYSIS OF FD
ACTIVITIES?
• COST OF ACTIVITIES – DIRECT AND INDIRECT
COSTS
• COSTS PER PERSON
• COST CONTAINMENT EFFORTS
• USAGE OF IN-HOUSE AND CONTRACTED
RESOURCES & MATERIALS
• RETURN ON INVESTMENT (ROI)

12. SATISFACTION
WHAT WAS PARTICIPANTS’ LEVEL OF SATISFACTION WITH THE FD
ACTIVITY?
• SATISFACTION SURVEY, AT END OF EVENT
• SATISFACTION SURVEY, LONGITUDINAL
• ANNUAL/PERIODIC CAMPUS-WIDE SURVEYS
• FOCUS GROUPS

13. LEARNING & DEVELOPMENT
DID THE PARTICIPANTS LEARN FROM THE FD ACTIVITY?
• APPLICATION ACTIVITIES AT THE EVENT
• PRE- AND POST-ASSESSMENTS
• SURVEYS, AT END OF EVENT
• SURVEYS, LONGITUDINALLY
• INDIVIDUAL INTERVIEWS
• FEEDBACK AND INTERVIEWS OF MENTOR, MENTEE,
CONSULTANTS

14. IMPACT ON TEACHING
DID PARTICIPANTS CHANGE THEIR TEACHING PRACTICES AS A RESULT
OF THE FD EVENT?
• SELF REPORTS BY FACULTY - REPORTS, SURVEYS, INTERVIEWS, FOCUS
GROUPS, PRESENTATIONS
• PRE- AND POST-EVENT CLASSROOM OBSERVATIONS
• MENTOR AND CONSULTANT OBSERVATIONS
• TEACHING PORTFOLIO REVIEWS
• PROGRAM CURRICULUM REVISIONS
• STUDENTS - RATINGS/EVALUATIONS, FEEDBACK, FOCUS
GROUPS, SMALL GROUP INSTRUCTIONAL DIAGNOSTICS

15. IMPACT ON RESEARCH & SCHOLARSHIP
DID PARTICIPANTS ADVANCE THEIR RESEARCH AND SCHOLARSHIP AS A
RESULT OF THE FD ACTIVITY?
• NUMBER OF GRANTS SUBMITTED; NUMBER OF GRANTS AWARDED
• NUMBER OF PUBLICATIONS & PRESENTATIONS
• NEW PROJECTS LAUNCHED
• CV & PORTFOLIO REVIEWS
• PROMOTION, TENURE, SABBATICAL DECISIONS
• PROJECTS INCLUDING STUDENTS
• NUMBER OF STUDENTS INVOLVED
• NUMBER OF STUDENTS INVOLVED IN PUBLICATIONS & PRESENTATIONS

16. IMPACT ON STUDENT LEARNING
DID STUDENT LEARNING OUTCOMES CHANGE AS A RESULT OF THE FD
ACTIVITY?
• ANNUAL/PERIODIC ACADEMIC PROGRAM ASSESSMENT REPORTS
• SELF REPORTS BY FACULTY - REPORTS, SURVEYS, INTERVIEWS, FOCUS
GROUPS, PRESENTATIONS
• STUDENTS
• RATINGS/EVALUATIONS
• FEEDBACK
• ASSIGNMENT/ASSESSMENT PERFORMANCE
• FINAL COURSE GRADES

17. IMPACT ON THE INSTITUTION
WAS THERE AN INSTITUTIONAL CHANGE AS A RESULT OF THE FD
ACTIVITY?
• INSTITUTIONAL ASSESSMENT DATA & NSSE RESULTS
• CHANGES IN PRACTICE, CULTURE, BEHAVIORS, ATTITUDES,
ENGAGEMENT, COLLABORATIONS
• REPORTS & RECOMMENDATIONS OF STANDING COMMITTEES
• NEW INITIATIVES – ACADEMIC, ADVISING, CO-CURRICULAR, SUPPORT,
ETC.
• FACULTY PROMOTION, TENURE, SABBATICAL DECISIONS
• STUDENTS
• FEEDBACK, RETENTION RATES, GRADUATION RATES

18. COMMON PROBLEMS DOCUMENTING IMPACT
• FD IS MULTIDIMENSIONAL - MULTIPLE UNITS AND INDIVIDUALS
• INTERNAL: FACULTY DEVELOPMENT UNIT, INSTRUCTIONAL DESIGN UNIT,
COMPUTING/TECHNOLOGY UNIT, HUMAN RESOURCES, LIBRARY, ASSESSMENT
UNIT, ACADEMIC DEPARTMENT/DISCIPLINE
• INTERNAL/EXTERNAL: CONTRACTED SERVICES
• EXTERNAL: PROFESSIONAL ASSOCIATIONS, COMMERCIAL
TRAININGS & WORKSHOPS
• DIFFERENT TYPES OF DATA COLLECTED BY DIFFERENT UNITS
• NO COMPREHENSIVE REPOSITORY OF DATA
• STAFFING, PLANNING, TIME

19. OVERCOMING DOCUMENTATION PROBLEMS
• USE AND BUILD ON EXISTING TOOLS, PROCESS, DATA
• ENGAGE INSTITUTIONAL ASSESSMENT OFFICE, TEAM, COMMITTEE
• ASSESS PERIODICALLY VS CONTINUALLY
• BUILD AN INTEGRATED ASSESSMENT AND REPORTING SYSTEM
• DEVELOP A COMMON REPOSITORY FOR FD ASSESSMENT
DOCUMENTS & EVIDENCE

20. RESOURCES – 1
• BOTHELL, TW, HENDERSON, T, EVALUATING THE RETURN ON INVESTMENT OF FACULTY
DEVELOPMENT. IN WEHLBURG, CM, CHADWICK-BLOSSEY, S, TO IMPROVE THE ACADEMY, VOL
22, 2003.
• CHISM, NVM, SZABO, BL, HOW FACULTY DEVELOPMENT PROGRAMS EVALUATE THEIR SERVICES.
JOURNAL OF STAFF, PROGRAM, AND ORGANIZATIONAL DEVELOPMENT 15(2): 55-62, 1997.
• GROSSE, E., CTL SELF-STUDY TOOL, 2013.
• HINES, SR, INVESTIGATING FACULTY DEVELOPMENT PROGRAM ASSESSMENT PRACTICES: WHAT’S
BEING DOE AND HOW CAN IT BE IMPROVED? JOURNAL OF FACULTY DEVELOPMENT 23(3): 5-19,
2007.
• HINES, SR, HOW TO EVALUATE YOUR FACULTY DEVELOPMENT SERVICES. ACADEMIC LEADER
27(2): 1&4, 2011.
• HINES, SR, HOW MATURE TEACHING AND LEARNING CENTERS EVALUATE THEIR SERVICES. IN MILLER,
J. & GROCIA, J. (EDS), TO IMPROVE THE ACADEMY, VOL 30, 2011.

21. RESOURCES – 2
• MILLOY, PM, BROOKE C, BEYOND BEAN COUNTING: MAKING FACULTY DEVELOPMENT NEEDS
ASSESSMENT MORE MEANINGFUL. IN WEHLBURG, CM, CHADWICK-BLOSSEY, S, TO IMPROVE THE
ACADEMY, VOL 22, 2003.
• PLANK, KM, KALISH, A, ROHDIECK, SV, HARPER, KA, A VISION BEYOND MEASUREMENT:
CREATING AN INTEGRATED DATA SYSTEM FOR TEACHING CENTERS. IN CHADWICK-BLOSSEY, S,
ROBERTSON, DR, TO IMPROVE THE ACADEMY, VOL 23, 2005.
• PLANK, KM, KALISH, A, PROGRAM ASSESSMENT FOR FACULTY DEVELOPMENT. IN GILLESPIE, KJ,
ROBERTSON, DL (EDS), A GUIDE TO FACULTY DEVELOPMENT, 2010.
• PROFESSIONAL AND ORGANIZATIONAL DEVELOPMENT NETWORK IN HIGHER EDUCATION, IMPACT
AND ASSESSMENT OF TEACHING CENTERS: RESEARCH, RESOURCES, AND POD MEMBER PRACTICES,
HTTPS://SITES.GOOGLE.COM/A/PODNETWORK.ORG/WIKIPODIA/HOME/TOPICS-FOR-
DISCUSSION/IMPACT-_OF_TEACHING_CENTERS#TOC-OVERVIEW-