Hinchliffe "Supporting Innovation: The Role of Assessment Training and Education"

SUPPORTING INNOVATION:
THE ROLE OF ASSESSMENT
TRAINING AND EDUCATION
Lisa Janicke Hinchliffe
LisaLibrarian@gmail.com
@lisalibrarian
NISO Webinar Series 2019

“inattentional blindness refers to the
common failure to notice plainly
visible items when attention is
otherwise preoccupied, even though
people look directly at them”
Steven B. Most, What’s “inattentional” about inattentional blindness?, Consciousness and Cognition, 19(4)
December 2010, p 1102-1104, http://dx.doi.org/10.1016/j.concog.2010.01.011.

"THE CONUNDRUM
FOR COMPANIES IS
THAT GOOD PRODUCTS
OR SERVICES AREN'T
ENOUGH."
Chip Heath and Dan Heath; Authors of Made to Stick
http://www.fastcompany.com/magazine/116/column-made-to-stick.html?partner=rss

“Arriving at meaningful solutions is an
inevitably slow and difficult process.
Nonetheless …. better is possible.
It does not take genius. It takes
diligence. It takes moral clarity. It takes
ingenuity. And above all, it takes a
willingness to try.”
Atul Gawande, Better: A Surgeon’s Notes on Performance

LISA’S TOUCHSTONE PRINCIPLES
Engagement
Innovation
Value

LISA’S TOUCHSTONE PRINCIPLES
Engagement
Innovation
Value
What do these
mean to you?

Engagement
taking part;
devoting effort and attention;
being in gear

“The MISSION of LIBRARIANS is to
IMPROVE SOCIETY through
FACILITATING KNOWLEDGE
CREATION in their COMMUNITIES”
R. David Lankes – Participatory Librarianship (see: http://www.newlibrarianship.org/)

Innovation
creation of something new;
re-creation of something existing

"Entrepreneurship is the recognition
and pursuit of opportunity without
regard to the resources you
currently control, with confidence
that you can succeed, with the
flexibility to change course as
necessary, and with the will to
rebound from setbacks."
Bob Reiss, Low-Risk, High-Reward: Starting and Growing Your Small Business With Minimal Risk

Value
degree of importance;
relationship between cost and expectation

CommunicateDocumentCreate
VALUE
<repeat>

CommunicateDocumentCreate
VALUE
<repeat>
Where does this
cycle break
down and why?

OPTIONS FOR PURSUING VALUE WORK
External Consultant
Internal Expert
Broad Competency

OPTIONS FOR PURSUING VALUE WORK
External Consultant - HIRE
Internal Expert – HIRE or TRAIN
Broad Competency - TRAIN

WHAT COMPETENCIES?
 ALA Core Competencies – “The concepts behind, and methods for,
assessment and evaluation of library services and their outcomes.”
(http://www.ala.org/educationcareers/careers/corecomp/corecompetences)
 ACRL Standards for Libraries in Higher Education – “7.6 The library plans
based on data and outcomes assessment using a variety of methods both
formal and informal” and “ 7.7 The library communicates assessment
results to library stakeholders.”
(http://www.ala.org/acrl/standards/standardslibraries)
 ACRL Competencies for Special Collections Professionals –
“III.E.8. Engages in short-term and long-term strategic planning, policy
development, and assessment strategies geared to special collections and
its integration into larger institutional, scholarly, and community contexts;
continually assesses progress toward strategic goals.”
(http://www.ala.org/acrl/standards/comp4specollect)
 ACRL Proficiencies for Assessment Librarians and Coordinators
(http://www.ala.org/acrl/standards/assessment_proficiencies)

Year+ Long
Intensive
Programs
ACRL
Assessment
in Action
(IMLS
Grant)
CARLI
Counts
(IMLS
Grant)
One-Day or Multi-Day Workshops
ACRL
Roadshows
–
Assessment
in Action
and
Standards
for
Librarians
in Higher
Education
ACRL
Immersion
Program -
Assessment
Immersion
LLAMA E-
Learning
Course on
Evidence-
Based
Decision
Making
Medical
Library
Association
Workshop
on
Engaging
Assessment
to Show
Value and
Make
Decisions
University
Library
(Illinois)
Instruction
Studio
Library
Coach
University
of South
Florida St.
Petersburg
Credit
Courses
School of
Information
Sciences
(Illinois)
590 EVO
Training Examples
(Examples Lisa Janicke Hinchliffe Has Delivered/Co-Delivered)

The Value of Academic Libraries
“Few libraries exist in a vacuum,
accountable only to themselves.
There is always a larger context
for assessing library quality,
that is, what and how well
does the library contribute
to achieving the overall goals
of the parent constituencies?”
Sarah Pritchard, Library Trends, 1996

Connect, Collaborate, and Communicate
Develop Professional Competencies
Foster Collaborative Relationships
Document Practices and Strategies

Cycle of Learning Activities/Focus

Taking It To the State Level
27

CARLI Counts: Overview
• A continuing education library leadership immersion program based on
the highly successful I-LEAD and Assessment in Action programs
• Prepares librarians to make effective use of research findings on the
impact of academic libraries on student learning success for the twin
purposes of service development and library advocacy
• Participants will learn how to use local library data analytics to
improve their services and demonstrate their value in competitive
campus budgeting processes, accreditation reports, and program reviews

CARLI Counts Goal:
Illinois academic librarians will be able to effectively and
systematically leverage national and local data in order to
communicate impact narratives that convey to stakeholders how their
libraries bolster student learning and success.
CARLI Counts Outcomes:
1) CARLI Counts participants are more confident in their skills and
abilities related to service design and library advocacy.
2) CARLI libraries are better equipped to demonstrate their value to
stakeholders.

CARLI Counts: Structure of Participation
Principles:
• Encourage Peer-to-Peer Learning
• Foster Ongoing Community of Practice
One Cohort Per Year for Two Years (2018-2019 and 2019-2020):
• Each Cohort Comprised of Multiple Teams
• Each Team Comprised of Five Members and One Mentor
Learning Experiences:
• Immersive Workshops (February and July)
• Monthly Webinars (CARLI Counts Staff or Guest Experts)
• Team Meetings (Led by Mentors)
• Poster Session from Each Team at CARLI Annual Meeting (November)
• Institutional Case Study from Each Participant (January)
• Case Study Due from Each Participant – January 15

PROGRAM LOGIC MODEL
Resources Activities Outputs Outcomes Impact
Your Planned Work Your Intended Results
“A logic model is a systematic and visual way to
present and share your understanding of the
relationships among the resources you have to
operate your program, the activities you plan, and
the changes or results you hope to achieve.”
Excerpted from W.K. Kellogg Foundation’s Logic Model Development Guide
Excerpted from W.K. Kellogg Foundation’s Logic Model Development Guide

Assessment in Action:
Demonstrating and Communicating
Library Contributions to Student
Learning and Success
http://www.ala.org/acrl/aiaroadshow

What is the roadshow?
In this day-long workshop on strategic and sustainable
assessment, participants will
• identify institutional priorities and campus partners
• design an assessment project grounded in action
research
• prepare a plan for communicating the project results
This workshop is based on the highly successful ACRL
Assessment in Action program curriculum.

Roadshow Outcomes
Participants will be able to:
• apply action research as a means to designing robust
assessment plans, practices, and processes.
• implement assessment practices that document the impact
of libraries on student learning, academic programs and
activities, and institutional initiatives.
• collaborate with key campus partners to plan and conduct
assessment that aligns library outcomes with institutional
initiatives, priorities, and assessment activities
• use the results of assessment and action research to foster
support for library contributions to student learning and
success.

Planning, Assessing, and
Communicating Library Impact
Putting the
Standards for Libraries in Higher Education
into Action
37

Workshop Learning Outcomes
Participants will be able to..
• Establish the library and institutional contexts to effective engage in planning,
self-study, accreditation, or program review processes.
• Use the ACRL Standards for Libraries in Higher Education and other
foundational documents as a framework to develop benchmarks, evaluate
quality and performance, and demonstrate value to the institutional mission.
• Evaluate various metrics and assessment tools to select the best approach for
a given situation.
• Examine the role of leadership in building a culture of assessment to engage
all librarians and staff members in effective decision-making.
38

39
The Assessment Immersion Track is based on two crucial assumptions: (1) that
assessment requires us to clearly articulate our goals, whether those are expressed
in terms of the learning that we want students to achieve or the impacts/outcomes
that our programs will accomplish, and (2) assessment is a developmental process
critical to fostering instructional and program development as much as, or more
than, instructional or program evaluation.
After the program, participants will be able to:
 Define assessment in terms of student learning in order to understand its
relationship to good teaching, library viability, and change
 Formulate a learning-centered philosophy of assessment in order to inform
development of information literacy program elements
 Explore and utilize multiple modes of assessment in order to build a culture
of evidence upon which to base programmatic development and change
 Critically examine a variety of assessment techniques and methodologies in
order to evaluate them for application in your institutional setting
 Examine the leadership role of the library in a collaborative IL assessment
effort in order to build support and trust among the stakeholder groups at
your institution

Competency:
Leaders make use of research
derived from trials, literature
reviews, or other activities that
provides objective information
on issues of concern in order to
help determine whether a
particular policy or program
will work at their organization
and to demonstrate its
effectiveness.
Participants will be able to:
 Explain the competency
and relate it to their
personal leadership
development
 Can apply the competency
in practice
 Demonstrate self-
knowledge of their
achievement of the
competency

ENGAGING ASSESSMENT TO SHOWYOUR VALUE
Medical Library AssociationWorkshop

A
G
E
N
D
A
•Program Logic Model for Service Design
•Assessment Cycle
•Assessment Facets of Service Design/Delivery
• NeedsAssessment
• Best Practices and Professional Standards
• Resource Adequacy
• Implementation Fidelity
• Program Quality
• Outcomes and Impact
•Planning, Change and Communication

Assessment Facets
of Service Design
and Delivery
(Handout)
Assessment Facets of Service Design and Delivery
Needs Assessment
Best Practices and
Professional Standards
Resource Adequacy
Implementation Fidelity
Program Quality
Outcomes/Impact

INSTRUCTION STUDIO (UIUC LIBRARY) –
THREE ONE-HOUR SESSIONS
instructional design and learning outcomes
learning activity design and assessment
reflection and decision-making for continuous improvement

COACH FOR USFSP
Included:
Weekly Conference Calls with Strategic Planning Committee
 Review of Existing Assessment Data
 Assessment Method Design and Data Plan Review
 Document Review and Feedback
 Onsite Retreats with Small Group Discussions
 Recommendations and Insights for Library Dean

UIUC IS 590 EVO: EVALUATION AND
ASSESSMENT OF LIBRARY SERVICES
Introduction to theories, approaches, and methodologies for evaluation and
assessment of library programs and services. Explores assessment and
evaluation models through lenses of user needs, policy and compliance,
continuous improvement, and performance review as well as communication with
stakeholders. Pre-/Co-requisites: None. Credit: 4 Graduate Hours.
Upon successful completion of this course, you will be able to:
 articulate the need for and benefits of program evaluation and
assessment for library services in a range of library settings;
 describe the characteristics and components of program evaluation and
assessment;
 determine which inquiry design and method(s) are most appropriate for
different evaluative contexts and contexts;
 analyze, interpret, and communicate assessment data; and
 analyze and assess current empirical literature presenting evaluations of
library services.

LIS COURSE DESIGN CONSIDERATIONS
Conceptual Organizing Approach
 Competency Development – Theory and Methods
 Evaluation Practices for Specific Library Services
 Project-Based
Challenges
 Students’ Existing Knowledge and Skills (Assessment and Librarianship)
 Meaningful Assignments
 Distinct from LIS Research Methods but Very Related
 Divergences in Practices Across Library Types
Readings/Resources
 Program Evaluation Generally
 Library Assessment Specifically
 Reports, Conference Proceedings, etc.
 Guest Speakers

Year+ Long
Intensive
Programs
ACRL
Assessment
in Action
(IMLS
Grant)
CARLI
Counts
(IMLS
Grant)
One-Day or Multi-Day Workshops
ACRL
Roadshows
–
Assessment
in Action
and
Standards
for
Librarians in
Higher
Education
ACRL
Immersion
Program -
Assessment
Immersion
LLAMA E-
Learning
Course on
Evidence-
Based
Decision
Making
Medical
Library
Association
Workshop
on Engaging
Assessment
to Show
Value and
Make
Decisions
University
Library
(Illinois)
Instruction
Studio
Library
Coach
University of
South
Florida St.
Petersburg
Credit
Courses
School of
Information
Sciences
(Illinois) 590
EVO
Training Examples
(Examples Lisa Janicke Hinchliffe Has Delivered/Co-Delivered)
What would
work for you/
your library?

NOTE: OTHER TRAINING PROGRAMS
 Institute for Research Design in Librarianship
(https://library.lmu.edu/irdl/)
 Canadian Library Assessment Workshop (http://www.carl-
abrc.ca/strengthening-capacity/workshops-and-training/canadian-
library-assessment-workshop/)
 Research Institute for Public Libraries (https://ripl.lrs.org/)
 Embedding Evaluation in Libraries
(https://www.rebeccateasdale.com/embedding-evaluation)

“THERE IS A VIBRANT
FUTURE FOR LIBRARIES AND
IT IS OURS TO GRASP.”

Hinchliffe "Supporting Innovation: The Role of Assessment Training and Education"

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Hinchliffe "Supporting Innovation: The Role of Assessment Training and Education"

Similar to Hinchliffe "Supporting Innovation: The Role of Assessment Training and Education" (20)

More from National Information Standards Organization (NISO)

More from National Information Standards Organization (NISO) (20)

Recently uploaded

Recently uploaded (20)

Hinchliffe "Supporting Innovation: The Role of Assessment Training and Education"