Building a Culture of Assessment @ ZSR
•
2 likes•709 views
This document discusses developing a culture of assessment in academic libraries. It defines assessment as a process to understand user needs and evaluate how well the library supports those needs to improve services. The key aspects of a culture of assessment are that decisions are based on research and facts, and services are planned and delivered to maximize positive outcomes. The document recommends libraries develop assessment skills among staff, ensure assessment aligns with institutional goals, and use assessment data in decision-making to improve practices and participate in campus-wide evaluation. It provides examples of factors that can facilitate or hinder developing such a culture, and suggests libraries implement both broad assessments and local team-based assessments as part of moving forward to create a culture of assessment.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (15)
Similar to Building a Culture of Assessment @ ZSR
Similar to Building a Culture of Assessment @ ZSR (20)
More from Susan Smith
More from Susan Smith (12)
Recently uploaded
Recently uploaded (20)
Building a Culture of Assessment @ ZSR
- 2. WHEN PEOPLE HEAR THE WORD “ASSESSMENT”
- 3. WHAT IS ASSESSMENT? Assessment: “Library assessment is a process undertaken by libraries to learn about the needs of users (and non-users) and to evaluate how well they support these needs, in order to improve library facilities, services and resources.” (Wikipedia!) How is assessment different from data gathering? Traditional: Inputs and Outputs Move To: Outcomes and Impacts Outcomes are “squishy” while outputs are hard
- 4. WHY ASSESS? Value of Academic Libraries Research Agenda • Align libraries with institutional outcomes • Empower libraries to carry out work locally • Create shared knowledge and understanding • Contribute to higher education assessment
- 5. WHAT IS A “CULTURE OF ASSESSMENT”? “A Culture of Assessment is an organizational environment in which decisions are based on facts, research, and analysis, and where services are planned and delivered in ways that maximize positive outcomes and impacts for customers and stakeholders. A Culture of Assessment exists in organizations where staff care to know what results they produce and how those results relate to customers’ expectations. Organizational mission, values, structures, and systems support behavior that is performance and learning focused.”
- 6. WHY DOES ZSR NEED TO DEVELOP THIS CULTURE? • We are excellent at narrating our story, that’s how we won our 2011 award • Today’s HE climate calls for evidence to support our stories
- 7. HOW? MacAyeal’s Mindsets • Assessment needs to live in the ongoing, daily work of everyone • Assessment skill sets need to be cultivated in more library staff • Assessment implies a willingness to change • Assessment needs to align with institutional goals • Assessment reporting allows libraries to participate in a campus-wide dialogue about evaluation
- 8. HOW? Factors that facilitate/hinder a culture of assessment: • ✔Campus-wide assessment initiative w/library involvement • Clear expectations for assessment in the library • ✔Assessment is a priority for administration • Library has adopted learning outcomes • Library has an assessment plan • Having a shared understanding of the purpose of assessment • ✔Library staff/faculty are supported in doing assessment
- 9. HOW ? Factors that facilitate/ hinder a culture of assessment: • ✔Education and training about assessment are supported • Library has assessment expertise in-house • ✔Library has systems/technologies to support assessment • ✔Leadership offers explicit support to get staff doing assessment • ✔Data is available to interested parties • ✔Leadership uses assessment data in decision-making • Librarians use results to improve practice • ✔Library culture is user-focused
- 10. MOVING FORWARD: CREATING THE CULTURE “Big Assessment” • LibQUAL+ Lite (October) • SACSCOC (Next 2 years) • AIA Project “Local Assessment” • FY15: Each team takes on one assessment goal • New programs/events should include a plan to assess • Think of the questions you want answered Educating Ourselves Develop an Assessment Plan Impact Starter Kit!
Editor's Notes
- Fear Frustration Exhaustion Puzzlement Tears Want to scream!
- Finally, the trend is a move from inputs and outputs, to outcomes and impact Inputs are the resources used for a program or activity (things and people) Outputs: These are the activities done. A library lecture series is an output. Outcomes: These are the observed effects of the outputs on the participant.. Outcomes (mid or long term) information gained (e.g. factual or bibliographical information) · problems solved · time saved (in studies or professional work) · information seeking skills improved · IT skills improved Long-term results of using library services might be: · information literacy · improved academic success · better career prospects · changes in behavior (e.g. frequency of reading, competent use of information) Impact: This is the degree to which the outcomes are attributable to its activities. Impact furthers the well-being of individual users or the quality of societal life. This includes features like: · democracy · social inclusion · cultural life · local identity · life-long learning
- Few libraries exist in a vacuum, accountable only to themselves. There is always a larger context for assessing library quality, that is, what and how well does the library contribute to achieving the overall goals of the parent constituencies? The Value of Academic Libraries: An ACRL Initiative: 2010 Purpose: Align libraries with institutional outcomes. • Empower libraries to carry out work locally. • Create shared knowledge and understanding. • Contribute to higher education assessment. We know that we contribute to: student enrollment – student retention and graduation rates – student success – student achievement – student learning – student experience Research Agenda: – faculty research productivity – faculty grant proposals and funding – faculty teaching – overall institutional reputation or prestige
- Think about culture as the “shared mental model” that the members of an organization hold and take for granted. “A Culture of Assessment is an organizational environment in which decisions are based on facts, research, and analysis, and where services are planned and delivered in ways that maximize positive outcomes and impacts for customers and stakeholders A Culture of Assessment exists in organizations where staff care to know what results they produce and how those results relate to customers’ expectations. Organizational mission, values, structures, and systems support behavior that is performance and learning focused.” Lakos, Phipps and Wilson (2002) as quoted in Lakos, A. & Phipps, S. E. (2004). "Creating a culture of assessment: A catalyst for organizational change." portal: Libraries and the Academy, 4(3), 345-361. Wendy F. Weiner: A culture of assessment is a set of attitudes, approaches, and understandings that support the evaluation of student learning outcomes. It's a set of mindsets that help create a positive response to the wider call for accountability
- Academic libraries everywhere are formalizing their approach to assessment by establishing committees, assigning new work to existing staff, or creating new positions entirely. Attendance data for the ARL Assessment Conferences lend credence to assessment as a growing focus. Our mission is to help faculty, students and staff succeed. We know it’s not about us, it’s about them. We also know that to keep succeeding, we need to continue to grow, change, learn, innovative and find creative solutions to the ever changing landscape. Assessment is a tool that can help us do this effectively. One challenge associated with creating a culture of assessment in libraries relates to professional values. We believe that what we do is a public good and because of this don’t feel the need to demonstrate outcomes and articulate impact. Isn’t “good” widely recognized? And universally appreciated? Intellectually, we’ve known the value of assessment for quite awhile. We have an assessment committee that is a good base for developing a culture of assessment. Members are the directors who can make sure that assessment activities take place. But they can’t do it alone!
- From Greg MacAyeal A culture of assessment article in C&RL News, June 2014: 1. Assessment needs to live in the ongoing, daily work of everyone. Committees, workgroups, or assigned staffers cannot "own" assessment and retain all responsibilities. Ideally, they act as resources for everyone else. Libraries, departments, and individuals need to include assessment as part of their expected work and build assessment activity into their goals. Initiatives should grow organically out of continuing work and should be completed by those engaged in that work. Assessment begins with questions "How will I know if this is successful?" "Has this improved," or "Is the result worth the time and expense?" All of us can ask these questions about what we're doing in the day-to-day of our work. (Our Assessment Committee, why it is set up the way it is) 2. Assessment skill sets need to be cultivated in more library staff. As stated above, there are now many librarians involved in assessment initiatives, but not all have developed appropriate skill sets. As we move forward, many of us find that our skills in research methods, data gathering, analysis, and reporting are not strong enough. We're learning on the job as the work dictates. Staff members who already possess assessment skills can be overwhelmed with consultations. The resulting bottleneck may cause work to slow or stop. More people working across all functional areas in libraries need to have well developed assessment skills. This needs to be a priority for training and staff development. 3. Assessment implies a willingness to change. Whatever we assess must be changeable, and at the onset of making any assessment, there must be an attitude that supports making changes. A valid outcome may be that no change is required or possible because of political or practical reasons, but even in such cases, the assessment itself still may be considered successful. Good assessment will lead to an informed decision. Efforts leading to anything less will be a waste of time. 4. Assessment needs to align with institutional goals. Libraries traditionally have been measured using data such as collection size, gate count, and staff size, but it’s now generally understood that this is not enough. Institutions of higher education create knowledge through student learning and faculty research. Traditional library assessment data does not demonstrate the impact a library has on student learning or faculty research. When traditional assessment data align with institutional goals, they remain valuable. If not, we need to stop committing precious time and scarce resources to collect and report it. 5. Assessment reporting allows libraries to participate in a campus-wide dialogue about evaluation. In many of the ways in which a college or university now looks at its own effectiveness, we see an increased need for accountability. There are many drivers for this, such as new accreditation standards, and it's felt in the library as campus leaders look to the library for indicators of student success.
- From (Lisa Janicke Hinchliffe, University of Illinois & Meredith Gorran Farkas, Portland State University) survey of library leaders: what factors facilitate or hinder a culture of assessment: 1. Campus-wide assessment initiative 2.Library involvement in campus-wide assessment initiative 3.Clear expectations for assessment in the library 4.Assessment is a priority of library administration 5.Library has adopted learning outcomes 6.Library has assessment plan 7.Library has shared understanding of the purpose of assessment 8.Library staff/faculty are supported in doing assessment 9.Education and training about assessment are supported 10.Library has assessment expertise in-house 11.Library has systems/technologies to support assessment 12.Library leadership offers explicit support to get staff doing assessment 13.Data is available to interested parties 14.Leadership uses assessment data in decision-making 15.Librarians use results to improve practice 16.Library culture is user-focused
- From (Lisa Janicke Hinchliffe, University of Illinois & Meredith Gorran Farkas, Portland State University) survey of library leaders: what factors facilitate or hinder a culture of assessment: 1. Campus-wide assessment initiative 2.Library involvement in campus-wide assessment initiative 3.Clear expectations for assessment in the library 4.Assessment is a priority of library administration 5.Library has adopted learning outcomes 6.Library has assessment plan 7.Library has shared understanding of the purpose of assessment 8.Library staff/faculty are supported in doing assessment 9.Education and training about assessment are supported 10.Library has assessment expertise in-house 11.Library has systems/technologies to support assessment 12.Library leadership offers explicit support to get staff doing assessment 13.Data is available to interested parties 14.Leadership uses assessment data in decision-making 15.Librarians use results to improve practice 16.Library culture is user-focused
- LibQual+ Since 2002 solicit, track, understand, and act upon users' opinions of service quality. The 22 core survey items measure user perceptions of service quality in three dimensions: Affect of Service, Information Control, and Library as Place. For each item, users indicate their minimum service level, desired service level, and perceived service performance. SACSCOC Southern Association of Colleges and Schools Commission on Colleges. regional body for the accreditation of degree-granting higher education institutions in the Southern states. Assessment in Action. As part of the Association of College & Research Libraries (ACRL) Value of Academic Libraries initiative, ZSR Library was selected to participate in Assessment in Action: Academic Libraries and Student Success (AiA), a grant program funded by the Institute of Museum and Library Services (IMLS). ZSR will develop and implement an action-learning project this year to examine how the library impacts student success. Students will be recruited to participate in this study, which will first identify what measures of student success are important to students at all points in their academic careers, and then determine how the library's programs, services, and study spaces help them to achieve this objective. The principal investigator of this 15-month study is Mary Beth Lock, Director of Access Services at ZSR. The project team consists of Ryan Shirey (Writing Center), Rachel Weaver (Student Employment Manager at The Bridge), Meghan Webb (ZSR Reference Coordinator), and John Champlin (Professional Development Center). Local Efforts” Assessment begins with questions "How will I know if this is successful?" "Has this improved," or "Is the result worth the time and expense?" All of us can ask these questions about what we're doing in the day-to-day of our work. Team goals: Access: utilize Qualtrics quiz after student training to assure learning; Qualtrics survey on Safety and Security training for students from Bootcamp Digital Scholarship: Conduct an institutional risk assessment for ZSR’s digital assets RIS: Develop LIB100 assessment pre-test/post-test Resource Services: try to figure out cost per use with video Special Collections: Technology: Fully implement Google Universal Analytics across ZSR web sites, in order to provide more granular data for assessing use of our online services and content. Figure out ways to assess what you are doing: Example: Hu and HvZ: giving paper survey at the end of the evening. Educating ourselves: Usually assessment sessions at every type of conference: it’s a hot topic Webinars Staff Development (perhaps)