1. 1
Wrapping the Gift?Wrapping the Gift?
Evaluating the School
Library Program
LIB 620 Library Management
Fall 2010

2. 2
What are school librariesWhat are school libraries forfor??
The Mission of the SchoolThe Mission of the School
LibraryLibrary
– The school library provides
information and ideas that are
fundamental to functioning
successfully in today’s information and
knowledge-based society. The school
library equips students with life-long learning
skills and develops the imagination, enabling
them to live as responsible citizens.
IFLA/UNESCO School Library Manifesto
2

3. 3
AASL MissionAASL Mission
The mission of the school libraryThe mission of the school library
media program is to ensure that themedia program is to ensure that the
students and staff are effective usersstudents and staff are effective users
of ideas and information. The schoolof ideas and information. The school
library media specialist empowerslibrary media specialist empowers
students to be critical thinkers,students to be critical thinkers,
enthusiastic readers, skillfulenthusiastic readers, skillful
researchers, and ethical users ofresearchers, and ethical users of
information . . .information . . .
– Empowering Learners (2009).
3

4. 4
What about school librarians?What about school librarians?
4
Roles and ResponsibilitiesRoles and Responsibilities
of the School Library Media Specialistof the School Library Media Specialist
– Leader
– Instructional Partner
– Information Specialist
– Teacher
– Program Administrator

5. 5
More on the role of the LMSMore on the role of the LMS
Guiding principlesGuiding principles
– Principle One: School libraries have no
boundaries.
– Principle Two: Library and information
professionals should be flexible.
– Principle Three: Ensure that students are
effective users of ideas and information.
– Principle Four: Information is everywhere,
essential, and central.
• Lowe, C. A.
The Role of the School Library Media Specialist in the 21st Centu
. ERIC Digest. 2000-11-00
5

6. 6
Another way to look at itAnother way to look at it
Three critical areas of servicesThree critical areas of services
1. Information access and delivery
• The school library media specialist who has a
solid foundation in evaluating information,
has technological expertise in retrieving and
organizing information, and maintains a
commitment to intellectual freedom, is able
to create an information-rich learning
environment within the school.
– Program Standards for School Library Media Specialist Prep
(2003), p. 7.
6

7. 7
More fromMore from NCATE StandardsNCATE Standards
Three critical areas of servicesThree critical areas of services
2. Teaching and learning
• Earlier versions of the school library media
specialist program focused on a consultancy
role and stressed locational guidance, but
current practice demands a true
partnership role, in which the school
library media specialist and classroom
teacher are engaged together throughout
the instructional process.
– Program Standards for School Library Media Specialist Pr
(2003), p. 7.
7

8. 8
More fromMore from NCATE StandardsNCATE Standards
Three critical areas of servicesThree critical areas of services
3. Program administration
• The school library media specialist must
understand management and change, must
communicate clearly, and must be prepared
to take tactical risks. In addition, librarians
must advocate for support and must create an
environment that contributes to student
achievement. . . . Therefore, the effective
school library media specialist will exhibit
leadership skills among colleagues . . .
– Program Standards for School Library Media Specialist Preparat
(2003), p. 7.
8

9. 9
LMS LeadershipLMS Leadership
9

10. 10
LeadershipLeadership
Are there many typesAre there many types
of leaders?of leaders?
– Appointed Leaders
– Expert Leaders
– Interpersonal Leaders
– Social/Informal Leaders
• Are you a “born leader”? If not, you can develop
skills to help you become an effective leader. Just
as you become a better teacher through
experience, you'll become more comfortable as a
leader over time.
• The Teacher Librarian as Manager
10

11. 1111

12. 12
The Kentucky Department of EducationThe Kentucky Department of Education
recommends the following resources:recommends the following resources:
– Characteristics of Highly Effective Teaching and Learning (Li
)
– School Library Media Program Rubric
– School Media Librarian Evaluation Instrument
12

13. 13
Evaluation of School Library Programs

14. 14
Preemptive Action:
The busy librarian’s guide to
program evaluation
Based in part on:Based in part on:
A workshop developed by theA workshop developed by the
Pennsylvania School Librarians AssociationPennsylvania School Librarians Association
Professional Development CommitteeProfessional Development Committee
((No longer available online)No longer available online)
indicates the slide was originally from the above presentation

15. 15
WhatWhat isis evaluation?evaluation?[[11]]
As defined by theAs defined by the
American Evaluation AssociationAmerican Evaluation Association::
– evaluation involves assessing the strengths and
weaknesses of programs, policies, personnel,
products, and organizations to improve their
effectiveness.
– Evaluation is the systematic collection and analysis
of data needed to make decisions
• Evaluation Definition: What is Evaluation?

16. 16
What aboutWhat about school library programschool library program evaluation?evaluation?
Evaluation ofEvaluation of
school library media centers:school library media centers:
– When you examine something or
someone, you examine the subject and
make a judgment about the quality,
significance, or condition of whatever is to
be evaluated.
• Emanuel T. Prostano and Joyce S. Prostano,
The School Library Media Center 5th
ed. (Englewood, CO:
Libraries Unlimited, 1999), 44.

17. 17
Purposes of evaluationPurposes of evaluation
To find out what is rightTo find out what is right
To find out what is wrongTo find out what is wrong
• Summary of definition in Blanche Woolls, The School
Library Media Manager 2nd
ed. (Westport, CT: Libraries
Unlimited, 1999).

18. 18
What can you evaluate?
Everything!Everything!
– Collection
– Personnel
– Library Layout
– Programs
– Curriculum collaboration
– Reference
service/collection:
• Interview skills
• Quality of the answers
provided
• Satisfaction of your
patrons
– Students or teachers—or
other school staff

19. 19
WhatWhat shouldshould you evaluate?you evaluate?
That depends . . .That depends . . .
– On your priorities
– On your economic needs and situation
– On the policies and procedures established by your
school, school district and/or state
– On how much time you have left over from your other
duties to devote to planning and executing an
evaluation project

20. 20
Why evaluate?Why evaluate?
Because you have toBecause you have to
– Continuous assessment of services can be an
ongoing requirement for accreditation/funding,
etc.
Because you want toBecause you want to
– You want to find out how well you’re doing
– You want to find out how well others think
you’re doing
– You want to find ways to improve your service

21. 21
Why Evaluate?
• Schools are being evaluated by student
academic achievement in reading,
writing and math.
• Recent studies show student
achievement correlates positively with
effective school library programs.

22. 22
Why Evaluate?Why Evaluate?
Studies show the positive correlation withStudies show the positive correlation with
student achievement occurs when there is anstudent achievement occurs when there is an
effective school library program [and]:effective school library program [and]:
– School librarians are full time and
– Librarians collaborate with teachers on
instructional planning.

23. 23
Why Evaluate?Why Evaluate?
Studies also show anStudies also show an
effective school library programeffective school library program
includesincludes
the following attributes:the following attributes:
– Quality collections;
– Increased hours of access beyond school day;
– Professional development for teachers and
librarians;
– Student access to technology; and
– Collaboration with other types of libraries.

24. 24
Why Evaluate? Because. . .Why Evaluate? Because. . .
To improve, the librarian mustTo improve, the librarian must
have baseline data about the program.have baseline data about the program.
To be relevant, the librarian must knowTo be relevant, the librarian must know
if the library services and resources areif the library services and resources are
aligned with the school’s goals foraligned with the school’s goals for
student achievement.student achievement.
To build a case for better support, theTo build a case for better support, the
librarian needs data that demonstrateslibrarian needs data that demonstrates
the value of the program.the value of the program.

25. 25
An evaluation enables you to:An evaluation enables you to:
– Determine success in attaining program goals.
– Determine students’ and teachers’ needs so they can
be incorporated into the program.
– Provide a basis for resource allocation.
– Recognize strengths and accomplishments.
– Examine the impact of the program on
student learning.
What Can an Evaluation
Do?
• Nancy Everhart,
Evaluating the School Library Media Center
, 1998)

26. 26
It makes sense—It makes sense—
 Focus data collecting andFocus data collecting and
evaluation on those components of aevaluation on those components of a
school library program that theschool library program that the
research shows correlate with studentresearch shows correlate with student
achievement.achievement.

27. 27
Types of Evaluation
 Formal
 Informal
 External
 Internal
 Formative
 During a program
 Summative
 After the program is completed

28. 28
Evaluation requires standardsEvaluation requires standards
How do you know if the media centerHow do you know if the media center
is “good”/ “not good” or even “good enough”?is “good”/ “not good” or even “good enough”?
– “Good enough” for what? For whom?
Guidelines or rubrics or objectives for collections,Guidelines or rubrics or objectives for collections,
performance, etc. can be found in:performance, etc. can be found in:
–

29. 29
Types of Data
QuantitativeQuantitative QualitativeQualitative
Number of OPACNumber of OPAC
searchessearches
Size of collectionSize of collection
Number ofNumber of
lessons plannedlessons planned
with teacherswith teachers
Students’ success rate inStudents’ success rate in
locating appropriatelocating appropriate
resources in OPAC searchesresources in OPAC searches
Collection supports theCollection supports the
curriculumcurriculum
Comprehensive andComprehensive and
collaborative planning is incollaborative planning is in
placeplace

30. 30
Standards require interpretationStandards require interpretation
Quantitative: relatively easyQuantitative: relatively easy
–

31. 31
Qualitative: more difficultQualitative: more difficult
–
Standards require interpretationStandards require interpretation

32. 32
Interpretation meansInterpretation means
operationalizationoperationalization
Operationalize:Operationalize:
– To define a concept in a way that can be measured. In
evaluation research, to translate program inputs,
outputs, objectives, and goals into specific measurable
variables.
• Program Evaluation Glossary
http://www.epa.gov/evaluate/glossary/o-esd.htm

33. 33
Interpretation means creating goals andInterpretation means creating goals and
objectivesobjectives
The purpose and study goals should determineThe purpose and study goals should determine
the types of methods and measures you use tothe types of methods and measures you use to
conduct the evaluationconduct the evaluation
Objectives will define your standard ofObjectives will define your standard of
excellence--the minimum level of appropriateexcellence--the minimum level of appropriate
service for your particular clienteleservice for your particular clientele
– Jo Bell Whitlach, Evaluating Reference Services

34. 34
Methods of EvaluationMethods of Evaluation
ObtrusiveObtrusive
– People are aware of the evaluation
• Self-evaluation
• Surveys
• Observation
UnobtrusiveUnobtrusive
– data collection that does not require intrusion into
the lives of participants by investigators
• Glossary for Responsible Conduct in Data Management
– People are unaware of the evaluation
• Unobtrusive measures of physical facilities
• Use of proxies—“mystery patrons”

35. 35
Obtrusive reference evaluations in a schoolObtrusive reference evaluations in a school
library contextlibrary context
Questionnaires orQuestionnaires or
interviews of students or teachersinterviews of students or teachers
Numbers gathering:Numbers gathering:
– Reference question counts
– Numbers/types of reference books used
– Circulation statistics
ObservationObservation
– By external observer
– Self-observation: Journal

36. 36
Evaluation ModelEvaluation Model
1.1. Define the question.Define the question.
2.2. Collect data--determine needed data andCollect data--determine needed data and
method of collection.method of collection.
3.3. Analyze the data.Analyze the data.
4.4. Formulate recommendations.Formulate recommendations.
5.5. Develop an action plan.Develop an action plan.

37. 37
Research FindingResearch Finding
Students whose school
librarian plays an
instructional role
tend to achieve higher
than average scores.
What data are
needed to
determine whether
or not the school
librarian plays an
instructional role
in the school?
The QuestionThe Question
Evaluation Model Step 1.
Define the Question

38. 38
Look at the rubricLook at the rubric

39. 39
Empowering LearnersEmpowering Learners
Building collaborative partnerships:Building collaborative partnerships:
– Guideline:
• The school library media program
promotes collaboration among members of
the learning community and encourages
learners to be independent, lifelong users
and producers of ideas and information
– Empowering learners, p. 20.
39

40. 40
Addressing multiple literacies:Addressing multiple literacies:
– Guideline:
• The school library media program
provides instruction that addresses
multiple literacies, including information
literacy, media literacy, visual literacy, and
technology literacy.
– p. 23.
According toAccording to Empowering LearnersEmpowering Learners

41. 41
Determine the type needed.Determine the type needed.
– Quantitative/Qualitative
Determine the method of collection.Determine the method of collection.
– Existing statistic
– Survey, questionnaire, focus group, observation
Evaluation Model Step 2.
Collect data

42. 42
– Identify areas of agreement and disagreement
among groups.
– Identify areas of strength and areas that need
improvement.
– Identify areas where more data is needed.
 How can the data be analyzed to determineHow can the data be analyzed to determine
whether or not the librarian plays an effectivewhether or not the librarian plays an effective
instructional role in the school?instructional role in the school?
Evaluation Model Step 3.
Analyze the data

43. 43
Agreement Example--Agreement Example--
Target Indicator:Target Indicator:
Information Literacy StandardsInformation Literacy Standards
are integrated into content learningare integrated into content learning..
Library Staff ResponseLibrary Staff Response
____Basic____Basic 100%100% Proficient ____ ExemplaryProficient ____ Exemplary
Administrative ResponseAdministrative Response
____Basic____Basic 100%100% Proficient ____ ExemplaryProficient ____ Exemplary
Teaching Staff ResponseTeaching Staff Response
12%12% BasicBasic 88%88% Proficient ____ ExemplaryProficient ____ Exemplary

44. 44
Disagreement Example--Disagreement Example--
Target Indicator:Target Indicator:
Curriculum development is modeled andCurriculum development is modeled and
promoted.promoted.
Library Staff ResponseLibrary Staff Response
100%100% Basic _____ Proficient ____ ExemplaryBasic _____ Proficient ____ Exemplary
Administrative ResponseAdministrative Response
____Basic____Basic 100%100% Proficient ____ ExemplaryProficient ____ Exemplary
Teaching Staff ResponseTeaching Staff Response
82%82%BasicBasic 18%18% Proficient ____ ExemplaryProficient ____ Exemplary

45. 45
Strength Area Example--Strength Area Example--
Target Indicator:Target Indicator:
Collaborative planning isCollaborative planning is
modeled and promoted.modeled and promoted.
Library Staff ResponseLibrary Staff Response
____Basic____Basic 100%100% Proficient ____ ExemplaryProficient ____ Exemplary
Administrative ResponseAdministrative Response
____Basic____Basic 100%100% Proficient ____ ExemplaryProficient ____ Exemplary
Teaching Staff ResponseTeaching Staff Response
____Basic____Basic 100%100% Proficient ____ ExemplaryProficient ____ Exemplary

46. 46
Weakness Area Example--Weakness Area Example--
Target Indicator:Target Indicator:
Students are engaged in reading, writing,Students are engaged in reading, writing,
speaking, viewing & listening for enjoyment,speaking, viewing & listening for enjoyment,
enrichment, & understanding.enrichment, & understanding.
Library Staff ResponseLibrary Staff Response
____Basic____Basic 100%100% Proficient ____ ExemplaryProficient ____ Exemplary
Administrative ResponseAdministrative Response
100%100% Basic ____ Proficient ____ ExemplaryBasic ____ Proficient ____ Exemplary
Teaching Staff ResponseTeaching Staff Response
82%82% BasicBasic 18%18% Proficient ____ ExemplaryProficient ____ Exemplary

47. 47
Contradictory DataContradictory Data ExampleExample——
Target Indicator:Target Indicator:
Effective teaching modeled & promoted.Effective teaching modeled & promoted.
LibraryLibrary Basic ____ Prof.Basic ____ Prof. 100%100% ExemplaryExemplary
Admin.Admin. ____ Basic ____ Prof.____ Basic ____ Prof. 100%100% ExemplaryExemplary
TeacherTeacher BasicBasic 60%60% Prof.Prof. 40%40% ExemplaryExemplary
Target Indicator: Student achievement isTarget Indicator: Student achievement is
assessed.assessed.
LibraryLibrary 100%100% BasicBasic Prof.Prof. ExemplaryExemplary
Admin.Admin. 100%100% Basic ____ Prof.Basic ____ Prof. ExemplaryExemplary
TeacherTeacher 90%90% BasicBasic 10%10% Prof.Prof. ExemplaryExemplary

48. 48
Back to Step 2Back to Step 2
Collecting More Data –Collecting More Data –
Focus GroupsFocus Groups
1.1. How are student products and performancesHow are student products and performances
assessed in units involving use of the library?assessed in units involving use of the library?
2.2. How do you determine the products andHow do you determine the products and
performances to demonstrate mastery of contentperformances to demonstrate mastery of content
in units involving use of the library?in units involving use of the library?
3.3. What is the role of reflection in student workWhat is the role of reflection in student work
involving use of the library?involving use of the library?

49. 49
• Review the data, formulate recommendations.Review the data, formulate recommendations.
• For example:For example:
– Because of the strong correlation between the
school librarian’s instructional role and student
achievement, the Advisory Committee establishes
a three year goal of reaching the Exemplary level
in all target indicators in the Teaching and
Learning Rubric
 Role of an Advisory Committee ofRole of an Advisory Committee of
stakeholders—stakeholders—
Evaluation Model Step 4.
Analyze the data

50. 50Example:Example:
Recommendations for First YearRecommendations for First Year
– Review the Standards for the
21st
Century Learner to ensure
they align with content standards
and set student expectations for analysis,
evaluation and inquiry.
– Meet with teaching teams to solicit
recommended revisions.
– Convene a workshop for new teachers to
review the standards.
 Standards for the 21Standards for the 21stst Century LearnerCentury Learner
Adapted from

51. 51Example:Example:
Recommendations for First YearRecommendations for First Year
Collaborative PlanningCollaborative Planning
– Establish regular common planning time for
teachers with the library media staff.
Curriculum DevelopmentCurriculum Development
– Review school policies to remove any barriers
that prevent librarians from participating in
building and district curriculum sessions.

52. 52Example:Example:
Recommendations for First YearRecommendations for First Year
Reading, Writing, Speaking, ViewingReading, Writing, Speaking, Viewing
– Provide a series of author workshops and develop
plans with teachers for student reading, writing,
speaking, and viewing responses to the author
workshops.
– Involve Parents.
Effective TeachingEffective Teaching
– Clarify conflicting feedback through focus groups
targeting use of assessment, differentiation, &
inquiry in instruction.
– Develop recommendations based on new data.

53. 53
Library staff develop an action plan for eachLibrary staff develop an action plan for each
recommendation.recommendation.
The Advisory Committee of StakeholdersThe Advisory Committee of Stakeholders
reviews and approves the action plans.reviews and approves the action plans.
Evaluation Model Step 5.
Develop an action
plan

54. 54
Action Plan Example:
Target Indicator:
Curriculum development is modeled and promoted
ObjectiveObjective ActivityActivity DocumentationDocumentation ParticipantsParticipants NewNew
ResourcesResources
CompletionCompletion
DateDate
Review barriersReview barriers
preventingpreventing
librarians fromlibrarians from
participating inparticipating in
curriculumcurriculum
sessions.sessions.
ReviewReview
policiespolicies
School CouncilSchool Council
minutesminutes
SchoolSchool
Council &Council &
admin.admin.
NoneNone
projectedprojected
Sept. 1Sept. 1
SecureSecure
scheduleschedule
Post schedule inPost schedule in
lib. & faculty rm.lib. & faculty rm.
PrincipalPrincipal NoneNone Sept. 10Sept. 10
ParticipateParticipate
in mtgs.in mtgs.
Meeting minutesMeeting minutes LibrariansLibrarians NoneNone
projectedprojected
June 1June 1
ReportReport
backback
Faculty mtg.Faculty mtg.
minutesminutes
LibrariansLibrarians NoneNone MonthlyMonthly
WrittenWritten
materialmaterial
In professionalIn professional
librarylibrary
LibrariansLibrarians NoneNone Within 5Within 5
days of mtgdays of mtg

55. 55
This example of an evaluation
was…
• Type (which one?)Type (which one?)
– Formal
– Internal
– Formative
• Data collection methods:Data collection methods:
– Research
– Rubric-based
questionnaire
– Focus Group
• Data type:Data type:
– Quantitative
– Qualitative

56. 56
Unobtrusive evaluation in schools 1Unobtrusive evaluation in schools 1
Unobtrusive measures of physical facilitiesUnobtrusive measures of physical facilities
– “The basic premise . . . is that you can learn a
great deal . . . by looking at how things wear
(‘erosion’), how things are left in the building
(‘traces’) and how things are rearranged
(‘adaptations for use’).”
• Nancy Everhart, Evaluating the School Library
Media Center: Analysis Techniques and
Research Practices. Libraries Unlimited, 1998.

57. 57
Unobtrusive evaluation in schools 2Unobtrusive evaluation in schools 2
Use of proxies—“mystery patrons”Use of proxies—“mystery patrons”
– Technique used more often in academic libraries
and government documents reference services
• Half-right reference: the 55% rule
• 5-minute rule
– Peter Hernon and Charles R. McClure, “Unobtrusive
Reference Testing: The 55 Percent Rule” Library Journal
April 15, 1986, 37-41.
• “It’s not true, and now we know why . . . the so-called
“55% rule” has never been tested against a truly
representative field sample.”
– John V. Richardson, Jr., “Reference Is Better Than We
Thought,” Library Journal April 15, 2002, 41-42.

58. 58
Importance of a sophisticated modelImportance of a sophisticated model
Richardson:Richardson:
– “The reference service performance model [that led to
the 55% rule] was overly simplistic, samples were way
too small, and the test questions were not representative
of real-world reference questions.”
– “Students of reference service should learn about the
existence of multiple performance outcomes (i.e.,
accuracy, utility, and satisfaction) and to recognize that
each outcome is driven by different factors.”
• “Reference Is Better Than We Thought.”

59. 59
Characteristics of good modelsCharacteristics of good models
Measures of reference service must beMeasures of reference service must be
– Valid
• They “accurately reflect the concept being studied.”
– Reliable
• They “are stable and dependable, and provide
consistent results with each repeated use.”
– Practical
• They “require that data be relatively easy to collect.”
– Useful
• They “provide information that can be used to
improve reference services.”
– Whitlach, Evaluating Reference Services.

60. 60
Problems of Evaluation in School ContextProblems of Evaluation in School Context
Often the school librarianis the only oneOften the school librarianis the only one
working in the libraryworking in the library
An evaluation project can be time-An evaluation project can be time-
consuming to plan and to put into actionconsuming to plan and to put into action

61. 61
Solutions to evaluation problemsSolutions to evaluation problems
Depends on your local situationDepends on your local situation
– Your resources:
• Time, money, available warm bodies
– You and your creativity
Make evaluation part of your routineMake evaluation part of your routine
– As you plan/prepare your program(s), include
an evaluation component
• Appropriate, affordable, accessible, accountable
Seek collaborative partners wheneverSeek collaborative partners whenever
possiblepossible

62. 62
Find out more in LIB 604!Find out more in LIB 604!
LIB 604 Libraries in theLIB 604 Libraries in the
School CurriculumSchool Curriculum
– An approach to understanding the role of the
library media specialist and the media center in the
school curriculum, with the aim of developing the
school as a learning community, stressing
collaboration between administrators, teachers and
librarians in planning learning experiences.
– Offered Spring semesters
– Includes a Collaborative Action
Research assignment.
– See What is Collaborative Action
Research?
62

63. 63
http://www.animationfactory.comhttp://www.animationfactory.com
63