Lecture presented by Marilou N. Andres at the 2nd Marina G. Dayrit Lecture Series 2016 held at Ateneo de Manila University on 11 March 2016

1. 2nd
Marina G. Dayrit Lecture Series 2016
Collection Analysis and Evaluation: Fundamentals of
Collection-Centered Assessment Techniques
March 11, 2016
Ateneo de Manila University
Loyola Heights, Quezon City
Marilou N. Andres
Philippine Association of Academic/Research Librarians, Inc.
(PAARL)

2. Outline
 Rationale
 Different collection-centered assessment
techniques
 Framework of identifying useful data for
collection assessment
 Preparation of collection assessment
studies/reports

3. 03/14/16
The Collection Development Process

4. 03/14/16
intended
curriculum
implemented
curriculum
attained
curriculum
G A P
management
of research
&
instructional
materials
COLLECTION
ANALYSIS
techniques
in
evaluation
status
of
collection

5. 03/14/16
Assessment
• examines or describes
collections either in their
own terms or relative to
other collections and
checklists.
Evaluation
• determines how well the
collection supports the
goals, needs, and
curriculum of the parent
organization.
COLLECTION
ANALYSIS
(Johnson ,2009)

6. 03/14/16
Collection Assessment
1. Analyzing what you have
a. How much?
b. How old?
c. How does it compare to others?
2. Analyzing how it is used
a. Circulation/in-library use
b. Comparing use
c. Electronic counting-website, databases
(Kachel, D. E.)

7. 03/14/16
“Culture of Evaluation”
 Way to demonstrate
 Relevance
 Value
 Impact
 Considered from the view of:
 Users
 Stakeholders
Lakos, A. & Phipps, S. (2004)

8. 03/14/16
A mechanism to determine:
 If the collection is meeting its objectives
 How well it is serving its users
 In which ways or areas it is deficient, and
what remains to done to develop the
collection
 If selectors are performing their
responsibilities effectively
 How to allocate collections/access funds
(Johnson ,2009)

9. 03/14/16
Who is the audience?
 Accreditation agencies
 Parent organization (administration,
board, senior management)
 Library administration
 Selector
 User community or communities
 Consortial partners
(Johnson ,2009)

10. 03/14/16
Collection-based Measures Look at:
 Size
 Growth
 Coverage (depth, breadth, balance)
(Johnson ,2009)

11. 03/14/16
Misconception
A “balanced” collection means having
resources on every topic.
Does NOT Mean:
• You purchase a few resources on
every topic known to mankind
•Replacing each weeded title with
something on the same topic (1 to 1
replacement)
•Purchasing resources “just in case”
there might be a need
Does mean:
•Having a balance of opinions on
controversial topics studied in the
curriculum
•Representing diversity in the topics
collected- includes “ethnic and racial
backgrounds, age, physical and cognitive
abilities, family status, sexual orientation,
socioeconomic status, religious and
spiritual values, and geographic
location.”
(Kachel, D. E.)

12. 03/14/16
Collection-based Measures
 Checking lists
 Evaluating the collection directly
 Comparative statistics
 Application of collection standards

13. 03/14/16
Research Methods
Qualitative Research
- A process of inquiry that draws data from the context
in which events occurs . . . using induction to derive
possible explanations based on observed phenomena.
Gorman and Clayton (2005)
Quantitative Research
-Involves the collection and analysis of data that is
quantifiable.

14. 03/14/16
Differences
Qualitative Quantitative
Strengths
Weaknesses
Count things (Titles, Circulation
transactions, Expenditures, E-
metrics, ILL transactions, Ratios
Provides context
Offers a way to understand the
attitudes that inform the statistics
Online or printed surveys
Interviews (structured or
unstructured)
Observation
Focus groups
(Johnson ,2009)

15. 03/14/16
Commonalities
Qualitative
(non-numerical)
Quantitative
(numerical data)
• Logic of triangulation
• Qualitative research
facilitates quantitative
research
• Quantitative research
facilitates qualitative
research
• Provide a general picture
• Researchers' and
subjects' perspectives Hughes, C.

16. 03/14/16
Different collection-centered assessment techniques
Method Technique
Use and user Centered Collection-Centered
Quantitative Interlibrary loan statistics Collection size and growth
Circulation statistics Materials budget and growth
In-house use statistics Collection size standards and formula
Qualitative User opinion survey List checking
Focus groups Citation analysis
Direct collection checking
Commercial Products
Collection mapping
(assigning conspectus levels)
Evans, (2012), Gregory, (2011), Johnson (2009)

17. STANDARDS FOR LIBRARY COLLECTION IN ACADEMIC LIBRARIES
PAASCU
Requirements
PACU-COA
Requirements
AACCUP
Requirement
PAARL
Requirements
CHED
Requirements
Basic book collection
of 6,000 well selected
titles with an enrollment
of 500 or less to support
its educational programs
Core book collection of 5,000 titles for a
college and 10,000 for a university
● A quality core book collection of 10,000
titles is maintained*
Core collection of at least 5,000
titles for a college and 10,000 for a
university
Core book collection of 5,000 well
selected titles for college libraries and
10,000 for university libraries
Minimum of 2 selected titles should be
provided for every student
Minimum of 5 professional titles per
student*;
3-5 book/journal titles for each
professional subjects
Minimum of 5 professional titles per
student shall be provided
Book Ratio 2- 5 titles per
professional course
For reserve books, 1 copy for every 30
students
For reserve books 1 copy per 25 students For reserve books 1 copy per 25
students
The provision of textbooks is not the
responsibility of the library but a
maximum of 5 copies of frequently
used materials shall be provided
● For reserve books, 1 copy per 25
students
Periodicals (local and foreign)
50 titles – 1,000 enrollees
75 titles – 1,001-3,000 enrollees
100 titles – 3,000 and more
● Every major field
should be covered
adequately by relevant
and appropriate journals
and periodicals
Periodicals (local and foreign)
50 titles – 1,000 enrollees
75 titles – 1,001-3,000 enrollees
100 titles – 3,000 and more
● For every major field
-6 titles of undergraduate concentration
in the major field
-6 titles for graduate course
-10 titles for doctoral course
● Core periodical/journal
collection of at least 100
titles are provided*
Periodicals (local and foreign)
50 titles – 1,000 enrollees
75 titles – 1,001-3,000
Enrollees 100 titles – 3,000 and more
● Every major field should be covered by
at least 3 titles of undergraduate
concentration of major
subject field, 6 titles of graduate
concentration and 10 titles of doctoral
work or equivalent
3-4 professional journals
Obille, K.L.B. (2007). An Evaluation of standards for academic libraries in the Philippines. Journal of Philippine Librarianship, 27(1&2), 109-150
AACCUP Survey Instrument

18. COLLECTION MAPPING
• EXISTING
COLLECTION
Input
• COLLECTION
MAPPING
Process
• COLLECTION
DEVELOPMENT
POLICY
Output

19. Elements in common
1. Subject based rather than descriptive of total.
2. Number of items.
3. Type or formats represented.
4. Age of items – mean, mode, medium.
5. Level, scholarship or audience for items.
6. Means of identifying appropriateness.
7. Necessity to view the collection –shelf scanning viewed
as essential.
(Bushing, 2005)

20. Appropriateness determined by . . .
1. Comparison with other similar or ideal library
collections.
2. Comparison with standard lists, bibliographies.
3. Relation to mission of the library.
4. Use statistics where applicable.
(Bushing, 2005)

21. 3 Basic Assumptions About Collections
that Relate to Collection Mapping
The collection should have breadth
It should have depth based on the needs of your students
and teachers
The collection should be well understood in order to be
effectively developed and used.
(Lamb and Johnson, 2014)

22. Collection maps are used for many collection-related projects
 showing strengths and weaknesses in the collection
 evaluating whether the strengths of the collection match
the curriculum focus
 tailoring purchasing decisions
 planning for future directions
 suggesting areas of weeding
 demonstrating areas of need and areas of excellence
 developing short and long term goals
 building a budget plan to match curriculum goals
 creating selection procedures that material collection
goals
(Lamb and Johnson, 2014)

23. What is a base collection and core collection?
BASE
• number of works recommended
for a minimal level collection
CORE
• associated with specific titles
(Lamb and Johnson, 2014)

24. Procedure for Collection Mapping
Divide the total size of the collection by the number of students in the
school
Divide the total size of each emphasis collection by the number of
students and chart the resulting items/student on a collection map
Decide which specific emphasis collection
Decide the general emphasis collections
http://libraryadmin.wikispaces.com/Collection+Mapping

25. Conspectus: a step in the evolution
• strengths of the collection
• acquisitions commitment
Input
• assessment
process
• desired collecting
intensity
output

26. How are results analyzed?
• examine to determine if the collection is evenly
distributed
• consider limiting selection
• develop promotional strategies
strength
• Select from retrospective selection bibliographies
• Watch for new items
weakness
• Circulation statistics
COMPare
http://libraryadmin.wikispaces.com/Collection+Mapping

27. Benefits of Conspectus
 a standard means of describing current collection strengths
 a standard means of describing future collection intentions
 assistance with collection rationalization by informing
librarians who else is collecting in a particular subject area
 assistance with resource sharing by listing collection strengths
for user who may then visit other libraries, and for library staff,
as a guide to interlibrary loans
 assistance with the prioritization of materials for preservation
activity, and recording of these priorities.
(Clayton ,2001)

28. 03/14/16
Collecting Levels
Collection depth indicators
Information Level Definition
Collecting
Level
Conspectus
A 5 Comprehensive A collection in a specifically defined field of knowledge
that strives to be exhaustive, as far as isreasonably
possible (i.e., a "special collection"), in all applicable
languages.
B 4 Research
A collection that contains the major published source
materials required for doctoral study and
independent research
C 3 Study
Collections that provide information about a subject in
a systematic way, but at a level of less than research
intensity, and support the needs of general library
users through college and beginning graduate
instruction

29. 03/14/16
Collection depth indicators
Information Level Definition
Collecting
Level
Conspectus
D 2 Basic
Collections that serve to introduce and define a
subject, to indicate the varieties of information
available elsewhere, and to support the needs of
general library users through the first two years of
college instruction
E 1 Minimal
Collections that support minimal inquiries about
this subject
O Out of Scope
Library does not intentionally collect materials in
any format for this subject.
http://www.ifla.org/files/assets/acquisition-collection-development/publications/gcdp-en.pdf

30. SAMPLE OF CONSPECTUS WORKSHEET FOR BUSINESS AND ECONOMICS
LC CLASS
(Dewey Dec.)
Subject Group Collecting Levels and Language
Coverage
Comments
ECS CCI DCI
HD4801-8942
(331)
HFI-4050
(380-382)
HF5001-6351
(650-659)
HG
(332)
Labor
Commerce
Business
Finance
2
2
2
1
2
2
2
2
2
3
2
2
Good GDoc Collection;
Meets most needs
Ave. Copyright 1979;
Good GDoc Collection
Low circ., low demand
47% Coverage/BPI
30% Harvard Core List;
High use/demand
Ave. Copyright 1981;
Low use/demand
(Wood and Strauch)

31. 03/14/16
Conspectus experience . . .
Like collection policies, the process is as valuable as the
end product.
– Discussing the appropriate data & gathering it.
– Sorting out the differences in information cultures from
discipline to discipline, from subject to subject.
– Deciding upon appropriate segments or subjects for
mapping.
– Identifying consistent vocabulary, codes, & notes.
The process is adaptable for every size & type of
collection.
(Bushing, 2005)

32. End results & benefits of mapping
 Extensive knowledge of collection details & character.
 Clarity of purpose.
 Identification of “problems” to be resolved.
 Evidence that can be communicated to others.
 Documentation for policy decisions.
 Staff with in-depth knowledge of discipline specific information
cultures.
 Confidence in making day-to-day decisions regarding information
resources.
(Bushing, 2005)

33. 03/14/16
Evaluating Electronic Resources
Data elementsNo. of sessions
No. of Queries
No. of Menu
selections
No. of full-
content units
examined
No. of turn-
aways, peak
simultaneous
users

34. 03/14/16
Where to start?
 Define the question or problem
 Determine metrics to use
 Decide:
 Where to locate the information
 Who will collect the information
 Who will analyze and report the information
 Who will act on the information

35. Framework of identifying useful data for collection
assessment
Problem Technique Data needed
1. Profile of the
collection
List checking Collection profiling (Inventory)
2. Compare
collection profile and
bibliographies
Bibliographies by recognized
subject experts, catalogs of libraries
with strong collections in the area, or
standard lists in a discipline.
Customized list may be prepared
by compiling references from a
number of authoritative sources
Collection profiling (Inventory)

36. 03/14/16
List Checking

37. 03/14/16
Framework of identifying useful data....
Problem Technique Data needed
1. Profile of the
collection
Citation
analysis:
books
Collection profiling
2. Compare
collection profile and
bibliographies
List of the bibliographic references
that have been cited by researchers
List of references to monographs
representing the materials that
should be available in a research-
level toxicology collection
Circulation data
Collection profiling (Inventory)

38. 03/14/16
COLLECTION EVALUATION AND DEVELOPMENT USING
CITATION ANALYSIS TECHNIQUES
Elizabeth Fuseler- McDowell
Colorado State University Libraries
Fort Collins, Colorado
Objectives:
1. Gather data for better collection management decisions;
2. Assess how well collection meets user needs;
3. Act as a check on the implementation of collection
development policies;
4. Review performance of current collection management
policies;
5. Reduce the inherent subjectivity of the selection/deselection
process.

39. 03/14/16

40. 03/14/16

41. 03/14/16
Framework of identifying useful data....
Problem Technique Data needed
1. Profile of the
collection
Citation
analysis:
journals
Cost-effectiveness factor
a. Subscription costs
b. Circulation data compiled for a
specified time period.
c. List of serials, ranked by annual
cost
2. Compare cost-
effectiveness and
impact factor
Impact factor (impact factor, along with
other data, can be found in ISI's Journal
Citation Reports (JCR).
List of serials

42. 03/14/16

43. 03/14/16
Framework of identifying useful data....
Problem Technique Data needed
Profile of
the
collection
Direct
collection
checking
Collection profiling
Shelf-list title count
Subject heading on the local catalog
Circulation records
Detailed information about imprints-
age, language of publication, percentage
of duplication, and subject coverage
Percentage of holdings by call number,
by copyright date

44. 03/14/16
Framework of identifying useful data....
Problem Technique Data needed
1. Profile of the
collection
Commercial
Products
Collection profiling (Inventory)
2. Compare collection
profile and analysis
tool
 Collection analysis tools
-WorldCat Collection
-Bowker’s Book Analysis System
- Ulrich’s Serial Analysis System
-Follet Library Resources’ Title Wise
- Sagebrush BenchMARC
- Spectra Dimension (Comparative Study)
 Comparative Statistics and Collection
Profile
 Gap Analysis

45. 03/14/16
Journals: Ulrich's Serials Analysis System
Wiersma, Gabrielle (2010). Collection Assessment in Response to Changing Curricula: An Analysis of the Biotechnology Resources at the University of
Colorado at Boulder. Retrieve from http://www.istl.org/10-spring/refereed1.html

46. 03/14/16
Journals: Ulrich's Serials Analysis System
Wiersma, Gabrielle (2010). Collection Assessment in Response to Changing Curricula: An Analysis of the Biotechnology Resources at the University of Colorado at
Boulder. Retrieve from http://www.istl.org/10-spring/refereed1.html

47. 03/14/16
Collection Profile using Spectra Dimension
Wiersma, Gabrielle (2010). Collection Assessment in Response to Changing Curricula: An Analysis of the Biotechnology Resources at the University of Colorado at
Boulder. Retrieve from http://www.istl.org/10-spring/refereed1.html

48. 03/14/16
Collection Profile using Spectra Dimension
Wiersma, Gabrielle (2010). Collection Assessment in Response to Changing Curricula: An Analysis of the Biotechnology Resources at the University of Colorado at
Boulder. Retrieve from http://www.istl.org/10-spring/refereed1.html

49. 03/14/16
Gap Analysis in Spectra Dimension

50. 03/14/16
Framework of identifying useful data....
Problem Technique Data needed
1. Profile of the
collection
Collection
mapping
Collection profiling (Inventory)
a. Number of items in the collection
treating the general topic
b. number of items in the collection treating
the specific topic.
2. Compare
collection profile and
analysis tool
Student population

51. 03/14/16
Framework of identifying useful data....
Problem Technique Data needed
1. Profile of the
collection
Collection
mapping
Collection profiling (Inventory)
a. Number of items in the collection
treating the general topic
b. number of items in the collection
treating the specific topic.
2. Collecting Level
(Conspectus)
Strengths of the collection
Acquisition Commitment
3. Total size of the
collection by the
number of students in
the school
Results of assessment (Problem No. 2)
Student population

52. Preparation of Collection Assessment Studies/Reports
Set purpose and objectives
Review previous research
Select data to be collected and methodology
Select population sample
Carry out a pilot study
Collect and analyze the data
Facilitate replication Clayton

53. 03/14/16
References:
Clayton, Peter. Managing Information Resources in Libraries: Collection Management in Theory and Practice. London:
Library Association Publishing, 2001.
Evans, G. Edwards. Collection Management Basic. 6th
ed. California: Libraries Unlimited, 2012.
Gregory, Vicki L. Collection Development and Management for 21st
Century Library Collections: An Introduction . New
York: Neal-Schuman Publishers, Inc., 2011.
Gorman and Clayton (2005). Qualitative Research for the Information Professional: A Practical Handbook, 2nd
ed. London:
Facet.
Hoffman, Frank and Wood, Richard. Library Collection Development Policies: Academic, Public and Special Libraries.
Maryland: The Scarecow Press, Inc. 2005.
Hughes, Christina. Qualitative and Quantitative Approaches to Social Research. Retrieve from
www2.warwick.ac.uk/.../researchprocess/quantitative_and_qualitative_ap
IFLA. Guidelines for a Collection Development Policy Using the Conspectus Model
http://www.ifla.org/files/assets/acquisition-collection-development/publications/gcdp-en.pdf
Johnson, Peggy. Fundamentals of collection development and management. 2nd
ed. Chicago: American Library
Association, 2009.
Johnson, Peggy. Collection Analysis: Overview. Retrieve from
www.ala.org/alcts/sites/ala.org.alcts/files/content/events/.../Johnson.ppt
Kachel, Debra E. Collection Evaluation: Knowing What You Have and What You Need, 2011. Retrieve from
https://www.webjunction.org/content/dam/.../ParticipantsDirections.pdf
Lakos A., Phipps S. Creating a culture of assessment: A catalyst for organizational change. Portal: Libraries & the Academy,
2004
Wood, Richard J. and Strauch, Katrina. Collection Assessment: A Look at the RLG Conspectus. New York: Haworth Press,
1992.
IFLA. Guidelines for a Collection Development Policy Using the Conspectus Model
http://www.ifla.org/files/assets/acquisition-collection-development/publications/gcdp-en.pdf

54. 03/14/16
Thank you very
much