Presentation at ELUNA, Richmond, 2009

1. Changing Data
Implementing Primo for the Tri-Universities Group (TUG)
Presentation at ELUNA
May, 2009
Alison Hitchens
Cataloguing & Metadata Librarian

2. Outline
 Background
 Loading data into Primo
 Normalization
 Testing

3. Where are we?

4.  Formed in 1995
 Shared resources and collaboration including:
 Shared storage facility
 Shared integrated library system (ILS)
 Reciprocal borrowing
 Document delivery
 Statistics portal
 Shared databases
 Collaborative functional committees

5.  Shared ILS and Catalogue
 TRELLIS (Voyager)
 No significant changes to interface in 10 years
 Search is limited to catalogue data

8.  One place to search
 Potential to include a variety of datasets:
 Library catalogue (currently loaded into Primo)
 Articles
 GIS information
 Our Ontario image bank
 Local repositories
 Deep search
Primo Advantages

9.  User-friendly interface
 XML compliant
 Avoids duplication of search results
 Groups together different editions of the
same work (FRBR)
 Interoperability with existing tools
Primo Advantages

12. The Primo Team

13. The Primo Team
 Team created in late January 2008
 Training held in late March 2008
 Primo Alpha launched to staff in July 2008
 Primo Beta launched to TUG community in
November 2008
 Goal: make Primo the primary search tool in
late May 2009

14. Phase Two
 Usability testing
 Naming & branding
 New data sources
 New books list
 Fine-tuning functionality
 Deep search

15. The Primo Team

16. Loading Data Into Primo
MARC

18. Loading Data Into Primo
MARC
MARC
XML
Extract

19. MARC XML

20. MARC XML

21. Loading Data Into Primo
MARC
MARC
XML
PNX (Primo
Normalized XML)
Extract
Normalization

22. PNX: Display area

23. PNX: Search area

24. PNX: Deduplication area

25. Loading Data Into Primo
MARC
MARC
XML
PNX (Primo
Normalized XML)
Deduplication, FRBR,
Didumean, Indexing
Front End
(user interface)
Extract
Normalization

26. Deduplication

27. Deduplication

28. Normalization: what is it?
 Massaging data
 Rules that tell the program how to get from MARC
XML to Primo Normalized XML (PNX)
 Filter that distributes the incoming data and places it in
different sections
 What MARC tags hold the title
 What MARC codes show the format
 What data should be included in searches
 What data should be available for display
 Transformation rules
 How that data should be formatted (dates, punctuation,
capitalization, etc.)

29. Normalization: what is it?
 Customization
 Fixing “bad” data
 Complex changes
 Consortial issues
 Lessons learned the hard way

30. Customization
 Search fields
 Created call number search
 Augmented title search with contents note (505 tag)
 Display fields
 Added subject tag used for slide collection subjects (654 tag)
 Added explanatory text in front of analytical titles
 FRBR
 Excluded “selections”
 Facets
 Used location names as collection facets

31. Adding contents note to title
search

32. Adding contents note to title
search

33. Adding contents note to title
search

34. Fixing “bad” data
 Old records lacking proper indicators
 Main author (100 tag) with invalid indicators (1st
indicator blank or |)
 Old records lacking subfield coding
 Uniform title (240 tag) missing subfields ($k)
 ISBNS with hyphens
 008 with invalid data in first 6 characters
 Blanks or letters instead of record creation date

35. 008 Workaround

36. 008 Workaround

37. 008 Workaround

38. 008 Workaround

39. Complex changes
 Tweaking delivery of online journals
 Delivery using SFX
 Exclude serials that no longer have an online holding
but record still coded as online
 Exclude government serials
 Exclude public microdata files
 Exclude databases (integrating resources)

40. Tweaking delivery of online
journals

41. Tweaking delivery of online
journals

42. Tweaking delivery of online
journals

43. Consortial issues
 Restricting online resources to individual institutions
 Which URL should be presented?
 Should restrictions be presented?
 Coping with shared locations
 e.g. GWINTER = Internet resource shared by Guelph and
Waterloo but not Wilfrid Laurier
 Instead of 2 separate locations UGINTER and UWINTER
 Creating search scopes for colleges and campuses
 e.g. ability to limit search to Architecture materials

44. Restricting Online Resources

45. Restricting Online Resources

46. Restricting Online Resources
 Problem 1:
 Which link belongs to which institution?
 Otherwise will simply present the first URL in the
record
 Need to add $$I based on ownership
 Location code isn’t extracted with the 856
 Problem 2:
 Restricting the each link to each institution
 Otherwise will give Online access message to users
who do not have access
 Need to add restricted delivery scope

47. Specify Institution for Online
Resources

48. Restricting Online Resources

49. Restricting Online Resources

50. Restricting Online Resources

51. Limiting searches to a location

52. Limiting searches to a location

53. Limiting searches to a location

54. Lessons learned the hard way
 If checking that a tag
exists, need to also
include subfields

55. Lessons learned the hard way
 If writing more than one
value, need more than
one rule
Result: $$I GUELPH

56. Lessons learned the hard way

57. Match current, match any?
Problem: includes title
from 245, author from 100,
publisher from 260 etc.

58. Match current, match any?
Match any
If any of the 880 tags have $6 505 then copy the 880
tag as is.
This means that if any of the 880s tag meet this
requirement, it will copy all of the 880 tags.
Match current: just analyse them one at a time and
only copy the one that meets the condition

59. Testing
 Staging database
 What am I testing?
 Testing sample records
 Testing the process
 Random testing

60. Test specific changes
 Changes to normalization rules
 Changes to front end display
 Changes to tables (e.g. new location codes)
 New release enhancements/bug fixes
Look for:
 What you were expecting
 Note any surprises!

61. Staging database
 Holds 200,000 records
 Random sample of our collection
 100 titles from each location code
 Random sample proportionate to records held by
each institution
 Combination of old pre-TRELLIS records and
newly created records
 Shakespeare call number range to test grouping
of editions (FRBR)

62. Test records
 100 records for repeated testing in front end
 Brief records (acquisitions, e-reserves, CODOC)
 Different formats (micro, music, video, electronic)
 Things to test holdings info (acc. material, multi-volume,
multiple items, multiple locations)
 Foreign language materials
 Duplicates
 Editions
 10 records
 For immediate test of normalization rules in back office

63. Test specific functionality
 Fulfillment cycle
 When the user finds the item that he wants, can
he actually get the item based on:
 The information presented in the results screen
 The information presented in the full display
 The linking provided
 The information presented in the holdings display

64. Testing: online resources
 What am I testing?
(what do I want to happen)
 Is online availability showing correctly in relation
to the user?
 Online access
 Online access is restricted
 Physical resource

65. Testing: online resources
 What am I testing?
(what do I want to happen)
 Does user receive a relevant link?
 SFX delivery
 Direct to resource
 Link appropriate to institution

66. Testing: online resources
 What am I testing?
(what do I want to happen)
 Can user view alternate/multiple links in the full
record display?

67. Testing: Online resources
 Variables for sample records
 E-journals, e-books, e-data, databases
 SFX delivery, online delivery
 Multi-volume sets
 Restricted to one institution
 Different institutions, different providers
 Online for one institution, physical for another

68. Testing: online resources
 Instructions for testers:
 Each tester should check the list of records and
verify that there is an online link
 For each record test that the link label is correct:
 Available online: check against TRELLIS to verify that
your institution has an online holding
 Online access is restricted: check against TRELLIS to
verify that your institution does NOT have an online
holding
 Link takes you to the correct place

69. Testing: online resources
 Random sampling
 Each tester should also do a search on a subject
of their choice and verify links using the first page
of results

70. Testing environment
 Test in all views
 Waterloo, Laurier, Guelph
 Test in different IP ranges
 Test off campus

71. General testing

72. Testing: feedback from users
Overall, I am VERY
impressed with Primo. It
is far more functional in
many ways.
When I find an online
journal, the Click here
for access link does
not work
I wonder if the alerts by
email are working?
Let me login using
my UWDIR id

73. Changing Data
 Thank you!
Alison Hitchens
Cataloguing & Metadata Librarian
University of Waterloo Library
ahitchen@library.uwaterloo.ca
http://www.lib.uwaterloo.ca