Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Discovery-ing Symphony MARC records
Who
• Gary Steele Library Systems Manager, Glasgow Caledonian University
• Bernard Sca...
Part A:
Getting bib records from
Symphony to the discover layer
Getting bib records from Symphony to Summon
• Summon 2.0
• Getting bib records from Symphony to Summon
• New/Updated recor...
4
Summon 2.0
5
Getting bib records from Symphony to Summon
• Content type, field and location mapping for Summon import
• Update and co...
6
New/Updated records report
• Use customised report created by Anne called gcalsummon.pl
• Selection:
– Item type: Exclud...
7
Export record
8
Deleted records report
• Use customised report I created under the guidance of Anne
• Selection:
– Records deleted on th...
9
1. Parse history logs
cat log.hist | seltrans -cFV -otJ | sort | uniq >allkeys
• Select all occurrence of “FV” (Remove I...
10
2. Lookup catalogue keys
cat allkeys | selcatalog -iC -oC | sort | uniq >presentkeys
• Lookup catalogue keys extracted ...
11
3. Compare files to find deleted keys
• comm command compares two
files
• -3 suppresses keys in both
files
• -1 suppres...
12
4. FTP flat file of deleted keys to Summon
tr -d '|' < deletedkeys >summonkeys
• File “translated” up to remove “|”
• R...
Getting bib records
from Symphony to
Primo
Bernard Scaife, Head of Technical
Services
Exporting…
Sirsidynix custom report – Export bibs (exportbibs.pl)
Criteria is:
selcatalog -f"~ILLLIB,EQUIP,MEMO,ROOM,SELEC...
Exporting…
News… Updates…
Timing…
Symphony (7 days per week):
0:45 – Find deleted records for Primo
1:00 – Export “New” marc records
1:10 - Export “...
Prime…
Special initialisation process
Symphony: Run same export.pl report – no date filter
Primo: Special “Prime” pipe (dr...
File management…
Reports save to /Xfer folder in Symphony
FTP is run as a part of the import routine from Primo, so we
had...
File format and processing
Primo consumers Marc exchange format (.mrc)
Classmark and location information is exported in t...
What about deletions?
More tricky…
getdeleteds.pl custom report compares today’s catkeys with yesterday’s:
Step A
system("...
Next part of getdeleteds.pl
Config
Lots of work to set up!
Mapping of marc fields to Primo normalised fields
Complex logic rules which can draw upon m...
Symphony and EBSCO Discovery
Kathy Sadler, Systems Librarian
@tatielane
Getting bib records from
Symphony to EDS
Getting bib records from
Symphony to EDS
Before my time at Cranfield:
• Started with Summon
• Commissioned a SirsiDynix cu...
Full data extraction custom report
I copied the Summon custom report and tweaked it for EDS
• It catalogdumps selected rec...
Item selections
Exclude any locations
that are shadowed
Title selections
Exclude interlibrary loans
and anything else that we
don’t want “discovered”
When indexing is complete
• EBSCO sends a confirmation email – usually same day, else next day
Daily adds/updates
Custom report ebscoftp-upd is a copy of the full data extraction report
• Only difference in script is ...
When the update has been indexed
• EBSCO sends a confirmation email, usually later same day
What about deletions?
• We keep it simple and don’t do anything special for EDS
• Items for deletion get checked out to on...
Our other catalogue does things
differently
• Koha catalogue at Cranfield Defence and Security in Shrivenham
• EBSCO harve...
Part B:
Real time availability in the
discover layer
Real time availability in Summon
• Availability in OPAC
• Availability in Summon
• Web Services
• Availability in Web Serv...
35
Availability in OPAC
36
Availability in Summon
37
Web Services
• Provide simplified remote access to features of Symphony
• Download available from Support Center
• Free...
38
Web Services xml
Base URL: http://193.62.224.60:8080/symws/
Protocol: rest/
Service: standard/
Request: lookupTitleInfo...
39
Availability in Web Services
40
Current location mapping
Location ID Location Description
CHECKEDOUT On Loan
ST0 Open Shelves - Level 0
ST1 Open Shelve...
41
Xml mapping
Real Time Availability
(RTA)
Bernard Scaife, Head of Technical
Services
Two types…
1. Via Symphony web services
2. Via the item details display
How RTA works on Primo
RTA (2): Item details tab
RTA (2): Item details tab
Typical Primo record:
RTA (2): Source record in ilink
RTA (2): Bits we want to hide
RTA (2): environment
RTA (2): User Profile
RTA (2): User Access
RTA (2): Circulation user
RTA (2): Find environment id
Run listpol report to get envn[nn].env
File is envn13.env /WebCat/Config/
RTA (2): create env file
Add a line
CUSTOMCSS|primo.css|
Points to file
Create primo.css and put in the css folder on
the ...
Do
The
Css!
RTA (2): How do we hide the
redundant areas? #3
In Primo, the calling url has &userid=PRIMO
44IOE_LMS_holdings
http://ioe....
Symphony and EBSCO Discovery
Kathy Sadler, Systems Librarian
@tatielane
Real time availability
We use Z39.50 for both catalogues
• May need to open port if not open already
SYSTEMCONFIG/MODULE/DISPLAY Catalog Zserver ...
Workflows configuration
• Workflows FAQ: “Entering item-level holdings for Z39.50 displays”
– 926 Item level holdings
– 92...
Opac Configuration
• Opac Configuration > Z39.50 Format > BIBMARC > ModifyAdd 926 + 927 Entry IDs
and entries to Z39.50 Fo...
Cataloguing Configuration
Create Entry ID policies
Cataloguing Configuration
Add 926+927 entries to
Catalog Formats you use
Configure RTAC headings in
EBSCOadmin
Multiple holdings
retrievable if clicked
Questions, comments, queries…
Summon
Gary Steele
Library Systems Manager, Glasgow Caledonian University
gary.steele@gcu.ac...
Upcoming SlideShare
Loading in …5
×

Discovery-ing Symphony MARC records

501 views

Published on

Presented by Gary Steele, Bernard Scaife and Kathy Sadler at COSI-EMEA Conference in Newcastle 10th July 2015

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Discovery-ing Symphony MARC records

  1. 1. Discovery-ing Symphony MARC records Who • Gary Steele Library Systems Manager, Glasgow Caledonian University • Bernard Scaife Head of Technical Services, UCL Institute of Education • Kathy Sadler Systems Librarian, Cranfield University What • Each use a different discovery layer; Summon, Primo, EDS • Get bib records from Symphony to the discover layer • Enable real time availability in the discover layer Why • All discovery layers rely on MARC records and holdings details but there’s more than one way to get this data out of Symphony…
  2. 2. Part A: Getting bib records from Symphony to the discover layer
  3. 3. Getting bib records from Symphony to Summon • Summon 2.0 • Getting bib records from Symphony to Summon • New/Updated records report • Deleted records report
  4. 4. 4 Summon 2.0
  5. 5. 5 Getting bib records from Symphony to Summon • Content type, field and location mapping for Summon import • Update and correct LDR, 007, and 008 fields in MARC records • Do not export equipment records (e.g. laptops) or eJournal records • Do export all books, ebooks, journal, ejournals, statutes, law reports, theses, etheses, dissertations, archive items, DVDs, software, and databases. • Schedule – Deleted records report: Every night at 23:00 – New/Updated records report: Every night at 23:10 – Full Catalogue dump: Every four months
  6. 6. 6 New/Updated records report • Use customised report created by Anne called gcalsummon.pl • Selection: – Item type: Exclude (~) items type (equipment and ejournals) – Date modified: Day report is run (D0:D0) • Process: – Select catkeys of records based on criteria above – Extract MARC records for selected catkeys – Rename file to gcal-catalog-updates-YYYY-MM-DD-HH-MM- SS.mrc – FTP file to Summon “Updates” folder
  7. 7. 7 Export record
  8. 8. 8 Deleted records report • Use customised report I created under the guidance of Anne • Selection: – Records deleted on the day the report is run • Process: – Parse history logs – Lookup catalogue keys – Compare files to find deleted keys – FTP flat file of deleted keys to Summon
  9. 9. 9 1. Parse history logs cat log.hist | seltrans -cFV -otJ | sort | uniq >allkeys • Select all occurrence of “FV” (Remove Item Part B) command code in the history log and output “tJ” datacode (Catalogue key number) • Sort results and remove duplicates • Write keys returned to new file (allkeys) E201507081255540077R ^S55FVFFDCR3^FEGCAL^FcNONE^FWDCR3^ NQ506021-1001^IQT610^IS1^NOY^ILENQUIRIES^INENQUIRIES^ NSGCAL^IGT^tJ506021^IUa506021^IKTHESIS^IF2008^Fv3000000^^O
  10. 10. 10 2. Lookup catalogue keys cat allkeys | selcatalog -iC -oC | sort | uniq >presentkeys • Lookup catalogue keys extracted from history logs • Catalogue keys for deleted records will return as errors • Sort results and remove duplicates • Write keys returned to new file (presentkeys) **error number 111 on catalog start, key=18622 flex=i9780406237002 **error number 111 on catalog start, key=198847 flex=i9780406237002 **error number 111 on catalog start, key=237053 flex=i9780406237002 **error number 111 on catalog start, key=237143 flex=i9780406237002 **error number 111 on catalog start, key=237145 flex=i9780406237002 **error number 111 on catalog start, key=237147 flex=i9780406237002 **error number 111 on catalog start, key=237148 flex=i9780406237002 **error number 111 on catalog start, key=505999 flex=a505863 **error number 111 on catalog start, key=506002 flex=a506000
  11. 11. 11 3. Compare files to find deleted keys • comm command compares two files • -3 suppresses keys in both files • -1 suppresses keys unique to presentkeys • Leaves keys that are unique to allkeys, and these keys are the deleted keys comm -31 presentkeys allkeys >deletedkeys
  12. 12. 12 4. FTP flat file of deleted keys to Summon tr -d '|' < deletedkeys >summonkeys • File “translated” up to remove “|” • Renamed to “gcal-catalog-YYYY-MM-DD-HH-MM-SS.deletes” • FTP file to Summon “Deletes” folder 18622 198847 237053 237143 237145 237147 237148 505999 506002
  13. 13. Getting bib records from Symphony to Primo Bernard Scaife, Head of Technical Services
  14. 14. Exporting… Sirsidynix custom report – Export bibs (exportbibs.pl) Criteria is: selcatalog -f"~ILLLIB,EQUIP,MEMO,ROOM,SELECT" -6"0" $Directives{'selcatalogoptions'} -oC | catalogdump -h -j -n"oemjunktags" -t"926" -ka998 -om -z 1>$xferdir/$exportedbibrecs Twice (new records, updated records)
  15. 15. Exporting… News… Updates…
  16. 16. Timing… Symphony (7 days per week): 0:45 – Find deleted records for Primo 1:00 – Export “New” marc records 1:10 - Export “Updated” marc records 1:15 – Export Primo catkeys Primo (7 days per week) 4:00 - 44IOE_LMS_ONGOING_ADD_PIPE 4:15 - 44IOE_LMS_ONGOING_UPDATE_PIPE 4:30 - 44IOE_LMS_ONGOING_DEL_PIPE
  17. 17. Prime… Special initialisation process Symphony: Run same export.pl report – no date filter Primo: Special “Prime” pipe (drops/recreates index) When do we need to do this? Timing: Symphony export (10m)
  18. 18. File management… Reports save to /Xfer folder in Symphony FTP is run as a part of the import routine from Primo, so we had to configure a Sirsi unix account and tell Primo where to pick up the file from and its name
  19. 19. File format and processing Primo consumers Marc exchange format (.mrc) Classmark and location information is exported in the 926 field for further processing Primo runs the following functions on the imported file: Normalisation, Enrichment, FRBR, Deduplication
  20. 20. What about deletions? More tricky… getdeleteds.pl custom report compares today’s catkeys with yesterday’s: Step A system("selcatalog -f"~ILLLIB,EQUIP,MEMO,ROOM,SELECT" -6"0" -oC $Directives{'selcatalogoptions'} 1>$xferdir/$exportedbibkeys"); Step B system("comm -3 $xferdir/$exportedbibkeys $xferdir/$bibkeysyesterday 1>$xferdir/$deletedreckeys");
  21. 21. Next part of getdeleteds.pl
  22. 22. Config Lots of work to set up! Mapping of marc fields to Primo normalised fields Complex logic rules which can draw upon multiple marc fields in order to create new fields. Flagged the source as “IOE Library Catalogue” Created a backlink to the full record using catkey
  23. 23. Symphony and EBSCO Discovery Kathy Sadler, Systems Librarian @tatielane Getting bib records from Symphony to EDS
  24. 24. Getting bib records from Symphony to EDS Before my time at Cranfield: • Started with Summon • Commissioned a SirsiDynix custom report to export to Summon • Never really implemented Summon fully • Did a market review, moved to EDS
  25. 25. Full data extraction custom report I copied the Summon custom report and tweaked it for EDS • It catalogdumps selected records and makes an export file in /tmp • It FTPs the file to EBSCO into the “full” directory We run it weekly on a Sunday at 05:00 • It takes about 23 minutes (292,000 records) • The finished report is emailed to my libsysadmin inbox
  26. 26. Item selections Exclude any locations that are shadowed
  27. 27. Title selections Exclude interlibrary loans and anything else that we don’t want “discovered”
  28. 28. When indexing is complete • EBSCO sends a confirmation email – usually same day, else next day
  29. 29. Daily adds/updates Custom report ebscoftp-upd is a copy of the full data extraction report • Only difference in script is that it FTPs into the “update” directory diff ebscoftp-full.pl ebscoftp-upd.pl < my $ftp_dir = "full"; --- > my $ftp_dir = "update"; • Runs daily except Sunday, after midnight • Takes 6 minutes
  30. 30. When the update has been indexed • EBSCO sends a confirmation email, usually later same day
  31. 31. What about deletions? • We keep it simple and don’t do anything special for EDS • Items for deletion get checked out to one of three shadowed locations [Auditors required us to adopt this practice many years ago] – WITHDRAWN [deliberately decided to withdraw] – GONE4GOOD [lost or missing, gave up searching] – DUFFITEM [test items and catalogue errors] • Cvtdiscard and Remdiscard run weekly before EBSCO Full Extract • This works fine for us and we haven’t experienced any issues
  32. 32. Our other catalogue does things differently • Koha catalogue at Cranfield Defence and Security in Shrivenham • EBSCO harvests the catalogue directly via OAI-PMH • Weekly full harvest, daily updates • No data extraction required, it’s all done for us!
  33. 33. Part B: Real time availability in the discover layer
  34. 34. Real time availability in Summon • Availability in OPAC • Availability in Summon • Web Services • Availability in Web Services • Web services xml • Current location mapping • Xml mapping
  35. 35. 35 Availability in OPAC
  36. 36. 36 Availability in Summon
  37. 37. 37 Web Services • Provide simplified remote access to features of Symphony • Download available from Support Center • Free to install and use for “standard” services • Setup guide explains how to install Web Services • SDK explains how to use Web Services • https://support.sirsidynix.com/zh-hans/node/55009
  38. 38. 38 Web Services xml Base URL: http://193.62.224.60:8080/symws/ Protocol: rest/ Service: standard/ Request: lookupTitleInfo? Client: clientID=SummonTiree Item ID: &titleID=201554 Parameter: &includeItemInfo=true http://193.62.224.60:8080/symws/rest/standard/lookupTitleInfo?clie ntID=SummonTiree&titleID=201554&includeItemInfo=true
  39. 39. 39 Availability in Web Services
  40. 40. 40 Current location mapping Location ID Location Description CHECKEDOUT On Loan ST0 Open Shelves - Level 0 ST1 Open Shelves - Level 1 ST2 Open Shelves - Level 2 ST3 Open Shelves - Level 3 ST4 Open Shelves - Level 4 B0 Compact Shelving - Level 0 B2 Compact Shelving - Level 2 B3 Compact Shelving - Level 3 B4 Compact Shelving - Level 4
  41. 41. 41 Xml mapping
  42. 42. Real Time Availability (RTA) Bernard Scaife, Head of Technical Services
  43. 43. Two types… 1. Via Symphony web services 2. Via the item details display
  44. 44. How RTA works on Primo
  45. 45. RTA (2): Item details tab
  46. 46. RTA (2): Item details tab Typical Primo record:
  47. 47. RTA (2): Source record in ilink
  48. 48. RTA (2): Bits we want to hide
  49. 49. RTA (2): environment
  50. 50. RTA (2): User Profile
  51. 51. RTA (2): User Access
  52. 52. RTA (2): Circulation user
  53. 53. RTA (2): Find environment id Run listpol report to get envn[nn].env File is envn13.env /WebCat/Config/
  54. 54. RTA (2): create env file Add a line CUSTOMCSS|primo.css| Points to file Create primo.css and put in the css folder on the server (Webcat/Config/Css/.)
  55. 55. Do The Css!
  56. 56. RTA (2): How do we hide the redundant areas? #3 In Primo, the calling url has &userid=PRIMO 44IOE_LMS_holdings http://ioe.sirsidynix.net.uk/uhtbin/cgisirsi/x/0/0/5 7/5/3?searchdata1={{control/addsrcrecordid}}{ CKEY}&searchfield1=GENERAL^SUBJECT^G ENERAL^^&user_id=PRIMO
  57. 57. Symphony and EBSCO Discovery Kathy Sadler, Systems Librarian @tatielane Real time availability
  58. 58. We use Z39.50 for both catalogues • May need to open port if not open already SYSTEMCONFIG/MODULE/DISPLAY Catalog Zserver 19/06/2015 (To return to previous level, press the ESCAPE key <ESC>.) ------------------------------------------------------------------------------- * Name : ZSERVER * Description : ZSERVER * Max Clients : 30 * Port Number : 2200 * Environment : Z3950 * User Access : Z3950 * Thesaurus : GENERAL * Allow Browsing : YES * Session Logging : YES * Text Logging : NO * Timeout : 300 * Initialization Timeout : 60 * User Library : CRANFIELD * User Clearance : NONE * Truncation Limit : 100 * Local Search : NO Press ENTER or RETURN to continue (CTL-F TO SCROLL)[]
  59. 59. Workflows configuration • Workflows FAQ: “Entering item-level holdings for Z39.50 displays” – 926 Item level holdings – 927 MARC holdings • Opac Configuration >Z39.50 format – Add 926 & 927 to BIBMARC and SIRSICAT formats • Cataloguing Configuration > Entry ID • Cataloguing Configuration > Catalog format – Add 926 & 927 to every format you use
  60. 60. Opac Configuration • Opac Configuration > Z39.50 Format > BIBMARC > ModifyAdd 926 + 927 Entry IDs and entries to Z39.50 Format for BIBMARC, SIRSICAT
  61. 61. Cataloguing Configuration Create Entry ID policies
  62. 62. Cataloguing Configuration Add 926+927 entries to Catalog Formats you use
  63. 63. Configure RTAC headings in EBSCOadmin
  64. 64. Multiple holdings retrievable if clicked
  65. 65. Questions, comments, queries… Summon Gary Steele Library Systems Manager, Glasgow Caledonian University gary.steele@gcu.ac.uk Primo Bernard Scaife Technical Services Manager, UCL Institute of Education b.scaife@ioe.ac.uk EDS Kathy Sadler Systems Librarian, Cranfield University k.e.sadler@cranfield.ac.uk @tatielane

×