SlideShare a Scribd company logo
Digital Content Integrated with
ILS Data for User Discovery:
Lessons Learned
(A Real Use Case for RDF!)
C ODE 4 L I B 2 0 1 5
PORTLAND , OR
FE B RUA RY 11
Naomi Dushay / ndushay@stanford.edu
Laney McGlohon / laneymcg@stanford.edu
Code4lib Digital Content Integrated with ILS Data for User Discovery: Lessons Learned
Indexing
Code Solr Master
Index
ILS DOR
XML
Data
Harvestor
Indexing Code
Course
Reserves
MARC
Data Sources
4
Digital Collections Have Items
Item
MODS
Collection
MODS
Item
MODS
Collection
MARC
Item
MODS
Collection
MARC
Item
MARC
To merge or not to merge?
Code4lib Digital Content Integrated with ILS Data for User Discovery: Lessons Learned
Our Data is Ugly (shhhhh ….)
We Have Dups in ILS data
• Data isn’t “work” based
• Catalog card data conversion
We Don’t Store Every Field in SearchWorks Solr
• Search performance, storage
• Can’t use atomic updates to Solr doc (only fields changed)
• Have to recreate Solr doc from MARC, with same code
• Digital Content Workflow Predated SearchWorks
• Multiple Workflows
• Poor Metadata QA

https://flic.kr/p/bQJRaZ

Indexing
Code Solr Master
Index
ILS DOR
XML
Data
Harvestor
Indexing Code
Course
Reserves
MARC
Coll Rec In|Outside ILS, Items Outside the ILS


https://flic.kr/p/7Q5RuB
Indexing
Code
Solr Master
Index
ILS DOR
XML
Data
Harvestor
Indexing Code
Course
Reserves
MARC
Merge Manager
App to Manage Multiple Sources
App to Manage Multiple Sources
Fail.
Why?
• Performance
• Storing Solr document pieces (in a database)
• Adding non-trivial app for writes to Solr
• Complexity

Indexing
Code Solr Master
Index
ILS DOR
XML
Data
Harvestor
Indexing Code
Course
Reserves
MARC
Coll Rec In|Outside ILS, Items In|Outside the ILS

Fail.
Why?
• ILS updates and steps on digital work info
• ILS feed granularity issues
• Repeated Re-indexing

Coll Rec In|Outside ILS, Items In|Outside the ILS
Solr Atomic Updates
We Don’t Store Every Field in SearchWorks Solr – Currently
• Search performance, storage
• Can’t use atomic updates to Solr doc (only fields changed)
• Have to recreate Solr doc from MARC, with same code
Would Require:
• Performance Testing / Load Testing
• Changes to stable ILS MARC  Solr code
• Changes to stable ILS course reserve  Solr code
• Changes to digital content metadata  Solr code

Indexing
Code Solr Master
Index
ILS DOR
XML
Data
Harvestor
Indexing Code
Course
Reserves
MARC
Let There Be (More) Dups!

Automagically Amend ILS MARC
ILS DOR
Course
Reserves
MARC
Solr Master
Index
Indexing
Code + UI code

StanfordSync & Embeddable Digital Objects
Thank You!
February 11th, 2015
ndushay@stanford.edu
laneymcg@stanford.edu

More Related Content

What's hot

WLIA - 2015 Fall Regional, Oshkosh WI
WLIA - 2015 Fall Regional, Oshkosh WIWLIA - 2015 Fall Regional, Oshkosh WI
WLIA - 2015 Fall Regional, Oshkosh WI
Wisconsin State Cartographer's Office
 
SAS Online Training
SAS Online TrainingSAS Online Training
SAS Online Training
Nagendra Kumar
 
Linq in asp.net
Linq in asp.netLinq in asp.net
C# Code Samples
C# Code SamplesC# Code Samples
C# Code Samples
astruck
 
Oracle apps technical online training in USA
Oracle apps technical online training in USAOracle apps technical online training in USA
Oracle apps technical online training in USA
Jeri Robinson
 
AngularJS 1.x - your first application (problems and solutions)
AngularJS 1.x - your first application (problems and solutions)AngularJS 1.x - your first application (problems and solutions)
AngularJS 1.x - your first application (problems and solutions)
Igor Talevski
 
Knowledge_Based_Systems_Siemens
Knowledge_Based_Systems_SiemensKnowledge_Based_Systems_Siemens
Knowledge_Based_Systems_Siemens
Vinay Bhat
 
Ciel, mes données ne sont plus relationnelles
Ciel, mes données ne sont plus relationnellesCiel, mes données ne sont plus relationnelles
Ciel, mes données ne sont plus relationnelles
Xavier Gorse
 
MarcEdit for Everyone with Katie Dunn
MarcEdit for Everyone with Katie DunnMarcEdit for Everyone with Katie Dunn
MarcEdit for Everyone with Katie Dunn
WiLS
 
Introduction to functional programming with JavaScript
Introduction to functional programming with JavaScriptIntroduction to functional programming with JavaScript
Introduction to functional programming with JavaScript
Farzaneh Orak
 
Business Analytics with R
Business Analytics with RBusiness Analytics with R
Business Analytics with R
Edureka!
 

What's hot (11)

WLIA - 2015 Fall Regional, Oshkosh WI
WLIA - 2015 Fall Regional, Oshkosh WIWLIA - 2015 Fall Regional, Oshkosh WI
WLIA - 2015 Fall Regional, Oshkosh WI
 
SAS Online Training
SAS Online TrainingSAS Online Training
SAS Online Training
 
Linq in asp.net
Linq in asp.netLinq in asp.net
Linq in asp.net
 
C# Code Samples
C# Code SamplesC# Code Samples
C# Code Samples
 
Oracle apps technical online training in USA
Oracle apps technical online training in USAOracle apps technical online training in USA
Oracle apps technical online training in USA
 
AngularJS 1.x - your first application (problems and solutions)
AngularJS 1.x - your first application (problems and solutions)AngularJS 1.x - your first application (problems and solutions)
AngularJS 1.x - your first application (problems and solutions)
 
Knowledge_Based_Systems_Siemens
Knowledge_Based_Systems_SiemensKnowledge_Based_Systems_Siemens
Knowledge_Based_Systems_Siemens
 
Ciel, mes données ne sont plus relationnelles
Ciel, mes données ne sont plus relationnellesCiel, mes données ne sont plus relationnelles
Ciel, mes données ne sont plus relationnelles
 
MarcEdit for Everyone with Katie Dunn
MarcEdit for Everyone with Katie DunnMarcEdit for Everyone with Katie Dunn
MarcEdit for Everyone with Katie Dunn
 
Introduction to functional programming with JavaScript
Introduction to functional programming with JavaScriptIntroduction to functional programming with JavaScript
Introduction to functional programming with JavaScript
 
Business Analytics with R
Business Analytics with RBusiness Analytics with R
Business Analytics with R
 

Similar to Code4lib Digital Content Integrated with ILS Data for User Discovery: Lessons Learned

Introduction to Object-Relational Mapping
Introduction to Object-Relational MappingIntroduction to Object-Relational Mapping
Introduction to Object-Relational Mapping
Ali Shakiba
 
ORM Methodology
ORM MethodologyORM Methodology
ORM Methodology
Ahmed Gomaa
 
Spark Summit EU talk by Bas Geerdink
Spark Summit EU talk by Bas GeerdinkSpark Summit EU talk by Bas Geerdink
Spark Summit EU talk by Bas Geerdink
Spark Summit
 
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
Jose Quesada (hiring)
 
TechEvent 2019: Oracle to PostgreSQL - a Travel Guide from Practice; Roland S...
TechEvent 2019: Oracle to PostgreSQL - a Travel Guide from Practice; Roland S...TechEvent 2019: Oracle to PostgreSQL - a Travel Guide from Practice; Roland S...
TechEvent 2019: Oracle to PostgreSQL - a Travel Guide from Practice; Roland S...
Trivadis
 
Intro
IntroIntro
AWS Customer Presentation: Freie Univerisitat - Berlin Summit 2012
AWS Customer Presentation: Freie Univerisitat - Berlin Summit 2012AWS Customer Presentation: Freie Univerisitat - Berlin Summit 2012
AWS Customer Presentation: Freie Univerisitat - Berlin Summit 2012
Amazon Web Services
 
AWS Summit Berlin 2012 Talk on Web Data Commons
AWS Summit Berlin 2012 Talk on Web Data CommonsAWS Summit Berlin 2012 Talk on Web Data Commons
AWS Summit Berlin 2012 Talk on Web Data Commons
Hannes Mühleisen
 
The Evolution of the Oracle Database - Then, Now and Later (Fontys Hogeschool...
The Evolution of the Oracle Database - Then, Now and Later (Fontys Hogeschool...The Evolution of the Oracle Database - Then, Now and Later (Fontys Hogeschool...
The Evolution of the Oracle Database - Then, Now and Later (Fontys Hogeschool...
Lucas Jellema
 
Solr @ eBay Kleinanzeigen
Solr @ eBay KleinanzeigenSolr @ eBay Kleinanzeigen
Solr @ eBay Kleinanzeigen
Lucidworks (Archived)
 
Echoes Project
Echoes ProjectEchoes Project
MarcEdit Tools and Workflows
MarcEdit Tools and WorkflowsMarcEdit Tools and Workflows
MarcEdit Tools and Workflows
WiLS
 
Tagging search solution design Advanced edition
Tagging search solution design Advanced editionTagging search solution design Advanced edition
Tagging search solution design Advanced edition
Alexander Tokarev
 
EDB's Migration Portal - Migrate from Oracle to Postgres
EDB's Migration Portal - Migrate from Oracle to PostgresEDB's Migration Portal - Migrate from Oracle to Postgres
EDB's Migration Portal - Migrate from Oracle to Postgres
EDB
 
Migrating from Redshift to Spark at Stitch Fix: Spark Summit East talk by Sky...
Migrating from Redshift to Spark at Stitch Fix: Spark Summit East talk by Sky...Migrating from Redshift to Spark at Stitch Fix: Spark Summit East talk by Sky...
Migrating from Redshift to Spark at Stitch Fix: Spark Summit East talk by Sky...
Spark Summit
 
Migration from Redshift to Spark
Migration from Redshift to SparkMigration from Redshift to Spark
Migration from Redshift to Spark
Sky Yin
 
Making RDA Easy(er) with MarcEdit
Making RDA Easy(er) with MarcEditMaking RDA Easy(er) with MarcEdit
Making RDA Easy(er) with MarcEdit
Terry Reese
 
Linked Data from a Digital Object Management System
Linked Data from a Digital Object Management SystemLinked Data from a Digital Object Management System
Linked Data from a Digital Object Management System
Uldis Bojars
 
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...
Lucas Jellema
 
From Legacy Web Application To SharePoint - a case study
From Legacy Web Application To SharePoint - a case studyFrom Legacy Web Application To SharePoint - a case study
From Legacy Web Application To SharePoint - a case study
Elizabeth Szabo
 

Similar to Code4lib Digital Content Integrated with ILS Data for User Discovery: Lessons Learned (20)

Introduction to Object-Relational Mapping
Introduction to Object-Relational MappingIntroduction to Object-Relational Mapping
Introduction to Object-Relational Mapping
 
ORM Methodology
ORM MethodologyORM Methodology
ORM Methodology
 
Spark Summit EU talk by Bas Geerdink
Spark Summit EU talk by Bas GeerdinkSpark Summit EU talk by Bas Geerdink
Spark Summit EU talk by Bas Geerdink
 
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
 
TechEvent 2019: Oracle to PostgreSQL - a Travel Guide from Practice; Roland S...
TechEvent 2019: Oracle to PostgreSQL - a Travel Guide from Practice; Roland S...TechEvent 2019: Oracle to PostgreSQL - a Travel Guide from Practice; Roland S...
TechEvent 2019: Oracle to PostgreSQL - a Travel Guide from Practice; Roland S...
 
Intro
IntroIntro
Intro
 
AWS Customer Presentation: Freie Univerisitat - Berlin Summit 2012
AWS Customer Presentation: Freie Univerisitat - Berlin Summit 2012AWS Customer Presentation: Freie Univerisitat - Berlin Summit 2012
AWS Customer Presentation: Freie Univerisitat - Berlin Summit 2012
 
AWS Summit Berlin 2012 Talk on Web Data Commons
AWS Summit Berlin 2012 Talk on Web Data CommonsAWS Summit Berlin 2012 Talk on Web Data Commons
AWS Summit Berlin 2012 Talk on Web Data Commons
 
The Evolution of the Oracle Database - Then, Now and Later (Fontys Hogeschool...
The Evolution of the Oracle Database - Then, Now and Later (Fontys Hogeschool...The Evolution of the Oracle Database - Then, Now and Later (Fontys Hogeschool...
The Evolution of the Oracle Database - Then, Now and Later (Fontys Hogeschool...
 
Solr @ eBay Kleinanzeigen
Solr @ eBay KleinanzeigenSolr @ eBay Kleinanzeigen
Solr @ eBay Kleinanzeigen
 
Echoes Project
Echoes ProjectEchoes Project
Echoes Project
 
MarcEdit Tools and Workflows
MarcEdit Tools and WorkflowsMarcEdit Tools and Workflows
MarcEdit Tools and Workflows
 
Tagging search solution design Advanced edition
Tagging search solution design Advanced editionTagging search solution design Advanced edition
Tagging search solution design Advanced edition
 
EDB's Migration Portal - Migrate from Oracle to Postgres
EDB's Migration Portal - Migrate from Oracle to PostgresEDB's Migration Portal - Migrate from Oracle to Postgres
EDB's Migration Portal - Migrate from Oracle to Postgres
 
Migrating from Redshift to Spark at Stitch Fix: Spark Summit East talk by Sky...
Migrating from Redshift to Spark at Stitch Fix: Spark Summit East talk by Sky...Migrating from Redshift to Spark at Stitch Fix: Spark Summit East talk by Sky...
Migrating from Redshift to Spark at Stitch Fix: Spark Summit East talk by Sky...
 
Migration from Redshift to Spark
Migration from Redshift to SparkMigration from Redshift to Spark
Migration from Redshift to Spark
 
Making RDA Easy(er) with MarcEdit
Making RDA Easy(er) with MarcEditMaking RDA Easy(er) with MarcEdit
Making RDA Easy(er) with MarcEdit
 
Linked Data from a Digital Object Management System
Linked Data from a Digital Object Management SystemLinked Data from a Digital Object Management System
Linked Data from a Digital Object Management System
 
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...
 
From Legacy Web Application To SharePoint - a case study
From Legacy Web Application To SharePoint - a case studyFrom Legacy Web Application To SharePoint - a case study
From Legacy Web Application To SharePoint - a case study
 

Recently uploaded

Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Muhammad Ali
 
Patch Tuesday de julio
Patch Tuesday de julioPatch Tuesday de julio
Patch Tuesday de julio
Ivanti
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
SynapseIndia
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
Matthias Neugebauer
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
Zilliz
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
Safe Software
 
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
Priyanka Aash
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
alexjohnson7307
 
Zaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdfZaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdf
AmandaCheung15
 
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
bellared2
 
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
Priyanka Aash
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
Ivanti
 
(CISOPlatform Summit & SACON 2024) Regulation & Response In Banks.pdf
(CISOPlatform Summit & SACON 2024) Regulation & Response In Banks.pdf(CISOPlatform Summit & SACON 2024) Regulation & Response In Banks.pdf
(CISOPlatform Summit & SACON 2024) Regulation & Response In Banks.pdf
Priyanka Aash
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
bhumivarma35300
 
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
Priyanka Aash
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
shanihomely
 
Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
Bhajan Mehta
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Nicolás Lopéz
 
Vulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive OverviewVulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive Overview
Steven Carlson
 
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
Priyanka Aash
 

Recently uploaded (20)

Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
 
Patch Tuesday de julio
Patch Tuesday de julioPatch Tuesday de julio
Patch Tuesday de julio
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
 
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
 
Zaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdfZaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdf
 
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
 
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
 
(CISOPlatform Summit & SACON 2024) Regulation & Response In Banks.pdf
(CISOPlatform Summit & SACON 2024) Regulation & Response In Banks.pdf(CISOPlatform Summit & SACON 2024) Regulation & Response In Banks.pdf
(CISOPlatform Summit & SACON 2024) Regulation & Response In Banks.pdf
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
 
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
 
Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
 
Vulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive OverviewVulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive Overview
 
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
 

Code4lib Digital Content Integrated with ILS Data for User Discovery: Lessons Learned

  • 1. Digital Content Integrated with ILS Data for User Discovery: Lessons Learned (A Real Use Case for RDF!) C ODE 4 L I B 2 0 1 5 PORTLAND , OR FE B RUA RY 11 Naomi Dushay / ndushay@stanford.edu Laney McGlohon / laneymcg@stanford.edu
  • 3. Indexing Code Solr Master Index ILS DOR XML Data Harvestor Indexing Code Course Reserves MARC Data Sources
  • 4. 4 Digital Collections Have Items Item MODS Collection MODS Item MODS Collection MARC Item MODS Collection MARC Item MARC
  • 5. To merge or not to merge?
  • 7. Our Data is Ugly (shhhhh ….) We Have Dups in ILS data • Data isn’t “work” based • Catalog card data conversion We Don’t Store Every Field in SearchWorks Solr • Search performance, storage • Can’t use atomic updates to Solr doc (only fields changed) • Have to recreate Solr doc from MARC, with same code • Digital Content Workflow Predated SearchWorks • Multiple Workflows • Poor Metadata QA 
  • 9. Indexing Code Solr Master Index ILS DOR XML Data Harvestor Indexing Code Course Reserves MARC Coll Rec In|Outside ILS, Items Outside the ILS 
  • 11. Indexing Code Solr Master Index ILS DOR XML Data Harvestor Indexing Code Course Reserves MARC Merge Manager App to Manage Multiple Sources
  • 12. App to Manage Multiple Sources Fail. Why? • Performance • Storing Solr document pieces (in a database) • Adding non-trivial app for writes to Solr • Complexity 
  • 13. Indexing Code Solr Master Index ILS DOR XML Data Harvestor Indexing Code Course Reserves MARC Coll Rec In|Outside ILS, Items In|Outside the ILS 
  • 14. Fail. Why? • ILS updates and steps on digital work info • ILS feed granularity issues • Repeated Re-indexing  Coll Rec In|Outside ILS, Items In|Outside the ILS
  • 15. Solr Atomic Updates We Don’t Store Every Field in SearchWorks Solr – Currently • Search performance, storage • Can’t use atomic updates to Solr doc (only fields changed) • Have to recreate Solr doc from MARC, with same code Would Require: • Performance Testing / Load Testing • Changes to stable ILS MARC  Solr code • Changes to stable ILS course reserve  Solr code • Changes to digital content metadata  Solr code 
  • 16. Indexing Code Solr Master Index ILS DOR XML Data Harvestor Indexing Code Course Reserves MARC Let There Be (More) Dups! 
  • 17. Automagically Amend ILS MARC ILS DOR Course Reserves MARC Solr Master Index Indexing Code + UI code 
  • 18. StanfordSync & Embeddable Digital Objects
  • 19. Thank You! February 11th, 2015 ndushay@stanford.edu laneymcg@stanford.edu

Editor's Notes

  1. Users want to be able to easily and intuitively discover objects of interest in our collections in the format they want with information about how to get the objects. Discover “work” Easily Intuitively In format desired Ability to get to it when needed
  2. Most institutions have integrated library systems (ILS) with bibliographic records, item level holdings and course reserves for print/analog objects. But, as we all know, our collections are growing in leaps and bounds with digitized and born-digital content.   Managing both analog and digital objects with an ILS can be a challenge. Either the digital object metadata is included in the ILS but the plan is to keep this metadata separate from the analog metadata, if one exists, or the two sets of metadata are merged into an individual record. At some institutions, the digital metadata is not in the ILS at all and there may be plans to include it in the ILS later based on pre-planning or, more likely, as an after-thought.
  3. collection record has item members some digital collections have no ILS records:  coll or item some digital collections have an ILS coll rec but only digital items (not in ILS) some digital collections have ILS coll rec and some/all ILS item recs
  4. User’s perspective Looks like multiple “works” Difficult to distinguish print versus digital Developer’s perspective Difficult to maintain up-to-date metadata From a user's point of view, it can be confusing if there it more than one record in their search results for what they may consider the same object or "work", just in different formats. As a researcher myself, I prefer to have merged metadata and a single search result to access both the analog and digital surrogate.   Whether to merge data or not has been bantered about for a while. Because of the nature of digital processing, managing the metadata as a specialized data source in it's own system is very tempting. However, from a developer's point of view, trying to manage multiple changing data sources and keep them in sync is challenging to say the least!
  5. So, I have been working in the last month on reverting back to unmerged records so that we can prepare for our next solution: using the embedding functionality that has been added to our discovery system. This embedding functionality will use specifically formatted 856 fields to pull required images and information about each digital collection into the rendering of the corresponding MARC record from the ILS.