Discovery platforms: Technology, tools and issues


Published on

Evolution of discovery tools and platforms.

Published in: Technology, Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Discovery platforms: Technology, tools and issues

  1. 1. Discovery PlatformsTECHNOLOGIES, TOOLS AND ISSUES Saiful Amin 39th Five Laws Lecture (2011)
  2. 2. Evolution of Discovery Tools Printed catalogues Traditional (Web)OPAC Integrated OPAC portals Federated search services Discovery interfaces Web-scale discovery services Integrated discovery platform
  3. 3. Printed catalogues Author browse Title browse Series browse Call Number browse Subject browse Shelf list (inventory)
  4. 4. Traditional (Web)OPAC (Web)Server Application ILS Database (Bibs)
  5. 5. Traditional (Web)OPACPros Cons Keyword search!  Uses database queries  Author, title, subject  „LIKE‟ statements  ISBN/LCCN search  Exact/partial match  Boolean queries  Limited use of search  Proximity search algorithm Browse index  No relevance ranking  Authority headings  Only physical collection  Title, Call Number and e-books Real-time item status!  Copies & availability info Link to URL (tag 856)
  6. 6. Integrated OPAC Portal Enrichment Web services Services Web Server ApplicationWebsite ILS Database ILS Databasecontent (Bibs) (Patrons)
  7. 7. Integrated OPAC PortalPros Cons All WebOPAC features  Uses database queries  Keyword search  „LIKE‟ statements  Headings browse  Exact/partial match  Availability info  Limited use of search Library website integration algorithms Patron empowerment  Circ/Account details  No relevance ranking  Online renewal  Still limited to only physical  Online hold placement collection & e-books  SDI services  New arrivals OPAC enrichment  Book cover/reviews Thesaurus integration
  8. 8. Federated Search Service360 Search dbWizResearch Pro Pazpar2 Full-text links Web Server Application Library Digital Science Catalog Repository ProQuest EBSCO Direct PubMed … Emerald
  9. 9. Federated Search Service Muse Content Architecture Supports 6300+ databases!
  10. 10. Federated Search ServicePros Cons Single search broadcast  Not all databases are Real-time search results standards compliant Based on standards  Requires custom search scripts  Z39.50, SRU/W  Requires metadata crosswalk  MARC, ISO2709, XML Supports large set of  Network intensive databases  Performance issues  7000+ in “360 Search”  Mostly available as hosted  6300+ in Muse platform Merging and sorting service No local index  Annual subscription (maintenance free!)
  11. 11. Discovery Interface Enrichment Web services Services Web Server ApplicationFull-text link Availability/Holds Digital Central Index Repository (Solr/Lucene) ILS Database DC XML data MARC Bib data
  12. 12. Discovery Interface Word stemming  Phrase query  „fishing‟, „fished‟, „fish‟,  „Did you mean?‟ „fisher‟ => „fish‟  Spell Checker Fuzzy search  insertion: cot  coat  Relevance ranking  deletion: coat  cot  TF-IDF / Term Vector  substitution: coat  cost  Term weights Auto-suggest  Lucene scores  N-gram, Edge N-gram  Faceted browsing analysis  Who are main authors and their count?  What are main subjects and their count?
  13. 13. Discovery InterfacePros Cons Google-like search box  Searches only locally hosted Advanced features collections  Fuzzy searching  Relevance ranking  Word stemming algorithms  Social tagging/reviews  “Did you mean?” feature  Auto-suggest (type ahead)  Faceted browsing Availability/Hold requests Metadata enrichment Linking  Amazon/Google/Wikipedia Digital repository integration
  14. 14. Can we combine the two? Modern discovery interface Local collections + Remote databases Unified search result
  15. 15. Web-scale Discovery Services EBSCO ProQuestABI Inform Web Server Application PubMed Availability Full-text link Science Direct Library … Central Index MARC data Catalog Full-text and metadata DigitalLexis-Nexis Repository DC data
  16. 16. Web-scale Discovery Services
  17. 17. Web-scale Discovery ServicesSummon Service Content types include: Library catalog records Conference proceedings E-journal articles Grey literature Institutional repositories Cited references Newspaper articles Reports E-books Digital library Dissertations Databases and more.
  18. 18. Web-scale Discovery ServicesPros Cons Google-like single search box  Supports limited number of Pre-indexed licensed content databases (1000-1500) Inclusion of local collection  Requires huge investment to  OAI-PMH, MARC updates maintain centralized index Advanced features  Publisher partnerships  Relevance ranking (Licensing/legal issues)  “Did you mean?”  Regular pre-publication indexing  Auto-suggest (type ahead)  Mostly hosted-only service  Faceted navigation  Content bias? (ranking) Availability/Full-text links  Vendor lock-in? Mobile friendly Web-service APIs  Annual subscription Easier off-campus access No installation/maintenance
  19. 19. Can we have best of both worlds?Modern discovery interface  Supports large number of databases Local collections +  Based on open standards Remote databases (extensible)  Can be maintained locally Unified search result (No subscription!) Web Server Application Remote Remote Remote Remote Digital ILS database database database database database Repository (Bibs) Remote Remote Remote Remote database database database database
  20. 20. Integrated Discovery Platform Semi-commercial Supports 1000+ databases
  21. 21. Integrated Discovery Platform Pazpar2 Architecture Open source (GPL) Build your own connector!
  22. 22. Conclusion Each platform has its own goals:  Pure library catalog can provide expressive search (high precision)  Federated search improves content coverage in single search  Discovery interfaces are designed to improve user experience for local collections  Web-scale discovery provides unified search experience for local and remote collections (still way short in content coverage)  Integrated platform provides extensibility (but requires significant effort in development and maintenance) One size does not fit all. No single system is perfect. As content becomes more open, the focus of discovery solutions should be on open platforms that are extensible as well as affordable.
  23. 23. Questions and Discussions