Download to read offline
Uploaded byMichael Cummings
ELUNA2013:Providing Voyager catalog data in a custom, open source web application, "Launchpad"
The document discusses a custom open-source web application called 'Launchpad' developed for providing Voyager catalog data, including features like HTTP request handling, retrieval methods, and integration with external APIs. It details the software environment and architecture, indicating it is built using Django and Python, and is available on GitHub with monthly updates. Additionally, it covers various functionalities such as JSON support, interlibrary loan features, and bibliographic information retrieval.
![[DSC Europe 25] Miodrag Pesovic & Vladislav Radonjic - Federated Data Archite...](https://cdn.slidesharecdn.com/ss_thumbnails/gsbe3y5it5uhndi4e08e-1-251212103249-f1008e0c-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Behzad Hosseini - AI Agents in the Wild: Deploying Models tha...](https://cdn.slidesharecdn.com/ss_thumbnails/3qtejajvsjqrzwfept2c-10-251212103250-7f2b1068-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Debmalya Biswas - Agentification: the art of transforming man...](https://cdn.slidesharecdn.com/ss_thumbnails/r5azlggvtqiaiiusrqdr-4-251212103249-5a12c89b-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Hans Kleinsman - The Compliance Gearbox: How Tax Tech Mediate...](https://cdn.slidesharecdn.com/ss_thumbnails/dxdytie1toel0hr90bjs-2-251212103250-174fdbe7-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Ivan Peric - Intelligence Swarm Logic and Techno-Functional M...](https://cdn.slidesharecdn.com/ss_thumbnails/7my7c97fsduiccadgavw-2-251212103249-5a03f7c6-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Dusan Nesic - Securing Tomorrow’s Infrastructure: Why Cyber-P...](https://cdn.slidesharecdn.com/ss_thumbnails/qikbszfftyowjm2q6duw-1-251211083848-8f2ead6b-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Kaja Kandare - LLM as a judge.pptx](https://cdn.slidesharecdn.com/ss_thumbnails/arxyccaxsdsd1ba99wjw-7-251212104007-2b4e3f64-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Marko Krstic - Understanding the AI Threat Landscape - Risks,...](https://cdn.slidesharecdn.com/ss_thumbnails/tiyim1ins5jvbrvzpzla-2-251209104645-c69d3553-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Branko Dzakula - From Defense to Attack: How AI Redefines Cyb...](https://cdn.slidesharecdn.com/ss_thumbnails/80bdzdxpr3ky2g0qvyk9-8-251211083048-ce5fc1ee-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Katherine Forrest - AI NOW: Understanding the Velocity of Cha...](https://cdn.slidesharecdn.com/ss_thumbnails/wvvbruqfrci0sfq9xwgb-4-251212104007-e5ad1987-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Vladimir Jelic - The AI-Driven Security Shift From Reactive D...](https://cdn.slidesharecdn.com/ss_thumbnails/6g5gj25mtjwayniqem1t-6-251209104645-7a5a5fc6-thumbnail.jpg?width=640&height=640&fit=bounds)
ELUNA2013:Providing Voyager catalog data in a custom, open source web application, "Launchpad"
- 1. Providing Voyager catalogdata in a custom, open source web application “Launchpad” ELUNA 2013 Athens, GA Michael Cummings, GWU
- 2. Launchpad ELUNA 2013 ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 3. Agenda ELUNA 2013 • Making Requests (http request) • Retrieval • Features (http response) • The software ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 4. Request: URL pattensELUNA 2013 The application responds to http requests. * By ‘item’ we mean the bibliographic id (bib_id) http://findit.library.gwu.edu/item/5642529 isbn/0521800900 issn/1448-4404 oclc/ocm46792617 item/2388259.json OpenURL requests are also accepted. ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 5. Request: Google ELUNA2013 Search term(s) gwu libraries Link to Launchpad Voyager bibid 2388259 ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 6. Request: schema.org ELUNA2013 The HTML source in our pages includes markup compliant with schema.org guidelines. This may help Google index and retrieve records from our index. <div itemscope itemtype="http://schema.org/Book"> <span itemprop="name">The Visual display of quantitative information / Edward R. Tufte.</span><br/> <span itemprop="author">Tufte, Edward R., 1942-</span> <span itemprop="publisher">Cheshire, Conn. : Graphics Press, c1983.</span> In <span itemprop="inLanguage" class="language">English.</span> ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 7. Request: Analytics ELUNA2013 ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 8. Request: 360 LinkELUNA 2013 Our journal finder page has a similar style or vice versa. ISSN link to Launchpad ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 9. Request: Worldcat ELUNA2013 Link to Launchpad ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 10. Request: Summon ELUNA2013 Title and holdings are Item links to Launchpad ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 11. Request: request JSONELUNA 2013 URL ends with json …/2388259.json ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 12. Retrieval ELUNA 2013 • Making Requests (http request) • Retrieval • Features (http response) • The software ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 13. Retrieval ELUNA 2013 All of these sources may be accessed in the background to retrieve title, holding, status, and other information using API’s, Z39.50, or SQLPlus. SQLPlus query Z39.50 query Innovative Interfaces Millenium WRLC Z39.50 query Holding, item status & bibliographic detail Holding, item status Holding, item status Google Book API Open Library API 360 Link API 360 LINK Book covers, full text online Journal Coverage date, provider ILL links for titles not held in consortium ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 14. Features ELUNA 2013 • Making Requests (http request) • Retrieval • Features (http response) • The software ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 15. Features: Bib InfoELUNA 2013 We intentionally limited the amount of information to display here. We could show more. A link to the classic catalog is provided for more detail if needed. ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 16. Feature: Holdings DetailELUNA 2013 Notes / Recent Issues detail Gelman Periodicals v.1(1986)-v.3(1988), v.12(1997)-v.22(2007) All volumes located off site (WRLC Center). To retrieve, please submit a Consortium Loan Request form. Bound volumes are Library Use Only. ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 17. Features: RTAC ELUNA2013 Availability Status via SQLPlus -- WRLC Voyager Z39.50 ----- GMU Voyager Z39.50 ----- GT Millenium ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 18. Features: Loan ELUNA2013 Link to Consortium Loan only IF item is eligible ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 19. Features: SMS linkELUNA 2013 ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 20. ELUNA 2013 InterlibraryLoan link Features: ILL ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 21. API: Book CoverELUNA 2013 Covers are displayed when they are available from OpenLibrary’s covers API. OpenLibrary cover API See: http://openlibrary.org/dev/docs/api/covers Example: http://covers.openlibrary.org/b/isbn/0521800900-M.jpg ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 22. API: Google BooksELUNA 2013 There was a search by ISBN No match found in the consortium catalog; Search, found in Google books. Provide an ILL link Google Books API https://developers.google.com/books/docs/v1/using ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 23. API: OpenLibrary fulltextELUNA 2013 While processing the request we check the OpenLibrary to see if a full text is available online. If so, we generate a link to the full text. Open Library Book API See: http://openlibrary.org/dev/docs/api/books Example: http://openlibrary.org/api/books?format=json&jscmd =data&bibkeys=LCCN:11025443 ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 24. API: 360 LinkELUNA 2013 The 360 Link XML API provides ‘From – to’ date range and Database provider name From 2003 to present Directory of Open Access Journals ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 25. JSON Catholic UniversityELUNA 2013 Consortium holdings, status Coming from Launchpad ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 26. JSON George MasonELUNA 2013 Primo Consortium print serial holdings select ‘More’ tab ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 27. The Software ELUNA2013 • Making Requests (http request) • Retrieval • Features (http response) • The software ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 28. Software: Environment ELUNA2013 • Prototyped using SQLPlus, with CGI-perl • Converted to a Django application framework • Programming in Python on Ubuntu Linux • Tested with Apache Bench ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 29. Software Open SourceELUNA 2013 • The ‘about Launchpad page’ at github is http://gwu-libraries.github.io/launchpad/ • The code is managed with git • Available on GW Libraries github repository • The code is available as open source • Updates are released on a monthly schedule ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 30. Software: github ELUNA2013 https://github.com/gwu-libraries/launchpad Select ‘Code’ tab, Readme.md for instructions ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 31. Software: Python codeELUNA 2013 The Python .py code, filed under / lp / ui is probably of interest to programmers… ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
- 32. Software: Learn moreELUNA 2013 About Launchpad http://gwu-libraries.github.io/launchpad/ Launchpad code on git hub https://github.com/gwu-libraries/launchpad ELUNA 2012 From Voyager to your website: Using Linux Shell scripts and Oracle SQL*Plus to generate web pages. https://github.com/cummingsm/VoyagerELUNA2012 Includes slides from 2012 presentation. They’re also on the ExLibris site. Contact Michael Cummings Library Systems Coordinator The George Washington University Libraries michaelc@gwu.edu ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
Editor's Notes
- #3 This is a screen shot of a typical book, in this case one containing information obtained from three different library catalog systems.
- #4 This presentation is only 30 minutes in total, perhaps 25 minutes after preliminaries. Subtract another 5 minutes for questions. So, figure about 5 minutes per topic on this agenda. This will be fairly high level, but the notes contained in the slides will have a lot of detail you can read later.
- #5 Using the word ‘item’ in the URL is sort of a misnomer,. It is the Voyager, or Millenium, bibid not the ‘item’ id. Each title has an item page. The other URL patterns are for look-up purposes. If a match is found based on one of the other standard identifiers, the page displayed will show the bib id in place of the standard number in the URL.
- #6 We generate an xml site map file, and Google indexes it. Processing is performed at roughly 50k / nightly, a limit imposed by Google. As of May 2013 about a million of our titles have been indexed (out of 12 million). To search in Google, enter search term(s) followed by ‘gwu libraries’.
- #8 We put the Google Analytics snippet on our pages. This is a screen shot from Google analytics for the first three months of 2013. Almost all requests are by the bibid.
- #9 The journal finder is at the same root URL, findit.library.gwu.edu We made the user interfaces of the journal finder and the launchpad look pretty much the same. This slide shows an OpenURL request to the journal finder.
- #10 Worldcat will link to GW Launchpad, if there is an ISBN or ISSN number.
- #11 Summon has been configured to link out to the GW Launchpad page when a user clicks on a title or holding of a library catalog record in the search results. * Ooops. This Discovery system isn’t always able to provide real-time availability in search results. That’s one reason we developed the Launchpad application. With Launchpad, we are always able to get real-time availability.
- #12 Our developers prefer working with JSON instead of XML. Although there are parsers for both, XML has a lot of extra text because of the tagging it uses. Two schools have programs that request this JSON output and then incorporate it into their discovery system using a combination of Php and Javascript.
- #13 Next, what happens when the Launchpad receives the http request.
- #14 The WRLC maintains a consortium Voyager catalog of approximately 12 million bib records including George Washington University, American, Catholic, District of Columbia, Gallaudet, George Mason (copy of bib), Georgetown (copy of bib), Howard, and Marymount. Our program retrieves bibliographic, holding, and item status data from the WRLC Voyager using SQLPlus. Georgetown has their own ILS, III Millenium, which we query using the Z39.50 protocol in order to obtain holding and item status. George Mason has their own which we query using the Z39.50 protocol in order to obtain holding and item status. We use API’s to retrieve other information, which is explained in the following pages.
- #15 After retrieval of the information, we have 100% control over the display. The following slides highlight some of the features we chose to implement so far.
- #16 We found that users want the location and availability to be prominent. We decided to show only a few elements of the bibliographic description. However, we could show more. Note: Our program is written in Python, and we can use the pymarc module to easily parse the MARC record.
- #17 Developers recommended a clean layout, one line per entry. However library staff insisted on the notes too. This example isn’t too bad, but in some cases the notes are extensive. In an upcoming release, we convert these notes to hide/show links.
- #18 This screen shot shows a title from all three integrated library system databases which we polled for real-time item status.
- #19 Conditional logic in our application determines eligibility for consortium loans. In this example, the item at George Mason Fenwick stacks is Charged and therefore the item is not eligible for a consortium loan request at this time.
- #20 There is a bit of Javascript loaded with the page which enables the user to click an icon and see the ‘Send to Mobile/Email window; the users may provide a cell number or email to themselves.
- #21 Launchpad opens the Interlibrary Loan form (ILLIAD), and populates some fields on the ILLIAD form. We recently updated our ILL code to handle both 0.1 and 1.0 versions of the OpenURL
- #22 Book covers are provided via OpenLibrary’s image API. This is a free service. OpenLibrary cover API See: http://openlibrary.org/dev/docs/api/covers Example: http://covers.openlibrary.org/b/isbn/0521800900-M.jpg
- #23 The Google Books API is checked when Launchpad can not find a record in the consortium catalog. If we find something via Google Books, we provide the user a link so that they can request the title through ILL. We could provide a link to Google Books too, but so far have elected not to do so. Google Books API https://developers.google.com/books/docs/v1/using
- #24 OpenLibrary / Internet archive full text is available for thousands of books. * Note: There are times when OpenLibrary is closed and the API call will fail. We had to make sure we coded our program to continue gracefully when requests to OpenLibrary failed. The API accepts queries based on standard numbers LCCN, OCLC, and OpenLibrary ID. While processing the request we check the OpenLibrary to see if a full text is available online. If so, we generate a link to the full text. Open Library Book API See: http://openlibrary.org/dev/docs/api/books Example: http://openlibrary.org/api/books?format=json&jscmd=data&bibkeys=LCCN:11025443
- #25 360 Link API provides coverage details. Often there are several providers with access to a different range of dates for the same publications. A setting in the application makes a full text online button for a preferred library – in this case Gw. A text link with a lock icon is used for full text holdings from other consortium member libraries that require their own authentication for online resources.
- #26 JSON data from Launchpad is read and reformatted by an application at CUA for display within their Summon Discovery system on the page where bibliographic details are provided.
- #27 JSON data from Launchpad is read and reformatted by an application at GM for display within their Exlibris Primo Discovery system. GM uses this for their bound periodicals because these are often held by other consortium library members, and patrons from any member library can use the consortium loan system to request them.
- #28 Next an overview of the software.
- #29 GW has been using SQLPlus in a Linux environment for a couple of years. “Using Oracle SQL*Plus to query Voyager in a Linux environment” was the topic of the presenter’s talk at ELUNA in 2012. A link to the presentation is provided later in these slides. We were able to do a rapid prototype of our concept using SQLPlus and CGI-Perl. We then converted the code to a more modern web application framework. Django manages cache, connections, and other aspects quite well. Normally you can model your database and then use Django to perform queries without writing SQL. However, due to the large Voyager schema and the fact that we already had the SQL from our prototype, we include those queries in our code. Python has many advantages, including modules that developers can import – such as pymarc for parsing MARC records. We stress tested the code with Apache Bench in coordination with the Voyager system administrator.
- #30 We involved staff from all departments in the design process. We specified the scope of the project as an application that will display information about a specific title and holdings of the title across the consortium. We have a very transparent process, with a public code repository on github. The code is officially available as open source.
- #31 We have several projects on this site. The readme file provides installation instructions. https://github.com/gwu-libraries/launchpad Select ‘Code’ tab, Readme.md for instructions
- #32 We obviously don’t have time for a code review, but when you have time you can peruse the Python code here. The Python .py code, filed under / lp / ui is probably of interest to programmers…
- #33 About page http://gwu-libraries.github.io/launchpad/ Code repository https://github.com/gwu-libraries/launchpad From Voyager to your website: Using Linux Shell scripts and Oracle SQL*Plus to generate web pages. https://github.com/cummingsm/VoyagerELUNA2012