• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
LITA Forum 2010
 

LITA Forum 2010

on

  • 1,175 views

Presentation given at the 2010 LITA National Forum

Presentation given at the 2010 LITA National Forum

Statistics

Views

Total Views
1,175
Views on SlideShare
1,152
Embed Views
23

Actions

Likes
0
Downloads
2
Comments
0

2 Embeds 23

http://milehighbrarian.net 18
http://ninermac.net 5

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    LITA Forum 2010 LITA Forum 2010 Presentation Transcript

    • “Quick Search” It Is Not: Testing Response Times of Traditional and NextGen Catalogs
      Nina McHale
      Margaret Brown-Sica
      LITA Forum 2010
    • Esteemed Researchers
    • Our Research
      Forthcoming:
      Margaret Brown-Sica, Jeffrey Beall, and Nina McHale, “Next-Generation Library Catalogs and the Problem of Slow Response Time,” Information Technology and Libraries, Volume 29/4,December 2010, 207-216.
    • Not-So-Quick-Search
    • Our Research Questions
      Are NextGencatalogs—or traditional catalogs that add NextGen content—too slow?
      Do 2.0/NextGen features slow them down too much?
    • Our Conclusions
      Yup.
      Features such as cover art, reviews, tagging, etc., can significantly increase the amount of data, and therefore time, required to return a catalog record page.
      Performance factors, particularly speed, should be required criteria for librarians and vendors evaluating and designing products.
    • Speed Standards?
      W3C does not set forth standards
      Jakob Nielsen
      0.1 of a second: feels “instantaneous”
      1.0 second: feels “uninterrupted”
      10 seconds:
      “About the limit for keeping the user’s attention focused on the dialogue.”
      Give the user “time-remaining” feedback
      Jakob Nielsen, Usability Engineering (San Francisco: Morgan Kaufmann, 1994) 135.
    • Our Method
      During a busy time during the semester, we recorded response times in seconds of permalinks for three catalog records
      Tested our classic/NextGen catalogs and three others
      3 books, 5 catalogs, 3 times per day for 13 days=585 data points
      Collecting several data points in this way using www.websitepulse.com ensured that data was consistent
    • Additional Catalogs Tested
      Library of Congress Catalog
      Voyager
      Traditional catalog
      University of Texas at Austin
      Innovative Interfaces
      Traditional catalog with added NextGen elements
      University of Southern California
      Sirsi/Dynix
      Traditional catalog with added NextGen elements
    • Books Used
      Hard Lessons: The Iraq Reconstruction Experience. Washington, DC: Special Inspector General, Iraq Reconstruction, 2009. (OCLC number 302189848)
      Ehrenreich, Barbara. Nickel and Dimed: On (Not) Getting by in America. 1st ed. New York: Metropolitan Books, 2001. (OCLC number 256770509)  
      Langley, Lester D. Simón Bolívar: Venezuelan Rebel, American Revolutionary. Lanham: Rowman & Littlefield Publishers, c2009. (OCLC number 256770509)  
    • Permalink Examples
      http://aurarialibrary.worldcat.org/oclc/302189848
      http://skyline.cudenver.edu/record=b2433301~S0
      http://lccn.loc.gov/2009366172
      http://catalog.lib.utexas.edu/record=b7195737~S29   
      http://library.usc.edu/uhtbin/cgisirsi/x/0/0/5?searchdata1=2770895{CKEY}
    • Testing Tools
      WebSitePulse™
      www.websitepulse.com
      Allows testing on any web page/site; does not require server installation
      Similar services:
      websiteoptimization.com
      browsermob.com
      tools.pingdom.com
      your favorite?
    • WebSitePulse™
    • WebSitePulse™ Results
      Horizontal bar:
      Gives visual representation of load time for each item (image files, javascript files, style sheets, etc.)
      Provides quick indication of “sticking points”
      Table:
      provides specifics about file size and delivery time for each
      DNS, Connect, Redirect, First Byte, Last Byte, Error
    • Numbers Crunched: Average Response Time in Seconds
      Auraria’s Skyline: 1.2930
      Auraria’s WCL: 11.5734
      Library of Congress: 2.1530
      University of Texas at Austin: 3.4997
      University of Southern California: 4.1085
    • Individual Catalog Test Results
      After data was analyzed, we took a closer look at each individual catalog, using the Hard Lessons catalog record
      WebSitePulse™ allowed us to take a glimpse at the inner workings of each catalog
      Findings confirmed that extra data and load times were from 2.0/NextGen content
    • “Skyline,” Auraria Library
    • Skyline Test Results: Graph
    • Skyline Test Results: Table
    • Skyline Findings
      Missing favicon (item 4)
      0.9172 seconds “uninterrupted” per Nielsen
      14 items, for a total of 84.64 K:
      9 GIFs
      2 CSS
      1 JavaScript
      Good performance, but an interface that only a librarian could love
    • WorldCat@Auraria
    • WorldCat@Auraria Results: Graph
    • WorldCat@Auraria Results: Table
    • WorldCat@Auraria Findings
      Reference & Instruction librarians’ observations corroborated
      10.3615 seconds
      31 items, for a total of 633.09 K, to load:
      10 CSS files
      10 JavaScript files
      8 GIFs/PNGs
      No single NextGen feature slowed down load time, but multitude of files created unacceptable delay
    • Library of Congress Catalog
    • Library of Congress Catalog Results: Graph
    • Library of Congress Catalog Results: Table
    • Library of Congress Catalog Findings
      Overall, second fastest of all five catalogs tested
      1.2900 seconds
      Only six items and 19.27 K to load:
      2 CSS files
      3 GIFs
      Like Skyline, fast, but has that “legacy look”
    • University of Texas at Austin
    • UT Austin Results: Graph
    • UT Austin Results: Table
    • University of Texas at Austin Findings
      Added NextGen features:
      Cover art
      LibraryThing’s Catalog Enhancement
      Supports recommendations, tag browsing, alternate editions/translations
      2.4674 seconds: user experience interrupted
      19 items, 138.84 K
      Cover art nearly doubles response time
      Item 14: script on ILS that queries Amazon for art
    • University of Southern California: HOMER
    • USC Results: Graph
    • USC Results: Table
    • USC Findings
      Slowest among traditional catalogs; Sirsi/Dynix takes longer to make initial connection (Item 1 on graph)
      8.7295 seconds (though average was 4.1085 seconds)
      16 items, 148.47 K
      While attractive and well-integrated, Syndetic Solutions content (cover art, summary, author biography, and table of contents) adds 1.2 seconds to load time
    • Isthe ContentWorth the Wait?
      “The new database seems based on Amazon.com. I don’t need suggestions, and poor ones at that, of related books when I use the library. I don’t need to see what other borrowers thought of the book. The information I need is poorly displayed. It is hard to cut and paste. It takes several screens to scan through, instead of the much quicker scroll in the traditional format…. It supplies distracting, if not useless information (a picture of the cover, the distance to other libraries—as if I need to know how far Provo is).”
      -Auraria Campus Faculty Member
    • Our Conclusions
      Make performance testing part of evaluation process for vendor products
      Adhere to industry standards for acceptable response times when testing
      Optimize delivery of 2.0/NextGen features as much as possible
      Conduct user testing to ensure that the content is “worth the wait” to their minds
    • Questions? Comments?
      Nina McHale
      @ninermac
      milehighbrarian.net