Usability Testing a Public ERM: Worth the Effort?


Published on

Reviews the overall usability testing process, then discusses the usability testing UConn Libraries completed in academic year 2006-07 and rolled out in March 2007. Presentation for Eastern Connecticut State Libraries, January 2008.

Published in: Technology, Education
1 Comment
  • After you got all the information on Fioricet, another point on your agenda should be the price for it. resolves this problem. Now you can make the decision to buy.
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • user focus usability principles what we did at UConn start with … why do we care? Graphic from Creating Passionate Users,
  • Usability Testing a Public ERM: Worth the Effort?

    1. 1. Usability Testing a Public ERM: Worth the Effort? Stephanie Willen Brown Electronic Resource Librarian University of Connecticut January 8, 2008
    2. 3. Act Like a User <ul><li>You need information on diabetes for a paper. </li></ul><ul><li>Two questions </li></ul><ul><ul><li>What’s the first thing you see? </li></ul></ul><ul><ul><li>Where can get material for your paper? </li></ul></ul><ul><li>Site 1 </li></ul><ul><li>Site 2 </li></ul><ul><li>Site 3 </li></ul>
    3. 4. Community College of Philadelphia
    4. 5. Salt Lake Community College
    5. 6. York County (ME) Community College
    6. 7. What *is* Usability? <ul><li>Usability … assesses how easy user interfaces are to use . The word “usability” also refers to methods for improving ease-of-use during the design process. </li></ul><ul><li>Jakob Nielsen, Usability 101 </li></ul>
    7. 8. “ Making Things Findable” <ul><li>UConn Libraries “Plan 2010” Goal 2: Scholar’s Portal says: </li></ul><ul><ul><li>“ Provide immediate, unmediated , and comprehensive access to digitized research and scholarly collections worldwide.” </li></ul></ul><ul><li>Web & database usage statistics greatly outweigh individual library-user contact </li></ul>
    8. 9. Components of Usability <ul><li>Goals </li></ul><ul><li>Learnability </li></ul><ul><li>Efficiency </li></ul><ul><li>Memorability </li></ul><ul><li>Error recovery </li></ul><ul><li>Satisfaction </li></ul><ul><li>Achieved by … </li></ul><ul><li>Thinking like a user </li></ul><ul><li>Consistency </li></ul><ul><li>Tweaking text </li></ul>
    9. 10. Thinking Like a User <ul><li>“… let’s … acknowledg[e] … the vital importance of empathy for the user . Only by understanding and caring about the perspective of the individual can we design useful, usable solutions.” </li></ul><ul><li>- Peter Morville, Ambient Findability </li></ul>
    10. 11. Consistency <ul><li>Color, graphics </li></ul><ul><li>Orientation & navigation </li></ul><ul><li>Language </li></ul>
    11. 12. Reading Online is Like … <ul><li>Reading Proust </li></ul><ul><ul><li>People like to read long sentences online because it’s easy to keep your place, follow complex trains of thought, and flip to the next screen of dense text. </li></ul></ul><ul><li>or </li></ul><ul><li>Skimming citations </li></ul><ul><ul><li>People skim titles & abstracts for keywords, take notes, and move to the next citation </li></ul></ul>
    12. 13. Reading Online … Proust <ul><li>For a long time I used to go to bed early. Sometimes, when I had put out my candle, my eyes would close so quickly that I had not even time to say &quot;I'm going to sleep.&quot; And half an hour later the thought that it was time to go to sleep would awaken me; I would try to put away the book which, I imagined, was still in my hands, and to blow out the light; I had been thinking all the time, while I was asleep, of what I had just been reading, but my thoughts had run into a channel of their own, until I myself seemed actually to have become the subject of my book: a church, a quartet, the rivalry between Francois I and Charles V. This impression would persist for some moments after I was awake; it did not disturb my mind, but it lay like scales upon my eyes and prevented them from registering the fact that the candle was no longer burning. Then it would begin to seem unintelligible, as the thoughts of a former existence must be to a reincarnate spirit; the subject of my book would separate itself from me, leaving me free to choose whether I would form part of it or no; and at the same time my sight would return and I would be astonished to find myself in a state of darkness, pleasant and restful enough for the eyes, and even more, perhaps, for my mind, to which it appeared incomprehensible, without a cause, a matter dark indeed. </li></ul>Swann’s Way / Marcel Proust
    13. 14. Skimming Citations Online … ERIC search: African Americans & mathematics
    14. 15. Which is Easiest to Understand? Type Books | Journals | Reports | Videos | Web Sites Sciences Biology | Chemistry | Microbiology | Physics | Zoology Biology Books Chemistry Journals Microbiology Physics Reports Videos Web Sites Zoology <ul><li>Type </li></ul><ul><li>Books </li></ul><ul><li>Journals </li></ul><ul><li>Reports </li></ul><ul><li>Videos </li></ul><ul><li>Web Sites </li></ul><ul><li>Sciences </li></ul><ul><li>Biology </li></ul><ul><li>Chemistry </li></ul><ul><li>Microbiology </li></ul><ul><li>Physics </li></ul><ul><li>Zoology </li></ul>
    15. 16. Arguably … <ul><li>Why? </li></ul><ul><li>Content arranged logically, with clear headings & bullets </li></ul><ul><li>Alphabetical within groups </li></ul><ul><li>Content goes down, as it does in traditional print </li></ul><ul><li>Words are “chunked” into pieces easy for eye to scan in one glance </li></ul><ul><li>Type </li></ul><ul><li>Books </li></ul><ul><li>Journals </li></ul><ul><li>Reports </li></ul><ul><li>Videos </li></ul><ul><li>Web Sites </li></ul><ul><li>Sciences </li></ul><ul><li>Biology </li></ul><ul><li>Chemistry </li></ul><ul><li>Microbiology </li></ul><ul><li>Physics </li></ul><ul><li>Zoology </li></ul>
    16. 17. Jargon for Librarians <ul><li>On travelocity, you need the cheapest round-trip fare from Boston to London. </li></ul><ul><li>These are your options – which is right? </li></ul><ul><ul><li>Flights & Prices </li></ul></ul><ul><ul><li>Fares </li></ul></ul><ul><ul><li>Three Best Itineraries </li></ul></ul>
    17. 18. Library Jargon to Students <ul><li>“ ERIC, I think it’s some kind of journal … some kind of citation.” </li></ul><ul><li>Reference Shelf “[It’s] very general. You don’t know what to expect as it could be anything.” </li></ul>
    18. 19. Usability Testing <ul><li>Define users </li></ul><ul><li>Design questions to mimic what users would realistically do </li></ul><ul><li>Do usability testing early & often </li></ul><ul><ul><li>3-5 users highlights 85% of errors </li></ul></ul><ul><ul><li>Better to test several small groups than 10-15 at once </li></ul></ul><ul><li>Note errors, redesign and retest </li></ul>
    19. 20. Redesigning the Research Database Locator <ul><li>External </li></ul><ul><ul><li>Literature review </li></ul></ul><ul><ul><li>Find sites we liked </li></ul></ul><ul><li>Internal </li></ul><ul><ul><li>Usage & query log analysis </li></ul></ul><ul><ul><li>Usability testing </li></ul></ul><ul><ul><li>Design  </li></ul></ul><ul><ul><ul><li>testing  </li></ul></ul></ul><ul><ul><ul><ul><li>redesign  </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>more testing </li></ul></ul></ul></ul></ul>
    20. 21. Literature Review <ul><li>Cobus, Laura, Frances D. Valeda, and Anita Ondrusek. “How Twenty-Eight Users Helped Redesign an Academic Library Web Site.” Information Technology & Libraries 44.3 (2005): 232-46. </li></ul><ul><li>Krug, Steve. Don’t Make Me Think! 2d ed, Indianapolis, Ind. : Que. 2006. </li></ul><ul><li>Nielsen, Jakob. “Usability 101.” Alertbox: Current Issues in Web Usability. </li></ul>
    21. 22. Sites We Liked <ul><li>University of Toronto ’s “Best research resources for your topic.” Shows article databases, research guides, and encyclopedias / dictionaries / handbooks. </li></ul><ul><li>NCSU’s “Browse Subjects” See Poultry Science e.g., and note tabs at the top for Summary, Books & More; Databases & Articles; Journals; Reference Tools. Full subject list at left, with Agriculture expanded on Poultry Science page. </li></ul><ul><li>Purdue’s “All Databases by Subject” Select main subjects from menu to see subcategories. </li></ul>
    22. 23. Usage Log Analysis <ul><li>March - May 2006, UConn patrons … </li></ul><ul><ul><li>Used the subject browse 18,000 times; </li></ul></ul><ul><ul><li>Performed a keyword search 15,800 times; and </li></ul></ul><ul><ul><li>Clicked on Databases by Title 6,600 times. </li></ul></ul>
    23. 24. Query Log Analysis <ul><li>Database searches </li></ul><ul><ul><li>america history and life </li></ul></ul><ul><ul><li>lexus nexus </li></ul></ul><ul><ul><li>infotrack </li></ul></ul><ul><li>Subject searches </li></ul><ul><ul><li>education </li></ul></ul><ul><ul><li>pharmacy medicine </li></ul></ul><ul><ul><li>anthropologyu </li></ul></ul><ul><li>Topic searches </li></ul><ul><ul><li>hamlet insane </li></ul></ul><ul><ul><li>adopted children of same sex couples </li></ul></ul><ul><ul><li>“ why doesn’t the us have a eurpean-style welfate state?” </li></ul></ul>Are these successful?
    24. 25. <ul><li>Subject browse: ~ 18,000 ; </li></ul><ul><li>Keyword search 15,800 ; and </li></ul><ul><li>Databases by Title: 6,600 </li></ul>housing market
    25. 26. Search “housing market”
    26. 27. Browse by Topic  Business
    27. 29. Usability Testing: 3 Rounds <ul><li>Who? </li></ul><ul><ul><li>3 undergraduates, 1 grad, 1 faculty in each </li></ul></ul><ul><ul><li>@ Storrs & regional campuses </li></ul></ul><ul><li>What? </li></ul><ul><ul><li>First tested old system </li></ul></ul><ul><ul><li>Major redesign </li></ul></ul><ul><ul><li>Tested redesign </li></ul></ul><ul><ul><li>Tweaked design </li></ul></ul><ul><ul><li>Tested again </li></ul></ul>
    28. 30. Usability Tasks <ul><li>Find articles about diabetes for your nursing class </li></ul><ul><li>Find a newspaper article on the day you were born </li></ul><ul><li>Your professor said “use a database named ERIC” </li></ul><ul><li>… plus 7 more … </li></ul>
    29. 32. Databases by Title – find ERIC
    30. 34. Design Goals <ul><li>Unmediated use of database locator </li></ul><ul><li>Focus on how most users think *** not librarians *** </li></ul><ul><li>Display 5 databases per subject </li></ul><ul><ul><li>Users can choose to see more </li></ul></ul><ul><li>Shortened database descriptions </li></ul>
    31. 35.
    32. 36. Best Bets in Library Science
    33. 37. Side Effect: License Data
    34. 38. All Databases in LIS
    35. 39. Shortened Descriptions <ul><li>From this … </li></ul><ul><li>ABI/Inform Global Full-text articles from 1800 journals covering business, finance, management and related functional areas. ABI/INFORM Global indexes a total of 2700 major publications. Subject coverage includes: business and management, including all functional areas. </li></ul><ul><li>To this … </li></ul><ul><li>ABI/Inform Global Articles in business, finance, management, accounting, advertising, banking, insurance, marketing, public administration, real estate, and telecommunications. 1991-current (full-text); 1971-current (index & abstracts). </li></ul>
    36. 40. Timeline … How Long?! <ul><li>Began Winter 2006: “database descriptions too long, fix” </li></ul><ul><ul><li>All agreed. But … </li></ul></ul><ul><ul><li>Rewriting database descriptions wouldn’t solve all problems. </li></ul></ul><ul><ul><li>“ Maybe we should do more” </li></ul></ul><ul><li>Ad hoc group started meeting spring 2006 & set up plan </li></ul>
    37. 41. From Ad Hoc to Rollout <ul><li>August 2006: first usability tests </li></ul><ul><li>Fall 2006: redesign & discussion </li></ul><ul><ul><li>Possible “quick wins” tested & rejected </li></ul></ul><ul><li>November 2006: 2d round of usability </li></ul><ul><ul><li>Redesign & more discussion </li></ul></ul><ul><li>January 2007 (early): 3d round of usability tests </li></ul><ul><li>January 2007 (mid): beta rollout </li></ul><ul><li>March 2007: rollout (spring break) </li></ul>
    38. 42. Final Round of Testing
    39. 43. UConn’s Next Steps <ul><li>Continue user analysis </li></ul><ul><ul><li>Review query logs, usage data </li></ul></ul><ul><ul><li>Solicit feedback from users, colleagues </li></ul></ul><ul><li>Add / tweak … </li></ul><ul><ul><li>User tags </li></ul></ul><ul><ul><li>“ My Databases” feature </li></ul></ul><ul><ul><li>Better searching? </li></ul></ul><ul><li>Note errors, redesign and retest </li></ul>
    40. 44. UConn Redesign Team <ul><li>Stephanie Willen Brown , electronic resource librarian & liaison to Communication Sciences </li></ul><ul><li>Susanna Cowan , undergraduate education & outreach librarian </li></ul><ul><li>Kate Fuller , reference collection maintenance coordinator and / administrative assistant </li></ul><ul><li>Jill Livingston , reference librarian/liaison to the school of allied health </li></ul><ul><li>Tom Wood , applications developer </li></ul>
    41. 45. Useful Usability Resources
    42. 46. “ PERM” FAQ
    43. 47. Usability Testing: Worth the Effort! Tony Dungy Coach of the Indianapolis Colts Upon winning 2007 Superbowl