Usability Testing a Public ERM: Worth the Effort? Stephanie Willen Brown Electronic Resource Librarian University of Connecticut January 8, 2008

Act Like a User You need information on diabetes for a paper. Two questions What’s the first thing you see? Where can get material for your paper? Site 1 Site 2 Site 3

You need information on diabetes for a paper.

Two questions

What’s the first thing you see?

Where can get material for your paper?

Site 1

Site 2

Site 3

Community College of Philadelphia http://www.ccp.edu/vpacaff/library/

Salt Lake Community College http://libweb.slcc.edu/index/library-resources

York County (ME) Community College http://ww.yccc.edu/library/library.asp

What *is* Usability? Usability … assesses how easy user interfaces are to use . The word “usability” also refers to methods for improving ease-of-use during the design process. Jakob Nielsen, Usability 101

Usability … assesses how easy user interfaces are to use . The word “usability” also refers to methods for improving ease-of-use during the design process.

Jakob Nielsen, Usability 101

“ Making Things Findable” UConn Libraries “Plan 2010” Goal 2: Scholar’s Portal says: “ Provide immediate, unmediated , and comprehensive access to digitized research and scholarly collections worldwide.” Web & database usage statistics greatly outweigh individual library-user contact

UConn Libraries “Plan 2010” Goal 2: Scholar’s Portal says:

“ Provide immediate, unmediated , and comprehensive access to digitized research and scholarly collections worldwide.”

Web & database usage statistics greatly outweigh individual library-user contact

Components of Usability Goals Learnability Efficiency Memorability Error recovery Satisfaction Achieved by … Thinking like a user Consistency Tweaking text

Goals

Learnability

Efficiency

Memorability

Error recovery

Satisfaction

Achieved by …

Thinking like a user

Consistency

Tweaking text

Thinking Like a User “… let’s … acknowledg[e] … the vital importance of empathy for the user . Only by understanding and caring about the perspective of the individual can we design useful, usable solutions.” - Peter Morville, Ambient Findability

“… let’s … acknowledg[e] … the vital importance of empathy for the user . Only by understanding and caring about the perspective of the individual can we design useful, usable solutions.”

- Peter Morville, Ambient Findability

Consistency Color, graphics Orientation & navigation Language

Color, graphics

Orientation & navigation

Language

Reading Online is Like … Reading Proust People like to read long sentences online because it’s easy to keep your place, follow complex trains of thought, and flip to the next screen of dense text. or Skimming citations People skim titles & abstracts for keywords, take notes, and move to the next citation

Reading Proust

People like to read long sentences online because it’s easy to keep your place, follow complex trains of thought, and flip to the next screen of dense text.

or

Skimming citations

People skim titles & abstracts for keywords, take notes, and move to the next citation

Reading Online … Proust For a long time I used to go to bed early. Sometimes, when I had put out my candle, my eyes would close so quickly that I had not even time to say "I'm going to sleep." And half an hour later the thought that it was time to go to sleep would awaken me; I would try to put away the book which, I imagined, was still in my hands, and to blow out the light; I had been thinking all the time, while I was asleep, of what I had just been reading, but my thoughts had run into a channel of their own, until I myself seemed actually to have become the subject of my book: a church, a quartet, the rivalry between Francois I and Charles V. This impression would persist for some moments after I was awake; it did not disturb my mind, but it lay like scales upon my eyes and prevented them from registering the fact that the candle was no longer burning. Then it would begin to seem unintelligible, as the thoughts of a former existence must be to a reincarnate spirit; the subject of my book would separate itself from me, leaving me free to choose whether I would form part of it or no; and at the same time my sight would return and I would be astonished to find myself in a state of darkness, pleasant and restful enough for the eyes, and even more, perhaps, for my mind, to which it appeared incomprehensible, without a cause, a matter dark indeed. Swann’s Way / Marcel Proust

For a long time I used to go to bed early. Sometimes, when I had put out my candle, my eyes would close so quickly that I had not even time to say "I'm going to sleep." And half an hour later the thought that it was time to go to sleep would awaken me; I would try to put away the book which, I imagined, was still in my hands, and to blow out the light; I had been thinking all the time, while I was asleep, of what I had just been reading, but my thoughts had run into a channel of their own, until I myself seemed actually to have become the subject of my book: a church, a quartet, the rivalry between Francois I and Charles V. This impression would persist for some moments after I was awake; it did not disturb my mind, but it lay like scales upon my eyes and prevented them from registering the fact that the candle was no longer burning. Then it would begin to seem unintelligible, as the thoughts of a former existence must be to a reincarnate spirit; the subject of my book would separate itself from me, leaving me free to choose whether I would form part of it or no; and at the same time my sight would return and I would be astonished to find myself in a state of darkness, pleasant and restful enough for the eyes, and even more, perhaps, for my mind, to which it appeared incomprehensible, without a cause, a matter dark indeed.

Skimming Citations Online … ERIC search: African Americans & mathematics

Which is Easiest to Understand? Type Books | Journals | Reports | Videos | Web Sites Sciences Biology | Chemistry | Microbiology | Physics | Zoology Biology Books Chemistry Journals Microbiology Physics Reports Videos Web Sites Zoology Type Books Journals Reports Videos Web Sites Sciences Biology Chemistry Microbiology Physics Zoology

Type

Books

Journals

Reports

Videos

Web Sites

Sciences

Biology

Chemistry

Microbiology

Physics

Zoology

Arguably … Why? Content arranged logically, with clear headings & bullets Alphabetical within groups Content goes down, as it does in traditional print Words are “chunked” into pieces easy for eye to scan in one glance Type Books Journals Reports Videos Web Sites Sciences Biology Chemistry Microbiology Physics Zoology

Why?

Content arranged logically, with clear headings & bullets

Alphabetical within groups

Content goes down, as it does in traditional print

Words are “chunked” into pieces easy for eye to scan in one glance

Type

Books

Journals

Reports

Videos

Web Sites

Sciences

Biology

Chemistry

Microbiology

Physics

Zoology

Jargon for Librarians On travelocity, you need the cheapest round-trip fare from Boston to London. These are your options – which is right? Flights & Prices Fares Three Best Itineraries

On travelocity, you need the cheapest round-trip fare from Boston to London.

These are your options – which is right?

Flights & Prices

Fares

Three Best Itineraries

Library Jargon to Students “ ERIC, I think it’s some kind of journal … some kind of citation.” Reference Shelf “[It’s] very general. You don’t know what to expect as it could be anything.”

“ ERIC, I think it’s some kind of journal … some kind of citation.”

Reference Shelf “[It’s] very general. You don’t know what to expect as it could be anything.”

Usability Testing Define users Design questions to mimic what users would realistically do Do usability testing early & often 3-5 users highlights 85% of errors Better to test several small groups than 10-15 at once Note errors, redesign and retest

Define users

Design questions to mimic what users would realistically do

Do usability testing early & often

3-5 users highlights 85% of errors

Better to test several small groups than 10-15 at once

Note errors, redesign and retest

Redesigning the Research Database Locator External Literature review Find sites we liked Internal Usage & query log analysis Usability testing Design  testing  redesign  more testing

External

Literature review

Find sites we liked

Internal

Usage & query log analysis

Usability testing

Design 

testing 

redesign 

more testing

Literature Review Cobus, Laura, Frances D. Valeda, and Anita Ondrusek. “How Twenty-Eight Users Helped Redesign an Academic Library Web Site.” Information Technology & Libraries 44.3 (2005): 232-46. Krug, Steve. Don’t Make Me Think! 2d ed, Indianapolis, Ind. : Que. 2006. Nielsen, Jakob. “Usability 101.” Alertbox: Current Issues in Web Usability. http://gslis.simmons.edu/mw/browns/Usability

Cobus, Laura, Frances D. Valeda, and Anita Ondrusek. “How Twenty-Eight Users Helped Redesign an Academic Library Web Site.” Information Technology & Libraries 44.3 (2005): 232-46.

Krug, Steve. Don’t Make Me Think! 2d ed, Indianapolis, Ind. : Que. 2006.

Nielsen, Jakob. “Usability 101.” Alertbox: Current Issues in Web Usability.

Sites We Liked University of Toronto ’s “Best research resources for your topic.” Shows article databases, research guides, and encyclopedias / dictionaries / handbooks. NCSU’s “Browse Subjects” See Poultry Science e.g., and note tabs at the top for Summary, Books & More; Databases & Articles; Journals; Reference Tools. Full subject list at left, with Agriculture expanded on Poultry Science page. Purdue’s “All Databases by Subject” Select main subjects from menu to see subcategories. http://gslis.simmons.edu/mw/browns/Usability/Sites

University of Toronto ’s “Best research resources for your topic.” Shows article databases, research guides, and encyclopedias / dictionaries / handbooks.

NCSU’s “Browse Subjects” See Poultry Science e.g., and note tabs at the top for Summary, Books & More; Databases & Articles; Journals; Reference Tools. Full subject list at left, with Agriculture expanded on Poultry Science page.

Purdue’s “All Databases by Subject” Select main subjects from menu to see subcategories.

Usage Log Analysis March - May 2006, UConn patrons … Used the subject browse 18,000 times; Performed a keyword search 15,800 times; and Clicked on Databases by Title 6,600 times.

March - May 2006, UConn patrons …

Used the subject browse 18,000 times;

Performed a keyword search 15,800 times; and

Clicked on Databases by Title 6,600 times.

Query Log Analysis Database searches america history and life lexus nexus infotrack Subject searches education pharmacy medicine anthropologyu Topic searches hamlet insane adopted children of same sex couples “ why doesn’t the us have a eurpean-style welfate state?” Are these successful?

Database searches

america history and life

lexus nexus

infotrack

Subject searches

education

pharmacy medicine

anthropologyu

Topic searches

hamlet insane

adopted children of same sex couples

“ why doesn’t the us have a eurpean-style welfate state?”

Subject browse: ~ 18,000 ; Keyword search 15,800 ; and Databases by Title: 6,600 housing market

Subject browse: ~ 18,000 ;

Keyword search 15,800 ; and

Databases by Title: 6,600

Search “housing market”

Browse by Topic  Business

Usability Testing: 3 Rounds Who? 3 undergraduates, 1 grad, 1 faculty in each @ Storrs & regional campuses What? First tested old system Major redesign Tested redesign Tweaked design Tested again

Who?

3 undergraduates, 1 grad, 1 faculty in each

@ Storrs & regional campuses

What?

First tested old system

Major redesign

Tested redesign

Tweaked design

Tested again

Usability Tasks Find articles about diabetes for your nursing class Find a newspaper article on the day you were born Your professor said “use a database named ERIC” … plus 7 more …

Find articles about diabetes for your nursing class

Find a newspaper article on the day you were born

Your professor said “use a database named ERIC”

… plus 7 more …

Databases by Title – find ERIC

Design Goals Unmediated use of database locator Focus on how most users think *** not librarians *** Display 5 databases per subject Users can choose to see more Shortened database descriptions

Unmediated use of database locator

Focus on how most users think *** not librarians ***

Display 5 databases per subject

Users can choose to see more

Shortened database descriptions

http://rdl.lib.uconn.edu/bySubject.php

Best Bets in Library Science

Side Effect: License Data

All Databases in LIS

Shortened Descriptions From this … ABI/Inform Global Full-text articles from 1800 journals covering business, finance, management and related functional areas. ABI/INFORM Global indexes a total of 2700 major publications. Subject coverage includes: business and management, including all functional areas. To this … ABI/Inform Global Articles in business, finance, management, accounting, advertising, banking, insurance, marketing, public administration, real estate, and telecommunications. 1991-current (full-text); 1971-current (index & abstracts).

From this …

ABI/Inform Global Full-text articles from 1800 journals covering business, finance, management and related functional areas. ABI/INFORM Global indexes a total of 2700 major publications. Subject coverage includes: business and management, including all functional areas.

To this …

ABI/Inform Global Articles in business, finance, management, accounting, advertising, banking, insurance, marketing, public administration, real estate, and telecommunications. 1991-current (full-text); 1971-current (index & abstracts).

Timeline … How Long?! Began Winter 2006: “database descriptions too long, fix” All agreed. But … Rewriting database descriptions wouldn’t solve all problems. “ Maybe we should do more” Ad hoc group started meeting spring 2006 & set up plan

Began Winter 2006: “database descriptions too long, fix”

All agreed. But …

Rewriting database descriptions wouldn’t solve all problems.

“ Maybe we should do more”

Ad hoc group started meeting spring 2006 & set up plan

From Ad Hoc to Rollout August 2006: first usability tests Fall 2006: redesign & discussion Possible “quick wins” tested & rejected November 2006: 2d round of usability Redesign & more discussion January 2007 (early): 3d round of usability tests January 2007 (mid): beta rollout March 2007: rollout (spring break)

August 2006: first usability tests

Fall 2006: redesign & discussion

Possible “quick wins” tested & rejected

November 2006: 2d round of usability

Redesign & more discussion

January 2007 (early): 3d round of usability tests

January 2007 (mid): beta rollout

March 2007: rollout (spring break)

Final Round of Testing

UConn’s Next Steps Continue user analysis Review query logs, usage data Solicit feedback from users, colleagues Add / tweak … User tags “ My Databases” feature Better searching? Note errors, redesign and retest

Continue user analysis

Review query logs, usage data

Solicit feedback from users, colleagues

Add / tweak …

User tags

“ My Databases” feature

Better searching?

Note errors, redesign and retest

UConn Redesign Team Stephanie Willen Brown , electronic resource librarian & liaison to Communication Sciences Susanna Cowan , undergraduate education & outreach librarian Kate Fuller , reference collection maintenance coordinator and / administrative assistant Jill Livingston , reference librarian/liaison to the school of allied health Tom Wood , applications developer

Stephanie Willen Brown , electronic resource librarian & liaison to Communication Sciences

Susanna Cowan , undergraduate education & outreach librarian

Kate Fuller , reference collection maintenance coordinator and / administrative assistant

Jill Livingston , reference librarian/liaison to the school of allied health

Tom Wood , applications developer

Useful Usability Resources http://gslis.simmons.edu/mw/browns/Usability

“ PERM” FAQ http://gslis.simmons.edu/mw/browns/Usability/UConn-FAQ

Usability Testing: Worth the Effort! Tony Dungy Coach of the Indianapolis Colts Upon winning 2007 Superbowl