SlideShare a Scribd company logo
Usability Testing of a Library Web Site: Librarians and Anthropologists Working Together Allison Cowgill & Amanda Dinscore Henry Madden Library  California State University, Fresno  California Association of Research Libraries April, 2010  Sacramento, CA
University of Rochester Study Ethnographic approach to studying campus libraries and their users
The Library Study at Fresno State September, 2008 - May, 2009 Goal:  “to discover information about student life that campus librarians could use to both increase library usage and improve student users’ experience of library services.”
“Draw How You Feel When You Write a Paper.”
Library Study Recommendations Provide different search boxes for different kinds of holdings. Attempt a redesign that aims for a less institutional, more youthful and content-rich feel. Move toward a Web 2.0 model. Still Needed to know… How do students use the site?  What tasks can students accomplish?  What assumptions are we making?  What can we discover that will inform our re-design?
Web Usability Workgroup (WUW) Decision to conduct a usability study and the formation of the Web Usability Workgroup (WUW) Involvement of Anthropology professor, Hank Delcore, and his students  in ANTH 111B (Intermediate Ethnographic Fieldwork)  “How people develop and use technological solutions (like the library's website) is one of the most complicated and important areas of modern life, so it's a natural anthropological attention.”                     Delcore, 2010
Usability Testing  Usability Testing demonstrates how users interact with the system.  Participants are asked to complete a series of pre-defined tasks  and to “think aloud” while completing them. 		 Benefits:  See what real users actually do (rather than what they say they do) Provides quantitative data rather than opinion or conjecture   Participants complete common tasks (finding a book, article, etc…)
Process  Develop a plan  Purpose Determine the user group to be tested Pre-survey/Consent form Tasks to be performed (number & time)  Script & Moderator Instructions—ensures consistency Test Design—how it will be conducted and how results are to be recorded Pre-test   Recruit & test Analyze the results
Nuts & Bolts    Subjects: Juniors   Sample Size: 42 (plus 10 pre-tests)  Recruitment: On-site People: 5 student moderators, 5 WUW members (Anthropology and library faculty) Facilities: 4 small rooms for testing; welcome area  Equipment: 4 Macs Software: Screenium Test Documents:  Pre-Survey/Consent Form  Introduction Script  Moderator Instructions Usability Test Activity Cost:  $636 ($520 for Starbucks gift cards; Screenium software site licenses $29 per computer)
Benefitted from pre-testing and multiple run-throughs with team members and moderators  Challenges with script, wording of questions, issues that we hadn’t previously considered (e.g. using Google)  Benefits of using student moderators (objective; offered valuable insight) Talk aloud method was sometimes challenging  Testing took less time  than we thought On-site recruitment worked better than expected  The $10 Starbucks card incentive worked well  Subjects took it seriously Software (Screenium) worked well Reflections
Coding & Data Entry  Subject Number, time stamp, task number (1-10)  Link Label (words clicked) & URL Notes (including quotes from subjects)  “Lostness”  Codes for task completion Codes for positive & negative comments Codes for non-verbal behaviors such as smiling, frowning, laughing Data Analysis (in process)
Discussion

More Related Content

What's hot

October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...
October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...
October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...
National Information Standards Organization (NISO)
 
Redesigning the Open Access Institutional Repository
Redesigning the Open Access Institutional RepositoryRedesigning the Open Access Institutional Repository
Redesigning the Open Access Institutional Repository
Edward Luca
 
Aect 2011 web_questincscl
Aect 2011 web_questincsclAect 2011 web_questincscl
Aect 2011 web_questincscl
Dabae Lee
 
Ethnographically-inspired usability testing
Ethnographically-inspired usability testingEthnographically-inspired usability testing
Ethnographically-inspired usability testing
Leah Emary
 
October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...
October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...
October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...
National Information Standards Organization (NISO)
 
Educ 526 presentation
Educ 526 presentationEduc 526 presentation
Educ 526 presentation
chervitz
 
Digital Library Poster
Digital Library PosterDigital Library Poster
Digital Library Poster
Nicholas Charles
 
Noteworks SI 622 Presentation Slides
Noteworks SI 622 Presentation SlidesNoteworks SI 622 Presentation Slides
Noteworks SI 622 Presentation Slides
kebuhc
 
Impact your Library UX with Contextual Inquiry
Impact your Library UX with Contextual InquiryImpact your Library UX with Contextual Inquiry
Impact your Library UX with Contextual Inquiry
Rachel Vacek
 
Getting Them There: A Small-Scale Usability Test of a University Library Website
Getting Them There: A Small-Scale Usability Test of a University Library WebsiteGetting Them There: A Small-Scale Usability Test of a University Library Website
Getting Them There: A Small-Scale Usability Test of a University Library Website
jsimon6
 
Straight from the user's mouth: Usability testing UNC's LibGuides
Straight from the user's mouth: Usability testing UNC's LibGuidesStraight from the user's mouth: Usability testing UNC's LibGuides
Straight from the user's mouth: Usability testing UNC's LibGuides
Sarah Joy Arnold
 
Using the internet to improve higher order thinking
Using the internet to improve higher order thinkingUsing the internet to improve higher order thinking
Using the internet to improve higher order thinking
Shawky Allam
 
Service Design: Towards a Holistic Assessment of the Library Experience
Service Design: Towards a Holistic Assessment of the Library ExperienceService Design: Towards a Holistic Assessment of the Library Experience
Service Design: Towards a Holistic Assessment of the Library Experience
Joe Marquez
 

What's hot (13)

October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...
October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...
October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...
 
Redesigning the Open Access Institutional Repository
Redesigning the Open Access Institutional RepositoryRedesigning the Open Access Institutional Repository
Redesigning the Open Access Institutional Repository
 
Aect 2011 web_questincscl
Aect 2011 web_questincsclAect 2011 web_questincscl
Aect 2011 web_questincscl
 
Ethnographically-inspired usability testing
Ethnographically-inspired usability testingEthnographically-inspired usability testing
Ethnographically-inspired usability testing
 
October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...
October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...
October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...
 
Educ 526 presentation
Educ 526 presentationEduc 526 presentation
Educ 526 presentation
 
Digital Library Poster
Digital Library PosterDigital Library Poster
Digital Library Poster
 
Noteworks SI 622 Presentation Slides
Noteworks SI 622 Presentation SlidesNoteworks SI 622 Presentation Slides
Noteworks SI 622 Presentation Slides
 
Impact your Library UX with Contextual Inquiry
Impact your Library UX with Contextual InquiryImpact your Library UX with Contextual Inquiry
Impact your Library UX with Contextual Inquiry
 
Getting Them There: A Small-Scale Usability Test of a University Library Website
Getting Them There: A Small-Scale Usability Test of a University Library WebsiteGetting Them There: A Small-Scale Usability Test of a University Library Website
Getting Them There: A Small-Scale Usability Test of a University Library Website
 
Straight from the user's mouth: Usability testing UNC's LibGuides
Straight from the user's mouth: Usability testing UNC's LibGuidesStraight from the user's mouth: Usability testing UNC's LibGuides
Straight from the user's mouth: Usability testing UNC's LibGuides
 
Using the internet to improve higher order thinking
Using the internet to improve higher order thinkingUsing the internet to improve higher order thinking
Using the internet to improve higher order thinking
 
Service Design: Towards a Holistic Assessment of the Library Experience
Service Design: Towards a Holistic Assessment of the Library ExperienceService Design: Towards a Holistic Assessment of the Library Experience
Service Design: Towards a Holistic Assessment of the Library Experience
 

Viewers also liked

ACRL 2011 Data-Driven Library Web Design
ACRL 2011 Data-Driven Library Web DesignACRL 2011 Data-Driven Library Web Design
ACRL 2011 Data-Driven Library Web Design
Amanda Dinscore
 
Stumble upon
Stumble uponStumble upon
Stumble upon
abudde
 
1 librarian 1,400 freshmen
1 librarian 1,400 freshmen1 librarian 1,400 freshmen
1 librarian 1,400 freshmen
Amanda Dinscore
 
Bruce Kraselsky
Bruce KraselskyBruce Kraselsky
Bruce Kraselsky
Carl Ford
 
Storyboard
StoryboardStoryboard
Storyboard
Amanda Dinscore
 
Dool's house 2
Dool's house 2Dool's house 2
Dool's house 2
Abhilash Manatkar
 
Chap009 managing diversifcation and group
Chap009  managing diversifcation and groupChap009  managing diversifcation and group
Chap009 managing diversifcation and group
Ajit Kumar
 
Five generic competitive strategies
Five generic competitive strategiesFive generic competitive strategies
Five generic competitive strategies
dragon888
 
Five generic competitive strategies
Five generic competitive strategiesFive generic competitive strategies
Five generic competitive strategies
yutkey
 
The Five Generic Competitive Strategies : Which One to Employ?
The Five Generic Competitive Strategies : Which One to Employ?The Five Generic Competitive Strategies : Which One to Employ?
The Five Generic Competitive Strategies : Which One to Employ?
Ami Sampath
 
Competitive strategies
Competitive strategiesCompetitive strategies
Competitive strategies
Geetika Madhur
 

Viewers also liked (11)

ACRL 2011 Data-Driven Library Web Design
ACRL 2011 Data-Driven Library Web DesignACRL 2011 Data-Driven Library Web Design
ACRL 2011 Data-Driven Library Web Design
 
Stumble upon
Stumble uponStumble upon
Stumble upon
 
1 librarian 1,400 freshmen
1 librarian 1,400 freshmen1 librarian 1,400 freshmen
1 librarian 1,400 freshmen
 
Bruce Kraselsky
Bruce KraselskyBruce Kraselsky
Bruce Kraselsky
 
Storyboard
StoryboardStoryboard
Storyboard
 
Dool's house 2
Dool's house 2Dool's house 2
Dool's house 2
 
Chap009 managing diversifcation and group
Chap009  managing diversifcation and groupChap009  managing diversifcation and group
Chap009 managing diversifcation and group
 
Five generic competitive strategies
Five generic competitive strategiesFive generic competitive strategies
Five generic competitive strategies
 
Five generic competitive strategies
Five generic competitive strategiesFive generic competitive strategies
Five generic competitive strategies
 
The Five Generic Competitive Strategies : Which One to Employ?
The Five Generic Competitive Strategies : Which One to Employ?The Five Generic Competitive Strategies : Which One to Employ?
The Five Generic Competitive Strategies : Which One to Employ?
 
Competitive strategies
Competitive strategiesCompetitive strategies
Competitive strategies
 

Similar to “Usability Testing of a Library Web Site: Librarians and Anthropologists Working Together”

Web quest presentation e_feig_edu643
Web quest presentation e_feig_edu643Web quest presentation e_feig_edu643
Web quest presentation e_feig_edu643
ellen feig
 
Ux Research Portfolio
Ux Research PortfolioUx Research Portfolio
Ux Research Portfolio
Tao Zhang
 
User experience at Imperial: a case study of qualitative approaches to Primo ...
User experience at Imperial: a case study of qualitative approaches to Primo ...User experience at Imperial: a case study of qualitative approaches to Primo ...
User experience at Imperial: a case study of qualitative approaches to Primo ...
Andrew Preater
 
How to Integrate Technology into the Curriculum
How to Integrate Technology into the CurriculumHow to Integrate Technology into the Curriculum
How to Integrate Technology into the Curriculum
Sabine Reljic
 
UCLALibrary_DResSUP.pptx
UCLALibrary_DResSUP.pptxUCLALibrary_DResSUP.pptx
UCLALibrary_DResSUP.pptx
robhill123
 
Evaluating Electronic Resources
Evaluating Electronic ResourcesEvaluating Electronic Resources
Evaluating Electronic Resources
Richard Bernier
 
Hci techniques from idea to deployment
Hci techniques from idea to deploymentHci techniques from idea to deployment
Hci techniques from idea to deployment
John Thomas
 
Presentation to 2014 University of Guelph Accessibility Conference Perspectiv...
Presentation to 2014 University of Guelph Accessibility Conference Perspectiv...Presentation to 2014 University of Guelph Accessibility Conference Perspectiv...
Presentation to 2014 University of Guelph Accessibility Conference Perspectiv...
Shawna Reibling
 
Fims student conference2012_sandygallipeau
Fims student conference2012_sandygallipeauFims student conference2012_sandygallipeau
Fims student conference2012_sandygallipeau
Sandra Gallipeau
 
Webquests
WebquestsWebquests
Webquests
ictgirl
 
Introduction to Qualitative Research - Syllabus Spring 2014
Introduction to Qualitative Research - Syllabus Spring 2014Introduction to Qualitative Research - Syllabus Spring 2014
Introduction to Qualitative Research - Syllabus Spring 2014
Joan E. Hughes, Ph.D.
 
project work on internet
project work on internetproject work on internet
project work on internet
XimenaBonilla
 
Owning the Discovery Experience for Your Patrons
Owning the Discovery Experience for Your PatronsOwning the Discovery Experience for Your Patrons
Owning the Discovery Experience for Your Patrons
Robert H. McDonald
 
New library management system slide
New library management system slideNew library management system slide
New library management system slide
Enzo Ivan
 
Web eval speed test lesson
Web eval speed test lessonWeb eval speed test lesson
Web eval speed test lesson
mimbyla
 
Ecer 2011
Ecer 2011Ecer 2011
Ecer 2011
walravenamber
 
Researching Researchers - Designing the User Experience at ProQuest
Researching Researchers - Designing the User Experience at ProQuestResearching Researchers - Designing the User Experience at ProQuest
Researching Researchers - Designing the User Experience at ProQuest
Mathias Burton
 
Collaborative learning presentation
Collaborative learning presentationCollaborative learning presentation
Collaborative learning presentation
ShiKia Carter
 
User Interface Prototyping Techniques: Low Fidelity Prototyping
User Interface Prototyping Techniques: Low Fidelity PrototypingUser Interface Prototyping Techniques: Low Fidelity Prototyping
User Interface Prototyping Techniques: Low Fidelity Prototyping
Hans Põldoja
 
Designing Problem Based Learning Online
Designing Problem Based Learning OnlineDesigning Problem Based Learning Online
Designing Problem Based Learning Online
David Jennings
 

Similar to “Usability Testing of a Library Web Site: Librarians and Anthropologists Working Together” (20)

Web quest presentation e_feig_edu643
Web quest presentation e_feig_edu643Web quest presentation e_feig_edu643
Web quest presentation e_feig_edu643
 
Ux Research Portfolio
Ux Research PortfolioUx Research Portfolio
Ux Research Portfolio
 
User experience at Imperial: a case study of qualitative approaches to Primo ...
User experience at Imperial: a case study of qualitative approaches to Primo ...User experience at Imperial: a case study of qualitative approaches to Primo ...
User experience at Imperial: a case study of qualitative approaches to Primo ...
 
How to Integrate Technology into the Curriculum
How to Integrate Technology into the CurriculumHow to Integrate Technology into the Curriculum
How to Integrate Technology into the Curriculum
 
UCLALibrary_DResSUP.pptx
UCLALibrary_DResSUP.pptxUCLALibrary_DResSUP.pptx
UCLALibrary_DResSUP.pptx
 
Evaluating Electronic Resources
Evaluating Electronic ResourcesEvaluating Electronic Resources
Evaluating Electronic Resources
 
Hci techniques from idea to deployment
Hci techniques from idea to deploymentHci techniques from idea to deployment
Hci techniques from idea to deployment
 
Presentation to 2014 University of Guelph Accessibility Conference Perspectiv...
Presentation to 2014 University of Guelph Accessibility Conference Perspectiv...Presentation to 2014 University of Guelph Accessibility Conference Perspectiv...
Presentation to 2014 University of Guelph Accessibility Conference Perspectiv...
 
Fims student conference2012_sandygallipeau
Fims student conference2012_sandygallipeauFims student conference2012_sandygallipeau
Fims student conference2012_sandygallipeau
 
Webquests
WebquestsWebquests
Webquests
 
Introduction to Qualitative Research - Syllabus Spring 2014
Introduction to Qualitative Research - Syllabus Spring 2014Introduction to Qualitative Research - Syllabus Spring 2014
Introduction to Qualitative Research - Syllabus Spring 2014
 
project work on internet
project work on internetproject work on internet
project work on internet
 
Owning the Discovery Experience for Your Patrons
Owning the Discovery Experience for Your PatronsOwning the Discovery Experience for Your Patrons
Owning the Discovery Experience for Your Patrons
 
New library management system slide
New library management system slideNew library management system slide
New library management system slide
 
Web eval speed test lesson
Web eval speed test lessonWeb eval speed test lesson
Web eval speed test lesson
 
Ecer 2011
Ecer 2011Ecer 2011
Ecer 2011
 
Researching Researchers - Designing the User Experience at ProQuest
Researching Researchers - Designing the User Experience at ProQuestResearching Researchers - Designing the User Experience at ProQuest
Researching Researchers - Designing the User Experience at ProQuest
 
Collaborative learning presentation
Collaborative learning presentationCollaborative learning presentation
Collaborative learning presentation
 
User Interface Prototyping Techniques: Low Fidelity Prototyping
User Interface Prototyping Techniques: Low Fidelity PrototypingUser Interface Prototyping Techniques: Low Fidelity Prototyping
User Interface Prototyping Techniques: Low Fidelity Prototyping
 
Designing Problem Based Learning Online
Designing Problem Based Learning OnlineDesigning Problem Based Learning Online
Designing Problem Based Learning Online
 

Recently uploaded

Skimbleshanks-The-Railway-Cat by T S Eliot
Skimbleshanks-The-Railway-Cat by T S EliotSkimbleshanks-The-Railway-Cat by T S Eliot
Skimbleshanks-The-Railway-Cat by T S Eliot
nitinpv4ai
 
Wound healing PPT
Wound healing PPTWound healing PPT
Wound healing PPT
Jyoti Chand
 
مصحف القراءات العشر أعد أحرف الخلاف سمير بسيوني.pdf
مصحف القراءات العشر   أعد أحرف الخلاف سمير بسيوني.pdfمصحف القراءات العشر   أعد أحرف الخلاف سمير بسيوني.pdf
مصحف القراءات العشر أعد أحرف الخلاف سمير بسيوني.pdf
سمير بسيوني
 
Haunted Houses by H W Longfellow for class 10
Haunted Houses by H W Longfellow for class 10Haunted Houses by H W Longfellow for class 10
Haunted Houses by H W Longfellow for class 10
nitinpv4ai
 
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
Nguyen Thanh Tu Collection
 
Stack Memory Organization of 8086 Microprocessor
Stack Memory Organization of 8086 MicroprocessorStack Memory Organization of 8086 Microprocessor
Stack Memory Organization of 8086 Microprocessor
JomonJoseph58
 
CIS 4200-02 Group 1 Final Project Report (1).pdf
CIS 4200-02 Group 1 Final Project Report (1).pdfCIS 4200-02 Group 1 Final Project Report (1).pdf
CIS 4200-02 Group 1 Final Project Report (1).pdf
blueshagoo1
 
Oliver Asks for More by Charles Dickens (9)
Oliver Asks for More by Charles Dickens (9)Oliver Asks for More by Charles Dickens (9)
Oliver Asks for More by Charles Dickens (9)
nitinpv4ai
 
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptxPrésentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
siemaillard
 
Level 3 NCEA - NZ: A Nation In the Making 1872 - 1900 SML.ppt
Level 3 NCEA - NZ: A  Nation In the Making 1872 - 1900 SML.pptLevel 3 NCEA - NZ: A  Nation In the Making 1872 - 1900 SML.ppt
Level 3 NCEA - NZ: A Nation In the Making 1872 - 1900 SML.ppt
Henry Hollis
 
Chapter wise All Notes of First year Basic Civil Engineering.pptx
Chapter wise All Notes of First year Basic Civil Engineering.pptxChapter wise All Notes of First year Basic Civil Engineering.pptx
Chapter wise All Notes of First year Basic Civil Engineering.pptx
Denish Jangid
 
Benner "Expanding Pathways to Publishing Careers"
Benner "Expanding Pathways to Publishing Careers"Benner "Expanding Pathways to Publishing Careers"
Benner "Expanding Pathways to Publishing Careers"
National Information Standards Organization (NISO)
 
MDP on air pollution of class 8 year 2024-2025
MDP on air pollution of class 8 year 2024-2025MDP on air pollution of class 8 year 2024-2025
MDP on air pollution of class 8 year 2024-2025
khuleseema60
 
Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...
PsychoTech Services
 
Temple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation resultsTemple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation results
Krassimira Luka
 
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
GeorgeMilliken2
 
Standardized tool for Intelligence test.
Standardized tool for Intelligence test.Standardized tool for Intelligence test.
Standardized tool for Intelligence test.
deepaannamalai16
 
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
Nguyen Thanh Tu Collection
 
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem studentsRHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
Himanshu Rai
 
Electric Fetus - Record Store Scavenger Hunt
Electric Fetus - Record Store Scavenger HuntElectric Fetus - Record Store Scavenger Hunt
Electric Fetus - Record Store Scavenger Hunt
RamseyBerglund
 

Recently uploaded (20)

Skimbleshanks-The-Railway-Cat by T S Eliot
Skimbleshanks-The-Railway-Cat by T S EliotSkimbleshanks-The-Railway-Cat by T S Eliot
Skimbleshanks-The-Railway-Cat by T S Eliot
 
Wound healing PPT
Wound healing PPTWound healing PPT
Wound healing PPT
 
مصحف القراءات العشر أعد أحرف الخلاف سمير بسيوني.pdf
مصحف القراءات العشر   أعد أحرف الخلاف سمير بسيوني.pdfمصحف القراءات العشر   أعد أحرف الخلاف سمير بسيوني.pdf
مصحف القراءات العشر أعد أحرف الخلاف سمير بسيوني.pdf
 
Haunted Houses by H W Longfellow for class 10
Haunted Houses by H W Longfellow for class 10Haunted Houses by H W Longfellow for class 10
Haunted Houses by H W Longfellow for class 10
 
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
 
Stack Memory Organization of 8086 Microprocessor
Stack Memory Organization of 8086 MicroprocessorStack Memory Organization of 8086 Microprocessor
Stack Memory Organization of 8086 Microprocessor
 
CIS 4200-02 Group 1 Final Project Report (1).pdf
CIS 4200-02 Group 1 Final Project Report (1).pdfCIS 4200-02 Group 1 Final Project Report (1).pdf
CIS 4200-02 Group 1 Final Project Report (1).pdf
 
Oliver Asks for More by Charles Dickens (9)
Oliver Asks for More by Charles Dickens (9)Oliver Asks for More by Charles Dickens (9)
Oliver Asks for More by Charles Dickens (9)
 
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptxPrésentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
 
Level 3 NCEA - NZ: A Nation In the Making 1872 - 1900 SML.ppt
Level 3 NCEA - NZ: A  Nation In the Making 1872 - 1900 SML.pptLevel 3 NCEA - NZ: A  Nation In the Making 1872 - 1900 SML.ppt
Level 3 NCEA - NZ: A Nation In the Making 1872 - 1900 SML.ppt
 
Chapter wise All Notes of First year Basic Civil Engineering.pptx
Chapter wise All Notes of First year Basic Civil Engineering.pptxChapter wise All Notes of First year Basic Civil Engineering.pptx
Chapter wise All Notes of First year Basic Civil Engineering.pptx
 
Benner "Expanding Pathways to Publishing Careers"
Benner "Expanding Pathways to Publishing Careers"Benner "Expanding Pathways to Publishing Careers"
Benner "Expanding Pathways to Publishing Careers"
 
MDP on air pollution of class 8 year 2024-2025
MDP on air pollution of class 8 year 2024-2025MDP on air pollution of class 8 year 2024-2025
MDP on air pollution of class 8 year 2024-2025
 
Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...
 
Temple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation resultsTemple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation results
 
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
 
Standardized tool for Intelligence test.
Standardized tool for Intelligence test.Standardized tool for Intelligence test.
Standardized tool for Intelligence test.
 
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
 
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem studentsRHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
 
Electric Fetus - Record Store Scavenger Hunt
Electric Fetus - Record Store Scavenger HuntElectric Fetus - Record Store Scavenger Hunt
Electric Fetus - Record Store Scavenger Hunt
 

“Usability Testing of a Library Web Site: Librarians and Anthropologists Working Together”

  • 1. Usability Testing of a Library Web Site: Librarians and Anthropologists Working Together Allison Cowgill & Amanda Dinscore Henry Madden Library California State University, Fresno California Association of Research Libraries April, 2010 Sacramento, CA
  • 2. University of Rochester Study Ethnographic approach to studying campus libraries and their users
  • 3. The Library Study at Fresno State September, 2008 - May, 2009 Goal: “to discover information about student life that campus librarians could use to both increase library usage and improve student users’ experience of library services.”
  • 4. “Draw How You Feel When You Write a Paper.”
  • 5. Library Study Recommendations Provide different search boxes for different kinds of holdings. Attempt a redesign that aims for a less institutional, more youthful and content-rich feel. Move toward a Web 2.0 model. Still Needed to know… How do students use the site? What tasks can students accomplish? What assumptions are we making? What can we discover that will inform our re-design?
  • 6. Web Usability Workgroup (WUW) Decision to conduct a usability study and the formation of the Web Usability Workgroup (WUW) Involvement of Anthropology professor, Hank Delcore, and his students in ANTH 111B (Intermediate Ethnographic Fieldwork) “How people develop and use technological solutions (like the library's website) is one of the most complicated and important areas of modern life, so it's a natural anthropological attention.” Delcore, 2010
  • 7. Usability Testing Usability Testing demonstrates how users interact with the system. Participants are asked to complete a series of pre-defined tasks and to “think aloud” while completing them. Benefits: See what real users actually do (rather than what they say they do) Provides quantitative data rather than opinion or conjecture Participants complete common tasks (finding a book, article, etc…)
  • 8. Process Develop a plan Purpose Determine the user group to be tested Pre-survey/Consent form Tasks to be performed (number & time) Script & Moderator Instructions—ensures consistency Test Design—how it will be conducted and how results are to be recorded Pre-test Recruit & test Analyze the results
  • 9. Nuts & Bolts Subjects: Juniors Sample Size: 42 (plus 10 pre-tests) Recruitment: On-site People: 5 student moderators, 5 WUW members (Anthropology and library faculty) Facilities: 4 small rooms for testing; welcome area Equipment: 4 Macs Software: Screenium Test Documents: Pre-Survey/Consent Form Introduction Script Moderator Instructions Usability Test Activity Cost: $636 ($520 for Starbucks gift cards; Screenium software site licenses $29 per computer)
  • 10. Benefitted from pre-testing and multiple run-throughs with team members and moderators Challenges with script, wording of questions, issues that we hadn’t previously considered (e.g. using Google) Benefits of using student moderators (objective; offered valuable insight) Talk aloud method was sometimes challenging Testing took less time than we thought On-site recruitment worked better than expected The $10 Starbucks card incentive worked well Subjects took it seriously Software (Screenium) worked well Reflections
  • 11. Coding & Data Entry Subject Number, time stamp, task number (1-10) Link Label (words clicked) & URL Notes (including quotes from subjects) “Lostness” Codes for task completion Codes for positive & negative comments Codes for non-verbal behaviors such as smiling, frowning, laughing Data Analysis (in process)

Editor's Notes

  1. Allison: Rochester Study…10 minutes Amanda: Fresno State Study… 10 minutes Amanda: Usability Study…20 minutes Allison: Talking Points (develop and lead discussion)…30 minutes Allison: Conclusion…5minutes
  2. Focus on design anthropology in which ethnographic study serves to aid in the design of products and services. Sought to engage students in their natural contexts. Library was closed and only offering limited services for the first half of the Library Study at Fresno State. New building opened in February, 2009. Study included observations of student life. Examples of some of the methods used were: “Floating Reference”—in which students acted out the role of the student and librarian at the reference desk in the Free Speech Area“Visions of Student Scholarship”—asked students to photograph student scholarship wherever they found it“Day Mapping” – with campus maps, disposable cameras, and notebooks. Ethnographic interviews—includinghome interviews (picture of lives beyond campus)“Bootlegging Workshops”—students brainstormed ideas about library services and acted out various scenes “Student Theatre Workshops”—research team devised a plot around a library service situation and directed the actors through the first half of a skit“Interior Design Workshops” –allowed students to design their own library space“Web Design Workshops”—from scratch or using “patchwork prototyping” (existing elements in new combinations)
  3. Images: Set up a large poster board in the Free Speech Area in front of the library and asked students to draw how they feel when they write a paper. Mostly negative responses indicating a great deal of stress and frustration. Negative impressions of the library and librarians were demonstrated in several of the skits and other activities that students performed. Students expect to have difficulty when doing research and writing assignments and generally don’t see the library or librarians as being very helpful.
  4. Some of the recommendations, particularly from the web design workshops, were useful for informing our redesign, including: separate search boxes for different types of holdings (doing this now) less institutional, more youthful and content rich feel move to a 2.0 model—general interest in personalization, class tie-ins, information about required texts for classes. Also requested online collaboration spaces like Google Docs--not practical. Integration with My Fresno State and FacebookGenerally—more “open-ended” observations and interactions. Did NOT seek statistical significance of findings. Observations about student life. Still needed to know…(read slide)
  5. Decided to form a subcommittee of the main library web group to investigate what we could do to get more concrete data.Decided that a usability test was the best option. At first, only comprised of library faculty but we knew we wanted to build upon what was discovered in the Library Study. This led to further discussion and involvement with Professor Delcore, one of the main authors of the Library Study.Volunteered his students.
  6. At first, we considered the use of focus groups, surveys, analysis of site usage logs—provided only limited information. Decided on usability testing. Previous decisions based on mostly anecdotal information, assumptions about user behavior and what students “should” know, and staff and faculty personal preferences. Needed data to support change
  7. Purpose:Data rather than assumptions about user behavior. User group: Juniors (not experts yet, but may have some familiarity with the site) Pre-Survey/Consent form: consent to be videotaped Tasks: 10 tasks requiring basic research (finding a book, article, interlibrary loan, etc…); 30-45 minutes Script/Moderator Instructions: had to be very precise, offer very specific instruction to moderators. Demonstrate speak aloud method“Please let me know what you’re thinking.” How to move on, if necessary. “Thank you for trying. Some of these questions are difficult to answer. Would you like to move on to the next question.” What to say if they went off site Really stressed that it was not a test of them but of the site Balance between establishing rapport and consistencyTest design: what data do we want? (audio & video? Screen capture?), what software to use? how participants would be recruited, where to test, what software to use to record data, what incentives to provide… Pre-test: (10) Very important. Learned a lot and changed the wording and our approach to several things based on this. Recruit & Test: where to find the participants? Who would recruit? Where will they go and when? Analyze the results: working on this now. Think big at first, then streamline to what is practical
  8. Subjects: Fresno State Juniors FTE: 4,081 95% Confidence LevelConfidence Interval of 15 Methods: On-site recruitment --$10 Starbucks gift cards—everyone recruited (but mostly Anthro. students). Did not take much time and was much easier than we thought it would be. Pre-Survey/Consent Form—Had to be over 18, a Junior, and had to give consent to be videotaped. Also asked about how often they used the library web site and how often, their major. Introduction Script—created a draft and then tweaked it throughout the process and through the pre-testing phase. This was read by one of 5 anthropology students who were recruited and trained as moderators. Usability Test Activity –consisted of 10 tasks that could be completed using the library’s web site. Consisted of such things as finding books, using interlibrary loan, searching for journal articles, etc… All were tasks that we expect most users to be able to accomplish with very little difficulty. This also took several meetings to draft and revise.
  9. “Lostness” calculated using: Number of different pages visited Total number of pages visited Minimum (or optimum) number of pages that must be visited to complete the taskFrom “Measuring the User Experience” Tullis and Albert