A challenge for library web designers is not just determining terminology that students will understand but also organizing terms in such a way that facilitates access to “needed information effectively and efficiently” (ACRL Information Literacy Standard 2). Library jargon can be confusing to students, so using terms that make sense to a librarian may actually hinder access to information on a website. A card sort exercise is one method that library web designers can use to determine the most appropriate terminology and structure to use on a website, which will result in user-centered information architecture. During a card sort exercise, participants organize terms on index cards in a manner that makes sense to them, creating categories of information with headings that best describe the groupings, along with choosing terms that best describe the information needed. This presentation will illustrate how to gather and correctly interpret assessment data with a card sort exercise in order to create a user-centered website that can effectively support information literacy initiatives, whether for a library website or for any educational website. The presenters will explain different types of card sort exercises, describe how they executed a card sort exercise, how they collected and analyzed the data, and discuss the results as well as lessons learned, such as what worked well, what went wrong, and what they would do differently for future card sort exercises and web site assessment projects.
2. Augusta State and Georgia Health
Sciences (MCG) Universities
announce plan to consolidate
3. Consolidation Moves Forward
JANUARY 12, 2012
University System of
Georgia (USG) Board
of Regents approve
merger of Augusta
Sate and Georgia
Health Sciences
University
Summer- Fall 2012
Library Work Team submit regular
updates to University
Consolidation Working Group.
Updates are made available to
public
December 11, 2012
SACS approves
consolidation
January 8, 2013
Consolidation
takes effect after
approval from
USG Board of
Regents
Fall 2013
Inaugural class of
GRU enrolled
4. Consolidation of Libraries
University Libraries
Reese Library
Undergraduates
liberal-arts
Teaching-focused
Greenblatt Library
Graduates and
Clinicians
health and science
Research-focused
5. Consolidated Libraries
Sub-Committee Formed
“to consolidate the Reese and Greenblatt
Libraries websites into one
Libraries website to best meet the needs of
the students, faculty, clinicians,
researchers and staff of New U by mid-
January 2013 when SACS approves
consolidation”
6. Consolidated Libraries
Sub-Committee Formed
Librarian
Library Systems
Greenblatt Library
Librarian
Research and Education
Greenblatt Library
Staff / Librarian
Research and Education
Greenblatt Library
Librarian
Reference and Education
Reese Library
Staff
Library Systems
Reese Library
Librarian
Reference and Education
Reese Library
8. Early Research and Review
Lehman, T., & Nikkel, T. (2008). Making library Web sites
usable: A LITA guide. New York: Neal-Schuman Publishers.
Cockrell, B., & Cockrell and Elaine A. Jayne. (2002). How do I
find an article? Insights from a web usability study. Journal of
Academic Librarianship, 28, 122-132.
Davidsen, S., & Yankee, E. (2004). Web Site Design with the
Patron in Mind : A Step-by-step Guide for Libraries. Chicago:
American Library Association
Norlin, E., & Winters, C. (2002). Usability testing for library
websites: A hands-on guide. Chicago: American Library
Association
Smith, S. S., & American Library Association. (2010). Web-
based instruction: A guide for libraries (3rd ed.). Chicago:
American Library Association.
Tolliver, R. L., Carter, D. S., Chapman, S. E., Edwards, P. M.,
Fisher, J. E., Haines, A. L., & ... Price, R. M. (2005). Website
redesign and testing with a usability consultant: lessons
learned. OCLC Systems & Services, 21(3), 156-166.
11. Gaining User Perspective: Survey
“It is too complicated
navigating through, it should be
more like common sense”
“lack of visual appeal”
“too many links”
“cluttered”
“(needs) better link to news
and events”
12. Gaining User Perspective: Survey
ASU users indicated that
GALILEO, GIL-Find, and
eJournals were the most
important
v.s.
GHSU users indicate eJournals
list and Database list most
important.
14. Gaining User Perspective: Next Steps
• Sub-Committee formed
• Committee review of professional literature and
aspirational sites
• Committee review of existing websites
• Gather user perspective
• Horizontal navigation bar solution
• Card-sort exercise
16. Overview
Materials
Setting
• Reese
– Study Rooms
• Greenblatt
– Lobby of the Health Sciences
Building
– Library Study Room
• Script
• Index Cards
• Sign in Sheet
• Incentives
• Camera
• Notepad
Type of Card Sort
• Closed Cart Sort
21. Results: Compiling Data
HELP ABOUT FIND INFORMATION SERVICES
LIBRARIES &
COLLECTIONS
HOW TO… ORGANIZATIONAL CHART CATALOG COMMUNITY SERVICES REESE LIBRARY
LIBANSWERS/FAQS DIRECTIONS/FLOOR PLAN DATABASES A-Z FACULTY SERVICES SPECIAL COLLECTIONS
TUTORIALS PHONE GIL EXPRESS BORROWING/CIRCULATIO
N
CURRICULUM CENTER
ASK A LIBRIARN EMAIL GOVERNMENT
DOCUMENTS
DOCUMENT DELIVERY/ILL EJOURNALS
CHAT STAFF
DIRECTORY/DEPARTMENT
LIST
LIBGUIDES AUDIO/VISUAL EBOOKS
TEXT/SMS POLICIES RESEARCH
CONSULTATION
SCHEDULE A ROOM HISTORICAL
COLLECTIONS/ARCHIVES
EVENTS CALENDAR OFF CAMPUS ACCESS GREENBLATT LIBRARIES
SUPPORT THE LIBRARIES
STRATEGIC PLAN
Terms which needed clarification User Remarks
LIAISONS/SUBJECT LIBRARIANS
SMS
STRATEGIC PLAN and ORGANIZATIONAL CHART not
specific enough – thought org chart was about floor
plans.
I would move text/sms to “Help” and
combine it with chat, making it one card
instead of two.
I have absolutely no idea what to do with
liaisons/subject librarians.
I would not have both “Help” and “Ask a
Librarian” – Instead use “Help” then have
subgroups, such as “Ask a Librarian” then
chat, etc. under that. To me ask a librarian
means that you have questions about the
library.
Change “find information” to “research” or
“research online.”
Tester Remarks
Near the end subject moved items to “Find
Information,” which seemed to be the most
confusing header.
Would ORDER BY IMPORTANCE.
23. Find
Resources
Services Get Help
Libraries &
Collections
About Us
Contact
Us
Catalog
Borrowing &
Renewal
LibAnswers
Curriculum
Center
Floor Plans
eBooks
For Community
Patrons
LibGuides
Government
Information
Give to the
Libraries
eJournals For Faculty
Off-Campus
Access
Greenblatt
Library
Maps &
Directions
GALILEO
For Patrons with
Disabilities
Research
Consultation
Historical
Collections &
Archives
Organizational
Chart
Health Sciences
Databases
GIL Express Tutorials Reese Library Policies
Interlibrary Loan
Special
Collections
Staff Directory
Strategic Plan
24. Lessons Learned
“There are often unforeseen results, though, depending on the
perceptions and experience of the users” (Brucker, 2010, p. 42)
Photo credit: Creative Commons
25. LESSON #1
Library Jargon
archives citation bibliography boolean operators catalog circulation interlibrary
loan database liaison subject specialist government documents course reserves
reference edition ezproxy endnote ejournals microforms primary source
organizational chart strategic plan GALILEO GIL universal catalog borrowing
renewing my library account institutional repository research consultation
repository curriculum materials special collections learning center citation style
guides data services patrons users discharged call number stacks OPAC digital
information literacy embedded librarian resources collections users library
instruction scholarly commons LibGuides scholarly voyager serials bound
volumes document delivery content management index patrons OCLC worldcat
26. “Some of these cards, when taken
out of the context of the Web site,
may convey little meaning to users”
(Brucker, 2010, p. 43).
28. Closed card sort: “…there is a real danger of skewing the results,
guiding users into categories that might not actually make any
sense to them, or categories they never would have created on
their own” (Brucker, 2010, p. 43).
29. “A closed card sort can be helpful in determining
where someone would place an item, but that
doesn’t mean that it illustrates which categories
users will look at in order to find information on an
actual Web site” (Brucker, 2010, pp. 47-48).
31. • Qualitative versus quantitative.
• Observational data (take good notes).
• Keep track of terms that confuse users.
• Follow up questions and discussion.
• Different users have different needs
(faculty/student/staff).
• Choose your software before you plan and
perform the sort.
32. • “Dedicated statistical software could be used
to uncover patterns in the data” (Brucker,
2010, p. 49).
o Statistical cluster analysis (XLSTAT)
o http://www.boxesandarrows.com/files/ban
da/card_sorting_a_definitive_guide/Card_S
ort_Analysis_Tmpl.xls (enhances
observational data; does not provide robust
statistical data)
33. No matter how much you plan, there
will be unforeseen results.
35. Find Resources Services Get Help Libraries &
Collections
About Us Contact Us
Catalog (GIL) For Students A-Z Index Greenblatt Library
(Health Sciences)
Floor Plans
eBooks For Faculty & Staff Ask Us! Reese Library
(Summerville)
Give to the
Libraries
eJournals For Clinicians Liaison/Subject
Librarians
Curriculum Center Maps & Directions
GALILEO
Databases
For Community
Patrons
LibGuides Government Information Organizational
Chart
Health Sciences
Databases
For Patrons with
Disabilities
Off-Campus Access Historical Collections
(Greenblatt)
Policies
Mobile Resources Borrowing &
Renewal
Research
Consultation
Special Collections
(Reese)
Staff Directory
GIL Express Tutorials Strategic Plan
Interlibrary Loan
36. Find Resources Services Get Help Libraries &
Collections
About Us Contact Us
GALILEO Databases For Students Ask Us! Greenblatt Library
(Health Sciences)
Maps & Directions
Health Sciences
Databases
For Faculty &
Staff
Liaison/Subject
Librarians
Reese Library
(Summerville)
Floor Plans
eJournals For Clinicians LibGuides Government Information Staff Directory
Catalog (GIL) For Community
Patrons
Library Instruction Historical Collections
(Health Sciences)
Organizational
Chart
eBooks For Patrons with
Disabilities
Off-Campus Access Special Collections
(Reese)
Policies
Mobile Resources Borrowing &
Renewing
Research
Consultation
Curriculum Center Strategic Plan
GIL Express Site Index Give to the
Libraries
Interlibrary Loan
(ILLiad)
Tutorials
37. Find Resources Services Get Help Libraries & Collections About Us Contact Us
Databases A-Z Students Ask Us! Greenblatt Library
(Health Sciences)
Maps &
Directions
eJournals Faculty & Staff Liaison/Subject
Librarians
Reese Library
(Summerville)
Policies
Health Sciences
eBooks
Clinicians Research
Consultation
Government
Information
Organizational
Chart
Catalog (GIL) Community Users LibGuides Historical
Collections & Archives
(Health Sciences)
Staff Directory
GALILEO Users with Disabilities Library
Instruction
Special
Collections
(Summerville)
Strategic
Plan
Borrowing & Renewing Tutorials Scholarly Commons Give to the
Libraries
Interlibrary Loan Off-Campus Access Curriculum
Center
A-Z Index
39. What we Would do Different
• Practice. Practice. Practice.
• Have a plan for analyzing data.
• Tell participants to go with first reaction and
don’t overthink.
• Don’t provide category headings (ask participants
to suggest heading names).
• Let them duplicate terms in different categories.
• Have definitions ready for each term (put on back
of index card using natural language).
40. “…observational data that might be more useful than
any statistical results” (Brucker, 2010, p. 51).
Photo credit: http://www.flickr.com/photos/nathanielhebert/5783879702/
45. References
Brucker, J. (2010). Playing with a bad deck: The caveats of card sorting as a web site redesign tool. Journal of Hospital
Librarianship, 10, 41-53.
Cobus, L., Dent, V., & Ondrusek, A. (2005). How Twenty-Eight Users Helped Redesign an Academic Library Web Site. Reference &
User Services Quarterly, 44(3), 232-246.
Cockrell, B., & Cockrell and Elaine A. Jayne. (2002). How do I find an article? Insights from a web usability study. Journal of
Academic Librarianship, 28, 122-132.
Davidsen, S., & Yankee, E. (2004). Web Site Design with the Patron in Mind : A Step-by-step Guide for Libraries. Chicago:
American Library Association.
Lehman, T., & Nikkel, T. (2008). Making library Web sites usable: A LITA guide. New York: Neal-Schuman.
Norlin, E., & Winters, C. (2002). Usability testing for library websites: A hands-on guide. Chicago: American Library Association
Smith, S. S., & American Library Association. (2010). Web-based instruction: A guide for libraries (3rd ed.). Chicago: American
Library Association.
Tolliver, R. L., Carter, D. S., Chapman, S. E., Edwards, P. M., Fisher, J. E., Haines, A. L., & ... Price, R. M. (2005). Website redesign
and testing with a usability consultant: lessons learned. OCLC Systems & Services, 21(3), 156-166.
Editor's Notes
Why did we redesign the website now? What was our motivator? “Next Great American Research University” http://chronicle.augusta.com/news/education/2012-04-23/jobs-new-names-parking-housing-concerns-aired-asu-and-ghsu-forums ZACH BOYDEN-HOLMES/STAFF JANUARY 12, 2012 University System of Georgia (USG) Board of Regents approve merger of Augusta Sate and Georgia Health Sciences University
Announcement of consolidation Spring 2012. Proposed timeline. Research I aims. Over 50 work teams, and numerous other taskforces. Each step of the consolidation brought significant changes to the library.
*THIS PAGE NEEDS WORK” LIBRARY WORK TEAM, Subcomittees- promotion and tenure, Content Management Peer Working group, (ibrary Work Team IT sub-committee) The Policy Peer Working Group, Library Work Team Governance sub-committee
Please also think about questions we should ask our users in a survey to determine what they like and what/how to improve. You may want to look at library literature or other library websites to see the survey questions they ask their users to improve their website. Our first meeting goal is to create the questions for web surveys for both libraries websites (many questions will be the same but some may be unique to Reese or Greenblatt) so the surveys can be reviewed by library leadership and we can put online to start getting user feedback. Please let me know if there are other crucial items that should be added to the agenda.
Sub-Committee formed in June 2012 to analyze website and offer recommendations. Good cross section of library to staff, reese to greenblatt, and library systems to reference and education. One member became earned her MLIS during process. Another new librarian joined. Early on committee members realized the uniqueness of their two user bases. Ginny joins in September
Looked towards comprable and aspirational schools. Buffalo ohio, miami and unc. Elements we like -> aesthetic, images, prominent search box, horizontal nav bar, consolidated singular page, and subpages for the library. Prominent Social Media
Review of library and website related literature. http://www.alastore.ala.org/images/nsimages/making-library-web-sites-usable--a-lita-guide-gallery-1-240x350.png
Discuss issues with Reese Library’s original website. Aesthetics, Nomenclature, format, static pages, multiple versions of website, old slideshow. Difficult to update
Discuss issues with Greenblatt’s original website Aesthetics, static pages, static image, too many links. Difficult to update
Users have problem finding links to the library websites from the university homepages.
Reese users liked multi-tab search box
GHSU users like the Quick links
Both cmapus request updated website links
Difficulty getting GALILEO password
Difficulty getting to ILL
Both mobile sites need more visibility
Survey reaffirmed early suspicions of issues such as aestheVast majority of ASU users indicated GALILEO databases, GIL, and eJournals were a priority.
GHSU users indicate eJournals and databases were priority. Seldom use of GIL and GALILEO.
tics, navigation, etc. “user-centered, intuitive, and user-friendly consolidated libraries web site”
HORIZONTAL Navigation Bar Solution
Type of Card sort
Pre-determined category headings.
Pre-determined terms.
Group terms under category headings.
Script for consistency.
Asked participants to think aloud.
Asked them to select category that best describes the term in hand.
Discard cards that are not important to Web site.
Set aside cards with no meaning – ask questions.
Follow-up Questions
Would you change any of the terms that appear on the yellow cards? Do you have suggestions for other categories? (Provide marker and yellow cards)
Would you change any of the terms that appear on the white cards? Do you have any suggestions for other resources and services? (Provide marker and white cards)
I see under the “services” category you have listed [7] cards. In what order would you put these terms if they appeared in a drop-down menu?
Do you have any remaining questions about the exercise or the redesign process?
Categories/headings in a different color to differentiate them from the links
This is an example of how we compiled our data for each users.
It’s important to note that we did not have this before we started the card sorting activity. It was only after we started that we realized we needed a better, systematic way of organizing the data we collected.
Terms that needed clarification with total number of instances (total number of participants: 36).
As a committee we reviewed data and decisions regarding our final categories and resources & services (terms)
Order of links in the drop downs-majority said A-Z
Be aware that your results may surprise you.
Every profession has its jargon; communicating with users;
Context – creating context with the categories; forcing items into these categories; too much context with headings; lack of context with cards.
During the card sort, everything is out there. In reality, on a website, the user only sees the headings, and if the headings don’t make sense to them, then they will get frustrated when looking for what they need. Is JSTOR under services or resources? Users don’t think like we (librarians and educators) do.
Plan ahead. Card sorting for library websites is not as cut and dried as say for a grocery store or bookstore.
We found that the qualitative data was extremely useful; take good notes. If you can, record the process. Your memory isn’t the most accurate tool.
Brucker recommends designing the card sort for the statistical tool that you have access to. Even then, “card sorting is an imprecise tool for libraries.” Why? Multiple user groups; jargon; vague terms (what is a resource to a patron?)
Lesson: Even if you come up with information architecture that you think will best represent what the users need, that doesn’t mean that it will be the final product.
What we presented to Library Work Team in December; order is alphabetical.
Revised dropdown based on meeting with Library Work Team; order is a combination of alphabetical and prioritization.
What we sent to the contractors and are still refining.
If we could go back and do it all over again.
First gut reaction, like when surfing on the web.
We found the comments extremely helpful. Again, qualitative data.
The nav bar on the home page in many forms.
Our original mock-up based on what we thought the university template was going to be.
Mock up we gave to contractors.
First and second mock-up from contractors
Third and last mock-up from contractors before they programmed the page.
Live page after more changes requested.
Live home page; not done; show live page via internet.