Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Merging Methodologies: Combining
Individual and Group Card Sorting
Robert L. Thomas and Ian Johnson
Liberty Mutual, Person...
Liberty Mutual Insurance
Our old-style navigation was no longer useful for a
major insurance company’s website
 History
o...
Liberty Mutual Insurance
We needed to make content discoverable and
easy to find
 Scope
o Create a new navigational
hiera...
Liberty Mutual Insurance
To meet the needs of the project, we needed to run a
card sort, a categorization exercise where u...
Liberty Mutual Insurance
We reviewed research on card sorting methodologies
and then developed a new methodology
Proprieta...
Liberty Mutual Insurance
Our methodology combined online card sorting,
physical card sorting, and focus group discussion w...
Liberty Mutual Insurance
Breaking into teams, we followed a four-step analysis
process to arrive at our final navigation c...
Liberty Mutual Insurance
Standardized labels and combined categories
Proprietary - Trade Secret (Competitively Sensitive I...
Liberty Mutual Insurance
Table 2. Final list of 11 in-person, group standardized categories
Proprietary - Trade Secret (Co...
Liberty Mutual Insurance
Group Card Sort Results Dendrogram
Proprietary - Trade Secret (Competitively Sensitive Informatio...
Liberty Mutual Insurance
Home
Our
Company
Policy, Billing
& Advice
Types of
Insurance
Individual Card Sort Site Map
Create...
Liberty Mutual Insurance
Home
About Us
Auto
Insurance
Home
Insurance
Insurance
Basics
Insurance
Products
Claims
Center
Gro...
Liberty Mutual Insurance
Finalized navigation categories
Proprietary - Trade Secret (Competitively Sensitive Information) ...
Liberty Mutual Insurance
In summary, the combination of individual card sorts,
group card sorts, and focus groups was succ...
Liberty Mutual Insurance
In conclusion, two years later, the site navigation has
had a positive effect on the business
 K...
Proprietary - Trade Secret (Competitively Sensitive Information) 16
Upcoming SlideShare
Loading in …5
×

Merging Methodologies: Individual and Group Card Sorting

853 views

Published on

This presentation is a case study describing how we combined individual card sorts with focus groups and group card sorting to improve the content hierarchy and organization of www.libertymutual.com, the personal insurance website of Liberty Mutual, which customers can visit to get an insurance quote, service their insurance policies, or find insurance-related information. We analyzed quantitative and qualitative data from 26 participants, on which we based our recommendations for a new hierarchy and site structure. Our paper will show how the results from the individual and group sorts differed, how the individual exercise informed the group exercise, and how the group exercise informed the recommendations. We believe this combination of individual sorting, group sorting, and focus group discussion makes this methodology unique.

The paper on which this presentation is based is available here: http://bit.ly/14jWa7G

Published in: Economy & Finance, Business
  • Be the first to comment

  • Be the first to like this

Merging Methodologies: Individual and Group Card Sorting

  1. 1. Merging Methodologies: Combining Individual and Group Card Sorting Robert L. Thomas and Ian Johnson Liberty Mutual, Personal Insurance, Boston, MA [robertl.thomas, ian.johnson]@liberty.mutual.com Personal Insurance
  2. 2. Liberty Mutual Insurance Our old-style navigation was no longer useful for a major insurance company’s website  History o www.liberymutual.com had moved over the years to a large site with multiple lines of business  Issues o Display of a total of only nine options in the top right corner of each page o Lack of a traditional navigation bar Proprietary - Trade Secret (Competitively Sensitive Information) 2
  3. 3. Liberty Mutual Insurance We needed to make content discoverable and easy to find  Scope o Create a new navigational hierarchy to replace the current one  High-Level Objectives o Focus on the navigation and organization of the website o Ensure the navigation labels are understandable o Make the site attractive to customers Proprietary - Trade Secret (Competitively Sensitive Information) 3
  4. 4. Liberty Mutual Insurance To meet the needs of the project, we needed to run a card sort, a categorization exercise where users group physical or virtual cards together • “Card sorting is excellent for situations where you want the users’ mental model to drive the information architecture of the product.” - Courage & Baxter (2005) • “Open sorts are used for discovery. Closed sorts are used for validation.” - Morville & Rosenfeld (2006) Proprietary - Trade Secret (Competitively Sensitive Information) 4
  5. 5. Liberty Mutual Insurance We reviewed research on card sorting methodologies and then developed a new methodology Proprietary - Trade Secret (Competitively Sensitive Information) • Repeated card sorting “Repeated sorting of a set of items to understand underlying dimensions or characteristics of a product or service” - Curran, Rugg, Corr (2005) • Delphi and modified Delphi card sorting “Participants build on the results from previous card sorts” - Paul (2008) • Focus group card sorting “Participants… complete the card sort individually. Then, [you] lead a discussion with [them] regarding the… strategies they used….” - Hawley (2008) 5 We developed a new methodology, one that had not been done before. Simple to run Requires only 10-20 people Useful for understanding emotional dimensions of a product Not useful for grouping information into different categories Reduces time Requires only 8-10 people, but… Subsequent participants are modifying a card sort others started UX research experts recommend 15-30 participants for a card sort Collect quantitative card sort data Collect qualitative insights Focus group discussion may be biased by group think Be careful about running a card sort with only 1 group of 8-15 participants
  6. 6. Liberty Mutual Insurance Our methodology combined online card sorting, physical card sorting, and focus group discussion with 26 participants split between 2 sessions Proprietary - Trade Secret (Competitively Sensitive Information) 6 Source: http://www.flickr.com/photos/cflc/2098395894 (far left); Schlesinger Associates, New Jersey (far right) • 13 laptops & 13 participants • 79 card sort labels • WebSort online card sort tool • 45-minute session • 15-minute break • 3 groups of 4-5 participants • 79 labels on Post-It’s • 1 moderator & 1 note-taker • Group spokesperson presented • 40-minute session • 1 moderator & 5 note-takers • Discussion topics • 20-minute session Individual, Online, Un-moderated, Open Card Sort Team-based, Physical, Moderated, Open Card Sort Focus Group Discussion
  7. 7. Liberty Mutual Insurance Breaking into teams, we followed a four-step analysis process to arrive at our final navigation categories Proprietary - Trade Secret (Competitively Sensitive Information) 7 Group Card Sort Team Individual Card Sort Team 1 Standardized Labels & Combined Categories 2 Compared Grouping of Labels 3 Created & Compared Site Maps 4 Finalized Navigation Categories Source: Illustrations by Jeffery Callender; Template by Livia Labate
  8. 8. Liberty Mutual Insurance Standardized labels and combined categories Proprietary - Trade Secret (Competitively Sensitive Information) 8 Sorter Original category Standardized category Sort20 About About the Company Sort7 About Our Company About the Company Sort17 About the Company About the Company Sort23 About the Company About the Company Sort19 About Us About the Company Sort21 About Us About the Company Sort8 ABOUT US About the Company Sort9 about us About the Company Sort22 company About the Company Sort1 Company and website info About the Company Sort10 Company Info About the Company Sort2 Company Info About the Company Sort24 company info About the Company Sort25 Company info/history About the Company 1 2 3 4 Table 1. Standardized Categories for Individual Card Sorts
  9. 9. Liberty Mutual Insurance Table 2. Final list of 11 in-person, group standardized categories Proprietary - Trade Secret (Competitively Sensitive Information) 9 Sorters Who Total Cards in Unique Standardized Category Used This This Category Cards Agreement Standardized labels and combined categories 1 2 3 4 Average of 13 cards (40÷3) in this category
  10. 10. Liberty Mutual Insurance Group Card Sort Results Dendrogram Proprietary - Trade Secret (Competitively Sensitive Information) 10 Compared grouping of labels 1 2 3 4 Information Tasks
  11. 11. Liberty Mutual Insurance Home Our Company Policy, Billing & Advice Types of Insurance Individual Card Sort Site Map Created and compared site maps 1 2 3 4
  12. 12. Liberty Mutual Insurance Home About Us Auto Insurance Home Insurance Insurance Basics Insurance Products Claims Center Group Card Sort Site Map Created and compared site maps 1 2 3 4
  13. 13. Liberty Mutual Insurance Finalized navigation categories Proprietary - Trade Secret (Competitively Sensitive Information) 13 1 2 3 4
  14. 14. Liberty Mutual Insurance In summary, the combination of individual card sorts, group card sorts, and focus groups was successful  Our new methodology allowed us to: o Obtain qualitative and quantitative data o Recruit a large sample size o Save money and time  The benefit of this unique method is that the individual card sort activity enriched the group card sort activity and focus group discussion Proprietary - Trade Secret (Competitively Sensitive Information) 14
  15. 15. Liberty Mutual Insurance In conclusion, two years later, the site navigation has had a positive effect on the business  Key performance indicators and web analytics demonstrate that customers are actively using the new navigation  In multiple usability tests since the navigation update, users have commented positively on the navigation, rating it as one of the best attributes of the site  J.D. Power and Associates stated that “Liberty Mutual provide[s] menus that [allow] shoppers to easily find relevant shopping tools on any page” Proprietary - Trade Secret (Competitively Sensitive Information) 15
  16. 16. Proprietary - Trade Secret (Competitively Sensitive Information) 16

×