1. The Science of Great Site Navigation:
Online Card Sorting + Tree Testing
Jeff Sauro | Measuring Usability LLC
@userzoom #uzwebinar

4. CARDS AS PSYCHOLOGICAL APPARATUS

5. Traditional Card Sort

6. Online Card Sorting

7. Card Sorting is as common as Lab based
Usability Testing
Source: 2011 UxPA Salary Survey

8. Terms & Concepts
• Open Sort: Users sort items into groups and give the
groups a name.
Closed Sort: Users sort items into previously defined
category names.
• Reverse Card Sort (Tree Test) : Users are asked to
locate items in a hierarchy (no design)
• Most Users Start Browsing vs Searching: Across 9
websites and 25 tasks we found on average 86% start
browsing
http://www.measuringusability.com/blog/card-sorting.php
http://www.measuringusability.com/blog/search-browse.php

9. Target.com Case Study

11. Method & FAQs
• We want to improve the Target.com navigation.
– Obtain a baseline set of metrics and problem spots
– Make improvements
– Compare against baseline
• Pick a variety of items to test the taxonomy
• Conduct an Open Sort then Tree-Test
• Use Product Names with some level of context

12. 40 Items Across Departments
1. Presto GranPappy Electric Deep Fryer
2. Black CleanAirBall Air Purifier
3. Nano Tech Shaver
4. Hamilton Beach FashionFirst Medium Iron
5. Brother International Sewing Machine-
XR9500PRW
6. T3 White Micro Featherweight Hair Dryer
7. Signature Sleep Silhouette 10” Queen
8. Thomas O’Brien Canterbury Blue Bath Rug
9. Fieldcrest Luxury Towel Collection
10. Privet House Sugar Sack Pillow
11. P Dog ZIP Lead Retractable Blue Leash
12. Kitchen Aid 3pc BBQ Set
13. Women’s Mossimo Kasey Boot
14. Women’s Mossimo Pearce Black Leather
Pumps
15. Men’s Brown Swiss Gear Jackson Hiker
16. Totes Clear Bubble Umbrella
17. Merona Yellow Scarf Tote
18. Wildkin Kaleidoscope Backpack
19. NBA Golden State Warriors Blue Water Bottle
20. Grey Stripe Merona Men’s Short-Sleeve Polo
21. Corellian Freighter Men’s Graphic Tee
22. Inset Mother of Pearl Stainless Steel Cuff Links
23. Xhilaration Black Large Sunglasses
24. Wrangler Mens Reversible Belt
25. Cherokee Boys Drawstring Cargo Short
26. Girls Danshuz Pink Short-Sleeve Sparkle Straight
Dress
27. Tiddliwinks Blue Dot Plush Monkey
28. Black Griffin Case for Apple iPod touch
29. Marvel POWER ATTACK AVENGER: IRONMAN
30. Titanic (2 Discs) (Blu-ray/DVD)
31. PC Diablo 3 Computer Game
32. PC Treasures FlipIt 10” Tablet Sleeve
33. Commodore Big And Tall Executive Oversize Chair
34. Classic Borders Wedding Invitation and Response
Card
35. Hello Kitty Party Favors 32-pc.
36. Women’s Obsession by Calvin Klein Eau de
Toilette
37. C9 by Champion Women’s Molded Cup Racer
Back Sports Bra
38. Coffee Shop Outerwear Women’s Plus-Size Spring
Twill Coat
39. Women’s One Shoulder Shantung Dress
40. $5-$1,000 Iconic Puppy GiftCard

13. Overview
• 50 Participants
– 65% Female
– 2 Excluded for Quality Control (Haphazard Sorts)
• Sort 40 Items into 10 Fixed Categories
• Median Study Time: 11 Minutes
• Output
– Dendrogram of Clusters
– Open Category Names
– Most Difficult to Sort

14. 12 Clusters
1. Men’s / Boys
2. Women’s / Girls
3. Accessories
4. Runt: NBA
5. Kitchenware
6. Bed + Bath
7. Runt : Furniture
8. Toys
9. Electronics
10. Runt: Shaver
11. Misc/Pets: Gift Card & Leash
12. Misc: Party Favors & Invites

15. Dendrogram

16. Most Popular Category Labels
Similar Category Names Combined
Percentages are % of 48 participants who labeled a category, so % adds up to more than 100%
Error Bars are 90% Confidence Intervals

17. Most Difficult Items to Categorize
Percentages are % of 48 participants who selected item, so % add up to more than 100. Error Bars are 90% Confidence Intervals

18. Dendrogram Close Up

19. Item 1st % 2nd % 3rd % 4th % 5th %
Black Griffin Case for Apple
iPod touch Electronics 65 Entertainment 15 Accessories 10 Misc 4 Gifts 2
PC Diablo 3 Computer Game Electronics 63 Entertainment 29 Misc 4 Other 2 Gifts 2
PC Treasures FlipIt 10”Tablet
Sleeve Electronics 58 Entertainment 13 Misc 10 Accessories 10 Gifts 2
Grey Stripe Merona Men’s
Short-Sleeve Polo Men 56 Clothing 40 Boys 2 Accessories 2
Corellian Freighter Men’s
Graphic Tee Men 56 Clothing 38 Sporting Goods 2 Accessories 2 Boys 2
C9 by Champion Women’s
Molded Cup Racer Back
Sports Bra Womens 54 Clothing 38 Girls 4 Sporting Goods 2 Accessories 2
Coffee Shop Outerwear
Womens Plus-Size Spring
Twill Coat Womens 54 Clothing 38 Girls 4 Homeware 2 Accessories 2
Black CleanAirBall Air Purifier Homeware 52 Electronics 13 Misc 8 Appliances 8 Bed & bath 2
Women’s Mossimo Kasey
Boot Womens 52 Clothing 27 Shoes 15 Girls 4 Accessories 2
Women’s Mossimo Pearce
Black Leather Pumps Womens 52 Clothing 27 Shoes 13 Accessories 4 Girls 4
Women’s One Shoulder
Shantung Dress Womens 52 Clothing 40 Girls 4 retail 2 Accessories 2
P Dog ZIP Lead Retractable
Blue Leash Pets 50 Misc 15 Accessories 10 Outdoor 8 Homeware 6
Titanic (2 Discs) (Blu-
ray/DVD) Electronics 50 Entertainment 40 Misc 4 Womens 2 Gifts 2
Men’s Brown Swiss Gear
Jackson Hiker Men 48 Clothing 35 Accessories 6 Shoes 4 Outdoor 4
Hamilton Beach FashionFirst
Medium Iron Homeware 46 Electronics 15 Appliances 10 Misc 8 Womens 6
Brother International Sewing
Machine-XR9500PRW Homeware 46 Electronics 19 Appliances 10 Misc 10 Womens 4
Xhilaration Black Large
Sunglasses Accessories 46 Womens 25 Clothing 10 Men 6 Misc 6
Wrangler Mens Reversible
Belt Men 46 Accessories 29 Clothing 19 Womens 2 Boys 2
Totes Clear Bubble Umbrella Accessories 42 Homeware 15 Womens 13 Misc 10 Outdoor 4
Items with > 40% of Votes as Primary Category

20. Item 1st % 2nd % 3rd % 4th % 5th %
Signature Sleep Silhouette
Queen Homeware 40 Bedroom 17 Bed & Bath 10 Furniture 10 Accessories 4
Privet House Sugar Sack
Pillow Homeware 40 Bedroom 17 Bed & Bath 13 Furniture 6 Misc 4
Cherokee Boys Drawstring
Cargo Short Clothing 40 Men 31 Kids 17 Boys 8 Homeware 2
Girls Danshuz Pink Short-
Sleeve Sparkle Straight Dress Clothing 40 Womens 35 Kids 15 Girls 8 Accessories 2
Nano Tech Shaver Electronics 38 Bath 21 Homeware 13 Men 10 Misc 4
Thomas O’Brien Canterbury
Blue Bath Rug Homeware 38 Bath 25 Bed & Bath 13 Misc 6 Bed bath 4
Kitchen Aid 3pc BBQ Set Homeware 38 Kitchen 33 Appliances 10 Outdoor 4 Misc 4
Women’s Obsession by Calvin
Klein Eau de Toilette Womens 38 Bath 17 fragrance 10 Accessories 10 Clothing 6
$5-$1,000 Iconic Puppy
GiftCard Pets 38 Misc 15 Gifts 13 Cards/Party 8 Accessories 4
Presto Gran Pappy Electric
Deep Fryer Homeware 35 Kitchen 31 Electronics 13 Appliances 10 Misc 6
Merona Yellow Scarf Tote Womens 35 Accessories 29 Clothing 21 Homeware 6 Luggage 4
Commodore Big And Tall
Executive Oversize Chair Homeware 35 Furniture 31 Misc 10 Office 6 Other 4
T3 White Micro Featherweight
Hair Dryer Homeware 33 Bath 17 Electronics 15 Misc 10 Appliances 6
Fieldcrest Luxury Towel
Collection Homeware 33 Bath 31 Bed & Bath 13 Bedroom 4 Misc 4
Inset Mother of Pearl Stainless
Steel Cuff Links Accessories 33 Men 31 Womens 15 Clothing 13 Jewelry 6
Classic Borders Wedding
Invitation and Response Card Cards/Party 33 Misc 17 Accessories 13 Gifts 6 Womens 6
Wildkin Kaleidoscope
Backpack Accessories 31 Kids 19 Misc 10 Clothing 10 Outdoor 8
Hello Kitty Party Favors 32-pc. Cards/Party 29 Misc 17 Kids 15 Gifts 6 Accessories 6
Marvel POWER ATTACK
AVENGER: IRONMAN Entertainment 27 Toys 19 Electronics 19 Kids 13 Misc 10
Tiddliwinks Blue Dot Plush
Monkey Toys 23 Kids 23 Misc 19 Homeware 4 Accessories 4
NBA Golden State Warriors
Blue Water Bottle Misc 21 Accessories 21 Sporting Goods 13 Kitchen 10 Men 10
Items with < 40% of Votes as Primary Category

21. Tree Test

22. Selected 14 Items From Open Sort
1. Wrangler Mens Reversible Belt
2. Marvel POWER ATTACK AVENGER: IRONMAN
3. Women’s One Shoulder Shantung Dress
4. Grey Stripe Merona Men’s Short-Sleeve Polo
5. Inset Mother of Pearl Stainless Steel Cuff Links
6. Women’s Obsession by Calvin Klein Eau de Toilette
7. Tiddliwinks Blue Dot Plush Monkey
8. Hello Kitty Party Favors 32-pc.
9. Nano Tech Shaver
10. Brother International Sewing Machine-XR9500PRW
11. Classic Borders Wedding Invitation and Response Card
12. Kitchen Aid 3pc BBQ Set
13. Men’s Brown Swiss Gear Jackson Hiker
14. Wildkin Kaleidoscope Backpack

23. Overview
• 50 Participants
• 68% Female
• Median Study Time 17 Minutes
• Task Based Metrics
• Confidence
• Difficulty (SEQ)
• Success
• Time

24. Post Task Questions in Userzoom
Single Ease Question: SEQ

25. Tree Testing in Userzoom (User’s View)

26. Successfully Located Item

27. Hardest Items to Locate

28. Brother Sewing Machine 64% -The sewing machine didn’t have a separate section that I could find
- it could fit into a lot of categories and it made it hard to nail down
Wildkin Kalidescope Backpack 38% - Not sure if it’s a child’s toy or real camping gear
- Expected to find it in the children’s section
Hello Kitty Party Favors 36% - I didn’t find a section for things like Cards or Party Favors Could be for
both men and women
- not apparent under what main heading it would be under.
Men’s Brown Swiss Gear Jackson Hiker 34% - Did not know what this was, a boot? A Jacket?
Classis Border Wedding Invitations 39% - Was expecting to find a stationary or cards section
Twiddledinks Blue Dot Plush Monkey 12% - Did not know if this was a toy
- No Stuffed animal section
Kitchenaid 3-pc BBQ set 10% -Did not know if it was Kitchen or a small appliance
- Could be categorized under kitchen or outdoors
Marvel Attack Avengers: Ironman 8% - Did not know what kind of Toy it was
- List of Toys was too long, it was hard to eliminate
Nano Tech Shaver 6% - Is it for a man or a women?
- Did not know if it was electronics or another category
Women’s Obsession by Calvin Klein 6% - Did not know which category to put it under
Mother of Pearl Stainless Steele Cuff links 2% - Could have fit into more than one category
Item % that found it difficult Reason it was difficult
Verbatim Reasons For Difficulty

29. How difficult or easy was it to complete the task
Very EasyVery Difficult
Percentages are normalized percentile ranks from across 200 tasks. 50% = Average.

30. Confidence
Not at all
Confident
Extremely
Confident

31. Confidence & Success
Disasters
Difficult
Deceptive

32. Hardest Items to Categorize – Tree Test Vs Open Sort
Difficulty sorting explains
about 16% of difficulty
finding.

33. Sample Size Considerations
Margin of Error
90% Confidence (+/-) Sample Size
24% 10
20% 15
15% 28
14% 32
13% 38
12% 45
11% 54
10% 65
9% 81
8% 103
7% 136
6% 186
5% 268
4% 421
3% 749
See Chapter 6 in Quantifying the User Experience

34. Conclusions
• Determine if new categories should be created
or consolidated
• Look to confirm or deny hypotheses about
categories and findability
– Gender Searching
– Sub Categories
• Watch for “false” sorts
• If “key” items are hard to find, move them,
test them
• It’s hard to do a Card Sort without a Tree Test

35. Find out More
MeasuringU.com
jeff@measuringu.com