The Science of Great Site Navigation: Online Card Sorting + Tree Testing

The Science of Great Site Navigation:Online Card Sorting + Tree TestingJeff Sauro | Measuring Usability LLC7/12/2012@MsrUsability@userzoom #uzwebinar

CARDS AS PSYCHOLOGICAL APPARATUS

Traditional Card Sort

Online Card Sorting

Card Sorting is as common as Lab basedUsability TestingSource: 2011 UxPA Salary Survey

Terms & Concepts• Open Sort: Users sort items into groups and give the groups a name. Closed Sort: Users sort items into previously defined category names.• Reverse Card Sort (Tree Test) : Users are asked to locate items in a hierarchy (no design)• Most Users Start Browsing vs Searching: Across 9 websites and 25 tasks we found on average 86% start browsing http://www.measuringusability.com/blog/card-sorting.php http://www.measuringusability.com/blog/search-browse.php

Target.com Case Study

Method & FAQs• We want to improve the Target.com navigation. – Obtain a baseline set of metrics and problem spots – Make improvements – Compare against baseline• Pick a variety of items to test the taxonomy• Conduct an Open Sort then Tree-Test• Use Product Names with some level of context

40 Items Across Departments1. Presto GranPappy Electric Deep Fryer 21. Corellian Freighter Men’s Graphic Tee2. Black CleanAirBall Air Purifier 22. Inset Mother of Pearl Stainless Steel Cuff Links3. Nano Tech Shaver 23. Xhilaration Black Large Sunglasses4. Hamilton Beach FashionFirst Medium Iron 24. Wrangler Mens Reversible Belt5. Brother International Sewing Machine- 25. Cherokee Boys Drawstring Cargo Short XR9500PRW 26. Girls Danshuz Pink Short-Sleeve Sparkle Straight6. T3 White Micro Featherweight Hair Dryer Dress7. Signature Sleep Silhouette 10” Queen 27. Tiddliwinks Blue Dot Plush Monkey8. Thomas O’Brien Canterbury Blue Bath Rug 28. Black Griffin Case for Apple iPod touch9. Fieldcrest Luxury Towel Collection 29. Marvel POWER ATTACK AVENGER: IRONMAN10. Privet House Sugar Sack Pillow 30. Titanic (2 Discs) (Blu-ray/DVD)11. P Dog ZIP Lead Retractable Blue Leash 31. PC Diablo 3 Computer Game12. Kitchen Aid 3pc BBQ Set 32. PC Treasures FlipIt 10” Tablet Sleeve13. Women’s Mossimo Kasey Boot 33. Commodore Big And Tall Executive Oversize Chair14. Women’s Mossimo Pearce Black Leather 34. Classic Borders Wedding Invitation and Response Pumps Card15. Men’s Brown Swiss Gear Jackson Hiker 35. Hello Kitty Party Favors 32-pc.16. Totes Clear Bubble Umbrella 36. Women’s Obsession by Calvin Klein Eau de17. Merona Yellow Scarf Tote Toilette18. Wildkin Kaleidoscope Backpack 37. C9 by Champion Women’s Molded Cup Racer19. NBA Golden State Warriors Blue Water Bottle Back Sports Bra20. Grey Stripe Merona Men’s Short-Sleeve Polo 38. Coffee Shop Outerwear Women’s Plus-Size Spring Twill Coat 39. Women’s One Shoulder Shantung Dress 40. $5-$1,000 Iconic Puppy GiftCard

Overview• 50 Participants – 65% Female – 2 Excluded for Quality Control (Haphazard Sorts)• Sort 40 Items into 10 Fixed Categories• Median Study Time: 11 Minutes• Output – Dendrogram of Clusters – Open Category Names – Most Difficult to Sort

12 Clusters1. Men’s / Boys2. Women’s / Girls3. Accessories4. Runt: NBA5. Kitchenware6. Bed + Bath7. Runt : Furniture8. Toys9. Electronics10. Runt: Shaver11. Misc/Pets: Gift Card & Leash12. Misc: Party Favors & Invites

Dendrogram

Most Popular Category LabelsSimilar Category Names CombinedPercentages are % of 48 participants who labeled a category, so % adds up to more than 100%Error Bars are 90% Confidence Intervals

Most Difficult Items to CategorizePercentages are % of 48 participants who selected item, so % add up to more than 100. Error Bars are 90% Confidence Intervals

Dendrogram Close Up

Items with > 40% of Votes as Primary Category Item 1st % 2nd % 3rd % 4th % 5th %Black Griffin Case for AppleiPod touch Electronics 65 Entertainment 15 Accessories 10 Misc 4 Gifts 2PC Diablo 3 Computer Game Electronics 63 Entertainment 29 Misc 4 Other 2 Gifts 2PC Treasures FlipIt 10”TabletSleeve Electronics 58 Entertainment 13 Misc 10 Accessories 10 Gifts 2Grey Stripe Merona Men’sShort-Sleeve Polo Men 56 Clothing 40 Boys 2 Accessories 2Corellian Freighter Men’sGraphic Tee Men 56 Clothing 38 Sporting Goods 2 Accessories 2 Boys 2C9 by Champion Women’sMolded Cup Racer BackSports Bra Womens 54 Clothing 38 Girls 4 Sporting Goods 2 Accessories 2Coffee Shop OuterwearWomens Plus-Size SpringTwill Coat Womens 54 Clothing 38 Girls 4 Homeware 2 Accessories 2Black CleanAirBall Air Purifier Homeware 52 Electronics 13 Misc 8 Appliances 8 Bed & bath 2Women’s Mossimo KaseyBoot Womens 52 Clothing 27 Shoes 15 Girls 4 Accessories 2Women’s Mossimo PearceBlack Leather Pumps Womens 52 Clothing 27 Shoes 13 Accessories 4 Girls 4Women’s One ShoulderShantung Dress Womens 52 Clothing 40 Girls 4 retail 2 Accessories 2P Dog ZIP Lead RetractableBlue Leash Pets 50 Misc 15 Accessories 10 Outdoor 8 Homeware 6Titanic (2 Discs) (Blu-ray/DVD) Electronics 50 Entertainment 40 Misc 4 Womens 2 Gifts 2Men’s Brown Swiss GearJackson Hiker Men 48 Clothing 35 Accessories 6 Shoes 4 Outdoor 4Hamilton Beach FashionFirstMedium Iron Homeware 46 Electronics 15 Appliances 10 Misc 8 Womens 6Brother International SewingMachine-XR9500PRW Homeware 46 Electronics 19 Appliances 10 Misc 10 Womens 4Xhilaration Black LargeSunglasses Accessories 46 Womens 25 Clothing 10 Men 6 Misc 6Wrangler Mens ReversibleBelt Men 46 Accessories 29 Clothing 19 Womens 2 Boys 2Totes Clear Bubble Umbrella Accessories 42 Homeware 15 Womens 13 Misc 10 Outdoor 4

Items with < 40% of Votes as Primary Category Item 1st % 2nd % 3rd % 4th % 5th %Signature Sleep SilhouetteQueen Homeware 40 Bedroom 17 Bed & Bath 10 Furniture 10 Accessories 4Privet House Sugar SackPillow Homeware 40 Bedroom 17 Bed & Bath 13 Furniture 6 Misc 4Cherokee Boys DrawstringCargo Short Clothing 40 Men 31 Kids 17 Boys 8 Homeware 2Girls Danshuz Pink Short-Sleeve Sparkle Straight Dress Clothing 40 Womens 35 Kids 15 Girls 8 Accessories 2Nano Tech Shaver Electronics 38 Bath 21 Homeware 13 Men 10 Misc 4Thomas O’Brien CanterburyBlue Bath Rug Homeware 38 Bath 25 Bed & Bath 13 Misc 6 Bed bath 4Kitchen Aid 3pc BBQ Set Homeware 38 Kitchen 33 Appliances 10 Outdoor 4 Misc 4Women’s Obsession by CalvinKlein Eau de Toilette Womens 38 Bath 17 fragrance 10 Accessories 10 Clothing 6$5-$1,000 Iconic PuppyGiftCard Pets 38 Misc 15 Gifts 13 Cards/Party 8 Accessories 4Presto Gran Pappy ElectricDeep Fryer Homeware 35 Kitchen 31 Electronics 13 Appliances 10 Misc 6Merona Yellow Scarf Tote Womens 35 Accessories 29 Clothing 21 Homeware 6 Luggage 4Commodore Big And TallExecutive Oversize Chair Homeware 35 Furniture 31 Misc 10 Office 6 Other 4T3 White Micro FeatherweightHair Dryer Homeware 33 Bath 17 Electronics 15 Misc 10 Appliances 6Fieldcrest Luxury TowelCollection Homeware 33 Bath 31 Bed & Bath 13 Bedroom 4 Misc 4Inset Mother of Pearl StainlessSteel Cuff Links Accessories 33 Men 31 Womens 15 Clothing 13 Jewelry 6Classic Borders WeddingInvitation and Response Card Cards/Party 33 Misc 17 Accessories 13 Gifts 6 Womens 6Wildkin KaleidoscopeBackpack Accessories 31 Kids 19 Misc 10 Clothing 10 Outdoor 8Hello Kitty Party Favors 32-pc. Cards/Party 29 Misc 17 Kids 15 Gifts 6 Accessories 6Marvel POWER ATTACKAVENGER: IRONMAN Entertainment 27 Toys 19 Electronics 19 Kids 13 Misc 10Tiddliwinks Blue Dot PlushMonkey Toys 23 Kids 23 Misc 19 Homeware 4 Accessories 4NBA Golden State WarriorsBlue Water Bottle Misc 21 Accessories 21 Sporting Goods 13 Kitchen 10 Men 10

Tree Test

Selected 14 Items From Open Sort1. Wrangler Mens Reversible Belt2. Marvel POWER ATTACK AVENGER: IRONMAN3. Women’s One Shoulder Shantung Dress4. Grey Stripe Merona Men’s Short-Sleeve Polo5. Inset Mother of Pearl Stainless Steel Cuff Links6. Women’s Obsession by Calvin Klein Eau de Toilette7. Tiddliwinks Blue Dot Plush Monkey8. Hello Kitty Party Favors 32-pc.9. Nano Tech Shaver10. Brother International Sewing Machine-XR9500PRW11. Classic Borders Wedding Invitation and Response Card12. Kitchen Aid 3pc BBQ Set13. Men’s Brown Swiss Gear Jackson Hiker14. Wildkin Kaleidoscope Backpack

Overview• 50 Participants • 68% Female• Median Study Time 17 Minutes• Task Based Metrics • Confidence • Difficulty (SEQ) • Success • Time

Post Task Questions in Userzoom Single Ease Question: SEQ

Tree Testing in Userzoom (User’s View)

Successfully Located Item

Hardest Items to Locate

Verbatim Reasons For Difficulty Item % that found it difficult Reason it was difficultBrother Sewing Machine 64% -The sewing machine didn’t have a separate section that I could find - it could fit into a lot of categories and it made it hard to nail downWildkin Kalidescope Backpack 38% - Not sure if it’s a child’s toy or real camping gear - Expected to find it in the children’s sectionHello Kitty Party Favors 36% - I didn’t find a section for things like Cards or Party Favors Could be for both men and women - not apparent under what main heading it would be under.Men’s Brown Swiss Gear Jackson Hiker 34% - Did not know what this was, a boot? A Jacket?Classis Border Wedding Invitations 39% - Was expecting to find a stationary or cards sectionTwiddledinks Blue Dot Plush Monkey 12% - Did not know if this was a toy - No Stuffed animal sectionKitchenaid 3-pc BBQ set 10% -Did not know if it was Kitchen or a small appliance - Could be categorized under kitchen or outdoorsMarvel Attack Avengers: Ironman 8% - Did not know what kind of Toy it was - List of Toys was too long, it was hard to eliminateNano Tech Shaver 6% - Is it for a man or a women? - Did not know if it was electronics or another categoryWomen’s Obsession by Calvin Klein 6% - Did not know which category to put it underMother of Pearl Stainless Steele Cuff links 2% - Could have fit into more than one category

How difficult or easy was it to complete the task Very Difficult Very EasyPercentages are normalized percentile ranks from across 200 tasks. 50% = Average.

Confidence Not at all Extremely Confident Confident

Confidence & Success Deceptive Difficult Disasters

Hardest Items to Categorize – Tree Test Vs Open Sort Difficulty sorting explains about 16% of difficulty finding.

Sample Size Considerations Margin of Error 90% Confidence (+/-) Sample Size 24% 10 20% 15 15% 28 14% 32 13% 38 12% 45 11% 54 10% 65 9% 81 8% 103 7% 136 6% 186 5% 268 4% 421 3% 749See Chapter 6 in Quantifying the User Experience

Conclusions• Determine if new categories should be created or consolidated• Look to confirm or deny hypotheses about categories and findability – Gender Searching – Sub Categories• Watch for “false” sorts• If “key” items are hard to find, move them, test them• It’s hard to do a Card Sort without a Tree Test

Find out More www.LeanUXDenver.com Sept 19-21st 2012 Hyatt Convention Center Denver, CO @LeanUXDenver MeasuringUsability.com jeff@measuringusability.com Twitter @MsrUsability

http://info.userzoom.com	244
https://na-b.marketodesigner.com	15
https://twitter.com	1

The Science of Great Site Navigation: Online Card Sorting + Tree Testing

by UserZoom on Jul 12, 2012

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 260

Statistics