Successfully reported this slideshow.

The Science of Great Site Navigation: Online Card Sorting + Tree Testing

2,962 views

Published on

  • Be the first to comment

The Science of Great Site Navigation: Online Card Sorting + Tree Testing

  1. 1. The Science of Great Site Navigation: Online Card Sorting + Tree Testing Jeff Sauro | Measuring Usability LLC @userzoom #uzwebinar
  2. 2. CARDS AS PSYCHOLOGICAL APPARATUS
  3. 3. Traditional Card Sort
  4. 4. Online Card Sorting
  5. 5. Card Sorting is as common as Lab based Usability Testing Source: 2011 UxPA Salary Survey
  6. 6. Terms & Concepts • Open Sort: Users sort items into groups and give the groups a name. Closed Sort: Users sort items into previously defined category names. • Reverse Card Sort (Tree Test) : Users are asked to locate items in a hierarchy (no design) • Most Users Start Browsing vs Searching: Across 9 websites and 25 tasks we found on average 86% start browsing http://www.measuringusability.com/blog/card-sorting.php http://www.measuringusability.com/blog/search-browse.php
  7. 7. Target.com Case Study
  8. 8. Method & FAQs • We want to improve the Target.com navigation. – Obtain a baseline set of metrics and problem spots – Make improvements – Compare against baseline • Pick a variety of items to test the taxonomy • Conduct an Open Sort then Tree-Test • Use Product Names with some level of context
  9. 9. 40 Items Across Departments 1. Presto GranPappy Electric Deep Fryer 2. Black CleanAirBall Air Purifier 3. Nano Tech Shaver 4. Hamilton Beach FashionFirst Medium Iron 5. Brother International Sewing Machine- XR9500PRW 6. T3 White Micro Featherweight Hair Dryer 7. Signature Sleep Silhouette 10” Queen 8. Thomas O’Brien Canterbury Blue Bath Rug 9. Fieldcrest Luxury Towel Collection 10. Privet House Sugar Sack Pillow 11. P Dog ZIP Lead Retractable Blue Leash 12. Kitchen Aid 3pc BBQ Set 13. Women’s Mossimo Kasey Boot 14. Women’s Mossimo Pearce Black Leather Pumps 15. Men’s Brown Swiss Gear Jackson Hiker 16. Totes Clear Bubble Umbrella 17. Merona Yellow Scarf Tote 18. Wildkin Kaleidoscope Backpack 19. NBA Golden State Warriors Blue Water Bottle 20. Grey Stripe Merona Men’s Short-Sleeve Polo 21. Corellian Freighter Men’s Graphic Tee 22. Inset Mother of Pearl Stainless Steel Cuff Links 23. Xhilaration Black Large Sunglasses 24. Wrangler Mens Reversible Belt 25. Cherokee Boys Drawstring Cargo Short 26. Girls Danshuz Pink Short-Sleeve Sparkle Straight Dress 27. Tiddliwinks Blue Dot Plush Monkey 28. Black Griffin Case for Apple iPod touch 29. Marvel POWER ATTACK AVENGER: IRONMAN 30. Titanic (2 Discs) (Blu-ray/DVD) 31. PC Diablo 3 Computer Game 32. PC Treasures FlipIt 10” Tablet Sleeve 33. Commodore Big And Tall Executive Oversize Chair 34. Classic Borders Wedding Invitation and Response Card 35. Hello Kitty Party Favors 32-pc. 36. Women’s Obsession by Calvin Klein Eau de Toilette 37. C9 by Champion Women’s Molded Cup Racer Back Sports Bra 38. Coffee Shop Outerwear Women’s Plus-Size Spring Twill Coat 39. Women’s One Shoulder Shantung Dress 40. $5-$1,000 Iconic Puppy GiftCard
  10. 10. Overview • 50 Participants – 65% Female – 2 Excluded for Quality Control (Haphazard Sorts) • Sort 40 Items into 10 Fixed Categories • Median Study Time: 11 Minutes • Output – Dendrogram of Clusters – Open Category Names – Most Difficult to Sort
  11. 11. 12 Clusters 1. Men’s / Boys 2. Women’s / Girls 3. Accessories 4. Runt: NBA 5. Kitchenware 6. Bed + Bath 7. Runt : Furniture 8. Toys 9. Electronics 10. Runt: Shaver 11. Misc/Pets: Gift Card & Leash 12. Misc: Party Favors & Invites
  12. 12. Dendrogram
  13. 13. Most Popular Category Labels Similar Category Names Combined Percentages are % of 48 participants who labeled a category, so % adds up to more than 100% Error Bars are 90% Confidence Intervals
  14. 14. Most Difficult Items to Categorize Percentages are % of 48 participants who selected item, so % add up to more than 100. Error Bars are 90% Confidence Intervals
  15. 15. Dendrogram Close Up
  16. 16. Item 1st % 2nd % 3rd % 4th % 5th % Black Griffin Case for Apple iPod touch Electronics 65 Entertainment 15 Accessories 10 Misc 4 Gifts 2 PC Diablo 3 Computer Game Electronics 63 Entertainment 29 Misc 4 Other 2 Gifts 2 PC Treasures FlipIt 10”Tablet Sleeve Electronics 58 Entertainment 13 Misc 10 Accessories 10 Gifts 2 Grey Stripe Merona Men’s Short-Sleeve Polo Men 56 Clothing 40 Boys 2 Accessories 2 Corellian Freighter Men’s Graphic Tee Men 56 Clothing 38 Sporting Goods 2 Accessories 2 Boys 2 C9 by Champion Women’s Molded Cup Racer Back Sports Bra Womens 54 Clothing 38 Girls 4 Sporting Goods 2 Accessories 2 Coffee Shop Outerwear Womens Plus-Size Spring Twill Coat Womens 54 Clothing 38 Girls 4 Homeware 2 Accessories 2 Black CleanAirBall Air Purifier Homeware 52 Electronics 13 Misc 8 Appliances 8 Bed & bath 2 Women’s Mossimo Kasey Boot Womens 52 Clothing 27 Shoes 15 Girls 4 Accessories 2 Women’s Mossimo Pearce Black Leather Pumps Womens 52 Clothing 27 Shoes 13 Accessories 4 Girls 4 Women’s One Shoulder Shantung Dress Womens 52 Clothing 40 Girls 4 retail 2 Accessories 2 P Dog ZIP Lead Retractable Blue Leash Pets 50 Misc 15 Accessories 10 Outdoor 8 Homeware 6 Titanic (2 Discs) (Blu- ray/DVD) Electronics 50 Entertainment 40 Misc 4 Womens 2 Gifts 2 Men’s Brown Swiss Gear Jackson Hiker Men 48 Clothing 35 Accessories 6 Shoes 4 Outdoor 4 Hamilton Beach FashionFirst Medium Iron Homeware 46 Electronics 15 Appliances 10 Misc 8 Womens 6 Brother International Sewing Machine-XR9500PRW Homeware 46 Electronics 19 Appliances 10 Misc 10 Womens 4 Xhilaration Black Large Sunglasses Accessories 46 Womens 25 Clothing 10 Men 6 Misc 6 Wrangler Mens Reversible Belt Men 46 Accessories 29 Clothing 19 Womens 2 Boys 2 Totes Clear Bubble Umbrella Accessories 42 Homeware 15 Womens 13 Misc 10 Outdoor 4 Items with > 40% of Votes as Primary Category
  17. 17. Item 1st % 2nd % 3rd % 4th % 5th % Signature Sleep Silhouette Queen Homeware 40 Bedroom 17 Bed & Bath 10 Furniture 10 Accessories 4 Privet House Sugar Sack Pillow Homeware 40 Bedroom 17 Bed & Bath 13 Furniture 6 Misc 4 Cherokee Boys Drawstring Cargo Short Clothing 40 Men 31 Kids 17 Boys 8 Homeware 2 Girls Danshuz Pink Short- Sleeve Sparkle Straight Dress Clothing 40 Womens 35 Kids 15 Girls 8 Accessories 2 Nano Tech Shaver Electronics 38 Bath 21 Homeware 13 Men 10 Misc 4 Thomas O’Brien Canterbury Blue Bath Rug Homeware 38 Bath 25 Bed & Bath 13 Misc 6 Bed bath 4 Kitchen Aid 3pc BBQ Set Homeware 38 Kitchen 33 Appliances 10 Outdoor 4 Misc 4 Women’s Obsession by Calvin Klein Eau de Toilette Womens 38 Bath 17 fragrance 10 Accessories 10 Clothing 6 $5-$1,000 Iconic Puppy GiftCard Pets 38 Misc 15 Gifts 13 Cards/Party 8 Accessories 4 Presto Gran Pappy Electric Deep Fryer Homeware 35 Kitchen 31 Electronics 13 Appliances 10 Misc 6 Merona Yellow Scarf Tote Womens 35 Accessories 29 Clothing 21 Homeware 6 Luggage 4 Commodore Big And Tall Executive Oversize Chair Homeware 35 Furniture 31 Misc 10 Office 6 Other 4 T3 White Micro Featherweight Hair Dryer Homeware 33 Bath 17 Electronics 15 Misc 10 Appliances 6 Fieldcrest Luxury Towel Collection Homeware 33 Bath 31 Bed & Bath 13 Bedroom 4 Misc 4 Inset Mother of Pearl Stainless Steel Cuff Links Accessories 33 Men 31 Womens 15 Clothing 13 Jewelry 6 Classic Borders Wedding Invitation and Response Card Cards/Party 33 Misc 17 Accessories 13 Gifts 6 Womens 6 Wildkin Kaleidoscope Backpack Accessories 31 Kids 19 Misc 10 Clothing 10 Outdoor 8 Hello Kitty Party Favors 32-pc. Cards/Party 29 Misc 17 Kids 15 Gifts 6 Accessories 6 Marvel POWER ATTACK AVENGER: IRONMAN Entertainment 27 Toys 19 Electronics 19 Kids 13 Misc 10 Tiddliwinks Blue Dot Plush Monkey Toys 23 Kids 23 Misc 19 Homeware 4 Accessories 4 NBA Golden State Warriors Blue Water Bottle Misc 21 Accessories 21 Sporting Goods 13 Kitchen 10 Men 10 Items with < 40% of Votes as Primary Category
  18. 18. Tree Test
  19. 19. Selected 14 Items From Open Sort 1. Wrangler Mens Reversible Belt 2. Marvel POWER ATTACK AVENGER: IRONMAN 3. Women’s One Shoulder Shantung Dress 4. Grey Stripe Merona Men’s Short-Sleeve Polo 5. Inset Mother of Pearl Stainless Steel Cuff Links 6. Women’s Obsession by Calvin Klein Eau de Toilette 7. Tiddliwinks Blue Dot Plush Monkey 8. Hello Kitty Party Favors 32-pc. 9. Nano Tech Shaver 10. Brother International Sewing Machine-XR9500PRW 11. Classic Borders Wedding Invitation and Response Card 12. Kitchen Aid 3pc BBQ Set 13. Men’s Brown Swiss Gear Jackson Hiker 14. Wildkin Kaleidoscope Backpack
  20. 20. Overview • 50 Participants • 68% Female • Median Study Time 17 Minutes • Task Based Metrics • Confidence • Difficulty (SEQ) • Success • Time
  21. 21. Post Task Questions in Userzoom Single Ease Question: SEQ
  22. 22. Tree Testing in Userzoom (User’s View)
  23. 23. Successfully Located Item
  24. 24. Hardest Items to Locate
  25. 25. Brother Sewing Machine 64% -The sewing machine didn’t have a separate section that I could find - it could fit into a lot of categories and it made it hard to nail down Wildkin Kalidescope Backpack 38% - Not sure if it’s a child’s toy or real camping gear - Expected to find it in the children’s section Hello Kitty Party Favors 36% - I didn’t find a section for things like Cards or Party Favors Could be for both men and women - not apparent under what main heading it would be under. Men’s Brown Swiss Gear Jackson Hiker 34% - Did not know what this was, a boot? A Jacket? Classis Border Wedding Invitations 39% - Was expecting to find a stationary or cards section Twiddledinks Blue Dot Plush Monkey 12% - Did not know if this was a toy - No Stuffed animal section Kitchenaid 3-pc BBQ set 10% -Did not know if it was Kitchen or a small appliance - Could be categorized under kitchen or outdoors Marvel Attack Avengers: Ironman 8% - Did not know what kind of Toy it was - List of Toys was too long, it was hard to eliminate Nano Tech Shaver 6% - Is it for a man or a women? - Did not know if it was electronics or another category Women’s Obsession by Calvin Klein 6% - Did not know which category to put it under Mother of Pearl Stainless Steele Cuff links 2% - Could have fit into more than one category Item % that found it difficult Reason it was difficult Verbatim Reasons For Difficulty
  26. 26. How difficult or easy was it to complete the task Very EasyVery Difficult Percentages are normalized percentile ranks from across 200 tasks. 50% = Average.
  27. 27. Confidence Not at all Confident Extremely Confident
  28. 28. Confidence & Success Disasters Difficult Deceptive
  29. 29. Hardest Items to Categorize – Tree Test Vs Open Sort Difficulty sorting explains about 16% of difficulty finding.
  30. 30. Sample Size Considerations Margin of Error 90% Confidence (+/-) Sample Size 24% 10 20% 15 15% 28 14% 32 13% 38 12% 45 11% 54 10% 65 9% 81 8% 103 7% 136 6% 186 5% 268 4% 421 3% 749 See Chapter 6 in Quantifying the User Experience
  31. 31. Conclusions • Determine if new categories should be created or consolidated • Look to confirm or deny hypotheses about categories and findability – Gender Searching – Sub Categories • Watch for “false” sorts • If “key” items are hard to find, move them, test them • It’s hard to do a Card Sort without a Tree Test
  32. 32. Find out More MeasuringU.com jeff@measuringu.com

×