Jared Spool (Founder, UIE) - Design Metrics That Matter
•
4 likes•3,677 views
This video for this talk from Business of Software Conference USA 2018 will be published here soon: http://businessofsoftware.org/videos
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Jared Spool (Founder, UIE) - Design Metrics That Matter
Similar to Jared Spool (Founder, UIE) - Design Metrics That Matter (20)
More from Business of Software Conference
More from Business of Software Conference (20)
Recently uploaded
Recently uploaded (20)
Jared Spool (Founder, UIE) - Design Metrics That Matter
- 1. @jmspool Jared M. Spool Is Design Metrically Opposed?
- 2. @jmspool Jared M. Spool Is Design Metrically Opposed?
- 3. 1Are we measuring the right thing?
- 6. @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ 33 Email Addresses @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ 77 Email Addresses @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @
- 7. @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ 33 Email Addresses 77 Email Addresses More email addresses are better @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ All email addresses are equal
- 8. @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ More email addresses are better All email addresses are equal Inferences Observations
- 11. Design DecisionsInferencesObservations More email addresses are better 2nd variant had more email addresses Use 2nd variant
- 12. Design DecisionsInferencesObservations Why we think it happened? What did we see? How will we improve the design?
- 13. Log file filled with “ ” (blanks)
- 14. Enter button focus was broken Didn’t enter any search text Was looking for advanced search Log file is broken
- 15. Design DecisionsInferencesObservations Broken focus Log file contained blanks Fix Focus No search text Advanced search Log file bug Add label to search box Build advanced search feature Fix bug
- 16. The best designers never stop at the first inference.
- 17. Log file contained blanks Design DecisionsInferencesObservations Broken focus Fix Focus No search text Advanced search Log file bug Add label to search box Build advanced search feature Fix bug Users didn’t know to type query into box Log file contained blanks
- 20. SnapChat AirBNB Uber Facebook MarketValuation InstancesofletterEonhomepage
- 25. Counting the letter E is a stupid metric.
- 26. Measure: Metric: Analytic: Something we can count. A measure we track. A measure software tracks.
- 27. Time on page
- 28. Time on page Bounce rate % Exit Pageviews Sessions w/search Time after search % Search Refinements
- 29. Bounce rate What should we do differently?
- 30. “ R.H. Coase, British economist If you torture data long enough, it will confess to anything you’d like.”
- 31. What Google Analytics can’t tell you: ‣What content was useful? ‣What people found confusing? ‣Who is your site’s biggest spender? ‣What do big spenders do that others don’t? ‣What should you do to improve your content? ‣Why did someone click?
- 32. What Google Analytics can’t tell you: Why?
- 33. Conversion Rate
- 34. M MATHEMATICAL CONTENT CONTAINS SOME FORMULAIC MATERIAL MAY BE INAPPROPRIATE FOR DESIGNERS THIS PRESENTATIONHAS
- 35. Conversion Rate Conversion Rate # of people who purchase # of people who visited = 1.0 % 10,000 1,000,000 = Conversion Rate # of people who purchase # of people who visited =
- 36. Conversion Rate 1.0 % 10,000 1,000,000 = 2.0 % 20,000 1,000,000 = 2.0 % 10,000 500,000 = 10,000
- 37. Conversion Rate 1,000,000 = 1,000,000 = 500,000 = 10,000 = x $100 $1,000,000 20,000 x $100 $2,000,000 = = 10,000 x $100 $1,000,000 1.0 % 2.0 % 2.0 % 10,00010,000 20,00020,000 10,00010,000
- 38. Conversion Rate 20,000 1,000,000 = 10,000 500,000 = $2,000,000= = $1,000,000 20,000 10,000 2.0 % 2.0 %
- 39. What should we design for? $2,000,000 $1,000,000 2.0 % 2.0 % Conversion Rate? Money?
- 40. Conversion Rate Conversion Rate # of people who purchase # of people who visited = Visit Visit Visit Purchase Conversion Rate = 25 % ? Conversion Rate = 100 % ?
- 41. What conversion rate can’t tell you: Why?
- 44. Achieving satisfaction is too low a bar to set for our designs. We can do better.
- 49. Pro tip: Nothing says “we don’t care” like a page of 10-point satisfaction scales.
- 50. 10-point scales make noise feel like science.
- 52. What satisfaction surveys and Net Promoter Score can’t tell you: Why?
- 53. We need metrics that help us improve our users’ experience.
- 56. ☹ Thingsusersfindfrustrating Content that is confusing ☹ Incomplete information ☹ Can’t remember password ☹ Hidden features ☹ Hidden navigation ☹ Confusing navigation ☹ Error messages
- 57. Thingsthatusersfindfrustrating Phone numbers can’t have dashes or spaces. ☹ Error messages The credit card security code is required (again). Username and password do not match.
- 58. Pro tip: Customize analytics by counting error message deliveries.
- 59. Case study: Improve checkout process for a major e-commerce site.
- 60. Shippinginformation Billinginformation Paymentinformation Confirmpurchase Done! 100% 50% Pageviews
- 64. Lost revenue from account sign-in issues: $300,000,000per year The team built a guest checkout (no sign-in required). Recovered the $300,000,000 within 1 year.
- 65. Qualitative usability research $300,000,000 Quantitative custom metrics + Mostimportantcustommetric: Unrealized shopping cart value from password issues.
- 66. Qualitative findings must drive our quantitative research agenda.
- 67. The team’s initial inferences ‣The checkout steps were where we’d find the biggest improvement. ‣It was “normal” to lose customers before checkout. ‣Cart review went straight to checkout without additional steps. ‣All of the screens were instrumented.
- 71. “Analytics are controlled by a different group.” Nolongeracceptable:
- 72. Owned by the UXTeam Owned by theAnalyticsTeam Under new management: the UX Team
- 73. DJ Patil, Former US Chief Data Scientist
- 74. “Analytics are controlled by a different group.” Data science is now an essential skill for every UX team. Nolongeracceptable:
- 75. Continually question what the metric is trying to tell you. Nolongeracceptable: “I don’t understand what the metrics mean.”
- 76. Keep the numbers simple,so you can focus on the behaviors. Nolongeracceptable: “I’m not good with numbers.”
- 77. Applying targeted metrics to qualitative research: a powerful addition to the UX designer’s toolkit.
- 86. “ Robert Fabricant The medium of design is behavior”
- 88. Design must drive metric collection, not the other way around.
- 89. Is Design Metrically Opposed? ‣We need metrics that help us improve the experience. ‣Avoid jumping from observations to inference too soon. ‣ Make sure you’re testing alternative inferences. ‣Customize metrics to match experience objectives. ‣Data science is now an essential UX skill.
- 90. @jmspool Go ahead! Follow me on the Twitters. jspool@uie.com Find me at: Copyright © 2015 User Interface Engineering Don’t forget to connect to me on the LinkedIn. uie.com