Metrics Driven Design by Joshua Porter
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Metrics Driven Design by Joshua Porter

on

  • 41,886 views

 

Statistics

Views

Total Views
41,886
Views on SlideShare
32,341
Embed Views
9,545

Actions

Likes
182
Downloads
882
Comments
6

73 Embeds 9,545

http://km 2522
http://km 2522
http://andrewchenblog.com 2281
http://www.stevenloi.com 283
http://andrewchen.co 274
http://madebymany.com 234
http://www.yucezerey.com 194
http://feeds.feedburner.com 194
http://acharoo.posterous.com 153
http://www.usmansheikh.com 123
http://ryanhoover.me 90
http://andreasdittes.com 81
http://www.marketing-pub.fr 53
http://runningleanhq.com 39
http://km.raysono.int 35
http://km.raysono.int 35
http://blog.raysono.int 34
http://blog.raysono.int 34
http://sylvansocial.com 31
http://paper.li 28
http://tomfisher.posterous.com 27
http://michaelbatistich.com 25
http://static.slidesharecdn.com 24
http://posterous.com 20
http://preciousforever.posterous.com 18
http://serve4impact.com 17
http://us-w1.rockmelt.com 16
https://twitter.com 15
http://www.badnima.com 15
http://tmdc.posterous.com 13
http://pollenizer.com 11
http://note.leminhhai.com 11
https://pramati.qontext.com 9
http://blog 7
http://flavors.me 6
http://www.netvibes.com 6
http://feedproxy.google.com 5
http://dashboard.bloglines.com 4
http://blog.leminhhai.com 4
http://moondaesa.tistory.com 4
http://s320192528.onlinehome.us 4
http://www.samthatte.com 4
http://translate.googleusercontent.com 3
http://www.paperblog.fr 3
http://www.mevora.com 2
http://t.kevinluddy.com 2
http://boutofcontext.com 2
http://localhost 2
http://trunk.ly 2
http://pinterest.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

15 of 6 Post a comment

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Metrics Driven Design by Joshua Porter Presentation Transcript

  • 1. Metrics-Driven Design by Joshua Porter Twitter: bokardo bokardo 52weeksofUX performable microcopy designing for the social web make them care! by Joshua PorterSaturday, March 12, 2011
  • 2. Saturday, March 12, 2011
  • 3. “Unfortunately for me, there was one small problem I didn’t see back then.”Saturday, March 12, 2011
  • 4. Doug Bowman on Design at Google “ Yes, it’s true that a team at Google couldn’t decide between two blues, so they’re testing 41 shades between each blue to see which one performs better. I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I can’t operate in an environment like that. Metrics-Driven Design Daring. Decisive. Conviction. SXSWSaturday, March 12, 2011
  • 5. 41 Shades of Blue Test link color Gmail Google Search 41 Bucket Split Test: ~2.5% of users each got a shade for 2 weeks.Saturday, March 12, 2011
  • 6. Saturday, March 12, 2011
  • 7. #2200C1 #0044CC Google BingSaturday, March 12, 2011
  • 8. “ That blue was worth at least $80 million. Paul Ray, UX Manager, Bing Metrics-Driven Design Microcopy SXSWSaturday, March 12, 2011
  • 9. Doug Bowman on Design at Google “ When a company is filled with engineers, it turns to engineering to solve problems. Reduce each decision to a simple logic problem. Remove all subjectivity and just look at the data. Data in your favor? Ok, launch it. Data shows negative effects? Back to the drawing board. And that data eventually becomes a crutch for every decision, paralyzing the company and preventing it from making any daring design decisions. Metrics-Driven Design Daring. Decisive. Conviction. SXSWSaturday, March 12, 2011
  • 10. Spectrum of Design Intuition-Driven Data-Driven Make best-guesses Every design choice is tested Rely on previous experience Takes others experience with a grain of salt Study what others are doing Design is a logic problem Use best practices, principles & patterns Rely on data for decision-making Aesthetics are integral Aesthetics are secondary Rely on our gut Never trust your gut Creative, visionary Cold, calculating Inherently risky Risk-averse Doug’s words: Assumed: instinctive, subjective, daring deliberate, objective, safeSaturday, March 12, 2011
  • 11. Imagine that your design is a mountain... Metrics-Driven Design The grass is always greener on other mountainsides. SXSWSaturday, March 12, 2011
  • 12. Imagine that your design is a mountain... EXISTING DESIGN Metrics-Driven Design The grass is always greener on other mountainsides. SXSWSaturday, March 12, 2011
  • 13. Imagine that your design is a mountain... Now EXISTING DESIGN Metrics-Driven Design The grass is always greener on other mountainsides. SXSWSaturday, March 12, 2011
  • 14. Imagine that your design is a mountain... Optimized (local maxima) Now EXISTING DESIGN Metrics-Driven Design The grass is always greener on other mountainsides. SXSWSaturday, March 12, 2011
  • 15. Imagine that your design is a mountain... Potential Improvement Optimized (local maxima) Now Metrics-Driven Design The grass is always greener on other mountainsides. SXSWSaturday, March 12, 2011
  • 16. Imagine that your design is a mountain... Goal BETTER DESIGN Current Design EXISTING DESIGN Metrics-Driven Design The grass is always greener on other mountainsides. SXSWSaturday, March 12, 2011
  • 17. Imagine that your design is a mountain... Goal Potential Improvement BETTER DESIGN Now EXISTING DESIGN Metrics-Driven Design The grass is always greener on other mountainsides. SXSWSaturday, March 12, 2011
  • 18. Optimization asks: What works best in the current model? Design innovation asks: What is the best possible model?Saturday, March 12, 2011
  • 19. What are Metrics?Saturday, March 12, 2011
  • 20. DEFINITION Metrics are simply numbers that measure the effectiveness of your business.Saturday, March 12, 2011
  • 21. 5 Reasons why Metrics are a Designer’s Best FriendSaturday, March 12, 2011
  • 22. 1. Metrics reduce arguments based on opinion.Saturday, March 12, 2011
  • 23. 2. Metrics give you answers about what really works.Saturday, March 12, 2011
  • 24. 3. Metrics show you where you’re strong as a designer.Saturday, March 12, 2011
  • 25. 4. Metrics allow you to test anything you want.Saturday, March 12, 2011
  • 26. 5. Clients love metrics.Saturday, March 12, 2011
  • 27. PRINCIPLE Your metrics will be as unique as your business.Saturday, March 12, 2011
  • 28. Saturday, March 12, 2011
  • 29. Vanity Metrics Metrics-Driven Design Ego SXSWSaturday, March 12, 2011
  • 30. The Usage LifecycleSaturday, March 12, 2011
  • 31. Interested Trial/beta User Customer Passionate CustomerSaturday, March 12, 2011
  • 32. Acquisition Conversion Engagement Satisfaction Interested Trial/beta User Customer Passionate CustomerSaturday, March 12, 2011
  • 33. With metrics you are essentially just measuring how well you move people along the lifecycle!Saturday, March 12, 2011
  • 34. Acquisition MetricsSaturday, March 12, 2011
  • 35. CPA - Cost per Acquisition If your CPA is higher than your LTV then you’re in trouble.Saturday, March 12, 2011
  • 36. Acquisition vs. Referral: Dropbox Ran Google Adwords campaigns to drive traffic to dropbox.com The traffic that completed the lifecycle: 1) Searched on a keyword 2) Visited their site 3) Signed-up for service 4) Became a customer cost them $233-$388 per person! (for a $99 product) Metrics-Driven Design LTV = Lifetime value SXSWSaturday, March 12, 2011
  • 37. Dropbox Lifecycle 30 days prior to April 2010, Dropbox users sent 2.8 million direct referral invites. Dropbox’s referral program with 2-sided incentive increased sign-ups by 60% permanently. Metrics-Driven Design Referral = Word of Mouth SXSWSaturday, March 12, 2011
  • 38. Performable Acquisition Metrics Comparative Metrics Revenue by Channel Revenue by Keyword Metrics-Driven Design Microcopy SXSWSaturday, March 12, 2011
  • 39. Conversion MetricsSaturday, March 12, 2011
  • 40. Trials / Conversions = Conversion RateSaturday, March 12, 2011
  • 41. Conversion Funnel Analysis A A 100% B B 60% C C 20% Metrics-Driven Design SXSWSaturday, March 12, 2011
  • 42. Sign-up Conversion Funnel Original Flow 1 2 3 4 confirm personal info add your friends invite others getting started Original Conversion Funnel 1 100% of the 100% of people who started the sign-up process, only 2 63% 14% made it to the getting started screen. 3 26% 4 14% Metrics-Driven Design SXSWSaturday, March 12, 2011
  • 43. Sign-up Conversion Funnel New Flow 2 4 add your friends getting started New Conversion Funnel 2 100% of the 100% of people who started the sign-up process, 86% made it to the getting started screen. 4 86% Metrics-Driven Design Removing stuff is a quick way to improve your funnel SXSWSaturday, March 12, 2011
  • 44. Engagement Matters: Twitter http://www.lukew.com/ff/entry.asp?1039 Metrics-Driven Design Speed without context is wasted. SXSWSaturday, March 12, 2011
  • 45. Engagement MetricsSaturday, March 12, 2011
  • 46. • Hits • Page views • Visits • Unique Visitors • Returning Visitors • Registered Users • Customers • Frequency • Time on Site • Daily Active UsersSaturday, March 12, 2011
  • 47. Cohort Analysis Engagement over time Month 1 Month 2 Month 3 Month 4 Month 5 Month 6 Month 7 Month 8 Month 9 Month 10 Month 11 Month 12 (Joined in) 100% 20% 19% 13% 13% 10% 12% 11% 7% 7% 7% ? January 100% 21% 16% 13% 11% 9% 9% 7% 7% 7% ? February 100% 24% 20% 17% 15% 13% 11% 10% 10% ? March 100% 31% 27% 24% 19% 15% 12% 12% ? April 100% 31% 27% 25% 21% 18% 16% ? May 100% 39% 28% 24% 20% 19% ? June 100% 40% 33% 27% 23% ? July 100% 47% 41% 32% ? August 100% 52% 43% ? September In this case engagement is improving nicely. 100% 53% ? Of the January cohort, only 20% were engaged in month 2. October Of the October cohort, 53% were engaged in month 2. 100% ? November ? December http://redeye.firstround.com/2008/01/after-the-techc.htmlSaturday, March 12, 2011
  • 48. Prevention: Facebook Deactivation Design changes to the deactivation page accounted for 1 million members not leaving the service. Julie Zhou, Facebook http://www.readwriteweb.com/archives/what_happens_when_you_deactivate_your_facebook_acc.php Metrics-Driven Design I will miss you after my talk today. SXSWSaturday, March 12, 2011
  • 49. Satisfaction MetricsSaturday, March 12, 2011
  • 50. Referral: Net Promoter Score How likely is it that you would recommend our company to a friend or colleague? 0 1 2 3 4 5 6 7 8 9 10 Metrics-Driven Design Do you know what your net promoter score is? SXSWSaturday, March 12, 2011
  • 51. Net Promoter Score How likely is it that you would recommend our company to a friend or colleague? 0 1 2 3 4 5 6 7 8 9 10 Detractors Passives Promoters Score = % Promoters - % Detractors Metrics-Driven Design Do you know what your net promoter score is? SXSWSaturday, March 12, 2011
  • 52. Mint.com & Net Promoter Score “ Maybe we didn’t have a high viral coefficient but we had a great net promoter score. Jason Putorti, Lead Designer, Mint.com Metrics-Driven Design SXSWSaturday, March 12, 2011
  • 53. Emergent MetricsSaturday, March 12, 2011
  • 54. Emergent Metrics: 5 Friends The magic number is 5. Once a FriendFeed user found five friends, they became active users. Bret Taylor, Friendfeed Metrics-Driven Design SXSWSaturday, March 12, 2011
  • 55. Is there one metric that drives others? “ At Blogger, we determined that our most critical metric was number of posts. An increase in posts meant that people were not just creating blogs, but updating them, and more posts would drive more readership, which would drive more users, which would drive more posts. Ev Williams founder of Blogger & Twitter Metrics-Driven Design Once you’re lucky, twice you’re good. SXSWSaturday, March 12, 2011
  • 56. Principles of Metrics-Driven Design 1. Optimize in small steps; innovate with daring leaps. 2. No design survives contact with the user. 3. Small improvements, taken together, yield amazing results.  4. Testing is empowering, reversion is cleansing. 5. Metrics are not creative: human beings are. 6. All team members are responsible for the user experience. 7. If metrics aren’t actionable, they aren’t useful. 8. Design is never done. Metrics-Driven Design Reminder: Principal = Person & Principle = Thing SXSWSaturday, March 12, 2011
  • 57. Saturday, March 12, 2011
  • 58. This Talk. http://bokardo.com/talks/metrics-driven-design/Saturday, March 12, 2011
  • 59. My Life. http://performable.com http://oneflightbooks.comSaturday, March 12, 2011