Web Directions South 2009, Oct 8-9
Session: Data Driven Design
Speaker: Luke Stevens (@lukestevens)

Sign up for the book!...
Hey, I’m Luke
1. Intro + The Problem
2. Quick case study
3. Cool technique
Data driven design
© Dominic Wilcox http://dominicwilcox.com
Who cares?
Data driven design
Performance
based design
No longer just an
aesthetic, usability
or experience
problem.
1. Coaches + athletes
2. Doctors + patients
Coach + Athlete
Athletes:
- Have a speci c goal
- Goal = single number
- Data driven
Michael Phelps’ breakfast:
- Three fried-egg sandwiches
- Five-egg omelet
- Bowl of grits (porridge)
- Three slices of Fre...
Performance encourages:
- Creativity
- The unexpected
- Risk
Doctors
1st priority?
NOT KILL YOU
(aka ‘do no harm’)
WARNING:
Side effects may include:
- Decreased sales
- Angry customers
- Lighter wallet
Getting harder
NOT to do harm
Design is maturing
http://www.microsoft.com/misc/features/features_ shbk.htm
2009
Even Microsoft
gets design
Well, visual design. On the web. For their home page.
Design maturity over time


 50




37.5




 25




12.5




  0
   1994   1997      2000          2003       2006   2009
Do no harm
One problem.
Many solutions.
Who chooses?
Flip a coin?
Wired:
Extreme Makeover:
Craigslist Edition
http://wired.com/entertainment/theweb/magazine/17-09/ff_craigslist_makeover
Original - craigslist today
Khoi Vinh & colleagues, NYTimes.com
SimpleScott, former design director of BarackObama.com
Luke Hayman and Lisa Strausfeld, Pentagram
1. Will it do harm?
2. Which is best?
“Your designs are
terrible, confusing and
you miss the point”
If we, as designers,
can’t pick the best
performer, then...?
THIS IS
THE PROBLEM
Out with the old
In with the new
Gutenberg Bible - 15th century
What we do is different
The most
measurable
form of design
in history
Recap - Part 1
1. We have the most
measurable form of
design in history
2. We should use that
Part II
Quick case study
Know your metric
What success looks like:
PhotographyBLOG.com - Original
~1 MM visitors
~3 MM page views
~10 blog stories a day
Home page traffic?
8%
(92% elsewhere)
Data > Assumptions
Know Google Analytics
Buy a book
advanced-web-metrics.com
Know web analytics
Read Avinash’s blog
kaushik.net/avinash/
PhotographyBLOG:
Camera reviews
Web analytics = strategy
PhotographyBLOG.com - v1
PhotographyBLOG.com - v2
PhotographyBLOG.com - v1 & v2
Test, don’t guess
A/B testing
Which design won?
v1 or v2? Pick one
Neither
Performance was almost identical
ANY result = learnings*


* ”learnings” is not a real word
E.g. Users did. not. care.
PhotographyBLOG.com - Original
PhotographyBLOG.com - Redesign
Performance based design:
Happy client &
Happy users
Testing is powerful
Testing will change the way
you think about design
But what to test?
YouTube - 1,024 (!)
combinations of
tiny details.
15% more sign ups
YouTube’s 1,024 combo experiment
                        using Google Website Optimizer




From: http://youtube-global.bl...
37signals tested
headlines.
30% more sign ups
Winner
30% lift


2nd place
27% lift


3rd place
15% lift


Original
(worst performer)

From: http://37signals.com/svn/pos...
Button text:
“Free Trial”
“Sign-up for Free Trial”
“See Plans and Pricing”
37signals tested a
button -
Doubled clicks

From: http://carsoni ed.com/blog/business/how-to-increase-sign-ups-by-200-perc...
We barely know where
the landscape is,
let alone where we
should be on it.
We all need to test
- All design is testable
- Testing takes time
- Can only test a few things
We all need to test
and SHARE
Current clearing houses:
@kissmetrics
@abtests
Part III
A cool technique
Google Website Optimizer
google.com/websiteoptimizer
The web standards
movement has given us
an incredible,
unexpected gift.
HTML + CSS =
The perfect platform to
test design
Use GWO to serve up
alternate CSS les
(But conversions don’t
suit everyone...)
Measure the results in
Google Analytics
gwotricks.com
CSS + GWO + GA =
Testing trinity
Objective web design
is possible right now
Performance
based design
builds on
standards
based design
It’s the future
of web design
BOOK!
Sign up to get noti ed:
http://msrmnt.com/book
follow @msrmnt

    luke@msrmnt.com | @lukestevens
Data Driven Design - Web Analytics & Testing for Designers (Web Directions South 2009)
Data Driven Design - Web Analytics & Testing for Designers (Web Directions South 2009)
Data Driven Design - Web Analytics & Testing for Designers (Web Directions South 2009)
Data Driven Design - Web Analytics & Testing for Designers (Web Directions South 2009)
Data Driven Design - Web Analytics & Testing for Designers (Web Directions South 2009)
Upcoming SlideShare
Loading in …5
×

Data Driven Design - Web Analytics & Testing for Designers (Web Directions South 2009)

6,113
-1

Published on

Slides from my "Data Driven Design" presentation from Web Directions South 2009.

This presentation looks at:
- How web designers can think about data driven, or "performance based" design
- What problem it addresses with how we currently do design
- A case study of how I applied "performance based design" to a project
- The 'testing trinity' of CSS + Google Analytics + Google Website Optimizer

The ideas in this presentation come from a book I'm working on, please sign up to get notified when it's released at:
http://msrmnt.com/book
or follow me on Twitter - @msrmnt for occasional book updates.

Published in: Design, Technology, Business

Data Driven Design - Web Analytics & Testing for Designers (Web Directions South 2009)

  1. 1. Web Directions South 2009, Oct 8-9 Session: Data Driven Design Speaker: Luke Stevens (@lukestevens) Sign up for the book! http://msrmnt.com/book/ http://twitter.com/msrmnt Questions? luke@msrmnt.com
  2. 2. Hey, I’m Luke
  3. 3. 1. Intro + The Problem 2. Quick case study 3. Cool technique
  4. 4. Data driven design
  5. 5. © Dominic Wilcox http://dominicwilcox.com
  6. 6. Who cares?
  7. 7. Data driven design Performance based design
  8. 8. No longer just an aesthetic, usability or experience problem.
  9. 9. 1. Coaches + athletes 2. Doctors + patients
  10. 10. Coach + Athlete
  11. 11. Athletes: - Have a speci c goal - Goal = single number - Data driven
  12. 12. Michael Phelps’ breakfast: - Three fried-egg sandwiches - Five-egg omelet - Bowl of grits (porridge) - Three slices of French toast - Three choc-chip pancakes - Two cups of coffee
  13. 13. Performance encourages: - Creativity - The unexpected - Risk
  14. 14. Doctors
  15. 15. 1st priority?
  16. 16. NOT KILL YOU (aka ‘do no harm’)
  17. 17. WARNING: Side effects may include: - Decreased sales - Angry customers - Lighter wallet
  18. 18. Getting harder NOT to do harm
  19. 19. Design is maturing
  20. 20. http://www.microsoft.com/misc/features/features_ shbk.htm
  21. 21. 2009
  22. 22. Even Microsoft gets design Well, visual design. On the web. For their home page.
  23. 23. Design maturity over time 50 37.5 25 12.5 0 1994 1997 2000 2003 2006 2009
  24. 24. Do no harm
  25. 25. One problem. Many solutions.
  26. 26. Who chooses?
  27. 27. Flip a coin?
  28. 28. Wired: Extreme Makeover: Craigslist Edition http://wired.com/entertainment/theweb/magazine/17-09/ff_craigslist_makeover
  29. 29. Original - craigslist today
  30. 30. Khoi Vinh & colleagues, NYTimes.com
  31. 31. SimpleScott, former design director of BarackObama.com
  32. 32. Luke Hayman and Lisa Strausfeld, Pentagram
  33. 33. 1. Will it do harm? 2. Which is best?
  34. 34. “Your designs are terrible, confusing and you miss the point”
  35. 35. If we, as designers, can’t pick the best performer, then...?
  36. 36. THIS IS THE PROBLEM
  37. 37. Out with the old
  38. 38. In with the new
  39. 39. Gutenberg Bible - 15th century
  40. 40. What we do is different
  41. 41. The most measurable form of design in history
  42. 42. Recap - Part 1
  43. 43. 1. We have the most measurable form of design in history
  44. 44. 2. We should use that
  45. 45. Part II Quick case study
  46. 46. Know your metric
  47. 47. What success looks like:
  48. 48. PhotographyBLOG.com - Original
  49. 49. ~1 MM visitors ~3 MM page views ~10 blog stories a day
  50. 50. Home page traffic?
  51. 51. 8% (92% elsewhere)
  52. 52. Data > Assumptions
  53. 53. Know Google Analytics
  54. 54. Buy a book advanced-web-metrics.com
  55. 55. Know web analytics
  56. 56. Read Avinash’s blog kaushik.net/avinash/
  57. 57. PhotographyBLOG: Camera reviews
  58. 58. Web analytics = strategy
  59. 59. PhotographyBLOG.com - v1
  60. 60. PhotographyBLOG.com - v2
  61. 61. PhotographyBLOG.com - v1 & v2
  62. 62. Test, don’t guess
  63. 63. A/B testing
  64. 64. Which design won?
  65. 65. v1 or v2? Pick one
  66. 66. Neither Performance was almost identical
  67. 67. ANY result = learnings* * ”learnings” is not a real word
  68. 68. E.g. Users did. not. care.
  69. 69. PhotographyBLOG.com - Original
  70. 70. PhotographyBLOG.com - Redesign
  71. 71. Performance based design: Happy client & Happy users
  72. 72. Testing is powerful
  73. 73. Testing will change the way you think about design
  74. 74. But what to test?
  75. 75. YouTube - 1,024 (!) combinations of tiny details. 15% more sign ups
  76. 76. YouTube’s 1,024 combo experiment using Google Website Optimizer From: http://youtube-global.blogspot.com/2009/08/look-inside-1024-recipe-multivariate.html
  77. 77. 37signals tested headlines. 30% more sign ups
  78. 78. Winner 30% lift 2nd place 27% lift 3rd place 15% lift Original (worst performer) From: http://37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page
  79. 79. Button text: “Free Trial” “Sign-up for Free Trial” “See Plans and Pricing”
  80. 80. 37signals tested a button - Doubled clicks From: http://carsoni ed.com/blog/business/how-to-increase-sign-ups-by-200-percent/
  81. 81. We barely know where the landscape is, let alone where we should be on it.
  82. 82. We all need to test
  83. 83. - All design is testable - Testing takes time - Can only test a few things
  84. 84. We all need to test and SHARE
  85. 85. Current clearing houses: @kissmetrics @abtests
  86. 86. Part III A cool technique
  87. 87. Google Website Optimizer google.com/websiteoptimizer
  88. 88. The web standards movement has given us an incredible, unexpected gift.
  89. 89. HTML + CSS = The perfect platform to test design
  90. 90. Use GWO to serve up alternate CSS les
  91. 91. (But conversions don’t suit everyone...)
  92. 92. Measure the results in Google Analytics gwotricks.com
  93. 93. CSS + GWO + GA = Testing trinity
  94. 94. Objective web design is possible right now
  95. 95. Performance based design builds on standards based design
  96. 96. It’s the future of web design
  97. 97. BOOK! Sign up to get noti ed: http://msrmnt.com/book follow @msrmnt luke@msrmnt.com | @lukestevens
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×