Web Directions South 2009, Oct 8-9
Session: Data Driven Design
Speaker: Luke Stevens (@lukestevens)
Sign up for the book!
http://msrmnt.com/book/
http://twitter.com/msrmnt
Questions?
luke@msrmnt.com

Hey, I’m Luke

1. Intro + The Problem
2. Quick case study
3. Cool technique

Data driven design

© Dominic Wilcox http://dominicwilcox.com

Who cares?

Data driven design
Performance
based design

No longer just an
aesthetic, usability
or experience
problem.

1. Coaches + athletes
2. Doctors + patients

Coach + Athlete

Athletes:
- Have a speci c goal
- Goal = single number
- Data driven

Michael Phelps’ breakfast:
- Three fried-egg sandwiches
- Five-egg omelet
- Bowl of grits (porridge)
- Three slices of French toast
- Three choc-chip pancakes
- Two cups of coffee

Performance encourages:
- Creativity
- The unexpected
- Risk

Doctors

1st priority?

NOT KILL YOU
(aka ‘do no harm’)

WARNING:
Side effects may include:
- Decreased sales
- Angry customers
- Lighter wallet

Getting harder
NOT to do harm

Design is maturing

http://www.microsoft.com/misc/features/features_ shbk.htm

2009

Even Microsoft
gets design
Well, visual design. On the web. For their home page.

Design maturity over time
50
37.5
25
12.5
0
1994 1997 2000 2003 2006 2009

Do no harm

One problem.
Many solutions.

Who chooses?

Flip a coin?

Wired:
Extreme Makeover:
Craigslist Edition
http://wired.com/entertainment/theweb/magazine/17-09/ff_craigslist_makeover

Original - craigslist today

Khoi Vinh & colleagues, NYTimes.com

SimpleScott, former design director of BarackObama.com

Luke Hayman and Lisa Strausfeld, Pentagram

1. Will it do harm?
2. Which is best?

“Your designs are
terrible, confusing and
you miss the point”

If we, as designers,
can’t pick the best
performer, then...?

THIS IS
THE PROBLEM

Out with the old

In with the new

Gutenberg Bible - 15th century

What we do is different

The most
measurable
form of design
in history

Recap - Part 1

1. We have the most
measurable form of
design in history

2. We should use that

Part II
Quick case study

Know your metric

What success looks like:

PhotographyBLOG.com - Original

~1 MM visitors
~3 MM page views
~10 blog stories a day

Home page traffic?

8%
(92% elsewhere)

Data > Assumptions

Know Google Analytics

Buy a book
advanced-web-metrics.com

Know web analytics

Read Avinash’s blog
kaushik.net/avinash/

PhotographyBLOG:
Camera reviews

Web analytics = strategy

PhotographyBLOG.com - v1

PhotographyBLOG.com - v2

PhotographyBLOG.com - v1 & v2

Test, don’t guess

A/B testing

Which design won?

v1 or v2? Pick one

Neither
Performance was almost identical

ANY result = learnings*
* ”learnings” is not a real word

E.g. Users did. not. care.

PhotographyBLOG.com - Original

PhotographyBLOG.com - Redesign

Performance based design:
Happy client &
Happy users

Testing is powerful

Testing will change the way
you think about design

But what to test?

YouTube - 1,024 (!)
combinations of
tiny details.
15% more sign ups

YouTube’s 1,024 combo experiment
using Google Website Optimizer
From: http://youtube-global.blogspot.com/2009/08/look-inside-1024-recipe-multivariate.html

37signals tested
headlines.
30% more sign ups

Winner
30% lift
2nd place
27% lift
3rd place
15% lift
Original
(worst performer)
From: http://37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page

Button text:
“Free Trial”
“Sign-up for Free Trial”
“See Plans and Pricing”

37signals tested a
button -
Doubled clicks
From: http://carsoni ed.com/blog/business/how-to-increase-sign-ups-by-200-percent/

We barely know where
the landscape is,
let alone where we
should be on it.

We all need to test

- All design is testable
- Testing takes time
- Can only test a few things

We all need to test
and SHARE

Current clearing houses:
@kissmetrics
@abtests

Part III
A cool technique

Google Website Optimizer
google.com/websiteoptimizer

The web standards
movement has given us
an incredible,
unexpected gift.

HTML + CSS =
The perfect platform to
test design

Use GWO to serve up
alternate CSS les

(But conversions don’t
suit everyone...)

Measure the results in
Google Analytics
gwotricks.com

CSS + GWO + GA =
Testing trinity

Objective web design
is possible right now

Performance
based design
builds on
standards
based design

It’s the future
of web design

BOOK!
Sign up to get noti ed:
http://msrmnt.com/book
follow @msrmnt
luke@msrmnt.com | @lukestevens