Evolution Not Revolution: Why Optimizing Beats Redesigning

Evolution, not Revolution. Why Optimizing Beats Redesigning. [presented by] Kate O’Neill of [meta]marketer Image credit: funfonix.com

Who am I to make these claims? And who are you to listen?

Why I’m a Believer • Helped lead rollout through data awareness of interactive redesign at Netﬂix that is still standard for e-commerce • Oversaw many multivariate tests at Magazines.com that resulted in 40% lift in conversion rate YOY • Now own a company that focuses on this kind of optimization for clients with great results

Why You May Choose to Believe • Working in a company torn by politics or conﬂicting interests in web design or feature? • Designing sites freelance and always encountering the same resistance? • Running personal web sites and blogs you’d like to see perform better?

How are most design decisions made?

HPPO • Highest Paid Person’s Opinion • Not irrelevant, but not comprehensive.

By Committee • “I need my department to be featured on the home page!” • “Can we use something other than red for the Buy buttons?”

Designer’s Aesthetic • Flash! • Prettiness over performance

Finger in the Wind • What seems to be trendy • What someone mentioned at a party • What’s reported in the news

In other words... in a vacuum.

How should design decisions be made?

You need user input. You are not your audience. Even if you are.

You need data. Users lie. Aggregate data doesn’t.

Data trumps opinions. Even highly paid ones. HPPO Use conversion-related metrics to determine executive-relevant strengths and weaknesses of the site.

If it isn’t interesting to the user, ditch it. Committee Use engagement metrics to determine what keeps users on the site.

Think users like your design? Prove it. Design for Design’s Sake If your success can’t be measured, you can’t defend it. And your input won’t be appreciated.

“Great idea! I’ll add it to the testing roadmap.” Finger in the Wind Trendy ideas are worth knowing about. But they may not work in your environment.

Balance objective and subjective input what people think or feel versus what people do

Balance qualitative and quantitative input what you can intuit versus what you can measure

How to Balance Input Gathering analytics data qualitative - - - - - - quantitative A/B or MVT results surveys focus groups usability studies / interviews subjective - - - - - - - - - - - - - - - - - - - - - - - - objective

What to do? ‣ Balance subjective & objective testing (And know that you may get it wrong) ‣ Find the story behind the story (But know that you may get it wrong) ‣ Look for a narrative in onsite testing (But know that you may get it wrong) ‣ Look for the unobvious AND the obvious (And know that you may get it wrong)

If you’re still going to get it wrong, why test? Because you can not only measure lift when you’re right... (Woo hoo!) you can also contain risk when you’re wrong. (And it just might save your job.)

Test your way to greatness.

How a Redesign Can Go Wrong: HPPO Edition

Netﬂix Branding Gaffe circa 2000 TV?! Static?! Totally wrong customer experience emphasis.

Improved Playing off of the movie theater experience.

How a Redesign Can Go Wrong: Usability Edition

Look Inside: Setup

Look Inside: Results • Visitors more likely to click on Preview offer (4.89% average) vs. Explore (3.85%) • Both variations resulted in a drop in conversion • Drop in conversion only slightly greater with Preview (-.87%) than with Explore (-.76%) • Overall, more established titles (Time, Sports Illustrated, People) had lowest Click % as well as below average drop in conversion of all viewer- enabled titles • Somewhat lesser-known titles (SI Kids, Health, StyleWatch, Money) had highest Click % and higher than average drop in conversion of all viewer- enabled titles

What to do? (Once more with feeling!) ‣ Balance subjective & objective testing (And know that you may get it wrong) ‣ Find the story behind the story (But know that you may get it wrong) ‣ Look for a narrative in onsite testing (But know that you may get it wrong) ‣ Look for the unobvious AND the obvious (And know that you may get it wrong)

If all else fails: [meta]marketer can help. Thank you! Kate O’Neill, Founder / CEO kate@metamarketer.com 615-852-META twitter: @kateo / @metamarketer

Evolution Not Revolution: Why Optimizing Beats Redesigning

by Kate O'Neill on Oct 17, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 31

Statistics

Evolution Not Revolution: Why Optimizing Beats Redesigning — Presentation Transcript

http://metamarketer.com	23
http://www.slideshare.net	8