×
  • Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
 

Design for Continuous Experimentation

by Lumberjack at Etsy on Dec 02, 2012

  • 106,180 views

Presented at Warmgun design conference, 2012. Video: http://mcfunley.com/design-for-continuous-experimentation

Presented at Warmgun design conference, 2012. Video: http://mcfunley.com/design-for-continuous-experimentation

Statistics

Views

Total Views
106,180
Views on SlideShare
35,587
Embed Views
70,593

Actions

Likes
245
Downloads
583
Comments
12

135 Embeds 70,593

http://mcfunley.com 41731
http://habrahabr.ru 10213
http://econsultancy.com 8206
http://webdesign.tutsplus.com 4279
http://www.jvetrau.com 916
http://blog.e-xanthos.co.uk 669
http://www.conseilsmarketing.com 393
http://localhost 374
https://econsultancy.com 371
http://feeds.feedburner.com 370
https://twitter.com 301
http://uiflow.com 300
http://4thbay.mc-j.com 274
http://www.e-xanthos.co.uk 204
http://tosabiten.blogspot.no 173
http://hive.interruptmarketing.com 170
http://webwash.net 167
http://www.sarahdoody.com 166
http://m.habrahabr.ru 166
http://www.Mcfunley.com 149
http://www.iphone-entreprise.com 93
http://tosabiten.blogspot.se 93
http://www.elprincipiodeincertidumbre.net 85
http://digitalministry.com 77
http://danielgruesso.com 58
http://tosabiten.blogspot.com 50
http://danielgruesso.wordpress.com 49
http://getmakeplay.com 36
http://bazqux.com 35
http://chrisjamesbk.tumblr.com 30
http://savepearlharbor.com 23
http://translate.googleusercontent.com 20
http://www.twylah.com 18
http://www.pvsm.ru 16
http://idevie.com 15
http://abtasty.com 14
http://yellow-rayed2.rssing.com 14
http://danielgruesso.svbtle.com 14
http://hub.tutsplus.com 12
http://interactive.vianet.vianow.com 12
https://si0.twimg.com 11
http://tutorials.onlineexchanger.com 9
http://tosabiten.blogspot.co.uk 9
http://t.co 9
http://coolstuff.soapcreative.us 9
http://mein.erfolgsebook.de 8
http://amitdhamu.com 7
http://vinodpandey.com 6
http://cradletime1.rssing.com 5
http://webdesignerblogs.collected.info 5
More...

Accessibility

Categories

Upload Details

Uploaded via SlideShare as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

110 of 12 previous next Post a comment

  • CharlyLeporc Charly Leporc, Product Manager at Lazada great presentation and showcase 6 months ago
    Are you sure you want to
    Your message goes here
    Processing…
  • georgeioannou George Ioannou, Head of Agency at Digital & Wise Great presentation. Testing is definitely the way to go and all serious online businesses should adopt this workflow. As an agency we do a lot of Multivariate testing for our clients and sometimes as you mentioned the results can be very surprising (positive and negative).

    Keep up the good work!
    1 year ago
    Are you sure you want to
    Your message goes here
    Processing…
  • danmckinley Dan McKinley, Lumberjack at Etsy @MarkLaw1 never used it, we have our own infrastructure for this 1 year ago
    Are you sure you want to
    Your message goes here
    Processing…
  • MarkLaw1 Mark Law, Operations Director at Wilson Cooke @danmckinley Great thanks for your response, this makes perfect sense and allows accurate measuring of a change. Have you ever used Google Analytics A/B split testing for this or is it something you have developed in-house into the Etsy system? 1 year ago
    Are you sure you want to
    Your message goes here
    Processing…
  • danmckinley Dan McKinley, Lumberjack at Etsy @MarkLaw1 we do randomized A/B testing, so there's always a control group not seeing a change and we compare the feature against their behavior. We don't compare behavior before/after releases because of the seasonal variation you mention. 1 year ago
    Are you sure you want to
    Your message goes here
    Processing…
  • MarkLaw1 Mark Law, Operations Director at Wilson Cooke Might be difficult to answer but wondered how you are able to accurately measure/track changes on the site by removing seasonal variations? For example if you roll out a change at Christmas when sales will naturally increase anyway, how do you even this out to see the actual impact of the change? 1 year ago
    Are you sure you want to
    Your message goes here
    Processing…
  • danmckinley Dan McKinley, Lumberjack at Etsy Video of this is here:
    http://mcfunley.com/design-for-continuous-experimentation
    1 year ago
    Are you sure you want to
    Your message goes here
    Processing…
  • cemre1 Cemre Güngör, Designer, Frontend at Startups Yay, the experiment UI I made last summer is still there it seems :) 1 year ago
    Are you sure you want to
    Your message goes here
    Processing…
  • danmckinley Dan McKinley, Lumberjack at Etsy @ronnykoh of course the reductio ad absurdum of 'performance doesn't matter' is that the pages never come back, and the website is nonfunctional. So obviously at some point it makes a difference. And if for no other reason, performance matters since CPU time costs money.

    I don't disbelieve any results Bing (or Google) has that show that performance matters. The key observation here is that Etsy's case is not identical to those and that the sensitivity of conversion on Etsy to performance is not as pronounced. This has implications for how much we prioritize work on performance. Quite likely many other sites are in this situation.
    1 year ago
    Are you sure you want to
    Your message goes here
    Processing…
  • ronnykoh ronnykoh Nice talk with great motivation. Be careful not to over-generalize.

    For example, at Bing we've shown over and over that performance matters. The same slowdown experiment described here definitely made a difference for many metrics.

    See http://exp-platform.com for paper, talks, and pitfalls.

    -- Ronny
    1 year ago
    Are you sure you want to
    Your message goes here
    Processing…

110 of 12 previous next

Post Comment
Edit your comment

Design for Continuous Experimentation Design for Continuous Experimentation Presentation Transcript