Is ab testing all you need? The answer is no.

510 views
413 views

Published on

Why A/B-testing is not enough, but it works as a complement to user research

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
510
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
1
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Is ab testing all you need? The answer is no.

  1. 1. Is A/B-testing all you need? The answer is no. … says Martin Christensen, UX designer & researcher
  2. 2. Martin Christensen RAND() A 42% conversion 27% conversion B
  3. 3. Martin Christensen RAND() A 42% conversion 27% conversion B
  4. 4. Martin Christensen RAND() A 27% conversion B 0% conversion
  5. 5. Martin Christensen What WHY!?!?!?!?!?! A B
  6. 6. Martin Christensen A
  7. 7. Martin Christensen
  8. 8. Martin Christensen
  9. 9. Martin Christensen • Incremental development • Incremental and iterative development
  10. 10. Martin Christensen User research 101
  11. 11. Martin Christensen User research 101 (measuring for success) = Learnability issue ”It took me a while to understand what 
 to do. That’s because 
 I’m stupid.” Do you imagine that most people in the business would learn to use this app quickly? Rate it on a scale from 1 to 7, please. Surely not. Tell me again why it took a while to understand. I did not know what to do when I opened the app. There was no clear way for me to proceed. Great. I understand. Thanks! Well, it was hard for me, so 3. This is the user This is you
  12. 12. Martin Christensen User research 101 (measuring for success) Ease of use Effectiveness Efficiency Satisfaction Sense of being in control Capability Efficacy Conviviality Consistency Simplicity Learnability Usability (mean value) 1 2 3 4 5 6 7
  13. 13. Martin Christensen Summary • A/B-testing requires a large sample. • A/B-tests can’t be changed or corrected on-the-fly without compromising the test. • A/B-testing does not answer ”why”, only ”what”. The rest is your assumptions. • You can not A/B-test if you only have A, or nothing at all.
 • Simple user research and A/B-tests complement each other. • Do enough research to understand. • Go to the users, shut up, ask why, listen really carefully. • Dazzle your boss with a nice chart.
  14. 14. Thank you! Bygg klokt med LeanUX 20 maj 2014 Eftersom livet är för kort för att bygga något som ingen vill ha 11 platser kvar se.linkedin.com/in/martinuxchristensen/ martin.christensen@crisp.se martin.ux.christensen @m8rt UX open 2014 24 oktober 2014 En knytkonferens inom UX Super-Early Bird = 600 kr exkl. moms

×