Slides from my talk at WordCamp Portland. It's been cleaned up to look nicer than than it did when I gave the talk.

1. You have NO Idea
what your users want
And how to learn
Evan Solomon | Automattic
@evansolomon

2. if( time() % 2 )
! $title = ‘Split testing’;
else
! $title = ‘A/B testing’;
setcookie( ‘title’, $title );
echo $title;

3. A/B testing
1.Try different things
2.Measure their effects
3.Profit!
Lots of good examples at blog.performable.com

4. What can it do?
Determine how likely one version
is to be better than another.
If you have enough data you can tell how much better, but you need quite a bit

5. Why do it?
It makes you
smart(er)!
Also turns out to be good for getting clients

6. Start basic
1.Talk to your customers
2.Listen
3.Profit!
Step 3 is always profit

7. Dashboard
Check out geckoboard.com

8. Running tests
We use Optimizely because:
1.Real time results (great for debugging)
2.Tests are written entirely in jQuery
3.Easy to target using WordPress logic

9. Version A

10. Version B

11. What happens?
B wins and
covers the spread
Signups increase ~25%

12. Handling data
Terms you should get used to:
“Confidence interval”
“Significance test”
Check out isvalid.org to calculate these stats
I made isvalid for myself so it’s a little rough, but very useful

13. isvalid.org

14. Keep in mind
• Question assumptions
• Unintended (or not) biases
• Test liberally, it’s nearly free
• Many (most) tests “fail”

15. Call for help
I’m working on a .org plugin
Based on the one we made for WordPress.com

16. @evansolomon