The document discusses web analytics and optimization through A/B and multivariate testing to enhance site and campaign performance. It highlights successful case studies, such as increased unique clicks from mobile versions and improved lead generation with sub-navigation, while advising on establishing evaluation criteria and recognizing confounding factors. Continuous optimization is emphasized as an ongoing program to yield benefits, with a note on the importance of patience for UX testing results.

1. Web Analytics 101
OPTIMIZATION
The power of A/B and
Multivariate testing.

2. Optimization is the practice of using
hypothesis testing and
experimentation to continuously
improve site and campaign
performance.

3. A/B Testing
Randomized experiment with two variants.

4. Multivariate Testing
Randomized experiment with multiple variants.

5. Benefits
Sustained performance
improvement.
Predictable, controlled routine.
Lower risk from changes.

6. Optimization Testing
Examples

7. Mobile vs. Online
Pre-header clicks

8. A
“Mobile”
B
“Online”

9. A
“Mobile”
Version A received a 173% increase in unique clicks.
Test was conducted after learning 25% of members read email on their
mobile devices.

10. Sub-Navigation
Leads

11. A Without Sub-Navigation B With Sub-Navigation

12. B With Sub-Navigation
Version B increased leads by
39%.
Noticed a correlation between
leads and visitors seeing 5
specific pages on the site.
Good site analytics leads to
strong hypotheses!

13. Images
Account sign-ups

14. A
Sarah
B
Blair

15. A
Sarah
Sarah drove 36% more account signups than Blair.

16. Tips
To keep in mind

17. Get agreement up front on the
Evaluation Criteria with Business
and IT decision-makers.
Targeting increases the power of
optimization.
Be aware of confounding factors
such as site changes that occur
independently of your experiments.
They can pollute your metrics and
lead to incorrect conclusions.

18. Tricks
To remember

19. Optimization should be an ongoing
program – you will continue to get
benefit as long as you do it.
As you get more sophisticated, you
can run many experiments at once.
But don’t get sloppy with the
samples.
Be patient – some tests (especially
UX changes) take many months to
show results.

20. Presented by
SOCIETY CONSULTING

21. Connect With Us
SOCIETY CONSULTING
@SOCIETY_CONSULT
SOCIETY CONSULTING
SOCIETYCONSULTING.COM