The document reports on two case studies conducted by Bryson Meunier of Vivid Seats on the impact of mobile usability fixes and Google's mobile friendly label. The first study found no statistically significant correlation between fixing mobile errors and changes in traffic, bounce rate or revenue. The second study found no statistically significant difference in click-through rate before and after the introduction of Google's mobile friendly label. The document concludes that assumptions about mobile usability and SEO are not necessarily supported by current data and more testing is needed.

1. March 3, 2015
Mobile SEO
Case Study:
Bryson Meunier
SEO Director, Vivid Seats
@BrysonMeunier

2. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
MOBILE FRIENDLY: A QUESTION WITH
MORE THAN ONE RIGHT ANSWER

3. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
DYNAMIC SERVING IS ONE RIGHT ANSWER

4. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
1. Does fixing the mobile usability errors drastically
increase mobile search traffic or revenue or
dramatically decrease bounce rate?
2. Does having a mobile friendly label in Google
increase click through rate in a statistically
significant way?
CASE STUDY: FIXING MOBILE USABILITY ERRORS

5. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
20,000 TO 1,000 ERRORS IN UNDER 4 MONTHS

6. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
20,000 TO 1,000 ERRORS IN UNDER 4 MONTHS
Identified
corresponding
template
Prioritized based
on revenue
impact
Redesigned
priority
templates

7. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
20,000 TO 1,000 ERRORS IN UNDER 4 MONTHS

8. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
20,000 TO 1,000 ERRORS IN UNDER 4 MONTHS

9. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
Start with list of 2000 mobile pages with errors fixed between 9/09 – 10/25
Keep only pages with sessions
Vast majority occur in October
500 pages remaining
Goals:
Are total number of fixed pages correlated to average number of sessions, bounce rate, or revenue?
Does average sessions/day, bounce rate/day, or revenue/day increase as mobile errors are fixed relative to randomized control sample?
METHODOLOGY

10. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
Control group comprised of 1000 randomly selected mobile landing pages with similar number of sessions but with unfixed errors
Control group linear fit has a slope consistent with zero
R = 0.03 (no linear correlation)
• Linear fit to test group also has a slope consistent with zero
R = 0.17 (no linear correlation)
NO statistically significant difference between correlation coefficient with control group (p = 0.66)
NO statistically significant difference between fit slope with control group (p = 0.48)
MEASURED CORRELATION BETWEEN FIXED ERRORS
AND BOUNCE RATE, SESSIONS AND REVENUE

11. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
• Investigated correlation between
sessions, bounce rate, or revenue vs.
total number of fixed mobile errors
In all cases, we found no linear
correlation
• Investigated average sessions/day,
bounce rate/day, or revenue/day
increase as mobile errors are fixed
relative to randomized control sample
In all cases, the correlation coefficients and
slopes of the test groups and control groups are
indistinguishable.
NO DIFFERENCE BEFORE AND AFTER
MOBILE USABILITY ERRORS FIXED

12. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
1. Fixing mobile usability errors has no immediate
effect on traffic, revenue, or bounce rate from
mobile search traffic
2. No apparent mobile usability algorithm exists as
of 2/17/2015. If it exists it may not be worth the
effort.
3. Going to continue measuring to see if effect is
delayed
DOES FIXING MOBILE USABILITY ERRORS
HELP WITH SEO OR USER EXPERIENCE?

13. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
DOES MOBILE FRIENDLY LABEL HELP CTR?

14. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
MFT introduced on 11/18/2014
Sample includes 21 landing pages having traffic between 11/13/2014 and 11/30/2014 with a variance in average page position of < 0.2
throughout that time
Average CTR before MFT: 13.5%
Average CTR AFTER MFT: 13.4%
DIFFERENCE IN AVERAGE CTR IS NOT STATISTICALLY SIGNIFICANT
DOES MOBILE FRIENDLY LABEL HELP CTR?

15. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
1. No statistically significant
difference in average CTR
before and after mobile-
friendly label
2. Limited data set
3. All competitors have
mobile friendly label, which
could affect results
4. Going to continue
monitoring to see if effect
is delayed.
DOES MOBILE FRIENDLY LABEL HELP CTR?

16. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
1. Assumptions about mobile usability and SEO are
not necessarily supported by data today
2. More testing is needed to support conclusions,
as this is one site and doesn’t account for
potential delay in benefit
3. Still recommend continuing with priority fixes
that aren’t cost prohibitive, and measuring
results
FINAL THOUGHTS

17. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D

18. February 16, 2015
Appendix: Effect of
Mobile Fixes on Sessions,
Bounce Rate, and
Revenue
Dr. David Pignotti
Data Scientist, Vivid Seats

19. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
Start with list of 2000 mobile pages with errors fixed between 9/09 – 10/25
Keep only pages with sessions
Vast majority occur in October
500 pages remaining
Goals:
Are total number of fixed pages correlated to average number of sessions, bounce rate, or revenue?
Does average sessions/day, bounce rate/day, or revenue/day increase as mobile errors are fixed relative to randomized control sample?

20. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
Average Sessions/Day Vs. Total Number of Fixed Pages
Correlation coefficient (R) = 0.26
No linear correlation

21. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
Average Sessions/day over time for test group and control group
Control group comprised of 1000 randomly selected mobile landing pages with similar number of sessions but with unfixed errors
Control group linear fit has a slight negative slope
R = 0.32 (weak linear correlation)
Linear fit to test group also has slight negative slope
R = 0.30 (weak linear correlation)
NO statistically significant difference between correlation coefficient with control group (p = 0.95)
NO statistically significant difference between fit slope with control group (p = 0.82)

22. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
Average Bounce Rate/Day Vs. Total Number of Fixed Pages
Correlation coefficient (R) = 0.07
No linear correlation

23. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
Average Bounce Rate/day over time for test group and control group
Control group comprised of 1000 randomly selected mobile landing pages with similar number of sessions but with unfixed errors
Control group linear fit has a slope consistent with zero
R = 0.03 (no linear correlation)
• Linear fit to test group also has a slope consistent with zero
R = 0.17 (no linear correlation)
NO statistically significant difference between correlation coefficient with control group (p = 0.66)
NO statistically significant difference between fit slope with control group (p = 0.48)

24. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
Average Revenue/Day Vs. Total Number of Fixed Pages
Correlation coefficient (R) = 0.18
No linear correlation

25. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
Average Bounce Rate/day over time for test group and control group
Control group comprised of 1000 randomly selected mobile landing pages with similar number of sessions but with unfixed errors
Control group linear fit has a slope consistent with zero
R = 0.05 (no linear correlation)
• Linear fit to test group also has a slope consistent with zero
R = 0.02 (no linear correlation)
NO statistically significant difference between correlation coefficient with control group (p = 0.90)
NO statistically significant difference between fit slope with control group (p = 0.86)

26. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
Summary
Investigated correlation between sessions, bounce rate, or revenue vs. total number of fixed mobile
errors
In all cases, we found no linear correlation
• Investigated average sessions/day, bounce rate/day, or revenue/day increase as mobile errors
are fixed relative to randomized control sample
In all cases, the correlation coefficients and slopes of the test groups and control groups are
indistinguishable.

27. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
Does introduction of Google’s “Mobile Friendly Tag” increase CTR?
Does introduction of Google’s “Mobile Friendly Tag” (MFT) increase CTR?
MFT introduced on 11/18/2014
Sample includes 21 landing pages having traffic between 11/13/2014 and 11/30/2014 with a variance in average page
position of < 0.2 throughout that time
Average CTR before MFT: 13.5%
Average CTR AFTER MFT: 13.4%
DIFFERENCE IN AVERAGE CTR IS NOT STATISTICALLY SIGNIFICANT

28. searchmarketingexpo.com
@BrysonMeunier
#SMX #21D
Does introduction of Google’s “Mobile Friendly Tag” increase CTR?
Does introduction of Google’s “Mobile Friendly Tag” (MFT) increase CTR?
MFT introduced on 11/18/2014
Sample includes 21 landing pages having traffic between 11/13/2014 and 11/30/2014 with a variance in average page
position of < 0.2 throughout that time
Weighted (impressions) average CTR before MFT: 9.6%
Weighted (impressions) average CTR AFTER MFT: 9.5%
DIFFERENCE IN AVERAGE CTR IS NOT STATISTICALLY SIGNIFICANT