Michael is the Head of SEO at Skyscanner where he's responsible for SEO and content strategy on a global basis. Michael's worked across some of the world’s largest blue chip sites in a range of agency environments. In his role at Skyscanner he gets to indulge his love of tech, content and understanding what users and search engines want from websites to create business value. Michael is a self-confessed geek who looks at where markets are going to focus on long-term successes through increased market penetration.

1. Testing for Understanding SEO
Michael Briggs

2. My Mate Nick

3. My Mate Nick
“We used
to Be
Mavericks”

4. Not Like This!!!

5. We used to Spam at Scale

6. We used to Spam Creatively

8. Loser

9. Liar

10. Fake

11. Phoney

12. How we solve this?
• Scientific Method
• Build -> Measure -> Learn
• The accidental test

13. Scientific Method
Lets see how accurately I can get to an answer

14. Is Google Flights Hiving Off Traffic

15. It Must Be

16. Problem
• Obviously isn’t good enough
• I work for a billion dollar plus company
• Your clients probably need more as they work in the same way
• Need to get to a definitive answer
– Or die trying

17. The Scientific Method

18. Google Flights Problems
• I hadn’t chosen to do the test

19. Google Flights Problems
• I hadn’t chosen to do the test
• Google was running the test

20. Google Flights Problems
• I hadn’t chosen to do the test
• Google was running the test
• Need to then fill in the blanks

21. Google Flights Problems
• I hadn’t chosen to do the test
• Google was running the test
• Need to then fill in the blanks
• Was looking backwards

22. The Real World Hypothesis
Google Flights launched and it’s changed the
search landscape and is affecting my CTR
from Organic

23. My Real Thoughts on the Matter
Google Flights launched and it’s hiving off
traffic because Google are a new
technological overlord.
If we don’t stop them they will take over the
world.

24. Meh
• Problem
–Search Volumes are growing (and inaccurate)
–Google Webmaster Tools (I am not ever calling it a
“search console”) is awful
–There is a wealth of data
–Where can I find something useful
–Where should I be looking

25. Hypothesis Development
• New Hypothesis
– Google flights launched and it is hiving off traffic. If click through rate changes and
positions stay stable then there is a correlation between the launch of Google flights

26. Hypothesis Development
• New Hypothesis
– Google flights launched and it is hiving off traffic. If click through rate changes and
positions stay stable then there is a correlation between the launch of Google flights
– If I have stable variables I can remove these from analysis
• I know on term set X my positions haven’t changed
– I’m lucky we’ve been number one for a range of things for a while
– And find the variables that change
• Traffic by device
• Search volume by device (error factor)
• Implied variable of CTR

27. The Graph

28. Problems
• Not enough data on the keyword set
– Longtail
– Google Keyword Tool is crap
– Data gap
• There were multiple changes on the historical data set
– Hummingbird
– SEM Changes - Ad extensions
– Listing had go site links added to it
– Site in position two changed
• Positional Data from rank tracking isn’t accurate
– Personalisation

29. Conclusion

30. Forward Looking Test
I think, therefore I’ll test, so I know

31. Build Measure Learn
Build
MeasureLearn

32. Forward Looking
• I think this to be true
• I will create a hypothesis
• I will test it
• I will make a conclusion (hopefully)

33. Accepted Truth

34. Unique Content
• Hypothesis
–If I add unique content to a page it will make
me rank better in Google because Google loves
unique content therefore I need unique content.

35. How do I build this as a significant test
• Take two sets of pages
–Test
–Control
• Change content on the test
• Measure the difference in the normalised
traffic

36. Worlds Most Boring Graph

37. Test Results

38. Conclusion

39. Why
• Content writers weren’t good enough?
• Google is a liar and we’ve been lied to?
• The signal balance had changed in a way we didn’t
understand?
• Something else?

40. Why
• Content writers weren’t good enough?
–Build new hypothesis
• Google is a liar and we’ve been lied to?
• The signal balance had changed in a way we didn’t
understand?
• Something else?

41. We Rebuilt The Content

42. Test Outcome
Unique Content only helps if someone who
knows wtf they are doing actually read the
content and makes sure it is in the form it is
meant to be in and is properly optimised.
Unique content isn’t enough.

43. The Accidental Test
Fail Forward

44. Web Engineering

45. What it really can look like

46. Sometimes it looks like this!

47. SEO’s Regress to This

48. Because Overnight

49. What this meant was some days of hell!

50. Failure is Always an Option

52. Direct got Similar Hit
90% of Direct Traffic on pages examined was really Organic

53. • Fail Hard
• Fail Fast
• Fail Frequently
• Learn from every failure
#failForward

54. Thanks
michael.ke.briggs@gmail.com