This document provides a guide to using Screaming Frog and XPath to analyze competitor websites. It introduces XPath as a query language for selecting nodes from XML documents. Examples are given for analyzing BBC.com and TheGuardian.com by counting elements like images, headings, and links to specific pages using relevant XPath selectors. Further examples compare Yourcat.co.uk and TheFoodaholic.co.uk by counting internal and external links within editorial content. The document emphasizes identifying the necessary data and using the correct XPath selector for analysis.

1. Screaming Frog + Xpath:
A Guide to Analyse the Pants
Off Your Competition
Sabine Langmann // sabine-langmann.com //
@SabTheLa https://slideshare.net/sabinelangmann

2. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

3. Level 1
12.04.2019 Sabine Langmann bit.ly/sfx-2019

4. What is this about?
12.04.2019 Sabine Langmann bit.ly/sfx-2019

5. We‘d like: to crawl specific
elements on our own web pages
or the ones of our competition
We use: Screaming Frog‘s
Custom Extraction + XPath
12.04.2019 Sabine Langmann bit.ly/sfx-2019

6. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

7. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

8. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

9. Level 2
12.04.2019 Sabine Langmann bit.ly/sfx-2019

10. Who am I?
https://www.sabine-langmann.com
https://www.linkedin.com/in/sabine-langmann/
@SabTheLa
12.04.2019 Sabine Langmann
Slides: bit.ly/sfx-2019
bit.ly/sfx-2019

11. Level 3Level 3
12.04.2019 Sabine Langmann bit.ly/sfx-2019

12. Xpath
12.04.2019 Sabine Langmann bit.ly/sfx-2019

13. XPath (XML Path Language) is a
query language
for selecting nodes
from an XML document.
Wikipedia
12.04.2019 Sabine Langmann bit.ly/sfx-2019

14. Simple Syntax
node every page element (e.g. H2, a, p, div)
// adresses a certain node
attribute attribute of a node (e.g. class, id)
@ adresses a certain attribute
count() counts addressed nodes
12.04.2019 Sabine Langmann bit.ly/sfx-2019

15. Simple Syntax
12.04.2019 Sabine Langmann
//node[@attribute="attribute_name"]
bit.ly/sfx-2019

16. Simple Syntax
12.04.2019 Sabine Langmann
//node[@attribute1="attribute_name1" and
@attribute2="attribute_name2"]
bit.ly/sfx-2019

17. Simple Syntax
12.04.2019 Sabine Langmann
count(//node[@attribute="attribute_name"])
bit.ly/sfx-2019

18. Level 4
12.04.2019 Sabine Langmann bit.ly/sfx-2019

19. Examples
12.04.2019 Sabine Langmann bit.ly/sfx-2019

20. BBC.com
vs.
TheGuardian.com
12.04.2019 Sabine Langmann bit.ly/sfx-2019

21. BBC.com
12.04.2019 Sabine Langmann bit.ly/sfx-2019

22. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

23. 12.04.2019 Sabine Langmann
How many images?
How many H2, H3, etc?
How many words?
How many links to which pages?
bit.ly/sfx-2019

24. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

25. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

26. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

27. What am I searching for?
12.04.2019 Sabine Langmann bit.ly/sfx-2019

28. In
I‘m searching for text (name of
the topic tag)
12.04.2019 Sabine Langmann bit.ly/sfx-2019

29. Suitable Xpath selector:
//li[
@class="tags-list__tags"
and
@data-entityid="topic_link_bottom"
]
12.04.2019 Sabine Langmann bit.ly/sfx-2019

30. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

31. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

32. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

33. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

34. 12.04.2019 Sabine Langmann
https://www.bbc.com/news/uk-politics-.*
bit.ly/sfx-2019

35. 12.04.2019 Sabine Langmann
Don‘t forget ;)
bit.ly/sfx-2019

36. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

37. 12.04.2019 Sabine Langmann
Result
bit.ly/sfx-2019

38. 12.04.2019 Sabine Langmann Bit.ly/abjsd
Ain‘t nobody got time
for Excel. Better listen
to Ben!
12.04.2019 Sabine Langmann bit.ly/sfx-2019

39. TheGuardian.com
12.04.2019 Sabine Langmann bit.ly/sfx-2019

40. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

41. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

42. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

43. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

44. What am I searching for?
12.04.2019 Sabine Langmann bit.ly/sfx-2019

45. In
<div class="submeta">
I‘m searching for
the topic tag names
12.04.2019 Sabine Langmann bit.ly/sfx-2019

46. Suitable Xpath selectors:
//div[@class="submeta__section-labels"])
and
//div[@class="submeta__keywords"]
12.04.2019 Sabine Langmann bit.ly/sfx-2019

47. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

48. 12.04.2019 Sabine Langmann
https://www.theguardian.com/sitemaps/news.xml
bit.ly/sfx-2019

49. 12.04.2019 Sabine Langmann
Result
bit.ly/sfx-2019

50. Yourcat.co.uk
vs.
TheFoodaholic.co.uk
12.04.2019 Sabine Langmann bit.ly/sfx-2019

51. Yourcat.co.uk
12.04.2019 Sabine Langmann bit.ly/sfx-2019

52. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

53. 12.04.2019 Sabine Langmann
How many links in editorial content?
How many internal/external?
bit.ly/sfx-2019

54. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

55. 12.04.2019 Sabine Langmann
<div class=“post-body-container”>
bit.ly/sfx-2019

56. “How many links in editorial content?”
Suitable Xpath selector:
count(//div[@class="post-body-container"]//p//a)
12.04.2019 Sabine Langmann bit.ly/sfx-2019

57. “How many internal editorial links?”
Suitable Xpath selector:
count(//div[@class="post-body-container"]//p//a[
starts-with(@href, "https://www.yourcat.co.uk/")])
12.04.2019 Sabine Langmann bit.ly/sfx-2019

58. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

59. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

60. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

61. Result
12.04.2019 Sabine Langmann bit.ly/sfx-2019

62. TheFoodaholic.co.uk
12.04.2019 Sabine Langmann bit.ly/sfx-2019

63. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

64. 12.04.2019 Sabine Langmann
How many links in editorial content?
How many internal/external?
bit.ly/sfx-2019

65. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

66. 12.04.2019 Sabine Langmann bit.ly/sfx-2019

67. “How many links in editorial content?”
Suitable Xpath selector:
count(//div[@itemprop="articleBody"]//p
//a[not(contains(@href, "wp-content"))])
12.04.2019 Sabine Langmann bit.ly/sfx-2019

68. “How many internal editorial links?”
Xpath selector:
count(//div[@itemprop="articleBody"]//p
//a[starts-with(@href, "http://www.thefoodaholic.co.uk")
and not(contains(@href, "wp-content"))])
12.04.2019 Sabine Langmann bit.ly/sfx-2019

69. Result
12.04.2019 Sabine Langmann bit.ly/sfx-2019

70. Level 5
12.04.2019 Sabine Langmann bit.ly/sfx-2019

71. Recap:
Which data do I need?
Can I crawl the respective elements?
What is the right Xpath selector?
That‘s it!
12.04.2019 Sabine Langmann bit.ly/sfx-2019

72. More Xpath Cases,
SERP-Crawling, Regex,
…
12.04.2019 Sabine Langmann bit.ly/sfx-2019

73. 12.04.2019 Sabine Langmann
Wait no more. Max
shows how!
bit.ly/sfx-2019

74. 12.04.2019 Sabine Langmann bit.ly/sfx-2019
Thanks!!