This deck is from my BrightonSEO talk dated 10/09 on a technical solution to content duplication. This covers the types of content duplication and the solutions and where these are most effectively attributed depending on circumstance.

1. A Technical Solution To
Content Duplication
Sophie Brannon
Absolute Digital Media
Head of SEO
@SophieBrannon

2. @SophieBrannon 2

3. Duplicate content refers to
content that is the same or
very similar across the same
domain or multiple

4. @SophieBrannon 4
Same Domain Duplicate
Content Can Include…

5. @SophieBrannon 5
URL Variations

6. @SophieBrannon 6
HTTP & HTTPS

7. @SophieBrannon 7
Boilerplate Content

8. @SophieBrannon 8
Cross-Domain
Duplicate Content Can
Include…

9. @SophieBrannon 9
Scraped Content

10. @SophieBrannon 10
Website
Migrations

11. @SophieBrannon 11
Competitors

12. @SophieBrannon 12
Duplicate content
won’t lead you into
a penalty

13. @SophieBrannon 13
But…

14. @SophieBrannon 14
It can significantly
hurt your search
rankings

15. @SophieBrannon 15
Search
engines won’t
always know
which page to
rank

16. @SophieBrannon 16
Search engines won’t
know how to split
authority

17. @SophieBrannon 17
And they won’t know which
version of a page to show for
a relevant search term

18. @SophieBrannon 18
If content duplication is so
bad, then why does it
happen?

19. @SophieBrannon 19
Up to 29% of the internet is
duplicate content

20. @SophieBrannon 20
And most of it is a
complete accident!

21. @SophieBrannon 21
URL variations are one of
the common causes of
content duplication

22. @SophieBrannon 22
URL variations include:
● Click tracking & analytics codes
● Session IDs
● Printer-friendly URLs

23. @SophieBrannon 23
But there are also lots of other
types of common content
duplication causes

24. @SophieBrannon 24
How To Fix Your Content
Duplication Issues

25. @SophieBrannon 25

26. @SophieBrannon 26

27. @SophieBrannon 27

28. @SophieBrannon 28
There are a number of different
solutions to consider but
understanding the reason the
issue exists will help you to find
the best solution rather than a
blanket fix

29. @SophieBrannon 29
Canonicalisation

30. @SophieBrannon 30
Questions to consider...

31. @SophieBrannon 31
Are the pages exact
duplicates?

32. @SophieBrannon 32
Is one of the pages
generating more traffic /
has more visibility?

33. @SophieBrannon 33
Does the page offer
additional value that
may not translate to SEO
value?

34. @SophieBrannon 34
If you answered yes,
yes, yes….
Then canonicalisation
may be your best bet.

35. @SophieBrannon 35
No Index

36. @SophieBrannon 36
Questions to consider...

37. @SophieBrannon 37
Are your crawl stats
suggesting Google’s
wasting a lot of valuable
time crawling these
pages?

38. @SophieBrannon 38
Do you need these
pages showing in
Google search results?

39. @SophieBrannon 39
Does the page offer
valuable information to
users?

40. @SophieBrannon 40

41. @SophieBrannon 41
If you answered yes, no,
yes, then you may want to
noindex if
canonicalisation isn’t an
option

42. @SophieBrannon 42
Redirects

43. @SophieBrannon 43
Questions to consider...

44. @SophieBrannon 44
Does the page need to
exist at all?

45. @SophieBrannon 45
If the answer is no, then
redirect it!

46. @SophieBrannon 46
Rewrites
Questions to ask....

47. @SophieBrannon 47

48. @SophieBrannon 48
Can you target the page
with a new search
intent?

49. @SophieBrannon 49
Do you have the
resource to rewrite?

50. @SophieBrannon 50
If you answered yes to
both, then rewrites may
be the better option.

51. @SophieBrannon 51
Or should you just live
with it?

52. @SophieBrannon 52

53. @SophieBrannon 53
If you can implement a
resolution, then that is
often better choice for
long-term SEO success

54. @SophieBrannon 54
Redirects
• HTTPS / HTTP domains
• Pages that are not valuable, are
outdated and irrelevant
• Non-www / www. versions

55. @SophieBrannon 55
Canonicalisation
● Exact duplicate pages that offer user value
and so need to remain
● Pages that cannot be rewritten

56. @SophieBrannon 56
Canonicalisation
● One page generates more traffic /
visibility than the other
● You can’t redirect because of technical
restrictions

57. @SophieBrannon 57
Content Rewrites
● You can target different key terms and
search intent within the copy
● You have the resource to implement

58. @SophieBrannon 58
NoIndex
● If you absolutely need to keep the page,
but it holds no SEO value.
● Bots are wasting valuable crawl budget &
301 redirects aren’t an option.

59. @SophieBrannon 59
Some other
considerations
for content
duplication

60. @SophieBrannon 60
Block crawling of
parameterized duplicate
content with the URL
Parameter Tool

61. @SophieBrannon 61
Keep your internal
linking consistent
/page/
/page
/page/index.html

62. @SophieBrannon 62
For country-
specific content,
Google advises the
use of CCTLD’s &
hreflang.

63. @SophieBrannon 63
Avoid the resource issues
with content automation
using OpenAI & GPT-3

64. Thank You
Follow me - @SophieBrannon

65. @SophieBrannon 65
Resources
https://developers.google.com/search/blog/2009/12/handling-legitimate-cross-domain
https://developers.google.com/search/blog/2009/10/reunifying-duplicate-content-on-your
https://www.google.com/webmasters/tools/crawl-url-parameters
https://twitter.com/BillieGeena/status/1428059594144817157
https://twitter.com/SophieBrannon/status/1427905326683148290