Migration Best-Practices: Successfully re-launching your website - SMX New York 2017
Oct. 25, 2017•0 likes•2,425 views
Report
Technology
My talk from SMX 2017 in New York covering best practices on how to successfully naviate through the various types of migrations (protocal migrations, frontend migrations, etc.) from an SEO perspective.
3. #SMX #22A2 @peakaceag
Every (big) change also means (loads of) opportunity!
But always keep in mind: the price tag for failure is absolutely immense!
Risk Reward
User acceptance “Once in a lifetime” opportunity
Project complexity Greenfield project: question everything
Resilience/Freeze Best chance to really “get shit done”
Interruption Opportunity to eliminate “legacy problems”
Politics Usually more agile vs. regular, daily business
Performance Rethink: RWD/dynamic serving, HTTPS, URL design, etc.
4. #SMX #22A2 @peakaceag
Migration types and their potential impact on SEO
Keep in mind: Often these types overlap – or multiple things are done at once…
Inspired by @jonoalderson: http://pa.ag/2xUCMnJ
Type Example
Hosting migrations You’re changing hosting or CDN provider(s)
You’re changing, adding, or removing server locations
You’re changing your tech stack/caching/lbs
Software migrations You’re changing CMS (or it's version/plugins, etc.)
You’re changing the language used to render the website
You’re merging platforms; e.g., a blog which operated on a separate domain
Domain migrations You’re changing the main domain of your website
You’re buying/adding new domains/subdomains to your ecosystem
You’re moving a website, or part of a website, between domains
Template migrations You’re changing the layout/structure/navigation of important pages
You’re adding or removing template components
You’re changing elements in your code, like title, canonical, or hreflang tags
… and there are more: design migrations, strategy migrations, content migrations, protocol migrations
5. #SMX #22A2 @peakaceag
Make it a granular, multi-step approach
Doing everything at once will make debugging & rolling back an almost impossible task!
Source: http://pa.ag/2yJqT1N
6. #SMX #22A2 @peakaceag
Thorough documentation, in-depth definition of requirements
and on-going testing are absolutely essential!
BE CRAZY ABOUT DETAILS!
9. #SMX #22A2 @peakaceag
What should go, what can stay? Complete vs. partial site move?
DEFINE YOUR MIGRATION STRATEGY
10. #SMX #22A2 @peakaceag
Consider your crawl budget during migration
Important: How long does it take to re-crawl my site completely (in theory)?
▪ In order to implement the protocol change,
Google needs to crawl the complete domain
and retrieve all URLs.
▪ For a domain with e.g. 100,000 subpages,
Google would have to process 200,000 URLs
(redirects).
▪ On average, Google crawls 30,000 pages/day
(see screenshot) which means this would take at
least 7 days.
▪ However, this assumption is only “half the truth”,
as Google doesn't retrieve all URLs one after
another.
11. #SMX #22A2 @peakaceag
You will need an arsenal of tools!
A lot comes down to personal taste: you’ll need at least one tool each for crawling, log
files analysis as well as search intelligence – and yes, this usually costs money!
12. #SMX #22A2 @peakaceag
Pre-migration site health check & clean up
A properly optimized domain migrates much easier and more efficiently.
Getting your house in order prior to the move minimizes the risk of losing rankings.
▪ Google Search Console: manual actions, server
errors (DNS, 5XX response codes), Markup
validation errors (AMP, schema.org, rich cards),
robots.txt
▪ Web crawl: internal redirects as well as redirect
chains, broken URLs, and internal links
▪ Log files: broken URLs, suspicious status codes,
crawler traps
▪ Algo issues? Relocating with Panda, Penguin &
Co. makes very little sense...
14. #SMX #22A2 @peakaceag
Gather all URLs including static assets
Must haves: log files, XML sitemaps as well as a full website crawl
Extras: analytics (top ranking URLs and/or URLs generating the most traffic)
Mode > List > Upload
small
domains large domains
15. #SMX #22A2 @peakaceag
Monitor your keyword rankings daily
Check critical rankings of keywords. Making this a daily task ensures that you are
only working with the most recent data.
17. #SMX #22A2 @peakaceag
Let’s start with <head>: Canonical tags & rel-alternate’s
Remember to annotate your dedicated mobile site and to adapt your RSS feeds
Canonical tag
<link href="http://www.example.com/deep/url" rel="canonical" />
<link href="https://www.example.com/deep/url" rel="canonical" />
Old
New
<link rel="alternate" media="only screen and (max-width: 640px)" href="http://m.example.com/page-1" />
<link rel="alternate" media="only screen and (max-width: 640px)" href="https://m.example.com/page-1" />
Old
New
Alternate tag
18. #SMX #22A2 @peakaceag
Multilingual setup? Customize hreflang target URLs!
Individual countries can be moved to HTTPS one-by-one, as long as the circular
reference remains intact
Hreflang
<link rel="alternate" hreflang="de" href="http://www.example.com/de/" />
Old
<link rel="alternate" hreflang="fr" href="http://www.example.com/fr/" />
Old
<link rel="alternate" hreflang="de" href="https://www.example.com/de/" />
New
<link rel="alternate" hreflang="x-default" href="http://www.example.com/" />
Old
<link rel="alternate" hreflang="fr" href="https://www.example.com/fr/" />
New
<link rel="alternate" hreflang="x-default" href="https://www.example.com/" />
New
19. #SMX #22A2 @peakaceag
Rel=next/prev pagination & Accelerated Mobile Pages
Are you using Google's recommendation for pagination? Or AMPs?
Pagination
<link href="http://www.example.com/deep/url?page=1" rel="prev" />
<link href="https://www.example.com/deep/url?page=1" rel="prev" />
Old
New
<link href="http://www.example.com/deep/url?page=3" rel="next" />
Old
<link href="https://www.example.com/deep/url?page=3" rel="next" />
New
AMP
<link rel="amphtml" href="http://www.example.com/amp/" />
<link rel="amphtml" href="https://www.example.com/amp/" />
Old
New
20. #SMX #22A2 @peakaceag
Update internal links
Simply relying on redirects is not a migration strategy
▪ Links to other internal
URLs
▪ Links to internal image
files
▪ Links to internal video
files
▪ Links to internal web
fonts
▪ Links to internal
JavaScript files
▪ Links to internal CSS
files
JS
HTML
Source code
21. #SMX #22A2 @peakaceag
Update internal links (within JavaScript files)
Simply relying on redirects is not a migration strategy
▪ Links to other internal
URLs
▪ Links to internal CSS
files
JAVASCRIPT
Files
▪ Links to internal image
files
22. #SMX #22A2 @peakaceag
Update internal links (within CSS files)
Simply relying on redirects is not a migration strategy
▪ Links to internal image
files
▪ Links to internal
web fonts
CSS
Files
▪ Any other internal links
23. #SMX #22A2 @peakaceag
Careful with internal redirects!
Avoid redirect chains: old HTTP URLs should lead directly to the new HTTPS URL
Source: Redirect Chain Report via DeepCrawl
24. #SMX #22A2 @peakaceag
Update XML sitemaps #1
Important: if you are using the sitemap index file, you need to update it.
If you reference the sitemap in robots.txt, the URL has to be changed there as well.
<url>
<loc>http://www.example.com/</loc>
<xhtml:link rel="alternate" hreflang="x-default" href="http://www.example.com/" />
<xhtml:link rel="alternate" hreflang=“de" href="http://www.example.com/de/" />
<xhtml:link rel="alternate" hreflang="fr" href="http://www.example.com/fr/" />
<xhtml:link rel="alternate" media="only screen and (max-width: 640px)" href="http://m.example.com/" />
<image:image>
<image:loc>http://www.example.com/i-am-an-image.jpg</image:loc>
</image:image>
<video:video>
<video:content_loc>http://www.example.com/i-am-a-video.flv</video:content_loc>
</video:video>
</url>
1
2
3
4
5
6
7
8
9
10
11
12
13
Old
25. #SMX #22A2 @peakaceag
Update XML sitemaps #2
Important: if you are using the sitemap index file, you need to update it.
If you reference the sitemap in robots.txt, the URL has to be changed there as well.
<url>
<loc>https://www.example.com/</loc>
<xhtml:link rel="alternate" hreflang="x-default" href="https://www.example.com/" />
<xhtml:link rel="alternate" hreflang=“de" href="https://www.example.com/de/" />
<xhtml:link rel="alternate" hreflang="fr" href="https://www.example.com/fr/" />
<xhtml:link rel="alternate" media="only screen and (max-width: 640px)" href="https://m.example.com/" />
<image:image>
<image:loc>https://www.example.com/i-am-an-image.jpg</image:loc>
</image:image>
<video:video>
<video:content_loc>https://www.example.com/i-am-a-video.flv</video:content_loc>
</video:video>
</url>
1
2
3
4
5
6
7
8
9
10
11
12
13
NewNew
26. #SMX #22A2 @peakaceag
Structured data update (schema.org)
Update your schema.org Markup references.
Caution: The short annotation “//schema.org” currently does not validate!
<script type="application/ld+json"> {
"@context": "http://schema.org", "@type": "Product", "aggregateRating": { "@type": "AggregateRating", "ratingValue": "4.5", "reviewCount":
"100" },
"description": “here goes the product description",
"name": “this is the product name",
"image": "http://www.example.com/i-am-the-image.jpg",
"offers": { "@type": "Offer", "availability": "http://schema.org/InStock", "price": "55.00", "priceCurrency": "EUR" }
} </script>
Old
1
2
3
4
5
6
7
8
<script type="application/ld+json"> {
"@context": "https://schema.org", "@type": "Product", "aggregateRating": { "@type": "AggregateRating", "ratingValue": "4.5",
"reviewCount": "100" },
"description": " here goes the product description ",
"name": " this is the product name ",
"image": "https://www.example.com/i-am-the-image.jpg",
"offers": { "@type": "Offer", "availability": "https://schema.org/InStock", "price": "55.00", "priceCurrency": "EUR" }
} </script>
New
1
2
3
4
5
6
7
8
27. #SMX #22A2 @peakaceag
Update CDN settings and resource hints
Requests for assets on a CDN need to be on HTTPS now, also any resource hints
(preconnect, dns-prefetch, etc.) need to be changed
<script src="http://ajax.googleapis.com/ajax/libs/jquery/3.1.0/jquery.min.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.0/jquery.min.js"></script>
Old
New
<script src="//ajax.googleapis.com/ajax/libs/jquery/3.1.0/jquery.min.js"></script>
CDN
Alternatively
Resource hints
<link rel="preconnect" href="http://cdn.example.com" pr="0.42">
<link rel="preconnect" href=" https:// cdn.example.com" pr="0.42">
Old
New
31. #SMX #22A2 @peakaceag
Inventory Sync: Crawl vs. XML sitemaps
Compare both inventories to ensure the same content is available on HTTPS as on the
HTTP original (HTTP vs. HTTPS e.g. via ScreamingFrog+Excel or DeepCrawl)
32. #SMX #22A2 @peakaceag
Make sure there are no more HTTP links
Before migrating, check that there are no more HTTP requests on the HTTPS system
33. #SMX #22A2 @peakaceag
Really helpful: HTTP’s specific crawl output
DeepCrawl also reports HSTS headers and non secure forms
35. #SMX #22A2 @peakaceag
Create HTTPS properties & merge them into a set
Properties for HTTP and HTTPS with and without www = 4 domains
If you use separate mobile domains, there are 6 domains in total!
Prerequisites for a migration:
▪ No manual actions
▪ No significant amount of crawl errors
(DNS, availability, etc.)
▪ No problems with the XML sitemaps
▪ Valid structured data / rich cards markup,
no significant AMP and hreflang errors
http://xxxxxxx.de/
https://xxxxxxx.de/
http://www.xxxxxxx.de/
https://www.xxxxxxx.de/
https://www.xxxxxxx.de/
http://www.xxxxx.de/
36. #SMX #22A2 @peakaceag
Transfer disavow links file in time
Especially for domains with a “questionable” link profile: GSC setup and disavow file
transfer should be done approximately 48 hours prior to going live!
37. #SMX #22A2 @peakaceag
Set your preferred domain and (if necessary) crawl rate
You don’t really need to set the crawl rate, but, if you have done so previously, I would
suggest transferring the original settings (for now)
38. #SMX #22A2 @peakaceag
Configure Geo-Targeting for gTLDs
Synchronize settings for handling URL parameters if necessary (i.e. for “.com” gTLDs)
39. #SMX #22A2 @peakaceag
Before you send to the index: test GSCs fetch & render!
Make sure Google includes and displays all requested components properly!
40. #SMX #22A2 @peakaceag
Test and re-submit all XML sitemaps
Also: synchronize URL parameter settings if you were using them or if they are needed
for your site’s functionality
42. #SMX #22A2 @peakaceag
After all this preparation, all you need to do now is to "just" redirect URLs.
301-REDIRECT ALL THE THINGS!
43. #SMX #22A2 @peakaceag
Even if, as some people say, all redirects are equal:
All we did for this client was change the (chained) 302/307 to 301 redirects!
44. #SMX #22A2 @peakaceag
Keep in mind: this only works for domain migrations!
USE THE GSC SITE MOVE FEATURE
45. #SMX #22A2 @peakaceag
Crawl old URLs again (301’s now) & manual SERP checks
Import HTTP URLs e.g. to ScreamingFrog (list mode), only 301s should appear now.
Additional manual check of indexed URLs e.g. via LinkClump Add-On in Chrome
1 3
4
2
53. #SMX #22A2 @peakaceag
Ideally it should look like this #1
Domain migration (“de” to “com”), no other changes; fully recovered after 5 weeks!
Source: Searchmetrics
54. #SMX #22A2 @peakaceag
Ideally it should look like this #2
Protocol migration: from HTTP to HTTPS; 2 weeks visibility dip (-25 %)
Source: Searchmetrics
56. #SMX #22A2 @peakaceag
#1 GSCs DNS verification can be pretty helpful
No need to worry about missing meta tags; plus you can also verify prior to actually
deploying a site/frontend to a new domain – and it is faster as well!
More: http://pa.ag/2yJ7xtH
57. #SMX #22A2 @peakaceag
#2 Also move your robots.txt file
When changing domains, make sure to transfer (the contents of) robots.txt as well!
58. #SMX #22A2 @peakaceag
#3 Don’t forget to redirect your images as well
When changing URLs/domains, make sure to implement redirect rules for images
Read the entire post: http://pa.ag/2yJtTLz
59. #SMX #22A2 @peakaceag
#4 Bulk testing all the things: mobile
Mobile-friendliness at scale: technicalseo.com
Check it out: https://technicalseo.com/seo-tools/mobile-friendly/
60. #SMX #22A2 @peakaceag
#5 Bulk testing all the things: hreflang tags
hreflang tags (in sitemaps) at scale: technicalseo.com
Check it out: https://technicalseo.com/seo-tools/hreflang/
61. #SMX #22A2 @peakaceag
#6 Bulk testing all the things: redirects & other headers
HTTP status codes (errors, redirects, etc.) at scale: httpstatus.io
Check it out: https://httpstatus.io/
62. #SMX #22A2 @peakaceag
#7 Simulate Googlebot for smartphone with JS-rendering
ScreamingFrog can do that easily at scale; again pay close attention to rendered
output! Also pretty cool: Extract > Xpath > //head/link[@rel="amphtml"]/@href
63. #SMX #22A2 @peakaceag
#8 HTTP 503 is your friend
Combine with “revisit-after” to throttle crawling; never use “noindex”/4xx instead!
Source: http://pa.ag/2xRiA5T
Webmasters should return a 503 HTTP header for all
the URLs participating in the blackout […] Googlebot's
crawling rate will drop when it sees a spike in 503 […]
as Googlebot is currently configured, it will halt all
crawling of the site if the site’s robots.txt file returns a
503 status code for robots.txt.
„
64. #SMX #22A2 @peakaceag
#9 Fix those redirect chains, especially on legacy sites…
…as multiple requests waste valuable performance and crawl budget!
65. #SMX #22A2 @peakaceag
#10 Careful: JavaScript frameworks are still tricky!
Check out Bartosz‘ massive research on crawlability and indexability!
Read more: http://pa.ag/2qLqlqH
66. #SMX #22A2 @peakaceag
https://pa.ag/smx17mig
Always looking for talent! Check out jobs.pa.ag
Bastian Grimm
bg@pa.ag
twitter.com/peakaceag
facebook.com/peakaceag
www.pa.ag
Found it useful? Here is the deck: