Migration Best-Practices: Successfully re-launching your website - SMX New York 2017

#SMX #22A2 @peakaceag
MIGRATION
BEST-PRACTICES:
Successfully re-
launching your
website
Key ingredients: Nerves of steel, loads of coffee & this deck!

NEVER CHANGE A RUNNING SYSTEM!

Every (big) change also means (loads of) opportunity!
But always keep in mind: the price tag for failure is absolutely immense!
Risk Reward
User acceptance “Once in a lifetime” opportunity
Project complexity Greenfield project: question everything
Resilience/Freeze Best chance to really “get shit done”
Interruption Opportunity to eliminate “legacy problems”
Politics Usually more agile vs. regular, daily business
Performance Rethink: RWD/dynamic serving, HTTPS, URL design, etc.

Migration types and their potential impact on SEO
Keep in mind: Often these types overlap – or multiple things are done at once…
Inspired by @jonoalderson: http://pa.ag/2xUCMnJ
Type Example
Hosting migrations You’re changing hosting or CDN provider(s)
You’re changing, adding, or removing server locations
You’re changing your tech stack/caching/lbs
Software migrations You’re changing CMS (or it's version/plugins, etc.)
You’re changing the language used to render the website
You’re merging platforms; e.g., a blog which operated on a separate domain
Domain migrations You’re changing the main domain of your website
You’re buying/adding new domains/subdomains to your ecosystem
You’re moving a website, or part of a website, between domains
Template migrations You’re changing the layout/structure/navigation of important pages
You’re adding or removing template components
You’re changing elements in your code, like title, canonical, or hreflang tags
… and there are more: design migrations, strategy migrations, content migrations, protocol migrations

Make it a granular, multi-step approach
Doing everything at once will make debugging & rolling back an almost impossible task!
Source: http://pa.ag/2yJqT1N

Thorough documentation, in-depth definition of requirements
and on-going testing are absolutely essential!
BE CRAZY ABOUT DETAILS!

…from someone who has successfully done this type of work before
ASK FOR HELP…

PLANNING & PREPARATION

What should go, what can stay? Complete vs. partial site move?
DEFINE YOUR MIGRATION STRATEGY

Consider your crawl budget during migration
Important: How long does it take to re-crawl my site completely (in theory)?
▪ In order to implement the protocol change,
Google needs to crawl the complete domain
and retrieve all URLs.
▪ For a domain with e.g. 100,000 subpages,
Google would have to process 200,000 URLs
(redirects).
▪ On average, Google crawls 30,000 pages/day
(see screenshot) which means this would take at
least 7 days.
▪ However, this assumption is only “half the truth”,
as Google doesn't retrieve all URLs one after
another.

You will need an arsenal of tools!
A lot comes down to personal taste: you’ll need at least one tool each for crawling, log
files analysis as well as search intelligence – and yes, this usually costs money!

Pre-migration site health check & clean up
A properly optimized domain migrates much easier and more efficiently.
Getting your house in order prior to the move minimizes the risk of losing rankings.
▪ Google Search Console: manual actions, server
errors (DNS, 5XX response codes), Markup
validation errors (AMP, schema.org, rich cards),
robots.txt
▪ Web crawl: internal redirects as well as redirect
chains, broken URLs, and internal links
▪ Log files: broken URLs, suspicious status codes,
crawler traps
▪ Algo issues? Relocating with Panda, Penguin &
Co. makes very little sense...

Be careful with broken pages & broken internal linking

Gather all URLs including static assets
Must haves: log files, XML sitemaps as well as a full website crawl
Extras: analytics (top ranking URLs and/or URLs generating the most traffic)
Mode > List > Upload
small
domains large domains

Monitor your keyword rankings daily
Check critical rankings of keywords. Making this a daily task ensures that you are
only working with the most recent data.

PROTOCOL MIGRATION

Let’s start with <head>: Canonical tags & rel-alternate’s
Remember to annotate your dedicated mobile site and to adapt your RSS feeds
Canonical tag
<link href="http://www.example.com/deep/url" rel="canonical" />
<link href="https://www.example.com/deep/url" rel="canonical" />
Old
New
<link rel="alternate" media="only screen and (max-width: 640px)" href="http://m.example.com/page-1" />
<link rel="alternate" media="only screen and (max-width: 640px)" href="https://m.example.com/page-1" />
Old
New
Alternate tag

Multilingual setup? Customize hreflang target URLs!
Individual countries can be moved to HTTPS one-by-one, as long as the circular
reference remains intact
Hreflang
<link rel="alternate" hreflang="de" href="http://www.example.com/de/" />
Old
<link rel="alternate" hreflang="fr" href="http://www.example.com/fr/" />
Old
<link rel="alternate" hreflang="de" href="https://www.example.com/de/" />
New
<link rel="alternate" hreflang="x-default" href="http://www.example.com/" />
Old
<link rel="alternate" hreflang="fr" href="https://www.example.com/fr/" />
New
<link rel="alternate" hreflang="x-default" href="https://www.example.com/" />
New

Rel=next/prev pagination & Accelerated Mobile Pages
Are you using Google's recommendation for pagination? Or AMPs?
Pagination
<link href="http://www.example.com/deep/url?page=1" rel="prev" />
<link href="https://www.example.com/deep/url?page=1" rel="prev" />
Old
New
<link href="http://www.example.com/deep/url?page=3" rel="next" />
Old
<link href="https://www.example.com/deep/url?page=3" rel="next" />
New
AMP
<link rel="amphtml" href="http://www.example.com/amp/" />
<link rel="amphtml" href="https://www.example.com/amp/" />
Old
New

Update internal links
Simply relying on redirects is not a migration strategy
▪ Links to other internal
URLs
▪ Links to internal image
files
▪ Links to internal video
files
▪ Links to internal web
fonts
▪ Links to internal
JavaScript files
▪ Links to internal CSS
files
JS
HTML
Source code

Update internal links (within JavaScript files)
▪ Links to other internal
URLs
▪ Links to internal CSS
files
JAVASCRIPT
Files
files

Update internal links (within CSS files)
files
▪ Links to internal
web fonts
CSS
Files
▪ Any other internal links

Careful with internal redirects!
Avoid redirect chains: old HTTP URLs should lead directly to the new HTTPS URL
Source: Redirect Chain Report via DeepCrawl

Update XML sitemaps #1
Important: if you are using the sitemap index file, you need to update it.
If you reference the sitemap in robots.txt, the URL has to be changed there as well.
<url>
<loc>http://www.example.com/</loc>
<xhtml:link rel="alternate" hreflang="x-default" href="http://www.example.com/" />
<xhtml:link rel="alternate" hreflang=“de" href="http://www.example.com/de/" />
<xhtml:link rel="alternate" hreflang="fr" href="http://www.example.com/fr/" />
<xhtml:link rel="alternate" media="only screen and (max-width: 640px)" href="http://m.example.com/" />
<image:image>
<image:loc>http://www.example.com/i-am-an-image.jpg</image:loc>
</image:image>
<video:video>
<video:content_loc>http://www.example.com/i-am-a-video.flv</video:content_loc>
</video:video>
</url>
1
2
3
4
5
6
7
8
9
10
11
12
13
Old

Update XML sitemaps #2
Important: if you are using the sitemap index file, you need to update it.
If you reference the sitemap in robots.txt, the URL has to be changed there as well.
<url>
<loc>https://www.example.com/</loc>
<xhtml:link rel="alternate" hreflang="x-default" href="https://www.example.com/" />
<xhtml:link rel="alternate" hreflang=“de" href="https://www.example.com/de/" />
<xhtml:link rel="alternate" hreflang="fr" href="https://www.example.com/fr/" />
<xhtml:link rel="alternate" media="only screen and (max-width: 640px)" href="https://m.example.com/" />
<image:image>
<image:loc>https://www.example.com/i-am-an-image.jpg</image:loc>
</image:image>
<video:video>
<video:content_loc>https://www.example.com/i-am-a-video.flv</video:content_loc>
</video:video>
</url>
1
2
3
4
5
6
7
8
9
10
11
12
13
NewNew

Structured data update (schema.org)
Update your schema.org Markup references.
Caution: The short annotation “//schema.org” currently does not validate!
<script type="application/ld+json"> {
"@context": "http://schema.org", "@type": "Product", "aggregateRating": { "@type": "AggregateRating", "ratingValue": "4.5", "reviewCount":
"100" },
"description": “here goes the product description",
"name": “this is the product name",
"image": "http://www.example.com/i-am-the-image.jpg",
"offers": { "@type": "Offer", "availability": "http://schema.org/InStock", "price": "55.00", "priceCurrency": "EUR" }
} </script>
Old
1
2
3
4
5
6
7
8
<script type="application/ld+json"> {
"@context": "https://schema.org", "@type": "Product", "aggregateRating": { "@type": "AggregateRating", "ratingValue": "4.5",
"reviewCount": "100" },
"description": " here goes the product description ",
"name": " this is the product name ",
"image": "https://www.example.com/i-am-the-image.jpg",
"offers": { "@type": "Offer", "availability": "https://schema.org/InStock", "price": "55.00", "priceCurrency": "EUR" }
} </script>
New
1
2
3
4
5
6
7
8

Update CDN settings and resource hints
Requests for assets on a CDN need to be on HTTPS now, also any resource hints
(preconnect, dns-prefetch, etc.) need to be changed
<script src="http://ajax.googleapis.com/ajax/libs/jquery/3.1.0/jquery.min.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.0/jquery.min.js"></script>
Old
New
<script src="//ajax.googleapis.com/ajax/libs/jquery/3.1.0/jquery.min.js"></script>
CDN
Alternatively
Resource hints
<link rel="preconnect" href="http://cdn.example.com" pr="0.42">
<link rel="preconnect" href=" https:// cdn.example.com" pr="0.42">
Old
New

Update HTTP header & customize cookie settings
If applicable, customize X-Robots header tags. Use Chrome DevTools!
X-Robots rel-canonical via Apache .htaccess
<Files theRequestedFile.pdf >
Header add Link 'https://www.example.com/ebook.html; rel="canonical"'
</Files>
Other HTTP header alternatives:
▪ Link: <https://www.example.com/de/>; rel="alternate"; hreflang="de"
▪ Link: <https://www.example.com/someurl>; rel="canonical"
▪ Link: <https://cdn.example.com>; rel="dns-prefetch"

You need to find every single reference to
your old HTTP URLs!

Better safe than sorry: let’s test crawl the new HTTPS test version
EVERYTHING UPDATED?

Inventory Sync: Crawl vs. XML sitemaps
Compare both inventories to ensure the same content is available on HTTPS as on the
HTTP original (HTTP vs. HTTPS e.g. via ScreamingFrog+Excel or DeepCrawl)

Make sure there are no more HTTP links
Before migrating, check that there are no more HTTP requests on the HTTPS system

Really helpful: HTTP’s specific crawl output
DeepCrawl also reports HSTS headers and non secure forms

WORK YOUR SEARCH CONSOLE!

Create HTTPS properties & merge them into a set
Properties for HTTP and HTTPS with and without www = 4 domains
If you use separate mobile domains, there are 6 domains in total!
Prerequisites for a migration:
▪ No manual actions
▪ No significant amount of crawl errors
(DNS, availability, etc.)
▪ No problems with the XML sitemaps
▪ Valid structured data / rich cards markup,
no significant AMP and hreflang errors
http://xxxxxxx.de/
https://xxxxxxx.de/
http://www.xxxxxxx.de/
https://www.xxxxxxx.de/
https://www.xxxxxxx.de/
http://www.xxxxx.de/

Transfer disavow links file in time
Especially for domains with a “questionable” link profile: GSC setup and disavow file
transfer should be done approximately 48 hours prior to going live!

Set your preferred domain and (if necessary) crawl rate
You don’t really need to set the crawl rate, but, if you have done so previously, I would
suggest transferring the original settings (for now)

Configure Geo-Targeting for gTLDs
Synchronize settings for handling URL parameters if necessary (i.e. for “.com” gTLDs)

Before you send to the index: test GSCs fetch & render!
Make sure Google includes and displays all requested components properly!

Test and re-submit all XML sitemaps
Also: synchronize URL parameter settings if you were using them or if they are needed
for your site’s functionality

WOHOOO… LETS DO IT!

After all this preparation, all you need to do now is to "just" redirect URLs.
301-REDIRECT ALL THE THINGS!

Even if, as some people say, all redirects are equal:
All we did for this client was change the (chained) 302/307 to 301 redirects!

Keep in mind: this only works for domain migrations!
USE THE GSC SITE MOVE FEATURE

Crawl old URLs again (301’s now) & manual SERP checks
Import HTTP URLs e.g. to ScreamingFrog (list mode), only 301s should appear now.
Additional manual check of indexed URLs e.g. via LinkClump Add-On in Chrome
1 3
4
2

POST MIGRATION TODOS

Focus on 4XX and 5XX status codes first, tackle those “live”!
LOG FILE, GSC & GA ERROR MONITORING

Migration performance monitoring #1
HTTP error status code monitoring (e.g. 40X for Googlebot & Bingbot)

HTTP redirects over time and split by user-agent containing *bot*

Top crawled pages breakdown (daily & weekly)

AMP, structured data and rich cards error monitoring

Keep an eye on the crawl frequency!

Ideally it should look like this #1
Domain migration (“de” to “com”), no other changes; fully recovered after 5 weeks!
Source: Searchmetrics

Ideally it should look like this #2
Protocol migration: from HTTP to HTTPS; 2 weeks visibility dip (-25 %)
Source: Searchmetrics

#10 TIPS TO MAKE YOUR LIFE EASIER

#1 GSCs DNS verification can be pretty helpful
No need to worry about missing meta tags; plus you can also verify prior to actually
deploying a site/frontend to a new domain – and it is faster as well!
More: http://pa.ag/2yJ7xtH

#2 Also move your robots.txt file
When changing domains, make sure to transfer (the contents of) robots.txt as well!

#3 Don’t forget to redirect your images as well
When changing URLs/domains, make sure to implement redirect rules for images
Read the entire post: http://pa.ag/2yJtTLz

#4 Bulk testing all the things: mobile
Mobile-friendliness at scale: technicalseo.com
Check it out: https://technicalseo.com/seo-tools/mobile-friendly/

#5 Bulk testing all the things: hreflang tags
hreflang tags (in sitemaps) at scale: technicalseo.com
Check it out: https://technicalseo.com/seo-tools/hreflang/

#6 Bulk testing all the things: redirects & other headers
HTTP status codes (errors, redirects, etc.) at scale: httpstatus.io
Check it out: https://httpstatus.io/

#7 Simulate Googlebot for smartphone with JS-rendering
ScreamingFrog can do that easily at scale; again pay close attention to rendered
output! Also pretty cool: Extract > Xpath > //head/link[@rel="amphtml"]/@href

#8 HTTP 503 is your friend
Combine with “revisit-after” to throttle crawling; never use “noindex”/4xx instead!
Source: http://pa.ag/2xRiA5T
Webmasters should return a 503 HTTP header for all
the URLs participating in the blackout […] Googlebot's
crawling rate will drop when it sees a spike in 503 […]
as Googlebot is currently configured, it will halt all
crawling of the site if the site’s robots.txt file returns a
503 status code for robots.txt.
„

#9 Fix those redirect chains, especially on legacy sites…
…as multiple requests waste valuable performance and crawl budget!

#10 Careful: JavaScript frameworks are still tricky!
Check out Bartosz‘ massive research on crawlability and indexability!
Read more: http://pa.ag/2qLqlqH

https://pa.ag/smx17mig
Always looking for talent! Check out jobs.pa.ag
Bastian Grimm
bg@pa.ag
twitter.com/peakaceag
facebook.com/peakaceag
www.pa.ag
Found it useful? Here is the deck:

Migration Best-Practices: Successfully re-launching your website - SMX New York 2017

More Related Content

What's hot

Viewers also liked

Similar to Migration Best-Practices: Successfully re-launching your website - SMX New York 2017

More from Bastian Grimm

Recently uploaded

Migration Best-Practices: Successfully re-launching your website - SMX New York 2017