SEARCH Y - Bastian Grimm - Migrations Best Practices

Disney’s Newport Bay Club®
L’Événement Search Marketing
PARIS 2019

pa.ag@peakaceag3
And who is Bastian Grimm?
Born in ’83, online since ’99 - and in love with SEO since 2001.
CEO of Peak Ace AG, based in Berlin (DE). Frequent traveler and proud dad to Isabella.

#1 Never change a running system!

pa.ag@peakaceag5
Every (big) change brings (lots of) opportunities!
But always keep in mind: the price tag for failure is immense!
Risk Reward
User acceptance “Once in a lifetime” opportunity
Project complexity Greenfield project: question everything
Resilience/freeze Best chance to really “get shit done”
Interruption Opportunity to eliminate “legacy problems”
Politics Usually more agile vs. regular, daily business
Performance Rethink RWD/dynamic serving, HTTPS, URL design, etc.

pa.ag@peakaceag6
What are your goals for migration?
The right mindset is really, really important!
I want to lose as little as
possible!” – isn’t really a
good migration objective!

Thorough documentation, in-depth definitions of requirements
and ongoing testing are essential!
Be crazy about the details

…from someone who has successfully
done this type of work before.
Ask for help…

The sooner your (SEO) team is involved, the better: making sure everyone
is on board, increasing time for prep work & more thorough planning, etc.
Get in (very) early

What should go, what can stay? Complete vs. partial site move?
Define your migration strategy

Use one of your smaller markets (revenue-wise) as a guinea pig
to test the process and gather real-world feedback.
Multinational setup?

Find a timeframe – which includes enough of a buffer – when business
in general is slower; also consider extraordinary (seasonal) events.
Migrate during a slow period

Rushing the process will guarantee failure! Ensure there's enough time for
Q&A, feedback loops and necessary changes to be implemented!
Be realistic about deadlines

pa.ag@peakaceag15
Pre-migration: site health check & clean up
A properly optimised domain migrates easier and more efficiently. Getting your house
in order before the move minimises the risk of losing beneficial rankings.
 Google Search Console: manual actions, server
errors (DNS, 4xx and 5xx response codes), mark-up
validation errors (AMP, schema.org, rich cards),
robots.txt
 Web crawl: internal redirects as well as redirect
chains, broken URLs, and internal links
 Log files: broken URLs, suspicious status codes,
crawler traps
 Further: perform a thorough audit, it’ll save you a
lot of time with regards to the post-migration
workload

pa.ag@peakaceag16
You will need an arsenal of tools!
A lot depends on personal taste, but you’ll need at least one tool each for crawling and
log file analysis as well as search intelligence – and yes, this costs money!
Google
Big Query

pa.ag@peakaceag17
At a minimum: fix broken pages & broken internal links

pa.ag@peakaceag18
Identify the pages to be eliminated during migration
But only if they neither receive any relevant traffic nor are externally linked anymore.
URLs that you want to
fall completely off the
map should return a
410 status code.”
Apache .htaccess example code:
Redirect gone "/some/file.html"
RedirectMatch 410 "/more/(.*).pdf$"
or
also add
ErrorDocument 410 /errors/410.html

Figure out well in advance how to gain access to all relevant server
access logs, e.g. via native access or by using a SaaS solution.
Gain access to server logs

pa.ag@peakaceag20
Gather all URLs including static assets #1
Must haves: log files, XML sitemaps as well as a full website crawl.
Extras: analytics (top ranking URLs and/or URLs generating the most traffic).
small domains
large domains

pa.ag@peakaceag21
Gather all URLs including static assets #2
Additionally: collect the URLs that are strongly linked, bring a lot of traffic and/or have
been shared the most, etc.

pa.ag@peakaceag22
Switch to monitor your keyword rankings, daily
Check critical keywords daily to ensure you are only working with the most recent data.

pa.ag@peakaceag23
Establish a status quo performance benchmark
Lighthouse (via Chrome DevTools) or webpagetest.org provide relevant metrics.
Important: also benchmark category/product pages, not just the homepage.

As a rule of thumb: 1-to-1 redirects from old to new!
Prepare for URL redirect mapping

pa.ag@peakaceag25
Provide staging/test server
Make sure the server is locked-down properly to ensure your content doesn’t get
indexed in advance (i.a. duplicate content problems).
Methodology Pros Cons
noindex (meta tag/header)
 External tools can access without
separate access rules
 URLs are definitely not indexed
 Indexing rules cannot fully be tested
(all noindex)
 Waste of crawl budget
robots.txt
 External tools can access without
separate access rules
 No crawl budget is wasted
 Indexing rules cannot fully be tested
(only with robots.txt override)
 If linked, test URLs may appear in the index
(without title/metas).
password secured (.htaccess)
 Everything can be tested properly
 External tools must be able to handle
password authentication.
IP-based access
 Everything can be tested properly
 External tools must be able to handle IP-based
authentication.
VPN  Absolutely safe!  So safe, only a few tools can handle it!

#3 Implement necessary changes

pa.ag@peakaceag27
Migration types and their potential impact on SEO
Often these types overlap – or multiple things are done simultaneously.
Inspired by @jonoalderson: http://pa.ag/2xUCMnJ
Type Example
Hosting migrations You’re changing hosting or CDN provider(s).
You’re changing, adding, or removing server locations.
You’re changing your tech stack/caching/lbs.
Software migrations You’re changing CMS (or its version/plugins, etc.).
You’re changing the language used to render the website.
You’re merging platforms; e.g. a blog which operated on a separate domain.
Domain migrations You’re changing the main domain of your website.
You’re buying/adding new domains/subdomains to your ecosystem.
You’re moving a website, or part of a website, between domains.
Template migrations You’re changing the layout/structure/navigation of important pages.
You’re adding or removing template components.
You’re changing elements in your code, like title, canonical, or hreflang tags.

pa.ag@peakaceag28
Make it a granular, multi-step approach
Doing everything at once will make debugging & rolling back an almost impossible
task!
Source: http://pa.ag/2yJqT1N

pa.ag@peakaceag29
Build your very own migration QA check-list
Depending on what type of changes you’re undergoing, this needs to be adapted.
Use Aleyda’s template for more inspiration: https://pa.ag/2H6bOLH
Canonical tags & other rel-alternate annotations
Remember to annotate your dedicated mobile site and to adapt
your RSS feeds
Stage: pre-migration Who? Bastian When? 30/01/19
Multilingual setup: customise hreflang target URLs
Keep in mind: various locations can be affected (e.g. head section,
server headers, xml sitemaps)
Update pel=next/prev pagination annotations
Are you using Google´s recommendation for pagination?
Update XML sitemaps
Sitemap index file also needs to be changed if you reference it in the
robots.txt
Structured data update (schema.org)
Update your schema.org mark-up references. Short annotations like
“//schema.org“ don’t validate!
Update CDN settings and resource hints
Update requests for assets to CDNs & any resource hints (preconnect,
dns-prefetch)
Update HTTP header & customise cookie settings
If applicable, customise X-Robots header tags. Use Chrome DevTools!

pa.ag@peakaceag30
Update internal links
Simply relying on redirects isn’t a migration strategy.
Links to other internal URLs Links to internal video filesLinks to internal JavaScript files
JS
HTML
source code
Links to internal image files Links to internal CSS files Links to internal web fonts

pa.ag@peakaceag31
Update internal links within JavaScript files
JAVASCRIPT
files
Links to other internal URLs Links to internal image files Links to internal CSS files

pa.ag@peakaceag32
Update internal links within CSS files
CSS
files
Links to internal web fontsLinks to internal image files
Links to other internal URLs

pa.ag@peakaceag33
Be careful with internal redirects!
Avoid redirect chains: old URLs should lead directly to the new corresponding URLs.
Source: Redirect Chain Report via DeepCrawl

pa.ag@peakaceag34
Tricky: don‘t miss “invisible“ HTTP headers!
If in use: update X-Robots header tags accordingly. Use Chrome Dev. Tools to visualise!
X-Robots rel-canonical using Apaches‘ .htaccess
<Files study.pdf>
Header add Link "<https://domain.com/html-landing/>; rel="canonical""
</Files>
Other HTTP header variants (for SEO):
 Link: <https://www.example.com/de/>; rel="alternate"; hreflang="de"
 Link: <https://www.example.com/someurl>; rel="canonical"
 Link: <https://cdn.example.com>; rel="dns-prefetch"

pa.ag@peakaceag35
Everything updated? Side-by-side comparison crawl!
Better safe than sorry: let’s test-crawl the staging server and run a comparison to make
sure all contents are available 1:1.

pa.ag@peakaceag37
Create new properties & eventually merge them into a set
For example, properties for HTTP and HTTPS with and without www = four domains.
If you use separate mobile domains, there are six properties to set up in total!

pa.ag@peakaceag38
Transfer the disavow links file in time
Especially for domains with a “questionable” link profile: GSC setup and disavow file
transfer should be done approximately 48 hours before going live!

pa.ag@peakaceag39
Set your preferred domain and crawl rate (if necessary)
You don’t need to set the crawl rate, but, if you have done so previously, I would
suggest transferring the original settings (for now).

pa.ag@peakaceag40
Configure geo targeting for gTLDs
Synchronise settings for international targeting (i.e. for “.com” gTLDs).

pa.ag@peakaceag41
URL parameter settings in GSC
Don’t forget to synchronise the URL parameter settings if you were utilising them, or if
you require them for your site’s functionality.

Get everyone into the same room before you push the button;
fixing things will be so much easier!
Set up a “war room”

And transfer the yfs
Remove access restrictions

After all this preparation, now you "just" need
to redirect all your old URLs.
301 redirect all the things!

pa.ag@peakaceag46
Even if, as some say, all redirects are equal…
…all we did for this client was change the (chained) 302/307 to 301 redirects!

pa.ag@peakaceag47
Make sure that fetch & render (in GSC) works properly
Perform manual checks at least for your homepage & for each of your templates (e.g.
category, product or landing page); don‘t forget to test smartphone vs. desktop as well!

pa.ag@peakaceag48
Manually check your robots.txt file
Check the http response code (200), including its syntax and directives. Also make sure
to use the GSC testing tool to double check things "from a Google perspective“.
Source: https://pa.ag/2RBSFcG
If there's no Bingbot section then they will pick up
the default (*) one. But if there is a Bingbot, they will
only look only at that specific section, not the (*) one!

pa.ag@peakaceag49
Test your XML sitemap files and their quality
First, check that the sitemaps response code (200) and its XML mark-up are valid.
Also: only indexable URLs that serve a 200 response must be in the sitemap file!
> e.g. use Screaming Frog
> Mode > List
> Upload > Download Sitemap
NO redirects, no URLs that are blocked via
robots.txt or meta robots, no URLs with a
different canonical tag!

pa.ag@peakaceag50
Resubmit all your XML sitemaps
Keep in mind: the new GSC no longer allows you to test the file before submitting it, so
make sure you‘re validating the XML mark-up beforehand.

pa.ag@peakaceag
Manual checks to verify: response code, canonicalization, indexing
directives, tracking codes, content, structured data & mark-ups.
Check your most important pages!

pa.ag@peakaceag52
List crawl of old URLs & manual SERP checks
Import old URLs, e.g. to ScreamingFrog (list mode) - only 301s should appear here.
Perform additional manual check of indexed URLs, e.g. via LinkClump add-on in Chrome.
1 2 3

Note: it only works for domain migrations!
Use the GSC “site move” feature

e.g. domain.de via GSC to domain.com/de/ won‘t work!
The destination can‘t be a gTLD folder!

Re-test: no broken URLs (4/5XXer), tracking in place,
correct metadata & other tags, indexing rules, etc.
Complete crawl of the new domain

56
Test and apply an “if... then... logic”
Based on your previously built QA checklist, make sure to double-check everything!
you work internationally,
then test hreflang tags/sitemap annotations.
PPC ads are shown,
then update and test the landing page URLs.
Google news listings are available,
then test the Google news sitemap.
ratings (featured snippets) are available,
then test the schema.org mark-up.
IF
SSL is also used on other subdomains,
then test these as well.
IF
IF
IF
IF

The most common problem with migrations gone awry?
Missing or incorrect redirects!
Seriously: check your redirects

Focus on 4XX and 5XX status codes first, tackle these “live”!
Log file, GSC & GA error monitoring

pa.ag@peakaceag64
Peak Ace log file auditing stack
Log files are stored in Google Cloud Storage, processed in Dataprep, exported to
BigQuery and visualised in Data Studio via BigQuery Connector.
8
Google Data Studio
Data
transmission
Display
dataImport
Google Dataprep
6 7
Google BigQuery
1
Log files
GSC
API v3
GA
API v4
GA
GSC
2
3
65
Google Apps Script
DeepCrawl
API
4

pa.ag@peakaceag65
AMP, structured data and rich cards error-monitoring

Google needs to recrawl ALL the relations - this takes time!
Consider hreflang to be broken

Short-term peaks are completely normal, dramatic drops are not!
Keep an eye on crawl frequency

pa.ag@peakaceag68
Compare performance side-by-side
HTTPs is usually a little bit slower (handshake etc.): compare your results.
Clear goal: never slower than before (always use HTTP/2 when switching to HTTPs).
Try it out: https://www.webpagetest.org/

pa.ag@peakaceag69
Adjusting/revising external links
Not every link source needs to be updated; focus on the strongest domains (e.g. via
LRT Power*Trust or Majestic Trust Flow).
Source: Majestic

pa.ag@peakaceag70
Watch your domain's Index Coverage in Search Console
Dive deep into the “Excluded” section and watch out for crawl anomalies.
It’s also worth double-checking: “crawled - currently not indexed”, “duplicate
without user-selected canonical” & “duplicate, Google chose different canonical
than user”.

Recrawling, recalculation & the inheritance of
”trust“ may take a few weeks…
First and foremost: be patient!

pa.ag@peakaceag72
Especially for new domains, two weeks is not enough:
Source: http://pa.ag/2htz26D
[…] it can take two weeks
for some sites but for
larger sites, it is more like
three months, not even
close to two weeks.

pa.ag@peakaceag
#6 Let’s talk tracking for a minute

pa.ag@peakaceag74
GA Checker to examine consistent pixel implementation
Crawl website for code implementation of GA, GTM, Ads, Optimize, etc.
Source: https://pa.ag/2TAGuu5

pa.ag@peakaceag75
Google Tag Assistant to check for correct implementation
Determine whether Google tools and pixels are functional and set up as recommended.
Source: https://pa.ag/2SM5Mpa

pa.ag@peakaceag76
WASP.inspector to investigate data transfer & pixels
Deep dive into e.g. variables and values transmitted per tool.
Source: https://pa.ag/2CXgFPw

pa.ag@peakaceag77
Pay attention to your error pages (e.g. custom 404)
Check for page titles using RegEx for e.g. “Page Not Found” or “404” or…

pa.ag@peakaceag78
Pay attention to your error pages (e.g. custom 404)
Send page URLs as well as referring URLs as events to Google Analytics.

pa.ag@peakaceag79
Check for JavaScript errors on web pages
Use the built-in GTM JavaScript error listener:

pa.ag@peakaceag80
Send JS error messages as well as error messages and URLs to Google Analytics.

pa.ag@peakaceag81
Examine each page for JS errors in Google Analytics. Locate the most important errors.

#7 Some tips to make your life easier

pa.ag@peakaceag83
Bulk test these things: redirects & other headers
HTTP status codes (errors, redirects, etc.) at scale, for free: httpstatus.io
Check it out: https://httpstatus.io/

pa.ag@peakaceag84
Bulk test these things: hreflang tags
Hreflang tags (in sitemaps) at scale, for free: technicalseo.com
Check it out: https://technicalseo.com/seo-tools/hreflang/

pa.ag@peakaceag85
Testing for mobile-friendliness and PageSpeed scores
Manually testing single pages is easily done using GSC (Mobile Usability) & Lighthouse.
Mass-testing a lot of URLs is tedious, however, URL Profiler can help!
Returns a massive spreadsheet with loads of info
(e.g. paint timings, etc.) as well as details concerning
mobile-friendliness (passed/failed) and a score per URL

pa.ag@peakaceag86
Simulate Googlebot for smartphones with JS rendering
ScreamingFrog can do that at scale, easily; pay close attention to rendered output!
Also pretty cool: Extract > Xpath > //head/link[@rel="amphtml"]/@href

pa.ag@peakaceag87
Don’t forget to redirect your images as well
When changing URLs/domains, make sure to implement redirect rules for images.
Read the entire post: http://pa.ag/2yJtTLz

pa.ag@peakaceag88
Move (and retest) your robots.txt file
When changing domains, make sure to transfer (the contents of) the robots.txt!

pa.ag@peakaceag89
HTTP 503 is your friend
Combine with “revisit-after” to throttle crawling; never use “noindex” or 4XX instead!
Source: http://pa.ag/2xRiA5T
Webmasters should return a 503 HTTP header for all
the URLs participating in the blackout […] Googlebot's
crawling rate will drop when it sees a spike in 503 […]
as Googlebot is currently configured, it will halt all
crawling of the site if the site’s robots.txt file
returns a 503 status code for robots.txt”

pa.ag@peakaceag90
GSC’s DNS verification can be pretty helpful
No need to worry about missing meta tags; plus you can verify before deploying a
site/frontend to a new domain – and it’s faster!
More: http://pa.ag/2yJ7xtH

pa.ag@peakaceag91
Successfully executed a protocol migration to HTTPs?
Make sure all your URLs are fully secure and don’t cause any “mixed content” warnings!
More: https://report-uri.com/home/generate
Mixed content occurs when
initial HTML is loaded over
a secure HTTPS connection,
but other resources (such as
images, videos, stylesheets,
scripts) are loaded over an
insecure HTTP connection
Content-Security-Policy-Report-Only: default-src
https: 'unsafe-inline' 'unsafe-eval’;
report-uri https://example.com/reportingEndpoint
If you don't want to configure a reporting
endpoint yourself, report-uri.com is a great help!

pa.ag@peakaceag92
Fix those redirect chains, especially on legacy sites…
…as multiple requests waste valuable performance and crawl budget!

pa.ag@peakaceag93
Don’t miss out on some of the non-obvious redirects
It’s not only protocol-specific and www vs. non-www redirects, but so many more…
Old URL / pattern New destination Please note
domain.com/robots.txt newdomain.com/robots.txt
domain.com/sitemap.xml newdomain.com/sitemap.xml
Redirect Sitemap Index (or *.gz files) respectively.
Don’t do this if you want to monitor de-indexation
domain.com/wp-content/uploads/*.jpg newdomain.com/images/*.jpg
If you’re moving (away) from e.g. WordPress, don’t forget your
images - all of them.
domain.com/someCRAZYurl.html newdomain.com/somecrazyurl.html
Unify to a lower-case URL; more importantly make sure your rules
on origin are case insensitive
domain.com/styles/old-*.css newdomain.com/styles/new-*.css Depending on crawl volume, also redirect old CSS and JS file URLs
Keep in mind that old URLs might come with or without query parameters –
again make sure your redirect mapping handles these accordingly.

pa.ag@peakaceag94
Some technical advice concerning (301) redirects
301 redirects are cached, so make sure you really know what you're doing!
Staged 302>301 redirects during a migration are more complex, but often worth it.
Once a 301 redirect has been implemented
(and seen by Googlebot), it‘ll be cached –
make sure the destination is 100% accurate
(and returning a HTTP 200)!

pa.ag@peakaceag95
Which redirect should you actually use and when?
An overview of the most common SEO use cases for redirects:
via audisto.com: https://pa.ag/2TqZACX
Use case/task Status Cacheable Possible status codes
Relaunch & structural changes permanent yes 301/308
Geo targeting & delivery temporary no 303/302/307
Device targeting & delivery temporary no 303/302/307
URL normalisation & unifying permanent yes 301/308
Campaign URLs (affiliate, PPC) temporary depends 302/307

pa.ag@peakaceag96
So, in a nutshell…
…whenever you perform a migration, make sure you:
01
Don‘t migrate
legacy!
Do a thorough health
check and clean up
first – it‘ll be super
hard to analyse errors
later on.
02
Collect all the
necessary data in
advance!
Make sure to fully
understand your URL
portfolio, crawl & GSC
data etc.
03
Get access to server
log files!
As much as I do love
the GSC, data comes
in with a delay and
you need instant
feedback on errors,
etc.
04
Check your
redirects, twice!
Make sure to review
your redirect mapping
multiple times, ideally
with another pair of
eyes.
05
Be patient and
don‘t panic!
Don‘t freak out,
seriously. Sometimes
things can just take a
bit longer for Google
to fully process (and
to pass along signals).

twitter.com/peakaceag
facebook.com/peakaceag
www.pa.ag
ALWAYS LOOKING FOR TALENT! CHECK OUT JOBS.PA.AG
WINNER
Bastian Grimm
bg@pa.ag
Slides? No problem:
https://pa.ag/searchy19
You want our log file setup (for free)?
e-mail us > hi@pa.ag

SEARCH Y - Bastian Grimm - Migrations Best Practices

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to SEARCH Y - Bastian Grimm - Migrations Best Practices

Similar to SEARCH Y - Bastian Grimm - Migrations Best Practices (20)

More from SEARCH Y - Philippe Yonnet Evénements

More from SEARCH Y - Philippe Yonnet Evénements (16)

Recently uploaded

Recently uploaded (20)

SEARCH Y - Bastian Grimm - Migrations Best Practices