Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Migration Best Practices - Search Y 2019, Paris

1,050 views

Published on

My talk from SEARCHY 2019 in Paris covering best practices on how to successfully navigate through the various types of migrations (protocol migrations, frontend migrations, website migration, cms migration, etc.) from an SEO perspective - mainly focussing on all things technical SEO.

Published in: Technology

Migration Best Practices - Search Y 2019, Paris

  1. 1. Bastian Grimm, Peak Ace AG | @basgr Everything you need to know to execute a successful migration Migration Best Practices
  2. 2. #1 Never change a running system!
  3. 3. pa.ag@peakaceag3 Every (big) change brings (lots of) opportunities! But always keep in mind: the price tag for failure is immense! Risk Reward User acceptance “Once in a lifetime” opportunity Project complexity Greenfield project: question everything Resilience/freeze Best chance to really “get shit done” Interruption Opportunity to eliminate “legacy problems” Politics Usually more agile vs. regular, daily business Performance Rethink RWD/dynamic serving, HTTPS, URL design, etc.
  4. 4. pa.ag@peakaceag4 What are your goals for migration? The right mindset is really, really important! I want to lose as little as possible!” – isn’t really a good migration objective!
  5. 5. Thorough documentation, in-depth definitions of requirements and ongoing testing are essential! Be crazy about the details
  6. 6. …from someone who has successfully done this type of work before. Ask for help…
  7. 7. #2 A lot of preparation
  8. 8. The sooner your (SEO) team is involved, the better: making sure everyone is on board, increasing time for prep work & more thorough planning, etc. Get in (very) early
  9. 9. What should go, what can stay? Complete vs. partial site move? Define your migration strategy
  10. 10. Use one of your smaller markets (revenue-wise) as a guinea pig to test the process and gather real-world feedback. Multinational setup?
  11. 11. Find a timeframe – which includes enough of a buffer – when business in general is slower; also consider extraordinary (seasonal) events. Migrate during a slow period
  12. 12. Rushing the process will guarantee failure! Ensure there's enough time for Q&A, feedback loops and necessary changes to be implemented! Be realistic about deadlines
  13. 13. pa.ag@peakaceag13 Pre-migration: site health check & clean up A properly optimised domain migrates easier and more efficiently. Getting your house in order before the move minimises the risk of losing beneficial rankings. ▪ Google Search Console: manual actions, server errors (DNS, 4xx and 5xx response codes), mark-up validation errors (AMP, schema.org, rich cards), robots.txt ▪ Web crawl: internal redirects as well as redirect chains, broken URLs, and internal links ▪ Log files: broken URLs, suspicious status codes, crawler traps ▪ Further: perform a thorough audit, it’ll save you a lot of time with regards to the post-migration workload
  14. 14. pa.ag@peakaceag14 You will need an arsenal of tools! A lot depends on personal taste, but you’ll need at least one tool each for crawling and log file analysis as well as search intelligence – and yes, this costs money! Google Big Query
  15. 15. pa.ag@peakaceag15 At a minimum: fix broken pages & broken internal links
  16. 16. pa.ag@peakaceag16 Identify the pages to be eliminated during migration But only if they neither receive any relevant traffic nor are externally linked anymore. URLs that you want to fall completely off the map should return a 410 status code.” Apache .htaccess example code: Redirect gone "/some/file.html" RedirectMatch 410 "/more/(.*).pdf$" or also add ErrorDocument 410 /errors/410.html
  17. 17. Figure out well in advance how to gain access to all relevant server access logs, e.g. via native access or by using a SaaS solution. Gain access to server logs
  18. 18. pa.ag@peakaceag18 Gather all URLs including static assets #1 Must haves: log files, XML sitemaps as well as a full website crawl. Extras: analytics (top ranking URLs and/or URLs generating the most traffic). small domains large domains
  19. 19. pa.ag@peakaceag19 Gather all URLs including static assets #2 Additionally: collect the URLs that are strongly linked, bring a lot of traffic and/or have been shared the most, etc.
  20. 20. pa.ag@peakaceag20 Switch to monitor your keyword rankings, daily Check critical keywords daily to ensure you are only working with the most recent data.
  21. 21. pa.ag@peakaceag21 Establish a status quo performance benchmark Lighthouse (via Chrome DevTools) or webpagetest.org provide relevant metrics. Important: also benchmark category/product pages, not just the homepage.
  22. 22. As a rule of thumb: 1-to-1 redirects from old to new! Prepare for URL redirect mapping
  23. 23. pa.ag@peakaceag23 Provide staging/test server Make sure the server is locked-down properly to ensure your content doesn’t get indexed in advance (i.a. duplicate content problems). Methodology Pros Cons noindex (meta tag/header) ▪ External tools can access without separate access rules ▪ URLs are definitely not indexed ▪ Indexing rules cannot fully be tested (all noindex) ▪ Waste of crawl budget robots.txt ▪ External tools can access without separate access rules ▪ No crawl budget is wasted ▪ Indexing rules cannot fully be tested (only with robots.txt override) ▪ If linked, test URLs may appear in the index (without title/metas). password secured (.htaccess) ▪ No crawl budget is wasted ▪ URLs are definitely not indexed ▪ Everything can be tested properly ▪ External tools must be able to handle password authentication. IP-based access ▪ No crawl budget is wasted ▪ URLs are definitely not indexed ▪ Everything can be tested properly ▪ External tools must be able to handle IP-based authentication. VPN ▪ Absolutely safe! ▪ So safe, only a few tools can handle it!
  24. 24. #3 Implement necessary changes
  25. 25. pa.ag@peakaceag25 Migration types and their potential impact on SEO Often these types overlap – or multiple things are done simultaneously. Inspired by @jonoalderson: http://pa.ag/2xUCMnJ Type Example Hosting migrations You’re changing hosting or CDN provider(s). You’re changing, adding, or removing server locations. You’re changing your tech stack/caching/lbs. Software migrations You’re changing CMS (or its version/plugins, etc.). You’re changing the language used to render the website. You’re merging platforms; e.g. a blog which operated on a separate domain. Domain migrations You’re changing the main domain of your website. You’re buying/adding new domains/subdomains to your ecosystem. You’re moving a website, or part of a website, between domains. Template migrations You’re changing the layout/structure/navigation of important pages. You’re adding or removing template components. You’re changing elements in your code, like title, canonical, or hreflang tags.
  26. 26. pa.ag@peakaceag26 Make it a granular, multi-step approach Doing everything at once will make debugging & rolling back an almost impossible task! Source: http://pa.ag/2yJqT1N
  27. 27. pa.ag@peakaceag27 Build your very own migration QA check-list Depending on what type of changes you’re undergoing, this needs to be adapted. Use Aleyda’s template for more inspiration: https://pa.ag/2H6bOLH Canonical tags & other rel-alternate annotations Remember to annotate your dedicated mobile site and to adapt your RSS feeds Stage: pre-migration Who? Bastian When? 30/01/19 Multilingual setup: customise hreflang target URLs Keep in mind: various locations can be affected (e.g. head section, server headers, xml sitemaps) Stage: pre-migration Who? Bastian When? 30/01/19 Update pel=next/prev pagination annotations Are you using Google´s recommendation for pagination? Stage: pre-migration Who? Bastian When? 30/01/19 Update XML sitemaps Sitemap index file also needs to be changed if you reference it in the robots.txt Stage: pre-migration Who? Bastian When? 30/01/19 Structured data update (schema.org) Update your schema.org mark-up references. Short annotations like “//schema.org“ don’t validate! Stage: pre-migration Who? Bastian When? 30/01/19 Update CDN settings and resource hints Update requests for assets to CDNs & any resource hints (preconnect, dns-prefetch) Stage: pre-migration Who? Bastian When? 30/01/19 Update HTTP header & customise cookie settings If applicable, customise X-Robots header tags. Use Chrome DevTools! Stage: pre-migration Who? Bastian When? 30/01/19
  28. 28. pa.ag@peakaceag28 Update internal links Simply relying on redirects isn’t a migration strategy. Links to other internal URLs Links to internal video filesLinks to internal JavaScript files JS HTML source code Links to internal image files Links to internal CSS files Links to internal web fonts
  29. 29. pa.ag@peakaceag29 Update internal links within JavaScript files Simply relying on redirects isn’t a migration strategy. JAVASCRIPT files Links to other internal URLs Links to internal image files Links to internal CSS files
  30. 30. pa.ag@peakaceag30 Update internal links within CSS files Simply relying on redirects isn’t a migration strategy. CSS files Links to internal web fontsLinks to internal image files Links to other internal URLs
  31. 31. pa.ag@peakaceag31 Be careful with internal redirects! Avoid redirect chains: old URLs should lead directly to the new corresponding URLs. Source: Redirect Chain Report via DeepCrawl
  32. 32. pa.ag@peakaceag32 Tricky: don‘t miss “invisible“ HTTP headers! If in use: update X-Robots header tags accordingly. Use Chrome Dev. Tools to visualise! X-Robots rel-canonical using Apaches‘ .htaccess <Files study.pdf> Header add Link "<https://domain.com/html-landing/>; rel="canonical"" </Files> Other HTTP header variants (for SEO): ▪ Link: <https://www.example.com/de/>; rel="alternate"; hreflang="de" ▪ Link: <https://www.example.com/someurl>; rel="canonical" ▪ Link: <https://cdn.example.com>; rel="dns-prefetch"
  33. 33. pa.ag@peakaceag33 Everything updated? Side-by-side comparison crawl! Better safe than sorry: let’s test-crawl the staging server and run a comparison to make sure all contents are available 1:1.
  34. 34. #4 Work your search console
  35. 35. pa.ag@peakaceag35 Create new properties & eventually merge them into a set For example, properties for HTTP and HTTPS with and without www = four domains. If you use separate mobile domains, there are six properties to set up in total!
  36. 36. pa.ag@peakaceag36 Transfer the disavow links file in time Especially for domains with a “questionable” link profile: GSC setup and disavow file transfer should be done approximately 48 hours before going live!
  37. 37. pa.ag@peakaceag37 Set your preferred domain and crawl rate (if necessary) You don’t need to set the crawl rate, but, if you have done so previously, I would suggest transferring the original settings (for now).
  38. 38. pa.ag@peakaceag38 Configure geo targeting for gTLDs Synchronise settings for international targeting (i.e. for “.com” gTLDs).
  39. 39. pa.ag@peakaceag39 URL parameter settings in GSC Don’t forget to synchronise the URL parameter settings if you were utilising them, or if you require them for your site’s functionality.
  40. 40. … enough of the prep work!
  41. 41. Get everyone into the same room before you push the button; fixing things will be so much easier! Set up a “war room”
  42. 42. And transfer the yfs Remove access restrictions
  43. 43. After all this preparation, now you "just" need to redirect all your old URLs. 301 redirect all the things!
  44. 44. pa.ag@peakaceag44 Even if, as some say, all redirects are equal… …all we did for this client was change the (chained) 302/307 to 301 redirects!
  45. 45. pa.ag@peakaceag45 Make sure that fetch & render (in GSC) works properly Perform manual checks at least for your homepage & for each of your templates (e.g. category, product or landing page); don‘t forget to test smartphone vs. desktop as well!
  46. 46. pa.ag@peakaceag46 Manually check your robots.txt file Check the http response code (200), including its syntax and directives. Also make sure to use the GSC testing tool to double check things "from a Google perspective“. Source: https://pa.ag/2RBSFcG If there's no Bingbot section then they will pick up the default (*) one. But if there is a Bingbot, they will only look only at that specific section, not the (*) one!
  47. 47. pa.ag@peakaceag47 Test your XML sitemap files and their quality First, check that the sitemaps response code (200) and its XML mark-up are valid. Also: only indexable URLs that serve a 200 response must be in the sitemap file! > e.g. use Screaming Frog > Mode > List > Upload > Download Sitemap NO redirects, no URLs that are blocked via robots.txt or meta robots, no URLs with a different canonical tag!
  48. 48. pa.ag@peakaceag48 Resubmit all your XML sitemaps Keep in mind: the new GSC no longer allows you to test the file before submitting it, so make sure you‘re validating the XML mark-up beforehand.
  49. 49. pa.ag@peakaceag Manual checks to verify: response code, canonicalization, indexing directives, tracking codes, content, structured data & mark-ups. Check your most important pages!
  50. 50. pa.ag@peakaceag50 List crawl of old URLs & manual SERP checks Import old URLs, e.g. to ScreamingFrog (list mode) - only 301s should appear here. Perform additional manual check of indexed URLs, e.g. via LinkClump add-on in Chrome. 1 2 3
  51. 51. Note: it only works for domain migrations! Use the GSC “site move” feature
  52. 52. e.g. domain.de via GSC to domain.com/de/ won‘t work! The destination can‘t be a gTLD folder!
  53. 53. Re-test: no broken URLs (4/5XXer), tracking in place, correct metadata & other tags, indexing rules, etc. Complete crawl of the new domain
  54. 54. 54 Test and apply an “if... then... logic” Based on your previously built QA checklist, make sure to double-check everything! you work internationally, then test hreflang tags/sitemap annotations. PPC ads are shown, then update and test the landing page URLs. Google news listings are available, then test the Google news sitemap. ratings (featured snippets) are available, then test the schema.org mark-up. IF SSL is also used on other subdomains, then test these as well. IF IF IF IF
  55. 55. The most common problem with migrations gone awry? Missing or incorrect redirects! Seriously: check your redirects
  56. 56. #5 Post-migration to-dos
  57. 57. Focus on 4XX and 5XX status codes first, tackle these “live”! Log file, GSC & GA error monitoring
  58. 58. pa.ag@peakaceag58
  59. 59. pa.ag@peakaceag59
  60. 60. pa.ag@peakaceag60
  61. 61. pa.ag@peakaceag61
  62. 62. pa.ag@peakaceag62 Peak Ace log file auditing stack Log files are stored in Google Cloud Storage, processed in Dataprep, exported to BigQuery and visualised in Data Studio via BigQuery Connector. 8 Google Data Studio Data transmission Display dataImport Google Dataprep 6 7 Google BigQuery 1 Log files GSC API v3 GA API v4 GA GSC 2 3 65 Google Apps Script DeepCrawl API 4
  63. 63. pa.ag@peakaceag63 AMP, structured data and rich cards error-monitoring
  64. 64. Google needs to recrawl ALL the relations - this takes time! Consider hreflang to be broken
  65. 65. Short-term peaks are completely normal, dramatic drops are not! Keep an eye on crawl frequency
  66. 66. pa.ag@peakaceag66 Compare performance side-by-side HTTPs is usually a little bit slower (handshake etc.): compare your results. Clear goal: never slower than before (always use HTTP/2 when switching to HTTPs). Try it out: https://www.webpagetest.org/
  67. 67. pa.ag@peakaceag67 Adjusting/revising external links Not every link source needs to be updated; focus on the strongest domains (e.g. via LRT Power*Trust or Majestic Trust Flow). Source: Majestic
  68. 68. pa.ag@peakaceag68 Watch your domain's Index Coverage in Search Console Dive deep into the “Excluded” section and watch out for crawl anomalies. It’s also worth double-checking: “crawled - currently not indexed”, “duplicate without user-selected canonical” & “duplicate, Google chose different canonical than user”.
  69. 69. Recrawling, recalculation & the inheritance of ”trust“ may take a few weeks… First and foremost: be patient!
  70. 70. pa.ag@peakaceag70 Especially for new domains, two weeks is not enough: Source: http://pa.ag/2htz26D […] it can take two weeks for some sites but for larger sites, it is more like three months, not even close to two weeks.
  71. 71. pa.ag@peakaceag #6 Let’s talk tracking for a minute
  72. 72. pa.ag@peakaceag72 GA Checker to examine consistent pixel implementation Crawl website for code implementation of GA, GTM, Ads, Optimize, etc. Source: https://pa.ag/2TAGuu5
  73. 73. pa.ag@peakaceag73 Google Tag Assistant to check for correct implementation Determine whether Google tools and pixels are functional and set up as recommended. Source: https://pa.ag/2SM5Mpa
  74. 74. pa.ag@peakaceag74 WASP.inspector to investigate data transfer & pixels Deep dive into e.g. variables and values transmitted per tool. Source: https://pa.ag/2CXgFPw
  75. 75. pa.ag@peakaceag75 Pay attention to your error pages (e.g. custom 404) Check for page titles using RegEx for e.g. “Page Not Found” or “404” or…
  76. 76. pa.ag@peakaceag76 Pay attention to your error pages (e.g. custom 404) Send page URLs as well as referring URLs as events to Google Analytics.
  77. 77. pa.ag@peakaceag77 Check for JavaScript errors on web pages Use the built-in GTM JavaScript error listener:
  78. 78. pa.ag@peakaceag78 Check for JavaScript errors on web pages Send JS error messages as well as error messages and URLs to Google Analytics.
  79. 79. pa.ag@peakaceag79 Check for JavaScript errors on web pages Examine each page for JS errors in Google Analytics. Locate the most important errors.
  80. 80. #7 Some tips to make your life easier
  81. 81. pa.ag@peakaceag81 Bulk test these things: redirects & other headers HTTP status codes (errors, redirects, etc.) at scale, for free: httpstatus.io Check it out: https://httpstatus.io/
  82. 82. pa.ag@peakaceag82 Bulk test these things: hreflang tags Hreflang tags (in sitemaps) at scale, for free: technicalseo.com Check it out: https://technicalseo.com/seo-tools/hreflang/
  83. 83. pa.ag@peakaceag83 Testing for mobile-friendliness and PageSpeed scores Manually testing single pages is easily done using GSC (Mobile Usability) & Lighthouse. Mass-testing a lot of URLs is tedious, however, URL Profiler can help! Returns a massive spreadsheet with loads of info (e.g. paint timings, etc.) as well as details concerning mobile-friendliness (passed/failed) and a score per URL
  84. 84. pa.ag@peakaceag84 Simulate Googlebot for smartphones with JS rendering ScreamingFrog can do that at scale, easily; pay close attention to rendered output! Also pretty cool: Extract > Xpath > //head/link[@rel="amphtml"]/@href
  85. 85. pa.ag@peakaceag85 Don’t forget to redirect your images as well When changing URLs/domains, make sure to implement redirect rules for images. Read the entire post: http://pa.ag/2yJtTLz
  86. 86. pa.ag@peakaceag86 Move (and retest) your robots.txt file When changing domains, make sure to transfer (the contents of) the robots.txt!
  87. 87. pa.ag@peakaceag87 HTTP 503 is your friend Combine with “revisit-after” to throttle crawling; never use “noindex” or 4XX instead! Source: http://pa.ag/2xRiA5T Webmasters should return a 503 HTTP header for all the URLs participating in the blackout […] Googlebot's crawling rate will drop when it sees a spike in 503 […] as Googlebot is currently configured, it will halt all crawling of the site if the site’s robots.txt file returns a 503 status code for robots.txt”
  88. 88. pa.ag@peakaceag88 GSC’s DNS verification can be pretty helpful No need to worry about missing meta tags; plus you can verify before deploying a site/frontend to a new domain – and it’s faster! More: http://pa.ag/2yJ7xtH
  89. 89. pa.ag@peakaceag89 Successfully executed a protocol migration to HTTPs? Make sure all your URLs are fully secure and don’t cause any “mixed content” warnings! More: https://report-uri.com/home/generate Mixed content occurs when initial HTML is loaded over a secure HTTPS connection, but other resources (such as images, videos, stylesheets, scripts) are loaded over an insecure HTTP connection Content-Security-Policy-Report-Only: default-src https: 'unsafe-inline' 'unsafe-eval’; report-uri https://example.com/reportingEndpoint If you don't want to configure a reporting endpoint yourself, report-uri.com is a great help!
  90. 90. pa.ag@peakaceag90 Fix those redirect chains, especially on legacy sites… …as multiple requests waste valuable performance and crawl budget!
  91. 91. pa.ag@peakaceag91 Don’t miss out on some of the non-obvious redirects It’s not only protocol-specific and www vs. non-www redirects, but so many more… Old URL / pattern New destination Please note domain.com/robots.txt newdomain.com/robots.txt domain.com/sitemap.xml newdomain.com/sitemap.xml Redirect Sitemap Index (or *.gz files) respectively. Don’t do this if you want to monitor de-indexation domain.com/wp-content/uploads/*.jpg newdomain.com/images/*.jpg If you’re moving (away) from e.g. WordPress, don’t forget your images - all of them. domain.com/someCRAZYurl.html newdomain.com/somecrazyurl.html Unify to a lower-case URL; more importantly make sure your rules on origin are case insensitive domain.com/styles/old-*.css newdomain.com/styles/new-*.css Depending on crawl volume, also redirect old CSS and JS file URLs Keep in mind that old URLs might come with or without query parameters – again make sure your redirect mapping handles these accordingly.
  92. 92. pa.ag@peakaceag92 Some technical advice concerning (301) redirects 301 redirects are cached, so make sure you really know what you're doing! Staged 302>301 redirects during a migration are more complex, but often worth it. Once a 301 redirect has been implemented (and seen by Googlebot), it‘ll be cached – make sure the destination is 100% accurate (and returning a HTTP 200)!
  93. 93. pa.ag@peakaceag93 Which redirect should you actually use and when? An overview of the most common SEO use cases for redirects: via audisto.com: https://pa.ag/2TqZACX Use case/task Status Cacheable Possible status codes Relaunch & structural changes permanent yes 301/308 Geo targeting & delivery temporary no 303/302/307 Device targeting & delivery temporary no 303/302/307 URL normalisation & unifying permanent yes 301/308 Campaign URLs (affiliate, PPC) temporary depends 302/307
  94. 94. pa.ag@peakaceag94 So, in a nutshell… …whenever you perform a migration, make sure you: 01 Don‘t migrate legacy! Do a thorough health check and clean up first – it‘ll be super hard to analyse errors later on. 02 Collect all the necessary data in advance! Make sure to fully understand your URL portfolio, crawl & GSC data etc. 03 Get access to server log files! As much as I do love the GSC, data comes in with a delay and you need instant feedback on errors, etc. 04 Check your redirects, twice! Make sure to review your redirect mapping multiple times, ideally with another pair of eyes. 05 Be patient and don‘t panic! Don‘t freak out, seriously. Sometimes things can just take a bit longer for Google to fully process (and to pass along signals).
  95. 95. twitter.com/peakaceag facebook.com/peakaceag www.pa.ag ALWAYS LOOKING FOR TALENT! CHECK OUT JOBS.PA.AG WINNER Bastian Grimm bg@pa.ag Slides? No problem: https://pa.ag/searchy19 You want our log file setup (for free)? e-mail us > hi@pa.ag

×