Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

BrightonSEO April 2018 Mobile-First & Crawl Budget

2,281 views

Published on

How does the transition to the Mobile-First Index impact crawl budget? And what can you achieve?

Published in: Marketing
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

BrightonSEO April 2018 Mobile-First & Crawl Budget

  1. 1. Crawl Budget Optimisation In a Mobile-First Index @Botify | @SearchMATH https://www.slideshare.net/MarkThomas114
  2. 2. AUDIT MOBILE SITE 1 OPTIMISE CRAWL BUDGET 2 GROW ORGANIC REVENUE 3
  3. 3. MOBILE- FIRST Case Study Context ✔ ✔ ✔ CRAWL BUDGET 2018
  4. 4. ORGANIC TRAFFIC 46% IN 8 WEEKS
  5. 5. Crawl Budget Optimisation
  6. 6. 46% Yielding Traffic Growth
  7. 7. AUDIT MOBILE SITE 1 OPTIMISE CRAWL BUDGET 2 GROW ORGANIC REVENUE 3
  8. 8. MOBILE-FIRST REFRESHER
  9. 9. W E B S E A R C H Internet Crawler Scheduler Parser Indexing SearchIndexURLs
  10. 10. C R A W L B U D G E T Internet Crawler Scheduler Parser Indexing SearchIndexURLs CRAWL DEMAND CRAWL RATE LIMIT CRAWL BUDGET
  11. 11. We'll use the mobile version of the content for indexing and ranking, to better help our – primarily mobile – users find what they're looking for. “ Mobile-First Index Explained
  12. 12. Internet Crawler Scheduler W E B C R A W L I N G Parser Indexing SearchIndexURLs BEFORE
  13. 13. M O B I L E F I R S T BEFORE Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
  14. 14. M O B I L E F I R S T AFTER Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
  15. 15. Internet Crawler Scheduler Parser Indexing SearchIndexURLs M O B I L E F I R S T AFTER
  16. 16. we transition sites slowly to ensure a good experience for site owners and users. “ Mobile-First Index Transition
  17. 17. We evaluate each site individually on its readiness for mobile-first indexing based on the best practices and transition the site when the site is ready. “ Mobile-First Index Transition
  18. 18. Mobile-First Transition M O B I L E F I R S T Index Index Index Index 2018 JANUARY 2018 JULY
  19. 19. MONITOR SERVER LOGS M O B I L E F I R S T MONITOR GSC NOTIFICATIONS
  20. 20. MONITOR SERVER LOGS M O B I L E F I R S T MONITOR GSC NOTIFICATIONS
  21. 21. AUDIT MOBILE SITE 1 OPTIMISE CRAWL BUDGET 2 GROW ORGANIC REVENUE 3
  22. 22. STEP 1 UNDERSTAND HOW MOBILE CRAWLERS VIEW YOUR SITE
  23. 23. M O B I L E T E S T I N G
  24. 24. Recommended Validation Approach M O B I L E T E S T I N G MOBILE FRIENDLY TEST MOBILE USABILITY GSC CHROME DEV TOOLS
  25. 25. M O B I L E T E S T I N G
  26. 26. M O B I L E T E S T I N G Crawl with Smartphone UA
  27. 27. Internet Crawler Scheduler Parser Indexing SearchIndexURLs M O B I L E F I R S T Test Crawl with Google’s Smartphone User Agent
  28. 28. Links Speed Content Metadata Structured Data Links Speed Content Metadata Structured Data M O B I L E T E S T I N G CRAWL C O M P A R E
  29. 29. BE CAREFUL, THIS IS INSUFFICIENT M O B I L E T E S T I N G
  30. 30. KPIS MOBILE DESKTOP DIFF Avg. Load Times 474,0 707,8 -33,0% Avg. Depth 34,0 4,9 591,0% % Internal PageRank 91,6% 92,1% -0,6% Avg. Follow Inlinks 22,5 24,3 -7,3% Avg. Follow Outlinks 24,1 24,3 -0,71% Avg. Anchors Variations 1,7 2,7 -36,9% Avg. Dup. Titles 512,8 0,4 141222,2% Avg. Dup. H1 386,2 2,3 16401,2% Avg. Dup. Descriptions 55,4 0,5 11043,8% Avg. Title Length 69,6 72,6 -4,2% Avg. Words (Not Ignored) 147,7 222,1 -33,5% Avg. % Unique N-Grams 10,9 14,8 -26,5% Sum. Similar Pages (>90%) 133902 42500 215,1% M O B I L E T E S T I N G GO DEEPER: Analyse Structural KPIs
  31. 31. M O B I L E T E S T I N G GO DEEPER: Analyse Structural KPIs
  32. 32. M O B I L E T E S T I N G Crawl with JavaScript enabled
  33. 33. JavaScript is the browser’s behaviour language M O B I L E T E S T I N G What is JavaScript? ● Tells the browser what to display ● Tree of nodes, described by tags ● Nodes can contain: ○ Text ○ Metadata ○ Links ○ (Assets) - not relevant ○ (Style) - not relevant ● Uses HTML to tell the browser what to display ● Allows a developer to dictate (‘Script’) a browser’s behaviour after receiving a Web Page ● Can make extra requests ● Can react to user interaction (clicks, mouse movement, keyboard, etc…) ● Can store information on the user’s computer ● Can modify the HTML HTML is the browser’s display language
  34. 34. M O B I L E T E S T I N G Simulate with JavaScript enabled
  35. 35. MIGRATEVALIDATEPLAN M O B I L E T E S T I N G Recommended Approach
  36. 36. AUDIT MOBILE SITE 1 OPTIMISE CRAWL BUDGET 2 GROW ORGANIC REVENUE 3
  37. 37. STEP 2 OPTIMISE MOBILE CRAWL BUDGET
  38. 38. ❏ Prune Content ❏ Clean Internal Redirects ❏ Avoid Inconsistent Signals ❏ Use Robots.txt to block sections of your site you don’t want crawled instead of page level noindex tags ❏ Improve your site structure ❏ 304 Status Code ❏ Use 410 to tell Google when content is permanently gone C R A W L B U D G E T Optimisation
  39. 39. AUDIT MOBILE SITE 1 OPTIMISE CRAWL BUDGET 2 GROW ORGANIC REVENUE 3
  40. 40. STEP 3 REVENUE GROWTH CASE STUDY
  41. 41. G R O W R E V E N U E Migrated to Mobile-First
  42. 42. Focus on profitability: # Crawls vs # Visits Do we have ? 1. few crawls vs. many visits 2. few crawls vs. few visits 3. many crawls vs. few visits - low frequency ? - thin content ? - a spider trap ? Allows us to focus on Active Page Ratio: - nb. URLs visited / nb. URLs crawled G R O W R E V E N U E Identify and cut the waste
  43. 43. G R O W R E V E N U E Analyse Architecture
  44. 44. The Results?
  45. 45. Crawl Budget Optimisation
  46. 46. March 15th 17.5% Indexable Pages Crawled April 15th 33.4% Indexable Pages Crawled
  47. 47. 46% Yielding Traffic Growth
  48. 48. THANK YOU!

×