Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

SEO for Progressive Web Apps (PWA) and JS Frameworks

4,468 views

Published on

Learn how to optimise Progressive Web Apps (PWA) for SEO. By Natzir Turrado and Christian Oliveira for "Congreso Web Zaragoza" and for the Lisbon SEO meetup (https://www.meetup.com/es-ES/Lisbon-SEO-Search-Engine-Optimization-Meetup/events/250906778/ )

Published in: Internet
  • Be the first to comment

SEO for Progressive Web Apps (PWA) and JS Frameworks

  1. 1. SEO for PWA What PWA are and how to SEO-optimize them By Christian Oliveira & Natzir Turrado
  2. 2. What we are going to learn 1. What is a PWA and how it works 2. How search engines deal with Javascript (JS) 3. Rendering approaches for JS sites 4. Good practices for PWA and JS sites 5. Experiment: PWA without prerender (CSR) 6. Conclusion
  3. 3. 1. What is a PWA and how it works
  4. 4. Native Apps $ Longer funnel Fewer and more expensive developers Support for different versions Device support Investment in another channel More time invested in development Native And PWA: Choices, Not Challengers! - https://goo.gl/fq6X2E
  5. 5. Native Apps How Much Does it Cost to Develop and Build an App - https://goo.gl/uQmoQm SIMPLE APP BBDD/API APP MULTI-FEATURE / ENTERPRISE GAME $10K $10K-50K $50K-150K $5K-250K 2-4 weeks 2-3 months 3-6 months Varies a lot
  6. 6. Native Apps How Much Does it Cost to Develop and Build an App - https://goo.gl/uQmoQm USER PROFILES IN-APP PURCHASES GEO-LOCATION SYNC DATA ACROSS DEVICES $7,5K-9,5K $5,5K-15K $7,5K-17K $5,5K-7,5K EMAIL LOGIN SOCIAL LOGIN SOCIAL INTEGRATION RATING SYSTEM $5,5K-7,5K $2K-3K $2K-15K $5,5K-20K
  7. 7. Native Apps “In a consumer mobile app, every step you make a user perform before they get value out of your app will cost you 20% of users.” Gabor Cselle, PM at @Google Every Step Costs You 20% of Users - https://goo.gl/QyaHUn
  8. 8. Mobile Web APPs $ Browser support Not indexable on App Stories Difficult RWD Device support No off-line navigation Web Development Cost / Rate Comparison - https://goo.gl/tQdXuZ
  9. 9. Progressive Web APPs (PWA) “App Experience” Faster transitions and purchases Push notifications Offline navigation Icon on home screen No installation Easily linkable and shareable Indexable on search engines Unified experience Indexable on App stories Progressive Web APPs - https://goo.gl/jrSk5u Progressive and Responsive
  10. 10. PWA use Javascript (JS) libraries and frameworks Huge community Safer Cheap / open-sourceEasier development Better performance
  11. 11. Top 5 frameworks, libraries and other technologies PWA use Javascript (JS) libraries and frameworks Stack Overflow Developer Survey '17 - https://goo.gl/P8S1X7 Top 5 programming languages
  12. 12. How a PWA works {i} JSON S/MPA Service Worker
  13. 13. How a PWA works Service Worker Beyond single-page apps: alternative architectures for your PWA (Google I/O '18) - https://youtu.be/X6yof_vIQnk Cache
  14. 14. How a PWA works Service Worker req Beyond single-page apps: alternative architectures for your PWA (Google I/O '18) - https://youtu.be/X6yof_vIQnk Cache req
  15. 15. How a PWA works Service Worker Beyond single-page apps: alternative architectures for your PWA (Google I/O '18) - https://youtu.be/X6yof_vIQnk Cache req req
  16. 16. How a PWA works Service Worker Beyond single-page apps: alternative architectures for your PWA (Google I/O '18) - https://youtu.be/X6yof_vIQnk Cache req res
  17. 17. How a PWA works {i} JSON Web App Manifest - https://goo.gl/SB2ur3
  18. 18. PWAs: building bridges to mobile, desktop, and native (Google I/O '18) - https://youtu.be/NITk4kXMQDw Browsers support for PWA features ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Add to home screen Service Workers Push notifications Credential Management API Payment Request API Meta Theme Color Supported Beta Development✔ ✔
  19. 19. PWA and speed Better first interaction performance and even better after first interaction The Definitive Guide to Ecommerce Performance - https://goo.gl/YVUngH
  20. 20. PWA and speed 35% faster purchases 49% search The Definitive Guide to Ecommerce Performance - https://goo.gl/YVUngH
  21. 21. PWA+AMP and speed Almost instant load from Search Engines The Definitive Guide to Ecommerce Performance - https://goo.gl/YVUngH
  22. 22. PWA+AMP and speed Service Worker loads on background <amp-install-serviceworker> The Definitive Guide to Ecommerce Performance - https://goo.gl/YVUngH
  23. 23. PWA+AMP and speed User enters the PWA The Definitive Guide to Ecommerce Performance - https://goo.gl/YVUngH
  24. 24. Google/SOASTA Research '17 - https://goo.gl/o1MWEK The importance of performance 1s - 3s the probability of bounce increases 32% 1s - 5s the probability of bounce increases 90% 1s - 6s the probability of bounce increases 106% 1s - 10s the probability of bounce increases 123% As page load time goes from:
  25. 25. Speed Matters Vol.3 - https://goo.gl/QM8cM4 & Ericsson ConsumerLab. Neurons Inc. 2015 - https://goo.gl/NyQd4Z Page load speedResponsive How attractive the site looks Findability The importance of performance What is more important for users? Simplicity 24 % 58 % 61 % 66 % 75 % Slow websites cause more stress than horror movies
  26. 26. 2. How search engines deal with Javascript (JS)
  27. 27. Only Google and ASK render JS, Baidu tries Baidu Adds New Spider with Rendering Job - https://goo.gl/Jd79qW
  28. 28. Crawling =! Rendering =! Indexing =! Ranking 49 indexed pages 28indexed pages 17 indexed pages 30indexed pages
  29. 29. JS rendering costs A LOT The Cost of JavaScript - https://goo.gl/6jwY3U 22s35s 14s 10% sites 25% sites 50% sites Time To Interaction (TTI) on mobile devices 170 KB of JS !== 170 KB of JPEG
  30. 30. JS rendering costs A LOT “To execute JS for every page at our scale [would] require 10,000–15,000 servers, and we believe our customers are not ready to pay for that yet.” Dmitry Gerasimenko, CEO Ahrefs Ahrefs crawlers are now rendering web pages and executing JavaScript - https://goo.gl/DQuero ~173 urls per second per server ~0,86 urls per second per server Ahrefs every 24 hours: ● Crawl 6 billion pages ● Render 30 million JS pages ● Which result in 250 million JS links ● With 400 servers
  31. 31. Google won’t wait indefinitely to render a page 5’ Time Google waits according to our own experiments, confirmed with other SEOs There is no official time - https://goo.gl/MhNXdd ’
  32. 32. Google won’t wait indefinitely to render a page 5’ Time Google waits according to our own experiments, confirmed with other SEOs There is no official time - https://goo.gl/MhNXdd ’ Use the snapshot of the Mobile Friendly Tool as a guide
  33. 33. Google renders JS in 2 phases CRAWL INDEX RENDER Instant, 1st wave of indexing 2nd wave of indexing New links to be crawled As rendering resources become available Deliver search-friendly JavaScript-powered websites (Google I/O '18) - https://youtu.be/PFwUbgvpdaQ
  34. 34. Google renders JS in 2 phases CRAWL INDEX RENDER Instant, 1st wave of indexing 2nd wave of indexing New links to be crawled As rendering resources become available Deliver search-friendly JavaScript-powered websites (Google I/O '18) - https://youtu.be/PFwUbgvpdaQ Google defers rendering because of lack of resources
  35. 35. Google renders JS in 2 phases CRAWL INDEX RENDER Instant, 1st wave of indexing 2nd wave of indexing New links to be crawled As rendering resources become available Deliver search-friendly JavaScript-powered websites (Google I/O '18) - https://youtu.be/PFwUbgvpdaQ Reads canonical, amphtml and https status only on initial fetch
  36. 36. Google renders JS in 2 phases CRAWL INDEX RENDER Instant, 1st wave of indexing 2nd wave of indexing New links to be crawled As rendering resources become available Deliver search-friendly JavaScript-powered websites (Google I/O '18) - https://youtu.be/PFwUbgvpdaQ If Noindex, Hreflang and rel=next/prev are not on the html, will read them on the rendered page
  37. 37. Google renders JS in 2 phases CRAWL INDEX RENDER Instant, 1st wave of indexing 2nd wave of indexing New links to be crawled As rendering resources become available Deliver search-friendly JavaScript-powered websites (Google I/O '18) - https://youtu.be/PFwUbgvpdaQ Google uses Chrome 41 for rendering. It is done by caffeine with a WRS subsystem
  38. 38. Types of events and links Google follows Deliver search-friendly JavaScript-powered websites (Google I/O '18) - https://youtu.be/PFwUbgvpdaQ Google crawls, index and pass PageRank to plain html links (<a href=...) ● <a href="/this-link"> will be crawled </a> ● <a href="/this-link" onclick="changePage('this-link')"> will be crawled </a> ● <a onclick="changePage('this-link')"> Not crawled </a> ● <span onclick="changePage('this-link')"> Not crawled </span>
  39. 39. Types of events and links Google follows https://goo.gl/rRPWcC & https://goo.gl/aZaECU & https://goo.gl/aZaECU & https://goo.gl/j1ccxF BUT: Google crawls and index anything that looks like a link, in order to discover, although it won’t pass PageRank: ● <a onclick="changePage('/this-link')"> will be crawled </a> ● <span onclick="changePage('/this-link')"> will be crawled </span> ● /this-link will be crawled
  40. 40. Types of events and links Google follows https://goo.gl/rRPWcC & https://goo.gl/aZaECU & https://goo.gl/aZaECU & https://goo.gl/j1ccxF BUT: Google crawls and index anything that looks like a link, in order to discover, although it won’t pass PageRank: ● <a onclick="changePage('/this-link')"> will be crawled </a> ● <span onclick="changePage('/this-link')"> will be crawled </span> ● /this-link will be crawled Onscroll events are followed sometimes, onmouseover events are not followed. If you have pagination which works with onscroll it may get indexed (only the first pages) Knowing what kind of events and links Google follows is important because all indexed URLs are taken into account to evaluate the site’s quality
  41. 41. 3. Rendering approaches for JS sites
  42. 42. Client Side Rendering (CSR) req res: JS JS+JSON=HTML req res: JSON
  43. 43. Server Side Rendering (SSR) JS+JSON = HTML req res: HTML
  44. 44. Hybrid Rendering Building a hybrid-rendered PWA - https://goo.gl/ZNfzr1 SSR CSR
  45. 45. Dynamic Rendering (DR) WEB SERVER Users Googlebot and others CSR or Hybrid rendering SSR Deliver search-friendly JavaScript-powered websites (Google I/O '18) - https://youtu.be/PFwUbgvpdaQ
  46. 46. Dynamic Rendering (DR) WEB SERVER Users Googlebot and others CSR or Hybrid rendering SSR Deliver search-friendly JavaScript-powered websites (Google I/O '18) - https://youtu.be/PFwUbgvpdaQ User-agent or Reverse DNS
  47. 47. Dynamic Rendering (DR) Googlebot Fetch & Render As Any Bot - https://goo.gl/1augcc
  48. 48. Dynamic Rendering (DR) Fetch & Render As Any Bot - https://goo.gl/1augcc UsersGooglebot
  49. 49. What is best for users? SSR
  50. 50. What is best for users? Netflix and SSR - https://goo.gl/ym6fWN
  51. 51. What is best for users? Walmart and SSR - https://goo.gl/EwmPyD SSR CSR FCP >
  52. 52. What is best for users? Walmart and SSR - https://goo.gl/EwmPyD SSR CSR TTFB <
  53. 53. What is best for IT? The Benefits of Server Side Rendering Over Client Side Rendering - https://goo.gl/EwmPyD CSR
  54. 54. What Google recommends? JS features Type of website > Chrome 41 < Chrome 41 Small and static SSR / DR CSR Small and dynamic SSR / DR CSR / SSR / DR Big and static SSR / DR SSR / DR Big and dynamic SSR / DR SSR / DR
  55. 55. What Google recommends? Barry Adams’ tweet - https://goo.gl/umSS1j
  56. 56. 4. Good practices for PWA and JS sites
  57. 57. PWA and performance audit https://developers.google.com/web/tools/lighthouse/ & https://sonarwhal.com
  58. 58. Mobile-Friendly Test / Screenshot Mobile-Friendly Test - https://goo.gl/FjHMk5
  59. 59. Mobile-Friendly Test / Rendered HTML Mobile-Friendly Test - https://goo.gl/FjHMk5
  60. 60. Mobile-Friendly Test / Page loading issues Mobile-Friendly Test - https://goo.gl/FjHMk5
  61. 61. Fetch and Render Fetch & Render is less tolerant waiting to render than the real WRS subsystem
  62. 62. Fetch and Render on any domain Fetch and Render of any domain - https://github.com/screamingfrog/fetch-and-render Add this meta to the original code!
  63. 63. Fetch and Render on any domain https://www.analistaseo.es/fetch-and-render/ ?url=https://www.ejemplo.com&type=mobile Fetch and Render of any domain - https://github.com/screamingfrog/fetch-and-render
  64. 64. Fetch and Render on staging 1. Noindex staging 2. Reverse DNS
  65. 65. Crawl and compare staging with production - https://www.fandangoseo.com/
  66. 66. Crawl emulating mobile and JS rendering - https://www.fandangoseo.com/
  67. 67. Check meta tags with other user agents Meta SEO inspector + User-Agent Switcher - https://goo.gl/tNo3ug & https://goo.gl/7r7rcn Google: User-Agent String: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Googlebot Mobile: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Facebook: facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
  68. 68. Check meta tags with other user agents Meta SEO inspector + User-Agent Switcher - https://goo.gl/tNo3ug & https://goo.gl/7r7rcn
  69. 69. Browse without JS with other user agents Web Developer - https://goo.gl/WqjjLa
  70. 70. Browse without JS with other user agents UsuariosGooglebot Web Developer - https://goo.gl/WqjjLa
  71. 71. Browse without JS with other user agents Web Developer - https://goo.gl/WqjjLa UsuariosGooglebot
  72. 72. Check console errors and links on Chrome 41 Descarga Chrome 41 - https://goo.gl/3tpxpH
  73. 73. ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ SEO+PWA Checklist for CSR 1. Canonicals + metadata do not depend on CSR 2. Page renders in less than 5 seconds 3. Page loads and is functional with Chrome 41 4. Rendered links are <a> elements with their real href 5. HTML+design is the expected on Mobile Friendly Tool 6. HTML+design is the expected on Fetch and Render with Search Console 7. Pages appear on Google after searching for specific text strings 8. Content is not replicated in different pages 9. Fragments (!#)are not used and every page has its own URL
  74. 74. ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ SEO+PWA Checklist for SSR / Dynamic Rendering 1. SSR is served to Googlebot and other specific bots (FB, TW…) 2. Content Googlebot receives is the same a normal user sees 3. Http response codes are the same in both cases 4. Meta tags are the same in both cases 5. Server is able to manage the rendering under high pressure 6. SSR version includes all the content and links in a correct way 7. All pages have the expected behaviour in both cases 8. Fragments (!#)are not used and every page has its own URL
  75. 75. ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ PWA Checklist UX & WPO Progressive Web App Checklist - https://goo.gl/jGj4D2 1. Site is served over HTTPS 2. Pages are responsive on tablets & mobile devices 3. All app URLs load while offline 4. Metadata provided for Add to Home screen 5. First load very fast even on 3G 6. Page transitions don't feel like they block on the network 7. Site uses cache-first networking 8. Site appropriately informs the user when they're offline 9. Content doesn't jump as the page loads 10. Pressing back from a detail page retains scroll position on the previous list page 11. When tapped, inputs aren't obscured by the on screen keyboard 12. Content is easily shareable from standalone or full screen mode 13. Any app install prompts are not used excessively
  76. 76. 5. Experiment: PWA without prerender (CSR)
  77. 77. Experiment: PWA without prerender (CSR) Worona - https://www.worona.org/ Before: classic wordpress After: PWA with Worona (CSR)
  78. 78. Experiment: PWA without prerender (CSR) CSR = No content if JS is not executed
  79. 79. Test 1: rendering Page renders on Mobile Friendly Tool Page renders on Search Console Google renders and index the page
  80. 80. Test 1: rendering Page renders on Mobile Friendly Tool Page renders on Search Console Google renders and index the page 15’ after “Request indexing” on SC
  81. 81. Test 1: rendering Observations: ● Google has been able to render and index the content ● The process has been relatively fast (15’)
  82. 82. Test 2: rankings
  83. 83. Test 2: rankings
  84. 84. Test 2: rankings
  85. 85. Test 2: rankings Observations: ● Rankings haven’t been lost after migrating to a PWA without SSR
  86. 86. Test 3: new content indexing We publish a new post We force the blog homepage to be reindexed
  87. 87. Test 3: new content indexing 4 days after it has not been indexed
  88. 88. Test 3: new content indexing Observations: ● If we don’t force the indexing, Google takes some time to index new content
  89. 89. Test 4: the two waves of indexing We add a made up word only on the html version of the page
  90. 90. Test 4: the two waves of indexing We use “Fetch” (not rendering) in SC + Request Indexing
  91. 91. Test 4: the two waves of indexing Minutes later… Google reindex the page and associate the word to it
  92. 92. On the other side, Google’s cache for this page was not updated Test 4: the two waves of indexing
  93. 93. Test 4: the two waves of indexing And the page keeps ranking for text strings from the previous version
  94. 94. Test 4: the two waves of indexing Days after, Google updated the cache with the rendered page
  95. 95. Test 4: the two waves of indexing And, from that moment, the page no longer ranks for the made up word
  96. 96. Test 4: the two waves of indexing Despite the fact that if we check the “Only text” version of the cache, the word is there
  97. 97. Observations: ● Before rendering the page Google has indexed the content on the page without JS (first wave) ● This new content has been associated to the previous version of the web (the one without PWA) ● Once rendered, the page only ranks for the content present on the rendered version (second wave indexing) Test 4: the two waves of indexing
  98. 98. 6. Conclusions
  99. 99. ● Up to date very few websites have migrated directly to a CSR PWA (no pre-rendering) for all user-agents. Those who did and lost traffic, had also other technical problems (e.g. hulu.com) ● New websites which are created directly as CSR PWA can rank, but they have problems with indexing ● Up to date, and regarding Google, Dynamic Rendering is the solution that works without any problems Conclusions
  100. 100. SEO for PWA THANKS! Natzir Turrado analistaseo.es hola@natzir.com @natzir9 Christian Oliveira christianoliveira.com christian@usal.es @christian_wilde

×