Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deep crawl the chaotic landscape of JavaScript

3,160 views

Published on

Find out the latest trends and research in the JavaScript SEO

Published in: Marketing

Deep crawl the chaotic landscape of JavaScript

  1. 1. <h2>The Chaotic Landscape of JavaScript </h4>
  2. 2. What is JavaScript SEO? What it takes for a modern JavaScript- powered website to be properly indexed by search crawlers?
  3. 3. 7 Random Statements 1. SEO was never this dynamic 2. Google is pushing more changes than ever, they aren’t as popular as Penguin and Panda 3. I still see too many SEOs who live in a fantasy world made out out of MOZ DA scores, keyword density and SEO “hacks” 4. That tiny pocket in jeans was designed to store pocket watches 5. Looking at some of the biggest websites in the world, it seems that most didn’t hear about Technical SEO yet. 6. McDonald’s once made bubblegum-flavored broccoli 7. JavaScript is here to stay and JavaScript SEO is not a geeky option anymore (YAY!)
  4. 4. Have you heard about the 2 waves of JavaScript indexing?
  5. 5. Web development HTML → HTML + CSS → HTML + a little JS → HTML + a lot of JS → JS
  6. 6. Read more: ele.ph/budget
  7. 7. Read more: ele.ph/googleio
  8. 8. Why isn’t this process instant?
  9. 9. Read more: ele.ph/budget
  10. 10. Source: Google I/O 2018
  11. 11. “…Rendering pages at the scale of the web requires a lot of time and computational resources. And make no mistake, this is a serious challenge for search crawlers, Googlebot included.”
  12. 12. “Rendering the JavaScript powered web pages takes processor power and memory. While Googlebot is very powerful, it doesn’t have infinite resources.”
  13. 13. 1 user = 60 Watt lightbulb lit for 3 hours* *Data from 2011
  14. 14. THE REAL COST of JavaScript
  15. 15. JavaScript “lives” in your CPU
  16. 16. JavaScript HTML
  17. 17. Enter JavaScript 1 100
  18. 18. USA TODAY Case Study
  19. 19. “The European Union Experience”
  20. 20. USA Today without JavaScript
  21. 21. The Guardian vs. CPU A Case Study
  22. 22. It is believed that The Guardian & Amazon are performance superstars.
  23. 23. “…Many of us still look at sites like Amazon’s or the Guardian’s in awe…”
  24. 24. How fast do you think The Guardian loads?
  25. 25. 137
  26. 26. But this performance comes with a price…
  27. 27. CPU
  28. 28. Go to Chrome Dev Tools → Performance CMD + OPT + I CTRL + ALT + I
  29. 29. HIGH END MACHINE - THE BEST CPU YOU CAN HAVE IN A MACBOOK - 10 seconds
  30. 30. Slower CPU Load Time – 90 seconds
  31. 31. CPUs 1-on-1
  32. 32. Source: https://medium.com/dev-channel/the-cost-of-javascript-84009f51e99e
  33. 33. Unfortunately not all websites are like The Guardian…
  34. 34. accuweather.com
  35. 35. SECONDS TO FIRST MEANINGFUL PAINT HIGH END CPU
  36. 36. 6sec
  37. 37. SECONDS TO FIRST MEANINGFUL PAINT (Moto G4’s CPU)
  38. 38. 13 SECOND DIFFERENCE BETWEEN A HIGH END CPU AND A SLOW CPU (!)
  39. 39. …enter
  40. 40. Rewriting JavaScript
  41. 41. • Technical SEO in 2018 - New metrics, new technologies and new challenges.
  42. 42. What did they need React for? Language switcher Buttons on the bottom of the page Client side login library
  43. 43. ~300 lines of code written ~200 kB removed
  44. 44. performance improvement
  45. 45. Netflix’s performance (and SEO) doesn’t depend on the client’s CPU that much
  46. 46. AMP – BBC.COM Netflix.com Netflix.com @ Moto G4
  47. 47. ~200 KB to process 4 seconds 0.065 seconds
  48. 48. How about ~400KB?
  49. 49. Source: Google I/O 2018
  50. 50. Read more: ele.ph/googleio
  51. 51. How do you ensure you are not caught up between the waves? You need to embrace chaos!
  52. 52. There are And a lot of those “documents” are getting more and more complex!
  53. 53. With the exponential growth of JavaScript frameworks...
  54. 54. With the exponential growth of JavaScript frameworks...
  55. 55. artoszCoin barti How to be cool in 2018
  56. 56. The Big Three Source: https://javascriptreport.com/the-ultimate-guide-to-javascript-frameworks/
  57. 57. Notable frameworks
  58. 58. Rest of the pack
  59. 59. Are they OK for SEO?
  60. 60. JavaScript Frameworks &
  61. 61. ? ? ? ?? ?? ?? ? ? ? ? ? ?? ?? ? ?? ? ??? ? ? ? ? ? ? Let’s hear the framework creators talk about JavaScript & SEO
  62. 62. ...if you care about SEO, you still need to have server-rendered content. Jeff Whelpley Angular U conference, June 22-25, 2015, Hyatt Regency, San Francisco Airport “Angular 2 Server Rendering” ele.ph/angularU
  63. 63. ...we are generally able to render and understand your web pages like modern browsers.
  64. 64. Chaos?
  65. 65. Indexing vs. Ranking
  66. 66. Can you rank with a JS website?
  67. 67. As Googlers would say “there are factors at play here”
  68. 68. client rendered JS
  69. 69. Let’s search for Hulu’s exclusive shows.
  70. 70. Ctrl+C
  71. 71. Ctrl+V
  72. 72. Everyone hates him for telling the truth! Here’s why you’re wasting your time on client-rendered JS websites… Findoutnow!
  73. 73. SEO – where the giants fight!
  74. 74. YouTube vs. Vimeo
  75. 75. Why is their visibility declining?
  76. 76. Vimeo’s homepage without JS!
  77. 77. I’m about to reveal the YouTube secret no one ever told you.
  78. 78. YouTube is a client-rendered JS website…
  79. 79. YouTube with JS disabled
  80. 80. …and they are prerendering for Googlebot
  81. 81. Why would YouTube prerender for Googlebot? Doesn’t Google render JS just fine?
  82. 82. SEO – where the giants fight
  83. 83. JavaScript vs. Crawler budget (crawl demand)
  84. 84. JavaScript vs. Crawler budget (crawl demand) Oh no! ele.ph/budget
  85. 85. JavaScript vs. Crawler budget (crawl demand) Oh no! (…) Also, crawling & indexing is currently a bit slower than static HTML (...) ele.ph/crawldemand
  86. 86. Crawling and indexing JavaScript is not a 0/1 thing
  87. 87. GSC fetch URL Indexed? Content Indexed? Test URL External Ajax callInline
  88. 88. JavaScript dozens of different frameworks their different configurations transpiling inline vs. external
  89. 89. On top of that, there are hundreds of factors affecting crawling and indexing
  90. 90. ?How about Search Engines other than
  91. 91. Get ready for real chaos!
  92. 92. Search Engines
  93. 93. SEO
  94. 94. statements reality guy @ Pubcon our experiment
  95. 95. THE BING THING Fabrice Canel of Bing said that Bing processes JS. True story! Patrick Stox @ DTD Conference 2018
  96. 96. Rich media cautions don’t bury links to content inside JavaScript
  97. 97. Rich media cautions Graceful degradation enable a clean down- level experience so crawlers can see your content
  98. 98. Rich media cautions Graceful degradation Down-level experience enhances discoverability avoid housing content inside Flash or JavaScript – these block crawlers from finding the content
  99. 99. It can process JS vs. It DOES process JS
  100. 100. Proof
  101. 101. 1. Go to Angular.io
  102. 102. 2. Copy any text from the website CTRL+C
  103. 103. 3. Try to find it in Bing.com CTRL+V 4. Find everything BUT the angular.io website
  104. 104. Or search for any content from jsseo.expert
  105. 105. Google vs. Bing
  106. 106. Thanks, Dan Petrovic!
  107. 107. ele.ph/beyondgoogle
  108. 108. 63,4% 22,8% 11,7% 1,3% 0,8% Google Bing Yahoo! Ask AOL Search Engine market share Source: https://www.statista.com/statistics/267161/market-share-of-search-engines-in-the-united-states/
  109. 109. 83,5% 11,1% 3,8% 0,8% 0,6% Google Bing Yahoo! MSN DuckDuckGo Search Engine market share Source: https://www.statista.com/statistics/280269/market-share-held-by-search-engines-in-the-united-kingdom/
  110. 110. 94,5% 4,2% 1,0% 0,1% 0,1% Google Bing Yahoo! T-Online Ask.com Search Engine market share Source: https://www.statista.com/statistics/445002/market-shares-leading-search-engines-germany/
  111. 111. ele.ph/beyondgoogle
  112. 112. Let’s take a closer look at Google
  113. 113. JavaScript is REALLY easy to break
  114. 114. It’s all very shaky, based on duct tape rather than…
  115. 115. Let’s start from Angular.io and their GitHub issue
  116. 116. noindex meta tag?
  117. 117. https://angular.io/guide/language-service https://angular.io/guide/security https://angular.io/guide/architecture https://angular.io/api/animations/group https://angular.io/guide/quickstart
  118. 118. https://angular.io/guide/language-service https://angular.io/guide/security https://angular.io/guide/architecture https://angular.io/api/animations/group https://angular.io/guide/quickstart https://angular.io https://angular.io/events https://angular.io/api https://angular.io/resources https://angular.io/about?group=Angular
  119. 119. https://angular.io/guide/language-service https://angular.io/guide/security https://angular.io/guide/architecture https://angular.io/api/animations/group https://angular.io/guide/quickstart https://angular.io https://angular.io/events https://angular.io/api https://angular.io/resources https://angular.io/about?group=Angular url / / url /
  120. 120. UPSIDE DOWN The only thing [those] pages (…) have in common is the number of path segments in the url. Igor Minar
  121. 121. SEO Visibility
  122. 122. http://yourdomain.com/category/shoes/ url / / url / How much would that cost you? http://yourdomain.com/category/
  123. 123. Prerender.io – is partial indexing in Google possible?
  124. 124. Visibility
  125. 125. vs. Developers SEOs vs. Search Engines vs. Framework creators
  126. 126. Developers are confused af…
  127. 127. …We’ve created what will be remembered as the world’s most confusing JavaScript survey ever.
  128. 128. …it contained only 1 question…
  129. 129. …but proved to be the most difficult question you can ask your developer.
  130. 130. Can client-side rendered JavaScript websites rank high in Google?
  131. 131. Node.js developers group 35 7
  132. 132. ReactJS group 6 21
  133. 133. Angular2+ group 4 9
  134. 134. JavaScript group 2 34
  135. 135. Solution? SEO team
  136. 136. Solution? SEO team
  137. 137. Can you rank well with a Client Rendered JavaScript Website?
  138. 138. Quick check
  139. 139. JavaScript Enabled
  140. 140. JavaScript Disabled
  141. 141. React Native
  142. 142. How about their rankings?
  143. 143. .com
  144. 144. #1 for
  145. 145. #1 for
  146. 146. Is there hope for client-rendered JS websites?
  147. 147. Well…
  148. 148. .No
  149. 149. Ranking with a CSR JavaScript website Ranking without it
  150. 150. Google Flights ele.ph/flights
  151. 151. Google Flights ele.ph/flights
  152. 152. A look behind the curtain
  153. 153. Source: Google I/O 2018
  154. 154. What is partial indexing? This means that if your site is using a heavy amount of client side JS, you can be tripped up at times when the content is being indexed due to this two phase indexing process. It’s possible that some details may be missed.
  155. 155. And this effectively means that if your site is using a heavy amount of client-side JavaScript for rendering, you could be tripped up at times when your content is being indexed due to the nature of this two phase indexing process. …it’s possible some details might be missed. Tom Greenaway
  156. 156. 2 waves WAVE 1 WAVE 2 • JS dependent content only • HTML content • Canonicals • Meta data • HTTP Codes
  157. 157. Indexing of modern JavaScript websites is a challenge. John Mueller
  158. 158. Rendering
  159. 159. Client Side rendering Source: Google I/O 2018
  160. 160. Server-side rendering Source: Google I/O 2018
  161. 161. Hybrid Rendering* *Google’s Long-Term recommendation Source: Google I/O 2018
  162. 162. Google’s policy change?
  163. 163. Dynamic Rendering (aka prerendering)
  164. 164. Prerendering/Dynamic Rendering issues 1. Computing power – A LOT of servers 2. Prone to issues (often load related) 3. Downtime = ranking loss 4. More complex and difficult from an SEO perspective (crawls, 2 sets of code, etc.) 5. Requires a lot of SEO knowledge 6. Requires a great dev team to make it run smoothly
  165. 165. When to use dynamic rendering (according to Google) A dynamic website “Is if your site is large and rapidly changing for, example if you have a news website.” Your website is relying on features that are not supported in Chrome 41 Libraries that cannot be transpiled back to ES5, APIs that don’t suport Chrome 41
  166. 166. Troubleshooting JavaScript indexing
  167. 167. Google Search Console – fetch and render
  168. 168. Google Search Console – fetch and render
  169. 169. Google mobile-friendly test g.co/MobileFriendly
  170. 170. Google mobile-friendly test ele.ph/hulu-test
  171. 171. Google mobile-friendly test ele.ph/hulu-test COPY THIS CODE
  172. 172. Diff Checker
  173. 173. Google mobile-friendly test
  174. 174. Google mobile- friendly test
  175. 175. Homework
  176. 176. #SMX #XXA @McCieslak 1. The Source Code vs. DOM Source code DOM
  177. 177. #SMX #XXA @McCieslak GSC fetch URL Indexed? Content Indexed? Test URL InlineInlineV2 Inline 2. Crawling - server log
  178. 178. #SMX #XXA @McCieslak
  179. 179. #SMX #XXA @McCieslak
  180. 180. Make sure your content is indexed ctrl+c ctrl+v hulu.com/casual
  181. 181. Under the right URL https://angular.io/features
  182. 182. To Do: 1. Diff Check is your new best friend 2. Experiment and make sure to monitor your server logs 3. Compare your setup with Chrome 41 4. Make sure that your content is indexed in Google 5. Under the right URL
  183. 183. Wrap up JS
  184. 184. Wrap up JS 1 JS can kill crawler budget
  185. 185. Wrap up JS 1 JS can kill a crawler budget 2 JS websites have ranking problems
  186. 186. Ranking well with CSR JavaScript websites is very hard if not impossible.
  187. 187. 2 yet Make sure you choose a right frawework
  188. 188. We are still looking for a pioneer!
  189. 189. Bartosz@Goralewicz.com @bart_Goralewicz

×