The Technical SEO Renaissance


Published on

Technical SEO is coming back

Published in: Marketing
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The Technical SEO Renaissance

  3. 3. An Abridged History of SEO From my perspective at least
  4. 4. IPULLRANK.COM @ IPULLRANK First There Were Webmasters Before the web world splintered into 50 million different jobs, there was just the Webmaster. In 1995 I was even one of them for Microsoft’s ActiveX team.
  5. 5. IPULLRANK.COM @ IPULLRANK Then Came Search Engines Before Search Engines we just found things on directories or resource pages.
  6. 6. IPULLRANK.COM @ IPULLRANK SEO Grew out of Webmaster/Info Retrieval World Basically, it was just a bunch of nerds trying to get money and succeeding.
  7. 7. IPULLRANK.COM @ IPULLRANK For A Long Time, It Was Just About Hacks and Tricks Remember bolding every single instance of a keyword on the page?
  8. 8. IPULLRANK.COM @ IPULLRANK Then Google Showed Up
  9. 9. IPULLRANK.COM @ IPULLRANK It Was Still About Hacks, Tricks and Testing Things Remember page rank sculpting?
  10. 10. IPULLRANK.COM @ IPULLRANK Then Google Got Better That was a bad day for a lot of people.
  11. 11. IPULLRANK.COM @ IPULLRANK And Better Panda ruined a lot more people’s days.
  12. 12. IPULLRANK.COM @ IPULLRANK And Better With Penguin a lot of people figured it was time to finally go white hat.
  13. 13. IPULLRANK.COM @ IPULLRANK At the Same Time More SEOs Came from Different Disciplines
  14. 14. IPULLRANK.COM @ IPULLRANK And It All Became About Content Now we are the people that make all of the things…right?
  15. 15. And People Started Saying You Don’t Need to Be Technical
  16. 16. That’s right. As an industry, we finally got on the ‘just make good content’ train.
  17. 17. IPULLRANK.COM @ IPULLRANK Yet, the Web is More Technical than Ever Before
  18. 18. We are being dragged into a technical SEO renaissance.
  19. 19. IPULLRANK.COM @ IPULLRANK Lucky for You This Is The Point Where I have My Own Agency Shameless plug.
  20. 20. Changes in Web Technologies What is pushing this renaissance?
  21. 21. IPULLRANK.COM @ IPULLRANK JavaScript is Not Going Away Nearly every site on the web is using JavaScript to perform some type of transformation on the page. It’s obvious that Google needed to solve for this.
  22. 22. IPULLRANK.COM @ IPULLRANK Growth of AngularJS AngularJS, the single page application, MVW framework continues to grow dramatically in the top million websites.
  23. 23. IPULLRANK.COM @ IPULLRANK AngularJS is Popular in Most Verticals It’s likely that you’ll run into an AngularJS site soon enough in your optimization duties
  24. 24. IPULLRANK.COM @ IPULLRANK ReactJS is Growing Too Similarly, Facebook’s MVW framework is growing as well. Although, it has a built in feature for making sure that it renders proper HTML, but it’s still a framework you should know how to optimize for. Check out this post
  25. 25. View Source is Dead
  26. 26. Inspect Element Only
  27. 27. IPULLRANK.COM @ IPULLRANK HTTP/2 – Perhaps the Most Important Thing All you need to know right now is that it’s fast as hell.
  28. 28. Adoption is Super Low For Now Though
  29. 29. We Need More from Our SEO Tools From my perspective at least
  30. 30. What are rankings in 2016?
  31. 31. So, You’re #1? Cool Story, Bro. Where they at tho?
  32. 32. IPULLRANK.COM @ IPULLRANK On What Device, OS and Location? Different browser and OS configurations will show you different results. So what truly is a mobile ranking? + + = Ranking #3+ Nexus 5 Android NYCChrome Safari + + = Ranking #1+ iPhone 6 iOS NYC
  33. 33. Are we measuring for context and actionability or just vanity?
  34. 34. Please Give Me Position 0 in My Rankings
  35. 35. Please Give Me My Results in Context of Paid #1 is really #8 in this SERP
  36. 36. What is cloaking in 2016?
  37. 37. IPULLRANK.COM @ IPULLRANK 304 Response Codes The Not-Modified response code tells clients that they do not need to download a page again if it has already been downloaded.
  38. 38. IPULLRANK.COM @ IPULLRANK Googlebot Respects the 304 What is cloaking when Google accepts 304 response codes and does not revisit that content after indexation?
  39. 39. IPULLRANK.COM @ IPULLRANK What is Cloaking in the Adaptive/Responsive Era? Many people show less or more content based on the context of the browser/user, so what is cloaking now?
  40. 40. Google accepts directives in HTTP headers.
  41. 41. IPULLRANK.COM @ IPULLRANK X-Robots-Tag HTTP Header Many of the popular crawling tools do not look for this in the response headers.
  42. 42. IPULLRANK.COM @ IPULLRANK Hreflang HTTP Header I have not seen a tool that looks for this HTTP header. However, this implementation is preferred over the many lines of hreflang tags on every page.
  43. 43. IPULLRANK.COM @ IPULLRANK Rel-Canonical HTTP Header Many tools do not look for the rel-canonical HTTP header.
  44. 44. IPULLRANK.COM @ IPULLRANK JavaScript Redirects for Mobile Google allows client side redirects for automation redirection of mobile to desktop and vice versa. Many crawling tools do not account for this.
  45. 45. Many SEO tools do not account for these things.
  46. 46. IPULLRANK.COM @ IPULLRANK Chrome DevTools Does Natively in Chrome’s Developer Tools you can see all of the HTTP Request and Response Headers in the Network tab section.
  47. 47. IPULLRANK.COM @ IPULLRANK Remote Debugging You can also see of this for a specific mobile devices with the remote debugging features of Chrome.
  48. 48. Crawlability From my perspective at least
  49. 49. IPULLRANK.COM @ IPULLRANK Google is Crawling Headless Make no mistake that Google has not been crawling with just text driven crawlers for a long time.
  50. 50. IPULLRANK.COM @ IPULLRANK Adam Audette Said It So You KNOW It’s True Adam ran a few tests a little under a year ago further proving out these capabilities
  51. 51. IPULLRANK.COM @ IPULLRANK Also an Ex-Googler Confirmed It A former Google developer said the headless crawling was his primary project from 2006 – 2010.
  52. 52. IPULLRANK.COM @ IPULLRANK I Said it in 2011
  53. 53. But Mike, what does it matter?
  54. 54. IPULLRANK.COM @ IPULLRANK Log Files Went out of Style? No, log file analysis has not gone out of style. In fact, it’s more valuable than ever before. LOOK AT YOUR LOG FILES!
  55. 55. IPULLRANK.COM @ IPULLRANK Log File Analysis We parse the logs and query them in MySQL, but you can use any number of tools for this.
  56. 56. IPULLRANK.COM @ IPULLRANK Search Console’s Crawl Stats are Mediocre I personally find what Google gives us to be pretty much useless. Don’t less this take the place of going through your logs.
  57. 57. IPULLRANK.COM @ IPULLRANK Log Files Help You Spot What Really Happens I had a client come last year that was unsure whether it was Penguin or changes in their offline media buying habits had led to the downturn in traffic. Layering the Googlebot logs made it clear. 0 50000000 100000000 150000000 200000000 250000000 0 500000 1000000 1500000 2000000 2500000 Unknown organic organic home organic dyor organic er organic home,dyor,er organic cushion organic net of home organic ritani organic diamonds direct home paid brand sessions tv spend web sessions (as reported by adaptv) Top 800 Monthly SV impressions Googlebot Visits Penguin Update
  58. 58. IPULLRANK.COM @ IPULLRANK Social Shares Influence Crawl Patterns We find that there is a higher correlation with crawling and social shares than crawling and links. Commonly held SEO knowledge would make you believe this is entirely dictated by links. 0 1000000 2000000 3000000 4000000 5000000 6000000 0 100000 200000 300000 400000 500000 600000 700000 Googlebot Visits Links Social Shares FB Twitter G+
  59. 59. IPULLRANK.COM @ IPULLRANK Prerendering is the Best Practice Using a solution like is the best practice.
  60. 60. IPULLRANK.COM @ IPULLRANK Prerender Can Cause Problems However, we’ve seen Prerender setups misidentify Googlebot as a human users and serve the wrong the AngularJS versions. 0 20000 40000 60000 80000 100000 120000 Googlebot Perceived as Human
  61. 61. IPULLRANK.COM @ IPULLRANK Prerender Setup Serving Non-200 Prepping the prerender cache can cause outages and lead to pages falling in and out of the index. 0 2000 4000 6000 8000 10000 12000 14000 16000 206 301 302 304 403 404 405 410 500 503 504 206 301 302 304 403 404 410 503 504 206 301 302 304 404 405 410 500 503 504 206 301 302 304 401 404 405 410 500 503 504 206 301 302 304 403 404 405 410 502 503 504 1 2 3 4 5 - hit miss pass
  62. 62. IPULLRANK.COM @ IPULLRANK Also Prerendering is Not a Requirement We got rid of prerender on 2 sections of the site and saw the traffic from Organic Search increase YoY. Google can index it just fine.
  63. 63. IPULLRANK.COM @ IPULLRANK AJAX Crawling Deprecated Google has gotten rid of the AJAX crawling scheme because they don’t need it anymore. They have perfected their ability to crawl headless at reasonable scale. They say don’t block the CSS or JS because it’s a browser.
  64. 64. They Say Don’t Prerender Just for Google
  65. 65. IPULLRANK.COM @ IPULLRANK Prerender Does Speed Up Crawling There is certainly value in prerendering for speed, but then that makes me wonder, why not just not use Angular?
  66. 66. Scraping The backbone of a lot of our analysis as SEOs
  67. 67. IPULLRANK.COM @ IPULLRANK Most Tools use cURL or Similar There are many similar libraries, but many SEO tools are built on library called cURL for managing HTTP requests. This is a text based crawler.
  68. 68. This is a fundamental flaw of many SEO tools because they are not constructing the DOM and its CSS/JS transformations. Search Engines are.
  69. 69. IPULLRANK.COM @ IPULLRANK PhantomJS – For Scraping the Modern Web There are many headless browsers out there such as Selenium, but PhantomJS is the go to.
  70. 70. IPULLRANK.COM @ IPULLRANK HorsemanJS – The Easy PhantomJS I prefer HorsemanJS which is a NodeJS module that is incredibly simple.
  71. 71. IPULLRANK.COM @ IPULLRANK In-Browser Scraping I wrote a post on how you can scrape using your browser’s Console.
  72. 72. IPULLRANK.COM @ IPULLRANK Scrape the Unscrapable Nothing is in unscrapable when you’re using a browser.
  73. 73. IPULLRANK.COM @ IPULLRANK Multi-Page In-browser Scraping If you want to scrape multiple pages check out ArtooJS
  74. 74. Content & Linking You knew I wasn’t going to get up here and not talk about these things, right?
  75. 75. IPULLRANK.COM @ IPULLRANK Google Looks at Entities First According to Paul Haahr, Google looks at entities first and it appears that we as an industry have talked about this in theory, but not truly worked it into the optimization workflow.
  76. 76. IPULLRANK.COM @ IPULLRANK AlchemyAPI I encourage you to work AlchemyAPI into your keyword research and content optimization process.
  77. 77. IPULLRANK.COM @ IPULLRANK Term Relevance There’s also been a lot of talk about co-relevance, topic modeling and TF*IDF. This is a great opportunity for technical elements to influence content creation.
  78. 78. IPULLRANK.COM @ IPULLRANK In Fact Just Do Everything Cyrus Says
  79. 79. IPULLRANK.COM @ IPULLRANK Moz’s New Tool Will Help Moz has made the related keywords concept more accessible by adding it to Moz Pro. Searchmetrics also has a similar solution.
  80. 80. IPULLRANK.COM @ IPULLRANK John Mueller Says 302s Pass PageRank Don’t let Google mislead you about 302s vs. 301s.
  81. 81. IPULLRANK.COM @ IPULLRANK These Rankings Say He’s Wrong We had a client with millions for 302s and all we did was change them to 301s June August September HyperTraffic (1-3) 1272 1,283 1,473 Traffic (4-10) 979 983 837 Striking Distance (11-20) 167 161 110 Emerging (21-40) 106 103 98 Developmental (41-100) 156 90 87 Unranked 0 60 75 0 200 400 600 800 1000 1200 1400 1600 301 redirects
  82. 82. IPULLRANK.COM @ IPULLRANK This Traffic Improvement Says He’s Wrong Traffic went up pretty dramatically despite seasonality. August 27th 301 redirects 11,039,188 10,455,517 11,348,902 12,049,481 11,250,148 12,902,847 11,040,192 13,158,296 11,287,790 12,203,998 12,508,164 13,158,296 14,041,313 15,609,390 0 2,000,000 4,000,000 6,000,000 8,000,000 10,000,000 12,000,000 14,000,000 16,000,000 18,000,000 April May Jun Jul Aug Sep Oct Organic Visits 2014 2015
  83. 83. IPULLRANK.COM @ IPULLRANK Internal Linking Structures Internal Linking Structures are still an incredibly powerful asset. I’d like to see more tools like what Portent has built to visualize them and understand where there is opportunity.
  84. 84. Google is becoming the presentation layer of the web. Help them do it.
  85. 85. IPULLRANK.COM @ IPULLRANK / JSON-LD is a lot more reasonable to implement now that they are allowing JSON-LD.
  86. 86. Page Speed Google’s next big play
  87. 87. IPULLRANK.COM @ IPULLRANK Google Expects Ludicrously Speedy Google has an expectation that the above the fold content on the page will load within 1 second. You basically get 400 ms to make that happen.
  88. 88. IPULLRANK.COM @ IPULLRANK The Critical Rendering Path
  89. 89. IPULLRANK.COM @ IPULLRANK DevTools Is Your Friend In the Timeline section, DevTools gives you a point by point break down of each operation and how it impacts the site’s load time. This is how you can determine what exactly is slowing down your pages.
  90. 90. IPULLRANK.COM @ IPULLRANK How Your Code Is Laid Out Matters A LOT
  91. 91. IPULLRANK.COM @ IPULLRANK This Is what PageSpeed Insights Helps With
  92. 92. IPULLRANK.COM @ IPULLRANK External Resources Also Slow Down the Page Chartbeat is typically a huge culprit and using rel-dns-prefetch can speed this up considerably.
  93. 93. IPULLRANK.COM @ IPULLRANK AMP is a Spec to Enforce the Critical Rendering Path Sound familiar?
  94. 94. IPULLRANK.COM @ IPULLRANK Responsive Sites are SLOOOOOOOW One of the key things you can do, aside from optimizing for the critical rendering path, is conditional loading.
  95. 95. IPULLRANK.COM @ IPULLRANK PreBrowsing Directives Browsers have preloading directives which allow the page to download elements in the background for later viewing or within the same session.
  96. 96. IPULLRANK.COM @ IPULLRANK Not to Be Confused with
  97. 97. IPULLRANK.COM @ IPULLRANK Google Uses It in the SERPs
  98. 98. IPULLRANK.COM @ IPULLRANK I Sent Thousands of Headless Visits
  99. 99. IPULLRANK.COM @ IPULLRANK Rel-Prerender’d Pages Generally Perform Better
  100. 100. IPULLRANK.COM @ IPULLRANK Use it With the GA API Get the pageviews of the ga:nextPagePath based on the ga:previousPagePath set to the current page that you’re on. You’ll get a list of the pages your users are most likely to visit next. Set one of these as the rel-prerender.
  101. 101. IPULLRANK.COM @ IPULLRANK Code. Inject rel-prerender into the page programmatically. You could also session the URLs the user is going to to ensure you are always prerendering a new page.
  102. 102. IPULLRANK.COM @ IPULLRANK Speed Improvements …
  103. 103. Don’t use it on mobile. Analytics packages aside from Google Analytics may show fake sessions.
  104. 104. IPULLRANK.COM @ IPULLRANK Defer JS Load with Page Visibility API …
  105. 105. IPULLRANK.COM @ IPULLRANK Rel-Preload for Same Page Preloading I haven’t played with it yet, but Ian says they’re using it on the Portent site.
  106. 106. How to Go Forward Things you should know and do to be prepared for the technical renaissance
  107. 107. IPULLRANK.COM @ IPULLRANK Understand the Document Object Model
  108. 108. IPULLRANK.COM @ IPULLRANK Critical Rendering Path
  109. 109. IPULLRANK.COM @ IPULLRANK Page Speed
  110. 110. IPULLRANK.COM @ IPULLRANK Log File Analysis
  111. 111. IPULLRANK.COM @ IPULLRANK Understand SEO for JavaScript Frameworks
  112. 112. IPULLRANK.COM @ IPULLRANK Learn DevTools
  113. 113. IPULLRANK.COM @ IPULLRANK Accelerated Mobile & Instant Pages Even if you’re not a publisher, I encourage you to watch these projects as I expect that Google and Facebook will grow the specs dramatically once they’re adopted.
  115. 115. IPULLRANK.COM @ IPULLRANK Get Back to Testing All The Things
  116. 116. Let’s stop chasing the content train and get back to making experiences that perform.
  117. 117. IPULLRANK.COM @ IPULLRANK I Just Feel Like Michelangelo Has a Place In This Deck
  118. 118. That’s All I’ve Got
  119. 119. THANK YOU Michael King Managing Director, iPullRank @iPullRank