Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

SearchLove Boston 2016 | Mike King | Developer Thinking for SEOs


Published on

Despite the huge shift to content marketing in recent years, the technical end of SEO has gotten increasingly complex and our tools are not keeping pace. As SEOs, we must develop strong working knowledge of the optimal usage of technology to get implementations accomplished. In this talk, Mike will walk through case studies, the impact of different technical implementations, and how to pull together small solutions when nothing on the shelf works for your needs.

Published in: Marketing
  • Dating direct: ♥♥♥ ♥♥♥
    Are you sure you want to  Yes  No
    Your message goes here
  • Dating for everyone is here: ♥♥♥ ♥♥♥
    Are you sure you want to  Yes  No
    Your message goes here
    Are you sure you want to  Yes  No
    Your message goes here
  • The ultimate acne system, Top ranked acne plan for download Unique clear skin strategies ◆◆◆
    Are you sure you want to  Yes  No
    Your message goes here

SearchLove Boston 2016 | Mike King | Developer Thinking for SEOs

  2. 2. Baby Sign Language …and how it relates to technical SEO
  3. 3. IPULLRANK.COM @ IPULLRANK So, I’m Having a Baby (She is too)… It’s a girl!
  4. 4. IPULLRANK.COM @ IPULLRANK Last Night I Learned Infants Can Learn Sign Language …
  5. 5. IPULLRANK.COM @ IPULLRANK Which is Great Because They Act like Drunks Otherwise …
  6. 6. IPULLRANK.COM @ IPULLRANK Which is Astonishingly Similar to the Perception of Marketers
  7. 7. SEOs generally know what they want, just not how to effectively ask for it.
  8. 8. As of Late People Started Saying You Don’t Need to Be Technical
  9. 9. IPULLRANK.COM @ IPULLRANK Yet, the Web is More Technical than Ever Before
  10. 10. The modern SEO needs to think like a developer because we are in a technical SEO renaissance.
  11. 11. IPULLRANK.COM @ IPULLRANK Lucky for You This Is The Point Where I have My Own Agency Shameless plug.
  12. 12. Changes in Web Technologies What is pushing this renaissance?
  13. 13. IPULLRANK.COM @ IPULLRANK JavaScript is Not Going Away Nearly every site on the web is using JavaScript to perform some type of transformation on the page. It’s obvious that Google needed to solve for this.
  14. 14. IPULLRANK.COM @ IPULLRANK Growth of AngularJS AngularJS, the single page application, MVW framework continues to grow dramatically in the top million websites.
  15. 15. IPULLRANK.COM @ IPULLRANK AngularJS is Popular in Most Verticals It’s likely that you’ll run into an AngularJS site soon enough in your optimization duties
  16. 16. IPULLRANK.COM @ IPULLRANK ReactJS is Growing Too Similarly, Facebook’s MVW framework is growing as well. Although, it has a built in feature for making sure that it renders proper HTML, but it’s still a framework you should know how to optimize for. Check out this post
  17. 17. IPULLRANK.COM @ IPULLRANK Ironically, React (Facebook) is SEO-ready, Angular (Google) is Not Angular 1 did not have anything to make it easier to crawl. React has ReactDOMServer.renderToString to allow React to serve fully rendered HTML from the server.
  18. 18. View Source is Dead
  19. 19. Inspect Element Only
  20. 20. IPULLRANK.COM @ IPULLRANK HTTP/2 – Perhaps the Most Important Thing All you need to know right now is that it’s fast as hell.
  21. 21. Adoption is Super Low For Now Though
  22. 22. We Need More from Our SEO Tools Many of them are far behind search engines
  23. 23. What are rankings in 2016?
  24. 24. So, You’re #1? Cool Story, Bro. Where they at tho?
  25. 25. IPULLRANK.COM @ IPULLRANK On What Device, OS and Location? Different browser and OS configurations will show you different results. So what truly is a mobile ranking? + + = Ranking #3+ Nexus 5 Android NYCChrome Safari + + = Ranking #1+ iPhone 6 iOS NYC
  26. 26. Are we measuring for context and actionability or just vanity?
  27. 27. Please Give Me Position 0 in My Rankings
  28. 28. Please Give Me My Results in Context of Paid #1 is really #8 in this SERP
  29. 29. What is cloaking in 2016?
  30. 30. IPULLRANK.COM @ IPULLRANK 304 Response Codes The Not-Modified response code tells clients that they do not need to download a page again if it has already been downloaded.
  31. 31. IPULLRANK.COM @ IPULLRANK Googlebot Respects the 304 What is cloaking when Google accepts 304 response codes and does not revisit that content after indexation?
  32. 32. IPULLRANK.COM @ IPULLRANK What is Cloaking in the Adaptive/Responsive Era? Many people show less or more content based on the context of the browser/user, so what is cloaking now?
  33. 33. Google accepts directives in HTTP headers.
  34. 34. IPULLRANK.COM @ IPULLRANK X-Robots-Tag HTTP Header Many of the popular crawling tools do not look for this in the response headers.
  35. 35. IPULLRANK.COM @ IPULLRANK Hreflang HTTP Header I have not seen a tool that looks for this HTTP header. However, this implementation is preferred over the many lines of hreflang tags on every page.
  36. 36. IPULLRANK.COM @ IPULLRANK Rel-Canonical HTTP Header Many tools do not look for the rel-canonical HTTP header.
  37. 37. All of these HTTP headers are better than clogging up the <head> of the code.
  38. 38. IPULLRANK.COM @ IPULLRANK JavaScript Redirects for Mobile Google allows client side redirects for automation redirection of mobile to desktop and vice versa. Many crawling tools do not account for this.
  39. 39. Many SEO tools do not account for these things
  40. 40. IPULLRANK.COM @ IPULLRANK Chrome DevTools Does Natively in Chrome’s Developer Tools you can see all of the HTTP Request and Response Headers in the Network tab section.
  41. 41. IPULLRANK.COM @ IPULLRANK Geolocation Emulation is Built-in to Chrome If you’re running into issues with finding local rankings now that Google has gotten rid of nearEquals you can just change your location on the browser level.
  42. 42. IPULLRANK.COM @ IPULLRANK Remote Debugging You can also see this for a specific mobile devices with the remote debugging features of Chrome.
  43. 43. Crawlability What can be crawled at this point?
  44. 44. IPULLRANK.COM @ IPULLRANK Google is Crawling Headless Make no mistake that Google has not been crawling with just text driven crawlers for a long time.
  45. 45. IPULLRANK.COM @ IPULLRANK Adam Audette Said It So You KNOW It’s True Adam ran a few tests a little under a year ago further proving out these capabilities
  46. 46. IPULLRANK.COM @ IPULLRANK Also an Ex-Googler Confirmed It A former Google developer said the headless crawling was his primary project from 2006 – 2010.
  47. 47. IPULLRANK.COM @ IPULLRANK I Said it in 2011
  48. 48. IPULLRANK.COM @ IPULLRANK Eric Wu has been doing tests to get a better determination of what it is that Google can actually see.
  49. 49. But Mike, what does it matter?
  50. 50. IPULLRANK.COM @ IPULLRANK Well I Captured Video of Headless Googlebot It’s a question of what does Googlebot do on your site. Using LuckyOrange I captured Googlebot sessions from Search Console’s Fetch and Render and then I submitted it to the index. They load the page and leave. The mouse never moves, the headless browser never came back.
  51. 51. IPULLRANK.COM @ IPULLRANK Get a Free Trial of LuckyOrange and Try it Yourself This code will only place LuckyOrange on the page if a JS-enabled Googlebot comes to your site. My test did not show what Googlebot from the wild does when it’s crawling.
  52. 52. IPULLRANK.COM @ IPULLRANK Log Files Went out of Style? No, log file analysis has not gone out of style. In fact, it’s more valuable than ever before. LOOK AT YOUR LOG FILES!
  53. 53. IPULLRANK.COM @ IPULLRANK Log File Analysis We parse the logs and query them in MySQL, but you can use any number of tools for this.
  54. 54. IPULLRANK.COM @ IPULLRANK Screaming Frog Log File Analyzer The Screaming Frog team rolled out an awesome new tool recently for log file analysis.
  55. 55. IPULLRANK.COM @ IPULLRANK Search Console’s Crawl Stats are Mediocre I personally find what Google gives us to be pretty much useless. Don’t less this take the place of going through your logs.
  56. 56. IPULLRANK.COM @ IPULLRANK Log Files Help You Spot What Really Happens I had a client come last year that was unsure whether it was Penguin or changes in their offline media buying habits had led to the downturn in traffic. Layering the Googlebot logs made it clear. 0 50000000 100000000 150000000 200000000 250000000 0 500000 1000000 1500000 2000000 2500000 Unknown organic organic home organic dyor organic er organic home,dyor,er organic cushion organic net of home organic ritani organic diamonds direct home paid brand sessions tv spend web sessions (as reported by adaptv) Top 800 Monthly SV impressions Googlebot Visits Penguin Update
  57. 57. IPULLRANK.COM @ IPULLRANK Social Shares Influence Crawl Patterns We find that there is a higher correlation with crawling and social shares than crawling and links. Commonly held SEO knowledge would make you believe this is entirely dictated by links. 0 1000000 2000000 3000000 4000000 5000000 6000000 0 100000 200000 300000 400000 500000 600000 700000 Googlebot Visits Links Social Shares FB Twitter G+
  58. 58. IPULLRANK.COM @ IPULLRANK Prerendering is the Best Practice Using a solution like is the best practice.
  59. 59. IPULLRANK.COM @ IPULLRANK Prerender Can Cause Problems However, we’ve seen Prerender setups misidentify Googlebot as a human users and serve the wrong the AngularJS versions. 0 20000 40000 60000 80000 100000 120000 Googlebot Perceived as Human
  60. 60. IPULLRANK.COM @ IPULLRANK Prerender Setup Serving Non-200 Prepping the prerender cache can cause outages and lead to pages falling in and out of the index. 0 2000 4000 6000 8000 10000 12000 14000 16000 206 301 302 304 403 404 405 410 500 503 504 206 301 302 304 403 404 410 503 504 206 301 302 304 404 405 410 500 503 504 206 301 302 304 401 404 405 410 500 503 504 206 301 302 304 403 404 405 410 502 503 504 1 2 3 4 5 - hit miss pass
  61. 61. IPULLRANK.COM @ IPULLRANK Also Prerendering is Not a Requirement We got rid of prerender on 2 sections of the site and saw the traffic from Organic Search increase YoY. Google can index it just fine.
  62. 62. IPULLRANK.COM @ IPULLRANK AJAX Crawling Deprecated Google has gotten rid of the AJAX crawling scheme because they don’t need it anymore. They have perfected their ability to crawl headless at reasonable scale. They say don’t block the CSS or JS because it’s a browser.
  63. 63. They Say Don’t Prerender Just for Google
  64. 64. IPULLRANK.COM @ IPULLRANK Prerender Does Speed Up Crawling There is certainly value in prerendering for speed, but then that makes me wonder, why not just not use Angular?
  65. 65. IPULLRANK.COM @ IPULLRANK A Better Idea Is a Library that Doesn’t Need Prerender Let’s stop going after the shiny object and go with the things that are built to work, shall we? Check out MeteorJS –
  66. 66. IPULLRANK.COM @ IPULLRANK Or go with Angular2 Brad Green is the Google engineering product manager for AngularJS. He says Angular2 is better in every way.
  67. 67. IPULLRANK.COM @ IPULLRANK Don’t Forget Google Wants Progressive Enhancement Thinking with the IoT in mind, progressive enhancement still makes sense. However, frameworks like AngularJS are completely contradictory in that they don’t offer graceful degradation.
  68. 68. Scraping The backbone of a lot of our analysis as SEOs
  69. 69. IPULLRANK.COM @ IPULLRANK Most Tools use cURL or Similar There are many similar libraries, but many SEO tools are built on library called cURL for managing HTTP requests. This is a text based crawler.
  70. 70. This is a fundamental flaw of many SEO tools because they are not constructing the DOM and its CSS/JS transformations. Search Engines are.
  71. 71. IPULLRANK.COM @ IPULLRANK PhantomJS – For Scraping the Modern Web There are many headless browsers out there such as Selenium, but PhantomJS is the go to.
  72. 72. IPULLRANK.COM @ IPULLRANK HorsemanJS – The Easy PhantomJS I prefer HorsemanJS which is a NodeJS module that is incredibly simple.
  73. 73. IPULLRANK.COM @ IPULLRANK In-Browser Scraping I wrote a post on how you can scrape using your browser’s Console.
  74. 74. IPULLRANK.COM @ IPULLRANK Scrape the Unscrapable Nothing is in unscrapable when you’re using a browser.
  75. 75. IPULLRANK.COM @ IPULLRANK Multi-Page In-browser Scraping If you want to scrape multiple pages check out ArtooJS
  76. 76. IPULLRANK.COM @ IPULLRANK WebScraper is another high-powered in-browser scraping tool. No code required.
  77. 77. IPULLRANK.COM @ IPULLRANK Or You Can Make Screaming Frog Headless You can make Screaming Frog headless by placing a proxy with a headless browser in front of it. From my initial findings, there is not much different aside from the order it identifies URLs and its ability to scrape content from pages.
  78. 78. IPULLRANK.COM @ IPULLRANK Here’s the Code Here’s the node code for how I did it. HorsemanJS is not the best tool for the proxy, but it’s great for a proof of concept. Screaming Frog pings for robots.txt a lot. Make sure you handle it with a cURL request rather than a headless one. For speed, don’t load images Make sure you’re setting the right headers before your return the HTML. HorsemanJS does not return headers.
  79. 79. IPULLRANK.COM @ IPULLRANK PHP Wrapper for PhantomJS For those of you that prefer PHP, the PHP PhantomJS wrapper is pretty decent as well. For speed, don’t load images
  80. 80. Content & Linking You knew I wasn’t going to get up here and not talk about these things, right?
  81. 81. IPULLRANK.COM @ IPULLRANK Google Looks at Entities First According to Paul Haahr, Google looks at entities first and it appears that we as an industry have talked about this in theory, but not truly worked it into the optimization workflow.
  82. 82. IPULLRANK.COM @ IPULLRANK AlchemyAPI I encourage you to work AlchemyAPI into your keyword research and content optimization process.
  83. 83. IPULLRANK.COM @ IPULLRANK Term Relevance There’s also been a lot of talk about co-relevance, topic modeling and TF*IDF. This is a great opportunity for technical elements to influence content creation.
  84. 84. IPULLRANK.COM @ IPULLRANK In Fact Just Do Everything Cyrus Says
  85. 85. IPULLRANK.COM @ IPULLRANK Moz’s New Tool Will Help Moz has made the related keywords concept more accessible by adding it to Moz Pro. Searchmetrics also has a similar solution.
  86. 86. IPULLRANK.COM @ IPULLRANK Onpage.Org’s TF-IDF Tool is Also Great
  87. 87. IPULLRANK.COM @ IPULLRANK John Mueller Says 302s Pass PageRank Don’t let Google mislead you about 302s vs. 301s.
  88. 88. IPULLRANK.COM @ IPULLRANK These Rankings Say He’s Wrong We had a client with millions for 302s and all we did was change them to 301s June August September HyperTraffic (1-3) 1272 1,283 1,473 Traffic (4-10) 979 983 837 Striking Distance (11-20) 167 161 110 Emerging (21-40) 106 103 98 Developmental (41-100) 156 90 87 Unranked 0 60 75 0 200 400 600 800 1000 1200 1400 1600 301 redirects
  89. 89. IPULLRANK.COM @ IPULLRANK This Traffic Improvement Says He’s Wrong Traffic went up pretty dramatically despite seasonality. August 27th 301 redirects 11,039,188 10,455,517 11,348,902 12,049,481 11,250,148 12,902,847 11,040,192 13,158,296 11,287,790 12,203,998 12,508,164 13,158,296 14,041,313 15,609,390 0 2,000,000 4,000,000 6,000,000 8,000,000 10,000,000 12,000,000 14,000,000 16,000,000 18,000,000 April May Jun Jul Aug Sep Oct Organic Visits 2014 2015
  90. 90. But. We need to stop saying ‘just do 301 redirects.’
  91. 91. IPULLRANK.COM @ IPULLRANK Instead Do Optimal 301 Redirects Pattern matching on the .htaccess level is always faster than a list of redirects and always better than doing it on the code/page level. Developers will respect that you understand what you’re talking about. ReWriteRule url_pattern file_reference [R=301, L] redirect 301 /relative/path/to/file.php redirect 301 /relative/path/to/file.php redirect 301 /relative/path/to/file.php >
  92. 92. IPULLRANK.COM @ IPULLRANK Internal Linking Structures Internal Linking Structures are still an incredibly powerful asset. I’d like to see more tools like what Portent has built to visualize them and understand where there is opportunity.
  93. 93. IPULLRANK.COM @ IPULLRANK Searchmetrics Does a Great Job of This I wish all the link indices had a stronger focus on internal linking structure and visualizing where there are opportunities to flow more link equity throughout the site. Searchmetrics’ Site Structure tool does this well.
  94. 94. IPULLRANK.COM @ IPULLRANK Paul’s Method is Good for Rolling Your Own
  95. 95. If Possible, Always Update your internal links on the database level WordPress Example UPDATE wp_posts SET post_content = REPLACE (post_content, '', '');
  96. 96. Google is becoming the presentation layer of the web. Help them do it.
  97. 97. IPULLRANK.COM @ IPULLRANK / JSON-LD is a lot more reasonable to implement now that they are allowing JSON-LD. Being in knowledge boxes and such yields a lot more traffic than it takes away.
  98. 98. Page Speed Google’s next big play
  99. 99. IPULLRANK.COM @ IPULLRANK Google Expects Ludicrously Speedy Google has an expectation that the above the fold content on the page will load within 1 second. You basically get 400 ms to make that happen.
  100. 100. IPULLRANK.COM @ IPULLRANK The Critical Rendering Path
  101. 101. IPULLRANK.COM @ IPULLRANK DevTools Is Your Friend In the Timeline section, DevTools gives you a point by point break down of each operation and how it impacts the site’s load time. This is how you can determine what exactly is slowing down your pages.
  102. 102. IPULLRANK.COM @ IPULLRANK How Your Code Is Laid Out Matters A LOT
  103. 103. IPULLRANK.COM @ IPULLRANK This Is what PageSpeed Insights Helps With
  104. 104. IPULLRANK.COM @ IPULLRANK External Resources Also Slow Down the Page Chartbeat is typically a huge culprit and using rel-dns-prefetch can speed this up considerably.
  105. 105. IPULLRANK.COM @ IPULLRANK AMP is a Spec to Enforce the Critical Rendering Path Sound familiar?
  106. 106. IPULLRANK.COM @ IPULLRANK Responsive Sites are SLOOOOOOOW One of the key things you can do, aside from optimizing for the critical rendering path, is conditional loading.
  107. 107. IPULLRANK.COM @ IPULLRANK PreBrowsing Directives Browsers have preloading directives which allow the page to download elements in the background for later viewing or within the same session.
  108. 108. IPULLRANK.COM @ IPULLRANK Rel-Preconnect Rel-preconnect will resolve the DNS, start the TCP handshake and negotiate the TLS tunnel beforehand, effectively shaving hundreds of milliseconds to several seconds in page load.
  109. 109. IPULLRANK.COM @ IPULLRANK When to Use Rel-Preconnect When you see a long time for connections to happen or a lot of idle time, it’s a good idea to use rel-preconnect.
  110. 110. <link rel=“preconnect” href=“domain name”>
  111. 111. IPULLRANK.COM @ IPULLRANK How Rel-Prerender Works It loads the page in an invisible tab to make the page appear instantly.
  112. 112. IPULLRANK.COM @ IPULLRANK Not to Be Confused with
  113. 113. IPULLRANK.COM @ IPULLRANK Google Uses It in the SERPs This could potentially be a way to definitively identify navigational queries. h/t @yoast
  114. 114. IPULLRANK.COM @ IPULLRANK I Sent Thousands of Headless Visits
  115. 115. IPULLRANK.COM @ IPULLRANK Rel-Prerender’d Pages Generally Perform Better
  116. 116. IPULLRANK.COM @ IPULLRANK Use it With the GA API Get the pageviews of the ga:pagePath based on the ga:previousPagePath set to the current page that you’re on. You’ll get a list of the pages your users are most likely to visit next. Set one of these as the rel-prerender.
  117. 117. IPULLRANK.COM @ IPULLRANK Code. Inject rel-prerender into the page programmatically. You could also session the URLs the user is going to to ensure you are always prerendering a new page.
  118. 118. IPULLRANK.COM @ IPULLRANK It Improved Our Site Speed 68.35% One line of code did this magic.
  119. 119. Don’t use it on mobile. Analytics packages aside from Google Analytics may show fake sessions.
  120. 120. IPULLRANK.COM @ IPULLRANK Defer JS Load with Page Visibility API …
  121. 121. IPULLRANK.COM @ IPULLRANK Rel-Preload for Same Page Preloading I haven’t played with it yet, but Ian says they’re using it on the Portent site.
  122. 122. How to Go Forward Things you should know and do to be prepared for the technical renaissance
  123. 123. IPULLRANK.COM @ IPULLRANK Understand the Document Object Model
  124. 124. IPULLRANK.COM @ IPULLRANK Critical Rendering Path
  125. 125. IPULLRANK.COM @ IPULLRANK Page Speed
  126. 126. IPULLRANK.COM @ IPULLRANK Log File Analysis
  127. 127. IPULLRANK.COM @ IPULLRANK Understand SEO for JavaScript Frameworks
  128. 128. IPULLRANK.COM @ IPULLRANK Learn DevTools
  129. 129. IPULLRANK.COM @ IPULLRANK Accelerated Mobile & Instant Pages Even if you’re not a publisher, I encourage you to watch these projects as I expect that Google and Facebook will grow the specs dramatically once they’re adopted.
  131. 131. IPULLRANK.COM @ IPULLRANK Get Back to Testing All The Things
  132. 132. Let’s stop chasing the content train and get back to making experiences that perform.
  133. 133. That’s All I’ve Got
  134. 134. THANK YOU Michael King Managing Director, iPullRank @iPullRank