Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

TechSEO Boost 2018: Extreme Full Contact Google APIs


Published on

When a client with a huge website had a tricky indexation issue, we needed to definitively know which pages were and were not indexed. Although Search Console has always taunted us with how many pages are indexed from your sitemap, they don't tell you which ones specifically. So, we built a tool using an undocumented Google API we discovered which allows us to check thousands of URLs in Google at a time, and released it to the public.

Published in: Marketing
  • We called it "operation mind control" - as we discovered a simple mind game that makes a girl become obsessed with you. (Aand it works even if you're not her type or she's already dating someone else) Here's how we figured it out... ◆◆◆
    Are you sure you want to  Yes  No
    Your message goes here
  • Earn a 6-Figure Side-Income Online... Signup for the free training HERE ♣♣♣
    Are you sure you want to  Yes  No
    Your message goes here
  • Legitimate jobs paying $40/h Tap into the booming online job, industry and start working now! ★★★
    Are you sure you want to  Yes  No
    Your message goes here

TechSEO Boost 2018: Extreme Full Contact Google APIs

  1. 1. Sean Malseed | @seanmalseed | #TechSEOBoost EXTREME FULL-CONTACT GOOGLE APIS – How Absentmindedly Watching Network Requests Led to our Most Popular SEO Tool
  2. 2. Sean Malseed | @seanmalseed | #TechSEOBoost About me I’m Sean Tech Director at Greenlane Extreme amateur programmer Hobbies include dark beer and long stretches of doing nothing in particular
  3. 3. Sean Malseed | @seanmalseed | #TechSEOBoost THERE WILL BE FREE SPAGHETTI
  4. 4. Sean Malseed | @seanmalseed | #TechSEOBoost
  5. 5. Sean Malseed | @seanmalseed | #TechSEOBoost SPAGHET YOU CAN DEPLOY TO THE CLOUD
  6. 6. Sean Malseed | @seanmalseed | #TechSEOBoost This kills the spaghetti meme.
  7. 7. Sean Malseed | @seanmalseed | #TechSEOBoost
  8. 8. Sean Malseed | @seanmalseed | #TechSEOBoost Send a request to a URL, and get back information in a simplified and easy- to-import format. Request URL:,smoothie,fruit Server gives back: {"ranks": {"banana":12,"smoothie":24,"fruit":48}} What an API actually is
  9. 9. Sean Malseed | @seanmalseed | #TechSEOBoost Modern API usage APIs are intended for two primary groups of people: • Professional developers building robust applications • Nosey marketing people who know just enough to be dangerous
  10. 10. Sean Malseed | @seanmalseed | #TechSEOBoost How to find APIs Two ways to find useful APIs: • Published APIs, such as Google’s excellent comprehensive API Explorer • Unpublished APIs, usually discovered by clicking around randomly
  11. 11. Sean Malseed | @seanmalseed | #TechSEOBoost Google, why aren’t you indexing all these pages? We Had a Problem
  12. 12. Sean Malseed | @seanmalseed | #TechSEOBoost Troubleshooting Indexation, “The Old Way” YAY NOT YAY
  13. 13. Sean Malseed | @seanmalseed | #TechSEOBoost Troubleshooting Indexation, “The Old Way” YAY NOT YAY
  14. 14. Sean Malseed | @seanmalseed | #TechSEOBoost But what if we have almost literally a bazillion pages to check? NOT YAY AT ALL
  15. 15. Sean Malseed | @seanmalseed | #TechSEOBoost We’re not alone in this problem
  16. 16. Sean Malseed | @seanmalseed | #TechSEOBoost Something like this could work
  17. 17. Sean Malseed | @seanmalseed | #TechSEOBoost Whoops
  18. 18. Sean Malseed | @seanmalseed | #TechSEOBoost Whoops
  19. 19. Sean Malseed | @seanmalseed | #TechSEOBoost GOOGLE WHY CAN’T YOU JUST BE COOL ABOUT THIS
  20. 20. Sean Malseed | @seanmalseed | #TechSEOBoost Everything is garbage, time to procrastinate
  21. 21. Sean Malseed | @seanmalseed | #TechSEOBoost Watching network requests because ¯_(ツ)_/¯
  22. 22. Sean Malseed | @seanmalseed | #TechSEOBoost You can find a of interesting things under XHR
  23. 23. Sean Malseed | @seanmalseed | #TechSEOBoost Hey, neat!
  24. 24. Sean Malseed | @seanmalseed | #TechSEOBoost This trick works on lots of websites
  25. 25. Sean Malseed | @seanmalseed | #TechSEOBoost Hey, neat!
  26. 26. Sean Malseed | @seanmalseed | #TechSEOBoost And once you have an API of it...
  27. 27. Sean Malseed | @seanmalseed | #TechSEOBoost “If it pops up, in, or out... good chance there’s an API of it.”
  28. 28. Sean Malseed | @seanmalseed | #TechSEOBoost Ok, back to the index thing
  29. 29. Sean Malseed | @seanmalseed | #TechSEOBoost I wonder if there’s something like the suggest API that can help me...
  30. 30. Sean Malseed | @seanmalseed | #TechSEOBoost What the heck is f.txt?
  31. 31. Sean Malseed | @seanmalseed | #TechSEOBoost Oh, that’s an API of the entire SERP. Hey, neat!
  32. 32. Sean Malseed | @seanmalseed | #TechSEOBoost FIRST COURSE OF SPAGHETTI
  33. 33. Sean Malseed | @seanmalseed | #TechSEOBoost Google Apps Scriptscetti with a Rustic World Wide Wine Sauce
  34. 34. Sean Malseed | @seanmalseed | #TechSEOBoost Oh boy! That’s helpful!
  35. 35. Sean Malseed | @seanmalseed | #TechSEOBoost Times out after running for 5 mins. How can I make it run a bazillion URLs?
  36. 36. Sean Malseed | @seanmalseed | #TechSEOBoost But what about...
  37. 37. Sean Malseed | @seanmalseed | #TechSEOBoost But what about... Requests to Google made from Google Apps Script are made from an internal Google IP address, not yours!
  38. 38. Sean Malseed | @seanmalseed | #TechSEOBoost Sample this spaghetti yourself! Google Sheet with full working script served with garlic bread and a world wide wine sauce:
  39. 39. Sean Malseed | @seanmalseed | #TechSEOBoost SECOND COURSE OF SPAGHETTI
  40. 40. Sean Malseed | @seanmalseed | #TechSEOBoost How about a daily automated snapshot of a keyword’s top 100 results?
  41. 41. Sean Malseed | @seanmalseed | #TechSEOBoost Sample this spaghetti yourself! Google Sheet with full working script served with a fresh pesto sauce. Make sure you go Tools → Script Editor, then Edit → Current Project’s Triggers and set a Time-driven trigger to run runRankings daily.
  42. 42. Sean Malseed | @seanmalseed | #TechSEOBoost THIRD COURSE OF SPAGHETTI
  43. 43. Sean Malseed | @seanmalseed | #TechSEOBoost “What if we use this API to make our own API?”
  44. 44. Sean Malseed | @seanmalseed | #TechSEOBoost Generic Inception Meme
  45. 45. Sean Malseed | @seanmalseed | #TechSEOBoost Google Script can run as a web app, and just return JSON-P to beat CORS
  46. 46. Sean Malseed | @seanmalseed | #TechSEOBoost Look ma, no CORS!
  47. 47. Sean Malseed | @seanmalseed | #TechSEOBoost Sample this spaghetti yourself! Google Apps Script ready to File → Make a Copy served with butter and a cors-free roll. Make sure you go File → Make a copy, then Publish → Deploy as web app and set a Execute the app as to Me and Who has access to the app to Anyone, even anonymous.
  48. 48. Sean Malseed | @seanmalseed | #TechSEOBoost This one simple trick works so well, you could build a whole back-end out of it
  49. 49. Sean Malseed | @seanmalseed | #TechSEOBoost Links – - Google Sheet with a formula to pull the live Google Suggest API - Bulk URL indexation tester sheet - Daily SERP top 100 search result snapshot sheet - JSONP keyword rank API in Apps Script that you can integrate anywhere - Google’s awesome list of official APIs - Experimental live keyword rank checker using the Apps Script API thing
  50. 50. Sean Malseed | @seanmalseed | #TechSEOBoost Thx! – Sean Malseed @seanmalseed