SearchLove Boston 2016 | Paul Shapiro | How to Automate Your Keyword Research

17,314 views

Published on

Are you tapping into automation for keyword research? If not, why not? When it comes to SEO, automation is awesome. For starters, it can help free up a lot of time that is normally spent on menial tasks. What’s more, it can also aid deep analysis, and even facilitate innovation. If you are still doing keyword research manually, this is a must-attend session. Paul will show you how to get started with automated keyword research, using some easy-to-use tools. You’ll see first-hand how they can help you uncover valuable insights automatically. Overall, you will walk away with an immediately actionable plan to start automating your keyword research today.

Published in: Marketing
1 Comment
24 Likes
Statistics
Notes
  • Great presentation man. Thanks for sharing. Learned a ton!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
17,314
On SlideShare
0
From Embeds
0
Number of Embeds
11,268
Actions
Shares
0
Downloads
149
Comments
1
Likes
24
Embeds 0
No embeds

No notes for slide
  • a CAR need GASOLINE before you SEE IT MOVE
  • Focus on medium size-sites
  • Average for this year vs last, over time
  • More time to do more impactful work
    Make better use of staff
    More data means you’re more likely to find gems others wouldn’t have uncovered.
  • SearchLove Boston 2016 | Paul Shapiro | How to Automate Your Keyword Research

    1. 1. HOW TO AUTOMATE YOUR KEYWORD RESEARCH Paul Shapiro – Organic Search Director at Catalyst SearchLove Boston – May 4th, 2016
    2. 2. @fighto 2 Without Automation
    3. 3. @fighto 3 With Automation
    4. 4. @fighto WHO AM I? 4
    5. 5. @fighto 5 Tool Data Sources Visualization + +
    6. 6. @fighto WHAT YOU WILL LEARN • The problems with the standard keyword research process • How you can save time on keyword research with automation • How to automate keyword research with a powerful, but easy-to-use tool • How APIs work and how to leverage them • What data sources are useful for keyword research automation • How to use data visualization to make keyword research decisions 6
    7. 7. MANUAL KEYWORD RESEARCH IS OFTEN BLAND & TIME CONSUMING 7
    8. 8. @fighto 8
    9. 9. @fighto 9 Most Keyword Research Looks Like This
    10. 10. @fighto 10 That’s just a Google Keyword Planner Export
    11. 11. @fighto TYPICAL TIME INVESTMENT FOR KEYWORD RESEARCH 5 8 12 21 21 6 10 18 22 28 0 5 10 15 20 25 30 Micro (0-49 pages) Small (50-99 pages) Medium (100-249 pages) Large (250-499 pages) Extra Large (>500 pages) Hours to Complete Keyword Research by Site Size Average (Low End) Average (High End) 11
    12. 12. @fighto 12 AIN’T NOBODY GOT TIME FOR MANUAL KEYWORD RESEARCH!
    13. 13. @fighto SIZE OF THE DATA SET 13 vs.
    14. 14. @fighto 14 We can do better!
    15. 15. WE CAN SAVE TIME WITH AUTOMATION
    16. 16. @fighto TYPICAL TIME INVESTMENT FOR KEYWORD RESEARCH 5 8 12 21 21 6 10 18 22 28 0 5 10 15 20 25 30 Micro (0-49 pages) Small (50-99 pages) Medium (100-249 pages) Large (250-499 pages) Extra Large (>500 pages) Hours to Complete Keyword Research by Site Size Average (Low End) Average (High End) 16
    17. 17. @fighto 17 If we do keyword research once per month, we will save around 24 to 45 full days over a 5-year period using automation. http://xkcd.com/1205/
    18. 18. WE CAN DO BETTER THAN AN EXCEL SPREADSHEET
    19. 19. @fighto THAT’S MORE LIKE IT! 19 h/t @victorpan
    20. 20. @fighto HOW TO USE FOR KEYWORD SELECTION / TARGETING 20 This top-right quadrant contains keywords with: • Low competition • Good growth Larger bubbles show higher search volumes. You can alternatively use current rank on the x-axis to signal organic market share like a traditional growth-share matrix.
    21. 21. @fighto 21 That’s much better! actionability.
    22. 22. MY KEYWORD RESEARCH AUTOMATION PROCESS
    23. 23. @fighto 23 https://www.knime.org It starts with a tool…
    24. 24. @fighto WHY KNIME? • Singular tool for data collection, manipulation, analysis, and reporting • Easy to use and no development skills necessary • If you are a developer, it provides faster prototyping • Large library of built-in “nodes” • Free/Open Source • Run on Windows/Mac/Linux • Infinitely expandable 24
    25. 25. THE BASICS OF KNIME
    26. 26. @fighto WHAT’S A NODE? • Nodes are prebuilt, drag and drop modules designed perform a singular task • Nodes are strung together like a chain to accomplish larger, more complex tasks • Nodes can be grouped together into “meta-nodes”, which can be configured in unison 26
    27. 27. @fighto HOW DO YOU ADD NODES & HOW DO THEY CONNECT? How do you add nodes to your “workflow”? How do you string nodes together? 27
    28. 28. @fighto HOW DO YOU CONFIGURE & RUN NODES? Configuring Nodes Running Workflows 28 OR
    29. 29. APIs 101 29
    30. 30. @fighto EASY & APPLICABLE API: GOOGLE AUTOSUGGEST http://suggestqueries.google.com/complete/search?output=toolbar&hl=en&q=ninja%20turtles 30 Simple API using “GET Request” This is a variable that changes
    31. 31. @fighto 31 Output = XML
    32. 32. A KNIME-API EXAMPLE 32
    33. 33. @fighto STEP 1: LIST OF KEYWORDS 33
    34. 34. @fighto STEP 2: ENCODE SPACES IN KEYWORD PHRASES 34 Space Character
    35. 35. @fighto STEP 3: CONCATENATE API URL + KEYWORDS 35
    36. 36. @fighto STEP 4: DOWNLOAD API OUTPUT 36
    37. 37. @fighto STEP 6: LOOP THROUGH EACH KEYWORD 37
    38. 38. @fighto STEP 7: PARSE THE XML OUTPUT FORM THE API 38
    39. 39. @fighto 39 Auto-Complete Suggestions
    40. 40. @fighto 40
    41. 41. @fighto FYI: I’M NOT A KNIME SALESMAN 41
    42. 42. DATA INPUTS / SOURCES
    43. 43. @fighto 43 # of primary data sources to aggregate
    44. 44. @fighto 44 http://searchwilderness.com/gwmt-data-python/ Google Search Analytics – Historical MySQL Database via Python Script
    45. 45. @fighto 45 AdWords - Search Query Report API or CSV
    46. 46. @fighto 46 SEMRush – API (if <100,000) or Domain vs. Domain: Competitor Keywords (Advanced Mode) https://www.semrush.com/info/domain_vs_domain
    47. 47. @fighto 47 GrepWords API – For Google Keyword Planner related query suggestions & search volumes http://grepwords.com/
    48. 48. @fighto 48 Google Autosuggest API or Emulation with Selenium
    49. 49. @fighto 49 Rank Checking/Full SERP API - GetSTAT https://getstat.com/
    50. 50. @fighto 50 MarketMuse – Semantic Keyword Recommendations https://www.marketmuse.com/ Alternative: http://pshapi.ro/SemanticKWR
    51. 51. @fighto 51 Moz API + Moz Nodes https://tech.knime.org/book/mmi-labs-nodes https://moz.com/products/mozscape/access
    52. 52. @fighto 52 Google Trends – Unofficial API https://github.com/GeneralMills/pytrends Script soon to follow: http://pshapi.ro/slopetrends
    53. 53. @fighto 53 More data sources: http://pshapi.ro/kwmetrics
    54. 54. THE DATA-TO-KNIME PROCESS 54
    55. 55. @fighto Filtering + Data Manipula- tions ONE BIG KEYWORD LIST Seed Keywords - List - GWMT - SEMRush Comp. KWs - SQR Keywords Keyword Planner Suggestions (via GrepWords) Google Autocomplete Semantic Keyword Recommendations (via MarketMuse) Google Autocomplete SEMRush Domain vs. Domain Keywords Google Autocomplete 55
    56. 56. @fighto STRING ‘EM ALL TOGETHER AND THEN… 56
    57. 57. @fighto DATA MANIPULATIONS / CALCULATIONS • Get top 10 results from rank checking API (i.e., GetSTAT) • Use Moz API nodes and find average PA to assess competiveness. • Optionally, use SEMRush’s Keyword Difficulty API Organic Competition Search Volume Keyword Trends 57
    58. 58. @fighto DATA MANIPULATIONS / CALCULATIONS • Get top 10 results from rank checking API (i.e., GetSTAT) • Use Moz API nodes and find average PA to assess competiveness. • Optionally, use SEMRush’s Keyword Difficulty API Organic Competition • Get Search Volumes via SEMRush API or via GrepWords APISearch Volume Keyword Trends 58
    59. 59. @fighto DATA MANIPULATIONS / CALCULATIONS • Get top 10 results from rank checking API (i.e., GetSTAT) • Use Moz API nodes and find average PA to assess competiveness. • Optionally, use SEMRush’s Keyword Difficulty API Organic Competition • Get Search Volumes via SEMRush API or via GrepWords APISearch Volume • Use 2 Years of Google Trends data to calculate slope and determine growing/declining keywordsKeyword Trends 59
    60. 60. @fighto 60
    61. 61. @fighto 61 Reusable 5-Minutes
    62. 62. @fighto 62 You can use this data to filter keywords and reduce massive list... OR, you can leverage data visualization!
    63. 63. VISUALIZING KEYWORD RESEARCH
    64. 64. @fighto 64 We have a few options…
    65. 65. @fighto OPTION: EXCEL + BUBBLE CHART 65
    66. 66. @fighto OPTION: TABLEAU 66
    67. 67. @fighto OPTION: PYTHON + BOKEH CHARTS IN KNIME 67
    68. 68. @fighto THEY’RE INTERACTIVE… 68
    69. 69. @fighto 69 Sample Python Code: http://pshapi.ro/SLBokeh
    70. 70. @fighto SO WHAT… • Free up time spend doing mundane, repetitive work and spend more doing impactful, strategic work • Free up staffing resources and utilize them in other ways • Extract and analyze a larger pool of data for greater data pool 70 From this… To this…
    71. 71. @fighto SUMMARY • You should rethink how your currently do keyword research. • Keyword research is often bland, time consuming, and not as actionable as it can be. • KNIME provides an easy-to-use interface for automating your keyword research • APIs are easier to use than they sound; Use them • There are great data sources you can leverage via APIs/built-in KNIME nodes • Visualize your keywords using a growth-share matrix like-graph to make keyword usage decisions • Emojis are awesome 71
    72. 72. @fighto RESOURCES: DATA SOURCES > KNIME 72 Data Source KNIME Node(s) Google Search Analytics Database Table Connector/Reader AdWords Search Query Report XLS Reader SEMRush Domain vs. Domain (or API if less than 100,000 keyword) XLS/CSV Reader or GET Resource & Read REST Representation Google Keyword Planner/GrepWords GET Resource & Read REST Representation Google Autosuggest GET Resource & Read REST Representation (or Selenium nodes) Rank Checking (via GetSTAT) GET Resource & Read REST Representation Semantic Keywords/MarketMuse POST Resource MOZ Moz API Connector/URL Metrics nodes Google Trends TBD – Either CSV or Python Script nodes
    73. 73. @fighto ADDITIONAL RESOURCES • KNIME Version to install: “KNIME Analytics Platform + all free extensions” • To install KNIME Add-ons, Choose Help->Install New Software • Moz nodes • Should come with Nightly Builds. If not can add http://update.knime.org/community- contributions/3.1 as a repository. • Add-on Selenium nodes • http://seleniumnodes.com/ • Search Engine Land article addressing additional data inputs: http://pshapi.ro/kwmetrics • Information about Growth-share matrices: https://en.wikipedia.org/wiki/Growth%E2%80%93share_matrix • Sanple Python + Bokeh script for an interactive keyword research visualization in KNIME: http://pshapi.ro/SLBokeh 73
    74. 74. @fighto Questions? 74
    75. 75. THANK YOU! @fighto http://searchwilderness.com http://catalystdigital.com

    ×