From a presentation at Text Analytics Forum, Washington, DC, Nov. 7, 2018. Keyword research allows companies to learn the voice of their customers and tune their marketing messages for them. One of the challenges in keyword research is to find collections of keywords that are topically relevant and in demand and therefore likely to draw search traffic and customer engagement. Data sources such as search logs and search engine result pages provide valuable sources of keywords, as well as insight into audience-specific language. Additionally, cognitive technologies such as natural language processing and machine learning provide capabilities for mining those sources at scale. With a few tools and some minimal coding, an analyst can generate clusters of best-bet keywords that are not only syntactically similar but semantically related. This how-to talk presents some practical techniques for automated analysis of keyword source data using off-the-shelf APIs.

1. Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation
Applying NLP & Machine Learning to Keyword Analysis
—
Dan Segal
Corporate Taxonomist, IBM

2. 2Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation
• Keyword governance
• Design of smart content platforms; e.g.,
• Automated tagging
• Assisted authoring
• Chatbots
• SEO platform migration, aka “The Big Sort”
• Over 300K keywords across 10 business units
• Out-of-date classifications
• In need of cleanup and pruning
Business Drivers

3. 3Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation
• Not a lot of text for ML to process
• Without context, keywords are just
strings
• Strings can be ambiguous:
• cloud
• hybrid
• paas
• storage
• Looking for semantic similarity, not just
syntactic
Challenges of Keyword Classification

4. 4Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation
Ranking URLs put Keywords into Context
• Page rank suggests
user intent
• Page presents the
keyword in context
• For text analysis,
the page is a proxy
for the keyword

5. 5Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation
Select Keywords
Refine training data to improve
performance
Classify Keywords with
Classifier API
• Goal: repeatable, scalable automated
classification
Solution Approach / Workflow
Train Classifier API
Keyword/Concept pairs
• Supervised learning using keyword / Concept
training sets
Use NLP API to scrape SERPs
and identify Concepts
• Pre-trained NLP for first-pass Concept
identification
• Grouping of co-occurring keywords by Concepts
Normalize Concepts to
taxonomy
• Concept cleanup and normalization

6. 6Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation
Concept Identification Use NLP API to scrape SERPs
and identify Concepts
• Compile list of keywords with top 10 SERPs for each
• Pass URLs to NLP API
• API returns a list of high-level Concepts for each URL
• Concepts are derived from pre-trained model
• High-throughput: 60-70 URLs / minute

7. 7Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation
Concept Cleanup and Normalization Normalize Concepts to
taxonomy
• Identify groups of keywords that
cluster around Concepts
• Reduce noise by filtering for:
• Relevance score
• Query volume
• Number of unique keywords per group
• Number of occurrences per keyword
• Duplicate and excluded Concepts
• Outputs:
• Training sets for ML classifier
• Seed Concepts for taxonomy

8. 8
Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation
Weeding Out the “Laughers”
Retrieved Concept:
X-Force
http://dbpedia.org/resource/X-Force
Keyword:
x-force
ranking URLs
8

9. Trained Classifier API
Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation
Keyword Classification Classify Keywords with
Classifier API
9
INPUT: keyword list (CSV) CLASSIFIED KEYWORDS (CSV, RDF)
post-processing
OUTPUT: keyword classes (JSON)

10. Keywords
Keyword Group
(from Topic taxonomy)
Primary Brand
(from Product taxonomy)
Connecting the Parts
10Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation
Business Unit
(from Product taxonomy)

11. Incorporating Keywords into the Ontology
11Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation
Keywords
Branded
keywords
Unbranded
keywords
keyword for
keyword for
keyword for
Business
Units
Products
markets
product
Topics
Industries
Job Role
for industry
solution for
has need

12. Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation
Execution
• Pre-trained models
enable first-pass, high-
throughput annotation
and sorting of hundreds of
thousands of keywords
and their SERPs
• For optimum results, the
process requires
customization,
parametrization, and
human review.
Solution
• Off-the-shelf API’s
provide rapid access to
NLP and ML tooling with
low technical debt.
• Keyword analysis time
reduced from months to
weeks.
Architecture
• A robust taxonomy is
essential for
consolidating API-
generated categories and
integrating keywords into
the ontology.
Next Steps
• Custom models for
domain-specific
annotation
• Applying the ontology;
e.g., site navigation,
search, autotagging, AI-
assisted authoring
Lessons Learned
12

13. Thank you
Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation
Dan Segal
Corporate Taxonomist
—
ibm.com
13

14. Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation

15. Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation
Backup slides
15

16. Toolset
16Text Analytics Forum / November 7, 2018 / © 2018 IBM Corporation
Tool Use Examples
Natural language processing (NLP) • Text extraction
• Concept identification
• Watson NLU (API)
Machine learning • Text classification
• Keyword clustering
• Watson NLC (API)
Data transformation • Data cleanup
• Data conversion; e.g., ML outputs
(JSON) to CSV and RDF
• OpenRefine
Ontology management • Domain modeling
• Keyword / Group associations
• TopBraid Composer
• TopBraid EVN
Sample scripts and data available at: https://github.com/dan-segal/keyword-classification/