Python for SEO

Hamlet Batista | @hamletbatista | #TechSEOBoost
Python for SEO
–
Programming As a Superpower

AGENDA
–
Practical SEO applications
of Python => 3.6 for:
Data extraction
–
Preparation
–
Analysis & Visualization
–
Machine learning
–
Deep learning

INTRO
–
Why program when you can hire
a programmer to do the work for you?

But before…

CHALLENGING SEO PROBLEMS
–
THAT NEED PROGRAMMING WORK

IBM WebSphere => SAP Hybris

IBM WebSphere Site
Category Page
(Links to one or more
Product Listing
Pages)
Product Listing Page
Product Pages)
Product Page
(Single SKU)

SAP Hybris Site
Category Page
Product Pages)
Product Page
(Single SKU)

Old Site
Product Pages
(717)
New Site
Product Pages
(442)
Product
Mapping
(3431)

Old Site
Category
Pages
(371)
New Site
Category
Pages
(147)
Category
Mapping
(712)

Category
Home
Product
Content
Videos
Other
NewUsersRevenuePageCount

Category
Home
Product
Content
Videos
Other
NewUsers

Winners vs Losers

Launch Jupyter Notebook in Google Colaboratory
https://colab.research.google.com/github/ranksense/open-
source/blob/master/Presentations/TechSEOBoost/2018/PythonforSEOTechSEOBoost2018_
Hamlet_Batista.ipynb

Ecommerce V3 => Shopify

https://github.com/plotly/plotly.py

Solution Part 1 – Steps
Step 1:
Pull Google Analytics Data
–
Step 2:
Store Data in Pandas DataFrame
–
Step 3:
Perform Data Preparation and
Perform Basic Set Operations
CHALLENGE: Find Which Pages Lost
SEO Traffic

Python – Basics
https://pandas.pydata.org/
Python for Data Science Cheat Sheet
https://s3.amazonaws.com/assets.datacamp.com/blog_assets/PythonF
orDataScience.pdf

Python – Jupyter
Google Colaboratory
https://colab.research.google.com/notebooks/
welcome.ipynb

Python – Pandas
https://pandas.pydata.org/
Cheat Sheet
https://pandas.pydata.org/Pandas_Cheat_Sheet.pdf
10 Minutes to pandas
https://pandas.pydata.org/pandas-docs/stable/10min.html
Intro to Pandas for Excel Super Users
https://towardsdatascience.com/intro-to-pandas-for-excel-
super-users-dac1b38f12b0

Python – Requests
WEB SCRAPING REFERENCE:
A Simple Cheat Sheet for Web Scraping with
Python
https://blog.hartleybrody.com/web-scraping-cheat-sheet/
http://docs.python-requests.org/en/master/

https://ga-dev-tools.appspot.com/query-explorer/

Pulling Google Analytics Data

Storing Data in a DataFrame

Transforming Data for Analysis
https://www.shanelynn.ie/merge-join-dataframes-python-pandas-index-1/
Left Join Full Outer Join Left Join (if NULL)
Inner Join Right Join Right Join (if NULL)

Transforming Data for Analysis

Pages That Lost SEO Traffic

Step 1:
We will crawl old pages to follow
redirects
–
Step 2:
We will group pages using regular
expressions
–
Step 3:
Repeat the previous analysis
CHALLENGE: Find Which Page Groups Lost
SEO Traffic (Manually)

Regular Expressions for
SEOs and Digital
Marketers (with Use
Cases)
https://netpeaksoftware.com/blog/
regular-expressions-for-seos-
and-digital-marketers-with-use-
cases
Regex101.com

Crawling Old Pages

Grouping with Regexes
Lookahead and Lookbehind Zero-Length Assertions
https://www.regular-expressions.info/lookaround.html

Page Groups That Lost SEO Traffic

Reverse Engineer Success Too

How Do We Generalize This?

Using Machine Learning!

Credit: Matt West
Why Are Dominicans So Good
at Baseball?

Hit a Vitilla? Hit Anything
https://www.youtube.com/watch?v=k8Aw2cBer84

Vitilla
https://en.wikipedia.org/wiki/Vitilla

Learn Machine Learning and Solve Any SEO Problem

Regex-> URL Matching
XPath-> Content
Matching

Step 1:
Collect training data
–
Step 2:
Prepare and split training data into
training, and testing
–
Step 3:
Find best model
CHALLENGE: Find Which Page Groups Lost
SEO Traffic (Automatically)

Python – BeautifulSoup
BeautifulSoup 4 Cheatsheet
http://akul.me/blog/2016/beautifulsoup-cheatsheet/
https://www.crummy.com/software/BeautifulSoup/bs4/download/
An SEO’s guide to XPath
https://builtvisible.com/seo-guide-to-xpath/

Python – Scikit-learn
https://scikit-learn.org/
Cheat Sheet
https://s3.amazonaws.com/assets.datacamp.com/blog_assets/Scikit_Lear
heat_Sheet_Python.pdf
Hands-On Introduction To Scikit-learn (sklearn)
https://towardsdatascience.com/hands-on-introduction-to-scikit-learn-
sklearn-f3df652ff8f2
Efficiently Searching Optimal Tuning
Parameters
https://www.ritchieng.com/machine-learning-efficiently-
search-tuning-param/

Data Scientist Bottom Up Solution
Inside the BloomReach Algorithm - Using
Machine Learning to Understand Page
Templates
https://www.bloomreach.com/en/blog/2018/07/using-machine-
learning-to-learn-page-templates.html

For most Ecommerce sites, the dimensions
and quantity of images and input form elements
change by page template.
Let’s use that as the features vector.
Hamlet’s Observation
and Simpler Solution

Hamlet’s Observation and Simpler Solution

Collecting Training Data

What is One Hot Encoding?
Why and when do you have to
use it?
https://hackernoon.com/what-is-one-
hot-encoding-why-and-when-do-you-
have-to-use-it-e3c6186d008f
Prepare and Split Data

Cross Validation and Grid Search
For Model Selection in Python
https://stackabuse.com/cross-validation-
and-grid-search-for-model-selection-in-
python/
Find Best Model

Simple guide to confusion matrix terminology
https://www.dataschool.io/simple-guide-to-confusion-matrix-terminology/
Confusion Matrix

But wait… We can do Better

Using Deep Learning!

Step 1:
Label a few thousand web page
screenshots with the visual features
you care about
–
Step 2:
Train a computer vision model to
predict more granular page groups
–
Step 3: Find best model
CHALLENGE: Learn More Granular Page
Groups that Lost SEO Traffic (Automatically)

https://www.tensorflow.org/
Keras Cheat Sheet
https://s3.amazonaws.com/assets.dataca
mp.com/blog_assets/Keras_Cheat_Sheet
_Python.pdf
TensorFlow Tutorial For
Beginners
https://www.datacamp.com/community/tut
orials/tensorflow-tutorial
Python – Tensorflow
& Keras

Bottleneck
The “Information
Bottleneck” Theory
https://www.quantamagazine.org/ne
w-theory-cracks-open-the-black-
box-of-deep-learning-20170921/

Encoder Bottleneck Decoder
Input Image Reconstructed Image
Latent Space
Representation
AUTOENCODER

14 x 14 Feature Map
1. Input Image 2. Convolutional
Feature Extraction
3. RNN with attention
over the image
4. Word by word
generation
LSTM
Encoder Bottleneck Decoder
Latent Space
Representation
Caption Generator

Python – Tensorflow Object Detection API
https://github.com/tensorflow/models/tree/master/research/object_detection

AutoML Vision API Tutorial
https://cloud.google.com/vision/automl/docs/tutorial
Google AutoML

Visually Labeling Screenshots

Don't Take Security
Advice from SEO Experts
or Psychics
https://www.troyhunt.com/dont-
take-security-advice-from-seo-
experts-or-psychics-neil-patel/

Launch Jupyter Notebook in Google
Colaboratory
https://colab.research.google.com/github/ranksense/open-
source/blob/master/Presentations/TechSEOBoost/2018/Pyt
honforSEOTechSEOBoost2018_Hamlet_Batista.ipynb

SUMMARY
–

Summary
Practical applications
of Python => 3.6
for:
Data extraction
–
Preparation
–
Analysis
–
Machine learning
–
Deep learning

Free Realtime SEO Monitor
–
Ongoing monitoring with no active crawls
–
Receive alerts about critical SEO issues
–
Apply quick, temporary fixes in Cloudflare
–
Create developer tickets for permanent solutions
ABOUT RANKSENSE
– Apply for Beta Access
www.ranksense.com

Python for SEO

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Python for SEO

Similar to Python for SEO (20)

More from Hamlet Batista

More from Hamlet Batista (18)

Recently uploaded

Recently uploaded (20)

Python for SEO

Editor's Notes