Demandbase uses machine learning to personalize marketing automation for business-to-business companies. They build a knowledge graph by crawling web pages to infer relationships between companies, analyze 700 billion web interactions to predict customer behavior, and recommend targeted content to potential customers based on their company profile. Their personalized recommendations increased key metrics like bounce rate, page views, time on site, and conversions by over 100% according to A/B tests. Demandbase is hiring data scientists to continue expanding their machine learning capabilities.

1. MARKETING AUTOMATION:
BEGGING FOR MACHINE LEARNING
Seth Myers
Senior Data Scientist
Demandbase
smyers@demandbase.com
Advanced Spark and TensorFlow Meetup

2. © 2016 DEMANDBASE｜SLIDE 2
What We Do
 We automate how companies market to, sell to, and advertise to other
companies (B2B)
 Old methods of B2B
 Target as many companies as possible
 Example: Send out generic spam sales emails
 New method
 Customized/Personalized approached to each customer
 Focus on the right companies instead of all companies
 Scaling personalized approach is huge opportunity for machine learning

3. © 2016 DEMANDBASE｜SLIDE 3
Examples
 Business Knowledge Graph
 A monthly crawl of 2 to 5 Billion web pages to build a network of companies, people, and
products
 Uses in-house-built NLP to infer relationships between entities from natural text (e.g.
which companies buy from/compete with/acquire other companies
 Technologies: Elastic MapReduce
 Predicting company buying behavior
 Currently tracking 700 Billion web interactions a year.
 Predicts when a company is shopping around for a new product.
 Technologies: Spark, BigQuery
 Predicting our client’s next customer (Who to Target)
 Takes current customers as positive training examples, trains a classifier
 Scores 14 Million known companies inside of 20 minutes (other solutions take months to
implement)
 Technologies: Spark/mllib
 Content Recommendations for our clients’ websites

4. © 2016 DEMANDBASE｜SLIDE 4
Generating Page Recommendations
 An important potential customer is on your website – how do we make
sure they find the right pages that will sell them?
 Our clients are big companies that sell a lot of products
 Different visitors are looking for very different things
 Using IP addresses and cookies, we know the company where each
website visitor works
 Based on visitor’s company, generate personalized recommendations

5. © 2016 DEMANDBASE｜SLIDE 5

6. © 2016 DEMANDBASE｜SLIDE 6
Learning to Recommend
 Build a supervised classifier that predicts which page a website visitor will
go to next: Prob(current page next page)
 Recommend the most likely next page.
 Training data: our client’s historical website visitor logs
 Feature vectors are extracted from:
 Visitor’s company (revenue, industry, buying behaviors, corporate structure, etc)
 Current and next pages
 Previous visitor behaviors
Visitor Page view sequence
Visitor 1 Page 7, Page 4, Page 8
Visitor 2 Page 2, Page 9

7. © 2016 DEMANDBASE｜SLIDE 7
Planning Sequences of Recommendations
 Our clients first reaction: “meh.”
 They only really care about traffic to certain target pages
 Signing up for webinars
 Downloading whitepapers
 Requesting a demo
 Becomes a reinforcement learning problem:
 Generate a sequence of page recommendations r1, r2, r3,… that maximize

8. © 2016 DEMANDBASE｜SLIDE 8
Closed-Form Solution
 Assume Markov property – next page is independent of previous pages
 Define V(pi) = Prob(eventually landing on target page | current page = pi)
 If pi is a target page, then V(pi) = 1
 The probability of going to a particular page pj, and then to the target page is
 V(pi) is the probability of going to any page times that page’s V()

9. © 2016 DEMANDBASE｜SLIDE 9
 If we recommend page pj, we assume the user will land on target page
with probability
 We recommend the best pj
Closed-Form Solution
 So Bounce rate,
measured directly
Original ML
Classifier

10. © 2016 DEMANDBASE｜SLIDE 10
Implementation
 For each client, train the Pr(pi pj ) classifier in Spark (mllib) from web
logs, and serialize the classifier in Redis.
 For each new visitor to the website – calculate Pr(pi pj ) between all
pages pi, pj on the site
 Solve for V(pi) for all pages.
 On each page view, generate next-page recommendations:

11. © 2016 DEMANDBASE｜SLIDE 11
But Does it work?
 A/B Test on Demandbase.com with recommendations
 Bounce Rate: -52%
 Page views per visitor: +107%
 Time on Site: + 231%
 Target Page # 1 – Sales Demo Requests: +311%
 Target Page # 2 – Downloading Whitepapers: +593%
 Similar results across all clients

12. © 2016 DEMANDBASE｜SLIDE 12
Shameless Plug
 We’re hiring very aggressively
 Pre-IPO
 Just raised $65M, mostly earmarked for expanding our data science
team and infrastructure
 We build customer-facing machine learning solutions – no revenue
modeling for the finance team here
 High impact – all of our recent hires are already building new products.
 Proprietary datasets - a lot of fun to work with
 Voted “Best 10 Places to Work” multiple times by Glassdoor and SF
Business Times
 Interested?
 Seth – smyers@demandbase.com