Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Marketing Automation: Begging for Machine Learning


Published on

Advanced Spark and TensorFlow Meetup

Published in: Technology
  • There is a useful site for you that will help you to write a perfect and valuable essay and so on. Check out, please ⇒ ⇐
    Are you sure you want to  Yes  No
    Your message goes here
  • Writing a good research paper isn't easy and it's the fruit of hard work. For help you can check writing expert. Check out, please ⇒ ⇐ I think they are the best
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Marketing Automation: Begging for Machine Learning

  1. 1. MARKETING AUTOMATION: BEGGING FOR MACHINE LEARNING Seth Myers Senior Data Scientist Demandbase Advanced Spark and TensorFlow Meetup
  2. 2. © 2016 DEMANDBASE|SLIDE 2 What We Do  We automate how companies market to, sell to, and advertise to other companies (B2B)  Old methods of B2B  Target as many companies as possible  Example: Send out generic spam sales emails  New method  Customized/Personalized approached to each customer  Focus on the right companies instead of all companies  Scaling personalized approach is huge opportunity for machine learning
  3. 3. © 2016 DEMANDBASE|SLIDE 3 Examples  Business Knowledge Graph  A monthly crawl of 2 to 5 Billion web pages to build a network of companies, people, and products  Uses in-house-built NLP to infer relationships between entities from natural text (e.g. which companies buy from/compete with/acquire other companies  Technologies: Elastic MapReduce  Predicting company buying behavior  Currently tracking 700 Billion web interactions a year.  Predicts when a company is shopping around for a new product.  Technologies: Spark, BigQuery  Predicting our client’s next customer (Who to Target)  Takes current customers as positive training examples, trains a classifier  Scores 14 Million known companies inside of 20 minutes (other solutions take months to implement)  Technologies: Spark/mllib  Content Recommendations for our clients’ websites
  4. 4. © 2016 DEMANDBASE|SLIDE 4 Generating Page Recommendations  An important potential customer is on your website – how do we make sure they find the right pages that will sell them?  Our clients are big companies that sell a lot of products  Different visitors are looking for very different things  Using IP addresses and cookies, we know the company where each website visitor works  Based on visitor’s company, generate personalized recommendations
  5. 5. © 2016 DEMANDBASE|SLIDE 5
  6. 6. © 2016 DEMANDBASE|SLIDE 6 Learning to Recommend  Build a supervised classifier that predicts which page a website visitor will go to next: Prob(current page next page)  Recommend the most likely next page.  Training data: our client’s historical website visitor logs  Feature vectors are extracted from:  Visitor’s company (revenue, industry, buying behaviors, corporate structure, etc)  Current and next pages  Previous visitor behaviors Visitor Page view sequence Visitor 1 Page 7, Page 4, Page 8 Visitor 2 Page 2, Page 9
  7. 7. © 2016 DEMANDBASE|SLIDE 7 Planning Sequences of Recommendations  Our clients first reaction: “meh.”  They only really care about traffic to certain target pages  Signing up for webinars  Downloading whitepapers  Requesting a demo  Becomes a reinforcement learning problem:  Generate a sequence of page recommendations r1, r2, r3,… that maximize
  8. 8. © 2016 DEMANDBASE|SLIDE 8 Closed-Form Solution  Assume Markov property – next page is independent of previous pages  Define V(pi) = Prob(eventually landing on target page | current page = pi)  If pi is a target page, then V(pi) = 1  The probability of going to a particular page pj, and then to the target page is  V(pi) is the probability of going to any page times that page’s V()
  9. 9. © 2016 DEMANDBASE|SLIDE 9  If we recommend page pj, we assume the user will land on target page with probability  We recommend the best pj Closed-Form Solution  So Bounce rate, measured directly Original ML Classifier
  10. 10. © 2016 DEMANDBASE|SLIDE 10 Implementation  For each client, train the Pr(pi pj ) classifier in Spark (mllib) from web logs, and serialize the classifier in Redis.  For each new visitor to the website – calculate Pr(pi pj ) between all pages pi, pj on the site  Solve for V(pi) for all pages.  On each page view, generate next-page recommendations:
  11. 11. © 2016 DEMANDBASE|SLIDE 11 But Does it work?  A/B Test on with recommendations  Bounce Rate: -52%  Page views per visitor: +107%  Time on Site: + 231%  Target Page # 1 – Sales Demo Requests: +311%  Target Page # 2 – Downloading Whitepapers: +593%  Similar results across all clients
  12. 12. © 2016 DEMANDBASE|SLIDE 12 Shameless Plug  We’re hiring very aggressively  Pre-IPO  Just raised $65M, mostly earmarked for expanding our data science team and infrastructure  We build customer-facing machine learning solutions – no revenue modeling for the finance team here  High impact – all of our recent hires are already building new products.  Proprietary datasets - a lot of fun to work with  Voted “Best 10 Places to Work” multiple times by Glassdoor and SF Business Times  Interested?  Seth –