Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Google Analytics and AdWords optimisation with GNU R

1,298 views

Published on

In the presentation we focused on brand keyword bidding. In this case, paid clicks potentially substitute organic traffic. The advertisement effect we observe is the total AdWords traffic which includes not only the truly gained traffic but also the clicks of individuals that would have visited the website even in absence of SEM. The latter is what economists call the missing counterfactual. Missing as we do not observe it. Without this data it is impossible to calculate the true ROI.

The way to find the missing counterfactual is through controlled random experiments. The presentation contains a complete AdWords optimisation toolbox in GNU R necessary to run controlled experiments.

Published in: Marketing
  • Be the first to comment

Google Analytics and AdWords optimisation with GNU R

  1. 1. Google Analytics and AdWords optimisation with GNU R Hinnerk Gnutzmann & Piotr Śpiewanowski flexponsive UG Booster Conference, 9th March 2016
  2. 2. About flexponsive UG • e-commerce consulting • Big Data focus • Qualitative user testing • Academic (PhD in economics) and programming background Contact • mailto: spiewanowski@flexponsive.net • web: https://www.flexponsive.net/ • t: @flexponsive
  3. 3. Topic of the day • Marketing outcomes • difficult to define • even more difficult to measure • Before Big Data: “Half the money I spend on advertising is wasted; the trouble is I don’t know which half.” (John Wanamaker, 1838 - 1922) • With Big Data: “AdWords brand keyword ads have no measurable short-term benefis” (Blake et al., 2015) - 100% wasted? • Open Questions: • Incrementality Debate: Do AdWords campaings cannibalise organic traffic? • Quality: Are bought visitors good or bad customers? • Heterogenity: Campaign effects differ between customers?
  4. 4. Agenda 1. Case study Brand Keyword: The Secret of vanishing AdWords ROI 2. What can we do? • attribution models • controlled experiments • GNU R & Analytics: A Dream Team 3. How to do that? • Google Core Reporting API & GNU R • GA Query Explorer • Configuring an experiment in AdWords 4. Analysis with GNU R • Data wrangling, sampling, etc. • GA replicate metrics • Regression Analysis 5. Case Study II: adClicks and rain in Bergen
  5. 5. Example - Skandiabanken
  6. 6. Example - Skandiabanken
  7. 7. What happened? • the AdWord is highly relevant to the search • Navigational Query: The visitor wants to visit Skandiabanken. • Customer knows the bank and maybe even has a service in mind • Result: Probably the best keyword in the account • Excellent CTR • Very good conversion on-site • CPC perhaps not so high • Any questions? • Organic result is the same! • What would you click if there was no ad?
  8. 8. What happened? • the AdWord is highly relevant to the search • Navigational Query: The visitor wants to visit Skandiabanken. • Customer knows the bank and maybe even has a service in mind • Result: Probably the best keyword in the account • Excellent CTR • Very good conversion on-site • CPC perhaps not so high • Any questions? • Organic result is the same! • What would you click if there was no ad?
  9. 9. What happened? • the AdWord is highly relevant to the search • Navigational Query: The visitor wants to visit Skandiabanken. • Customer knows the bank and maybe even has a service in mind • Result: Probably the best keyword in the account • Excellent CTR • Very good conversion on-site • CPC perhaps not so high • Any questions? • Organic result is the same! • What would you click if there was no ad?
  10. 10. Example - Skandiabanken (without AdWords)
  11. 11. ROI Problem: SEM expenditure a function not only of the campaign, but also of the behavior and intent of consumer
  12. 12. The eBay study • Blake et al. (2015), “Consumer Heterogeneity and Paid Search Effectiveness: A Large Scale Field Experiment” • Field Experiment: Does AdWords work for eBay? • Very controversial results: 1. Conventional methods used to measure the causal (incremental) impact of SEM vastly overstate its effect. 2. True effectiveness of SEM is small for a well-known company like eBay 3. Click substition: When the brand keyword AdWord disappeared, almost all the users click on the organic result 4. Informative Advertising: AdWords work if a visitor gains additional information through advertisement - AdWords had almost no effect on revenues from existing customers - They found their own way to eBay!
  13. 13. What can be done? Attribution modelling But how to know the true channel’s impact?
  14. 14. Attribution modelling • a way to divide the “credit” for a sale between different marketing channels • if you don’t know what attribution model you are using, it’s “last click” => you believe the sale only depends on the last ad the customer saw before purchasing • probably that’s not true: perhaps the customer had been following the company blog for a long time, heard friends talk off-line about the product, or saw many banner ads on different sides before making a purchase • problem: no good way to decide how to “attribute” between different marketing channels • results depend a lot on assumptions, which you cannot test • similar problem: if you advertise your brick-and-mortar store on TV and on radio, what drives the customer to your store?
  15. 15. What can be done? Controlled experiments • Select by random treatment and control group, for example: • Per user: A / B Testing • By Geographical Region • Assumption: Without experiment, both groups behave similarly • Evaluation: difference in differences • difference in the control group: Noise • difference in treatment group: Effect + Noise • Metrics: ∆TREATED − ∆UNTREATED • Advantages of a geographical experiment: • no multi-device tracking necessary • easy integration with external data • Caveat: Geographical groups really need to be comparable (e.g. commuters)
  16. 16. Difference in Differences
  17. 17. GNU R and Google Analytics: Dream Team 1. Selection of the treated and control group • Install R, generate a sample with GNU R • Export: Copy & paste to AdWords 2. Data collection • Google Analytics already configured 3. Aggregation and query • In the cloud: Google Analytics Query Explorer • Integration with RGoogleAnalytics 4. Evaluation: Estimation and Visualization • All necessary functions available as packages in R
  18. 18. About R • Programming language and software environment for statistical computing and graphics, a dialect of S • Quite lean; functionality is divided into modular packages • Graphics better than in most stat packages. • Useful for interactive work, but contains a powerful programming language for developing new tools (user -> programmer) • Very active and vibrant user community; R-help and R-devel mailing lists and Stack Overflow • Markdown packages for reproducable research and automated reporting • It’s free!
  19. 19. Install R • Open Source for Windows / Mac / Linux etc. • GNU R: https://www.r-project.org/ • RStudio IDE: http://www.rstudio.com • Cheat Sheets to help! • R Reference Card • RStudio cheatsheets • Package management via CRAN install.packages('RGoogleAnalytics', repos = "http://cran.no.r-project.org"); install.packages('plm', repos = "http://cran.no.r-project.org"); install.packages('ggplot2', repos = "http://cran.no.r-project.org");
  20. 20. Selecting Treatment Group download.file('https://goo.gl/qVgiYp', destfile='geoid.csv'); #Kommune level selection, but Fylke level also possible regions <- read.csv('geoid.csv'); norway<-regions[which(regions$Country.Code == 'NO' & regions$Target.Type == 'County' & regions$Status == 'Active'),]; set.seed(1); norway$isTreatment <- sample(c(0,1), nrow(norway), replace =T) write.csv(norway, file='norway.csv'); # paste into AdWords writeLines(as.vector( norway[which(norway$isTreatment == '1'),]$Canonical.Name), file('treatment.csv'));
  21. 21. Configuring Google AdWords I
  22. 22. Configuring Google AdWords II
  23. 23. Configuring Google AdWords III
  24. 24. Configuring Google AdWords IV
  25. 25. Configuring Google AdWords V
  26. 26. Done!!
  27. 27. Wait . . . for the results
  28. 28. Google Analytics Core Reporting API & R 1. Create an “app” • Google Developers page • Enable Google Analytics API • Create Credentials: OAuth client ID, Application type: Other • Result: Client ID and Client Secret 2. Find your GA Profile ID
  29. 29. Setting up GNU R client.id <- 'xxxxxxxxxxxxxxx.apps.googleusercontent.com'; client.secret <- 'xxxxxxxxxxxxxxx'; analyticsProfileId <- '111111111'; # redirect to google, paste, code require(RGoogleAnalytics); token <- Auth(client.id, client.secret) # save save(token, file = 'gatoken.txt'); # next time token <- load("./gatoken.txt") ValidateToken(token);
  30. 30. Create a query query.list <- Init(start.date = "2015-10-01", end.date = "2016-02-29", dimensions = "ga:region,ga:date,ga:medium", metrics = "ga:sessions,ga:transactionRevenue", filter = "ga:country==Norway", max.results = 50000, sort = "-ga:date,ga:region", table.id = paste0("ga:",analyticsProfileId)); ga.query <- QueryBuilder(query.list); ga.data <- GetReportData(ga.query, token);
  31. 31. Real Data Example - www.flexponsive.net kable(head(ga.data)) region date medium country sessions transactio Brussels 20160229 referral Belgium 1 State of Parana 20160229 referral Brazil 1 Baden-Wurttemberg 20160229 organic Germany 1 Baden-Wurttemberg 20160229 referral Germany 1 Rhineland-Palatinate 20160229 referral Germany 1 (not set) 20160229 (none) Hong Kong 5
  32. 32. Tip: Query Explorer
  33. 33. Tip2: Dimensions & Metrics Explorer
  34. 34. Tip3: Avoiding sampling > ga.data <- GetReportData(ga.query, token) Status of Query: The API returned 1393 results The query response contains sampled data. It is based on XX.XX % of your visits. You can split the query day-wise in order to reduce the effect of sampling. Set split_daywise = T in the GetReportData function Note that split_daywise = T will automatically .... • “Sampling occurs automatically when more than 500,000 sessions (25M for Premium) are collected for a report, allowing Google Analytics to generate reports more quickly for those large data sets.”
  35. 35. Data Integration • Wide Format: for each region and time a row • Long Format: Region / time / dimension one line (EAV) require (reshape2); ## Loading required package: reshape2 w <- reshape (ga.data, timevar = 'medium', idvar = c( 'region', 'date'), direction = 'wide');
  36. 36. Data Integration: Almost finished • Merge: Who is in which group? ds <- merge (w, norway[, c ( 'Name', 'isTreatment')], by.x = 'region', by.y = 'Name', all.x = T) • Data set is ready! • Comfortable DSL for data manipulation • Use packages to minimize code
  37. 37. Case Study: Wanderlust
  38. 38. Case Study: Wanderlust • an app “developed” for this presentation • mysterious weekend getaway and short holidays booking engine • supports inventory management of hotels and airlines • seasonal demand fluctuations
  39. 39. Evaluation • Simulated data for illustration: 3 summer months • 1st August: experiment starts in 10 random provinces (fylke) - AdWords stopped • 1st August: start of school, search volume falls everywhere by 50% • Scenario: 100% of visitors click organically when the AdWord invisible • Randomization has decided: • Sor-Trondelag (Trondheim): In the treatment group - from 1st August no AdWords • Hordaland (Bergen): In the control group - AdWords continue
  40. 40. Revenues in Sor-Trondelag (treatment) 60 80 100 120 Jun 01 Jun 15 Jul 01 Jul 15 Aug 01 Aug 15 Sep 01 date transactionRevenue.total
  41. 41. Revenues in Hordaland (control) 60 80 100 120 Jun 01 Jun 15 Jul 01 Jul 15 Aug 01 Aug 15 Sep 01 date transactionRevenue.total
  42. 42. Revenues in both Fylke 60 80 100 120 Jun 01 Jun 15 Jul 01 Jul 15 Aug 01 Aug 15 Sep 01 date transactionRevenue.total region Hordaland Sor−Trondelag
  43. 43. ROI Calculation - standard regression require(stargazer); out <- lm(transactionRevenue.total ~ isTreatment.cpc, data = sd.w) stargazer(out, header=FALSE, type='latex') Table 2 Dependent variable: transactionRevenue.total isTreatment.cpc −48.358∗∗∗ (1.350) Constant 111.350∗∗∗ (0.569) Observations 1,748 R2 0.424
  44. 44. ROI Calculation - standard regression • Standard OLS regression with binary variable == comparing means • But not the right ones. In this case: Revenues = β0 + β1 ∗ treatment • The treatment takes value 1 for the treatment group after the AdWords were stopped in Sor-Trondelag, otherwise 0 • As a result β1 represents the difference between the average revenues in Sor-Trondelag in August and average revenues in Hordaland and Sor-Trondelag in June and July • That’s clearly now what we are looking for!!
  45. 45. Difference in Differences
  46. 46. ROI Calculation - Differences in Differences require(plm) out <- plm(transactionRevenue.total ~ isTreatment.cpc, data=sd.w, index=c("region", "date"), model="between stargazer(out, header=FALSE, type='latex') Table 3 Dependent variable: transactionRevenue.total isTreatment.cpc 0.189 (0.254) Constant 102.741∗∗∗ (0.062) Observations 19 R2 0.032
  47. 47. ROI Calculation - Difference in Differences • Difference in Differences estimator using fixed effects model with binary varaibles allows to calculate the true effect of the treatment • Econometrically we estimate this equation: Revenues = β0 + β1 ∗ treatment + β2 ∗ before + γ ∗ fylke • fylke is a matrix of binary variables for each district • before is a binary variable takes value 0 in a period in which AdWords were running in all districts and value 1 in period in which experiment was started in some regions • treatment takes value 1 for the treatment group in the preiod in which the experimetn was started, i.e. after the AdWords were stopped in Sor-Trondelag, otherwise 0 • The estimation result reveals the true impact of AdWords on revenues in this data set
  48. 48. Discussion • The Missing counterfactual - we do not know what else could be happening - help: Experiment • Challenge: Big Data without Big Code - Google Analytics & GNU R - Very rich toolbox • Result: Differences in Differences can work - note assumptions
  49. 49. Table of Contents Intro Brand Keywords The eBay Study Calculating the true ROI Brand keywords with R Configuring Experiment Using Google Analytics API AdWords experiment: an example Regression Results

×