Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Advertising Quality Science

399 views

Published on

Native advertising is a specific form of online advertising where ads replicate the look-and-feel of their serving platform. In such context, providing a good user experience with the served ads is crucial to ensure a positive user experience and hence long-term user engagement. In this talk, I will describe work at Yahoo aiming at understanding the user experience on ads in the mobile context and building learning frameworks to identify and account for ads of low quality while ensuring a return of investment to advertisers.

Slides for the Invited Talk at BigData Innovators Gathering (BIG), co-located with WWW 2017, Perth 2017 (https://big2017.org). Earlier versions of this talk were given at various venues in London.

Published in: Internet
  • Be the first to comment

  • Be the first to like this

Advertising Quality Science

  1. 1. Advertising Quality Science Mounia Lalmas
  2. 2. This talk 4-year effort across research, engineering and product at Yahoo to measure the quality of ads served on Gemini, Yahoo native advertising network Not just measuring but taking actions to improve user experience as well as providing feedbacks to advertisers à no deep learning but large scale predictive analytics Focus of the talk: the post-click experience on native ads à the quality of the landing page… if it is seen
  3. 3. 3 The advertising world is fun J
  4. 4. Online advertising is big business Values in $billions
  5. 5. Advertising is how Yahoo (and many other Internet companies) makes money… and what keeps Yahoo services free for its customers 90% of Yahoo’s revenue is from advertising: 2016 search advertising revenue – $2.67B (52% of total revenue) 2016 display including native advertising – $1.98B (38% of total) Scale: billion ads served daily (Source: Yahoo 2016 10k annual report)
  6. 6. Online advertising is about connecting supply & demand Search Native Display Video Brand Direct Response Yahoo own & operated sites Publisher Partners SUPPLY (publishers) DEMAND (advertisers)
  7. 7. Advertising (ad) quality science Develop predictive models that characterise the quality of ads shown to and clicked by users. Maximise revenue and guaranteeing ROI to advertisers without negatively impacting user experience. Publishers Advertisers Users Ad inventory ad network Being able to help advertisers improve the quality of their ads
  8. 8. Ad Quality: Scope Non-intentional Ad quality Intentional Ad compliance
  9. 9. Major shift in how users access the Internet comScore 2015 UK Internet users
  10. 10. Native advertising (Source: Sharethrough.com & IPG Media Lab Study: Na;ve Adver;sement Effec;veness) Visually Engaging Higher user attention Higher brand lift Social sharing
  11. 11. The quality of the post-click experience the quality of the landing page on mobile
  12. 12. The post-click experience: Dwell time dwell time proxy of post-click ad quality experience predictive post-click ad quality models ad quality ratings & recommendations accidental clicks identification proxy of accidental clicks Metric Ad serving User engagement Advertiser ROI
  13. 13. dwell time proxy of post-click ad quality experience predictive post-click ad quality models ad quality ratings & recommendations accidental clicks identification proxy of accidental clicks Metric Ad serving User engagement Advertiser ROI The post-click experience journey
  14. 14. Quality of the post-click experience Best experience is when conversion happens No conversion does not mean a bad experience Proxy metric of post-click experience: dwell time on the ad landing page tad-click tback-to-publisher dwell time = tback-to-publisher – tad-click Positive post-click experience (“long” clicks) has an effect on users clicking on ads again
  15. 15. dwell time proxy of post-click ad quality experience predictive post-click ad quality models ad quality ratings & recommendations accidental clicks identification proxy of accidental clicks Metric Ad serving User engagement Advertiser ROI The post-click experience journey
  16. 16. Optimise for high quality ads Estimating P(hq|click) = quality score P(dwell time > t) Build predictive models that predict if an ad is of high quality = predicted dwell time above a given threshold t ➔  high quality = high dwell time revenue = 𝓕 (bid, CTR, quality) P(hq|click) logistic regression, gradient descent boosting, random forest, survival random forest
  17. 17. Landing page features ●  window_size ●  view_port ●  media_support content mobile support requested information multimedia mobile optimized out-going connectivity interactivity textual content in-coming connectivity ●  description ●  keywords ●  title meta information ●  num_forms ●  num_input_radio ●  num_input_string ●  ... readability multimedia significant effect of text readability and page structure
  18. 18. A/B testing dwell time increased by 20% bounce rate decreased by 7% revenue = bid x CTR x quality (Lalmas etal, 2015; Barbieri, Silvestri & Lalmas, 2016)
  19. 19. dwell time proxy of post-click ad quality experience predictive post-click ad quality models ad quality ratings & recommendations accidental clicks identification proxy of accidental clicks Metric Ad serving User engagement Advertiser ROI The post-click experience journey
  20. 20. Landing page rating: Low, Average or High landing pages quality score q … L H L and H are customisable: e.g., LOW=[0,25%), AVG=[25%,75%], HIGH=(75%,100%] 2 cut-off points (L, H) that divide distribution of quality scores q into 3 regions: -  LOW: q < L -  AVG: L <= q <= H -  HIGH: q > H (L, H) ad ratingq LOW
  21. 21. Improving landing pages Exploiting the features for recommending improvements mobile optimized out-going connectivity interactivity textual content in-coming connectivity meta information readability multimedia ●  num_forms ●  num_input_radio ●  num_input_string ●  ... interactivity ●  mediannum_forms ±ε ●  mediannum_input_radio ±ε ●  mediannum_input_string ±ε ●  ... for each feature compute median and confidence interval for each ad feature compute the distance from the confidence interval given an ad num_input_radio num_forms num_input_string ...
  22. 22. There might be too few/much textual content There might be too few/many entities There might be too few/many images Landing Page Content n. of words n. of Wikipedia en;;es n. of images Landing Page Layout height/width resizability (fit to mul;ple screen size) Landing Page Structure n. of drop-down menus n. of checkboxes n. of input strings Landing Page Readability content summarizability The height/width of the landing page might be too small/large The landing page might not be adapted to different screen sizes There might be too many drop-down menus There might be too many checkboxes There might be too much information requested from the users The textual content might be further summarised to make it more readable Examples of recommendations
  23. 23. dwell time proxy of post-click ad quality experience predictive post-click ad quality models ad quality ratings & recommendations accidental clicks identification proxy of accidental clicks Metric Ad serving User engagement Advertiser ROI The post-click experience journey
  24. 24. peak on app X ●  accidental clicks do not reflect post-click experience ●  not all clicks are equal app X The quality of a click on mobile apps peak on app Y dwell time distribution of apps X and Y for a given ad app Y
  25. 25. dwell time proxy of post-click ad quality experience predictive post-click ad quality models ad quality ratings & recommendations accidental clicks identification proxy of accidental clicks Metric Ad serving User engagement Advertiser ROI The post-click experience journey
  26. 26. Fitting the data into a mixture model The number of mixture components is determined using the BIC criterion which selects the model that fits best the data while avoiding overfitting Time period 1 Time period 2 (after UI change) Bayesian information criterion (BIC) bouncy clicks accidental clicks
  27. 27. Accidental clicks threshold for app X Min 1st Quartile Median Mean 3rd Quartile Max Distribution of the medians as computed on the first component of each ad Applications -  discount accidental clicks using economics models -  train click models discarding accident clicks -  input to UI design ads with all three components
  28. 28. dwell time proxy of post-click ad quality experience predictive post-click ad quality models ad quality ratings & recommendations accidental clicks identification proxy of accidental clicks Metric Ad serving User engagement Advertiser ROI Ad quality: The post-click experience journey Acknowledgments: Marc Bron, Ayman Farahat, Andy Haines, Miriam Redi, Gabriele Tolomei, Guy Shaked, Ke (Adam) Zhou, Fabrizio Silvestri, Michele Trevisiol, Ben Shahshahani, Puneet M Sangal and many others

×