# SmartNews TechNight Vol.5 : AD Data Engineering in practice: SmartNews Ads裏のデータシステム

SmartNews, Inc.
Jan. 14, 2016
1 of 29

### SmartNews TechNight Vol.5 : AD Data Engineering in practice: SmartNews Ads裏のデータシステム

• 1. Data Engineering In Practice: SmartNews Ads裏のDMP System Lan
• 2. Who am I • Lan • Veteran hacker but new in AD world • someone who can make a computer do what he wants—whether the computer wants to or not. (http://paulgraham.com/gba.html) • ex-{Rakuten, GREE} • Distribution System, Info Retrieval, ML
• 3. Today’s Talk • DMP in SmartNews Ads • #1. Prediction • #2. Targeting • Future Work & Summary
• 4. DMP = Data Management Platform
• 5. DMP in SmartNews Ads • Private DMP ( 90%+1st-party data ) • Data Collect, Clean, Aggregation • ID Mapping • User Proﬁling • User Clustering • CTR / CVR Prediction • Lookalike • Custom Audience
• 7. #1 Prediction
• 8. Pick up an AD to feed here
• 9. Similar to Recommendation but DIFFERENT • optimization goal • accuracy of the probability
• 10. More than Ranking • When we do AD auction • eCPM (effective Cost per Mille) = CTR (Click Through Rate) x CPC (Cost per Click) • Suppose we have • CTRad1=0.05 > CTRad2=0.04 > CTRad3=0.03 • CPCad1 = 10JPY, CPCad2 = 13JPY, CPCad3 = 20JPY(winner) • but if: pCTRad1 = 0.2 (winner) > pCTR’ad2 = 0.1 > pCTR’ad3 = 0.03 • then we lost 0.1JPY potential income
• 11. The CTR(CVR) prediction Problem μ(a, u, c) = p(click | a,u,c)
• 12. CTR Prediction v1 • Train and scoring daily • One GBDT (Gradient Boosting Decision Tree) model per AD campaign • using ~1month’s data • Hundreds of small batches inside Hadoop Yarn • Quick and Simple • dev in 1 month • pick up best features for every campaign • minutes ~ 1 hour for model training • explainable Tree models • no need for AD feature • Same approach for CVR prediction (CPC / CVR = CPA (Cost Per Acquisition) ) delivery result User Features generate samples Yarn Users predictions sample model scoring sample model scoring sample model scoring …
• 13. Metrics • NE (Normalized Cross- Entropy) • the average log loss when using predicted CTR / the average log loss per impression • https://facebook.com//download/321355358042503/adkdd_2014_camera_ready_junfeng.pdf • AUC (Area under the ROC curve, AUROC) • measure ranking quality • others: Precision/Recall, ECS(Effective catalog size), CTR / CVR / Sales, etc
• 14. Review of CTR Prediction v1 • Marked improvement, moderate AUC & NE • And • hard to do overall tuning • hard to prediction online (feature set differs) • latency for new campaigns • relatively poor performance to new campaigns (cold start) • lost the connections between campaigns even for the same advertiser • …
• 15. CTR Prediction v2 • A simple model for all • AD feature added • Dynamic features extraction • All calculation distributed • GBDT + LogisticRegression • Train once per day, scoring twice
• 16. About the Features • >1M unique features, sparse • GBDT provides great feature engineering • (sometimes) feature engineering is kind of intuition and trial-and-error • demographic, device, location, reading interests… • AD history is helpful • Feature Hashing, Binarization & Discretization, …
• 17. Performance improvement
• 18. #2 Targeting Watabe TamTam Komiya Takei Ikeishi Nagase Lan Niku Game Beer Snack Costume Gourmet Princess
• 19. It’s difﬁcult comparing to
• 20. Proﬁling User by Statistics and ML • Gender Prediction (precision: 0.90+), Age Prediction, … • News Channel / Source Preference • AD Slot Preference • …
• 21. Standard Targeting • Female in Kansai who subscribes Travel Channel
• 22. Lookalike Targeting
• 23. Lookalike Targeting • Our solution • Solve it as an classiﬁcation problem • Seed user as Positive Sample • While all targeting candidates as Negative Sample (w/ random sampling ) • based on Spark MLlib Logistic Regression • 30%~50% CVR↑ comparing to normal targeting
• 24. Article Keyword Targeting Keyword Realtime Calculating Reach UU Only user who exceeds a certain read-time threshold will be included
• 25. Custom Audience SmartNews AD tracker Send any custom event (S2S req, web beacon, etc) Event Audience BloomFilter Obj Updating per Several Minutes Your Service / App / Site SmartNews AD Delivery Cluster AD targeting / Delete Targeting Lookalike Lookalike Targeting
• 26. Future Work
• 27. Targeting Audience by Interests
• 28. Collect Negative Signal to Optimize UX
• 29. Summary of My 1st SmartNews Year • Challenge place. We’re startup so we can move quick and break things • Learn from the industry leaders. Keep trial-and-error. • Number don’t lie. Don’t trust your intuition over number. • But if you really doubt the number, look closely. there may be BUG hidden.