Computational advertising in Social Networks

6,443 views

Published on

Computational Advertising Workshop @ ICML 2012

Published in: Business, Technology

Computational advertising in Social Networks

  1. 1. Computational   Advertising     in    Social  Networks   Anmol Bhasin Sr. Manager Analytics Engineering www.linkedin.com
  2. 2. Core  MessageWe  live  in  fascinating  times.  Two  new  nascent  technological  disciplines   are   coming   together   to   transform   how   the  marketers   go   about   their   business   of   reaching   consumers,  be  it  businesses  or  end  users.   It   is   time   for   the   practitioners   in   these   disciplines   to   push  the   envelope   by   creating   innovative   products   and  sophisticated  algorithms  to  define  what  the  future  will  hold  in  this  new  digitally  social  era.. Anmol  Bhasin
  3. 3. Source  :  www.140proof.com
  4. 4. Our Mission Connect the world’s professionals to make them more productive and successful.Our VisionCreate economic opportunity for every professional in the world.Value propositionTo make professionals better in theJobs that they are already in.
  5. 5. World’s  largest  professional  network   Over  60%  of  members  are  now  international 161+ M 90 ~2/sec New  Members  joining 82% * Fortune 100 Companies use LinkedIn to hire 55 >2M * 32 Company Pages 17 2 4 8 ~4.2B Professional searches in 20112004 2005 2006 2007 2008 2009 2010 LinkedIn  Members  (Millions)   *as of March 31, 2012 6
  6. 6. Challenges  in  Social  Network  Advertising §  Information  Seeking  vs  Information  Consumption §  Dedicated  marketing  channels  for  brand  awareness  required §  Hypertargeting §  Blending  organic  and  sponsored  content §  Mobile  ?
  7. 7. Sponsored  Search  vs  Social  Advertising §  No  Search  Queries                Q  :  Home  Remodel  San  Francisco  Bay  Area E( pCTR(clicka | qi , u j,C)) >> E( pCTR(clicka | u j, C))
  8. 8. Sponsored  Search  vs.    Social  Network  Advertising    Source  :    Marin  Software        www.marinsoftware.com
  9. 9. Dedicated  Marketing  Channels
  10. 10. Hypertargeting E(max[c1, c2, c3.....c n ]) > E(max[c1, c2, c3.....c n−1 ])
  11. 11. Blending  Organic  and  Sponsored
  12. 12. The  thing  called  Mobile.. §  Cannibalizing   website   page  views §  Small  form  factor §  ~10%  views  from   Mobile  but  only  ~1%   monetizable §  Blending  organic  and   sponsored  essential §  Impression  &   conversion  tracking   loop  hard  to  close
  13. 13. The  good  news.. Hey  user..  I  know  thee
  14. 14. The  good  news.. And  your  friends.. source:  h]p://inmaps.linkedinlabs.com https://inmaps.linkedinlabs.com 19
  15. 15. The  good  news.. We  also  know  what  you  read  .. And  how  much  you  liked  it..
  16. 16. Advertising  @  LinkedIn LinkedIn  Marketing  Solutions Performance   Brand Organic   Marketing Marketing Marketing (LinkedIn  Ads) (Display  Ads) (Company  Pages)
  17. 17. Advertiser  spectrum Higher  Educa-on   Tech  B2B  MBA  Programs    |    Masters  &  Graduate  Programs   CRM    |    SoGware/Biz  Hardware  |    ERP    |    Sales  Tools     Online  Degrees    |  Execu-ve  Leadership       Marke-ng  Automa-on    |    SaaS         Internet  Services   Staffing  &  Hiring   Website  Hos-ng  |    Video  Conferencing   Staffing  Agencies    |    Recrui-ng  SoGware     Prin-ng    |    Phone  Systems         Corporate  Recrui-ng    |  Job  Boards          
  18. 18. Campaign  creation
  19. 19. The  basics  -­‐‑  Ad  Ranking §  Given U j , {(c0, b0 ), (c1, b1 ), (c2, b2 ), (c3, b3 )..(cn, bn )}, H§  Objective argmax( pCTR i *bidi ) i∈C§  Goal: §  Increase revenue §  Respect daily budgets of Advertisers §  Good user experience
  20. 20. Virtual  Profiling Title  : Title  :  Eng  Mgr   Sr.  SE<1>,  Eng  Mgr<1>,  Company  :  LinkedIn Eng  Dir<1> Location  :  CA,USA   Skills  :    ML,  RecSys Company  :   LinkedIn<2>,  Google<1>, Title  :  Sr.  SE Company  :  Google Location  :  PA,  USA Location  :   Skills  :    ML,  DM          CA,USA  <2>,  PA,  USA<1>   Skills  :   Title  :  Eng  Dir  ML<2>,  RecSys<1>,   Company  :  Linkedin  Stats<1>,  DM<1> Location  :  PA,  USA Skills  :    ML,  Stats,  DM
  21. 21. Virtual  Profiling Title  :  Eng  Mgr Company  :  LinkedIn Location  :  CA,USA   Skills  :    ML,  RecSys Title  :  Vice  President Company  :  Twi]er Location  :  CA,USA   Skills  :    DM,  ML,   RecSys                    ……………….
  22. 22. Virtual  Profiling Information  Gain §  Pick  Top  K  overrepresented  features  from  the   clicker  distribution  vs  the  target  segment A  representative  projection  of  the  item  in  the   member  feature  space
  23. 23. CTR  Prediction  –  CF  Similarity Ranker MEMBER  FEATURES AD  CREATIVE  VIRTUAL  PROFILE Creative   Score  to   features pCTR   pCTRi correction §  L2  regularized  Logistic  Regression  (Liblinear,  VW,  Mahout,  ADMM) §  Frequency  or  conditional  smoothed  oCTR  as  feature  values  from   activated  features  in  the  Virtual  Profile §  For  new  ad  creatives  back-­‐‑off  to  the  advertiser  /  ad  category  nodes  till   they  reach  critical  impression/click  volume  (explore/exploit)
  24. 24. What  about  Hypertargeting  ? Done  via §  Transitions  probability §  Profile  collocation  analysis §  Co-­‐‑Targeted  segments §  Virtual  Profile  Similarity §  A/B  tested  for  most  effective   solutions
  25. 25. Recommendations:  What  are  they  worth?  Think  50% §  > 50% of connections are from recommendations (PYMK)§  > 50% of job applications are from recommendations (JYMBII)§  > 50% of group joins are from recommendations (GYML) RecLS 32
  26. 26. Hiring  Solutions  –  Self  Serve  Jobs  Postings
  27. 27. Sponsored  Recommendations
  28. 28.  Talent  Match
  29. 29. Recommendation  Algorithm Job Corpus  Stats Matching Transition  probabilities Connectivity title geo industry description … Binary        Exact  matches: yrs  of  experience  to  reach  title   education  needed  for  this  title … company functional area        geo,  industry,        … Soft User  Base Similarity   (candidate  expertise,  job  description)          transition Filtered 0.56          probabilities, Candidate          similarity,          …   Similarity   (candidate  specialties,  job  description) 0.2 Transition  probability 0.43 Text (candidate  industry,  job  industry) General Current Position expertise title 0.8 specialties summary Title  Similarity education tenure lengthheadline industrygeo functional area 0.7 Similarity  (headline,  title) experience … deriv ed . . .
  30. 30. Feature  Engineering  –  Entity  Resolution §   Companies ‘IBM’ has 8000+ variations -  ibm – ireland -  ibm research -  T J Watson Labs -  International Bus. Machines K-Ambiguous -  Deep Blue•   Huge  impact  on  the    business  and  UE •   Ad  targeting •   TalentMatch •   Referrals 37
  31. 31. Feature  Engineering  –  Entity  Resolution §   Binary  classifier  (LR),  not  ranker §   P({position,  company  entity}  is   a  match) §   Features:   §   Content  –  name  similarity   features,  industry  match,   location  match,  email  domain   match,  company  size §   Social  Graph  -­‐‑  #  connections   at  company  entity §   Behavior  -­‐‑  #  of  invitations   received  from  company  entity   members §  Company  candidate  set   97%  Precision   leveraged  from  Social  graph   at  50%  Coverage and  cosine  similarity Asonam’11, KDD’11 38
  32. 32. Feature  Engineering  –  Sticky  locations §   Zip  code  mapped  to  Regions §   How  sticky  are  those  locations? §   Huge  impact  on  the  business  and  UE • Job  Seeker,  Recruiter
  33. 33. Feature  Engineering  –  Sticky  locations §   Open  to  relocation  ? §   Region  similarity  based  on  profiles  or  network §   Region  transition  probability §  predict  individuals  propensity  to  migrate  and   most  likely  migration  target   §  Impact  on  job  recommendations §  20%  lift  in  views/viewers/applications/applicants
  34. 34. The  Network  effect 41
  35. 35. What  should  you  transition  to  &  when  ? 42
  36. 36. Multiple  Objective  Optimization Applicable  in  multiple  contexts §  Online  Dating §  Click  Shaping §  Revenue  vs  CTR  optimization  tradeoff §  Talent  Match   Luiz  Pizzato,  Tomek  Rej,  Thomas  Chung,  Irena  Koprinska,  Kalina  Yacef,  and  Judy  Kay.  2010.  Reciprocal  recommender  system  for  online  dating.  In  Proceedings  of  the  fourth  ACM  conference  on  Recommender  systems  (RecSys  ʹ10).  ACM,  New  York,  NY,  USA,  353-­‐‑354.   Deepak  Agarwal,  Bee-­‐‑Chung  Chen,  Pradheep  Elango,  and  Xuanhui  Wang.  2011.  Click  shaping  to  optimize  multiple  objectives.  In  Proceedings  of  the  17th  ACM  SIGKDD  international  conference  on  Knowledge  discovery  and  data  mining  (KDD  ʹ11).  ACM,  New  York,  NY,  USA,  132-­‐‑140 Mario  Rodriguez,  Christian  Posse,  Ethan  Zhang.2012.  Multiple  Objective  Optimization  in  Recommender  Systems.  To  appear  in  Proceedings  of  the  Sixth  ACM  conference  on  Recommender  systems  (RecSys  ʹ12)
  37. 37. Multiple  Objective  Optimization JobSeeker   Intent TalentMatch Member  Z,  0.89,   job Active … MOO Member  X,  0.98,  0.98,   +40%  InMail  Response   NonSeeker Rate Member  Y,  0.91,  0.91,   NonSeeker … Formalism
  38. 38. Multiple  Objective  Optimization §  TalentMatch §  Logistic  Regression  model §  JobSeeker  Intent §  Ordered  Logistic  Regression  model §  Active/Passive/NonSeekers §  Outputs  propensity  score §  MOO  (Multiple  Objective  Optimization) §  Grid  Search  on  Objective  function §  sMOOth  for  large  parameter  spaces
  39. 39. Multiple  Objective  Optimization Formalism 1.  Rank  top  K’  >  K  semantically  rank  results 2.  Perturb  the  ranking  with  a  parametric  function  parameterized  by  α  ,  β   which  leads  to  inclusion  of  the  secondary  objective 3.  Measure  the  perturbation  using  a  delta  function  wrt  to  the  primary   objective 4.  Create  a  framework  to  quantify  the  tradeoff  between  the  two  objectives
  40. 40. Multiple  Objective  Optimization
  41. 41. Multiple  Objective  Optimization
  42. 42. Social  Referral
  43. 43. Social  Referral §  Order  recommendations  by §  Connection  Strength  between  two  users  => (ui , u j ) σ §  Recommendation  Strength  for  the  target  user  => R(u j , gk ) §  Combination  thereof Linkedin  Group:  Text  Analytics From:  Deepak  Agarwal  –  Engineering  Director,  LinkedIn I  found  this  group  interesting,  and  I  think  you  will  too Deepak 2X  conversion Linkedin  Group:  Text  Analytics >  2X  Conversion Mohammad  Amin,  Baoshi  Yan,  Sripad  Sriram,  Anmol  Bhasin,  Christian  Posse.  2012.  Social   Referral  :  Using  network  connections  to  deliver  recommendations.  To  appear  in  Proceedings  of   the  Sixth  ACM  conference  on  Recommender  systems  (RecSys  ʹ12)
  44. 44. Follow  Ecosystem
  45. 45. Recommended  Followers  Targeting Task  :  To  identify  a  set  (usually  Millions)  of   users  likely  to  follow  the  given  company Scorer MEMBER  PROFILE  FEATURES COMPANY  FOLLOWER  VIRTUAL   PROFILE Global   Company     p( follow | ci , u j ) popularity Other  rankings  – 1.  User’s  login  probability  in  next  X  days 2.  User’s  PVs  in  the  next  X  days 3.  User’s  propensity  to  follow  any  company Weighted  Borda  count  to  for  Information  Fusion  &  A/B  Test  combinations h]p://www.colorado.edu/education/DMP/voting_b.html  -­‐‑  loss  of  information  in  plurality  votes
  46. 46. A/B  Testing Is  Option  A  Be]er  Than  Option  B?  Let’s  Test Beware  of   §  novelty  effect §  Cannibalization A/B  allows  seemingly  subjective  questions  of  design—color,  layout,  image   §  potential  biases  (time,  targeted  population) selection,  text—to  become  incontrovertible  ma]ers  of  data-­‐‑driven  social  science.   §  random  sampling  destroying  the  network    -­‐‑    Dan  Siroker,  Digital  Advisor  to  Barack  Obama’s  election  campaign  -­‐‑2008 effect h]p://www.wired.com/business/2012/04/ff_abtesting/ Don’t forget to A/A test first Enjoy testing furiously!. Hundreds of tests live on LinkedIn at all times.. job$views$per$5%$bucket$range$5$6/5/11$ job$views$6/19/11$ 9,000" 7,000" 8,000" 7,000" 6,000" 6,000" 5,000" 5,000" 4,000" job"views"per"5%"bucket"range"?" 4,000" 6/5/11" 3,000" job"views"6/19/11" 3,000" 2,000" 2,000" 1,000" 1,000" 0" 0" 0" 5" 10" 15" 20" 25" 0" 5" 10" 15" 20" 25" (“Seven  Pitfalls  to  Avoid  when  Running  Controlled  Experiments  on  the  Web”,  KDD’09 “Framework  and  Algorithms  for  Network  Bucket  Testing”  WWW’12  submission) 53 53
  47. 47. Demand  exceeds  supply Supply Demand Newer  advertise r     acquisition Page  view  growth  ,  highest   Social  media   in  mobile frenzy #  of  pages  with   Newer  Ad   ads products  like   “fans”  &   “follows” 55
  48. 48. Real  Time  Bidding Fun  problems §  When  not  to  bid  ? §  CTR  prediction  on  the  publisher   §  What  auction  does  the  exchange  run  ? §  Onsite  vs  Offsite  impression  tradeoff   for  impression  capped  campaigns
  49. 49. Onsite/Offsite  tradeoff LinkedIn Ads shown to LinkedIn Member – zillow.com
  50. 50. Other  initiatives.. §  Audience  Forecasting §  Bid  Landscaping §  Lookalike  Modeling §  Publisher  DNA §  Auto  ad  creative  generation  from  landing  pages §  Explore  Exploit  strategies And  more..
  51. 51. New  Product! §  New  guaranteed  display  ad  product §  Impressions  guaranteed  =  1 §  eCPI    >  $[0-­‐‑9]{1}000
  52. 52. Picture yourself with this New Job: You Applied Researcher / Research Engineer
  53. 53. Credits Engineering  :  Abhishek  Gupta,  Adam  Smyczek,  Adil  Aijaz,  Alan  Li,  Baoshi  Yan,  Bee-­‐‑Chung  Chen,  Deepak  Agarwal,  Ethan  Zhang,  Haishan  Liu,  Igor  Perisic,  Jonathan  Traupman,  Liang  Zhang,  Lokesh  Bajaj,  Mario  Rodriguez,  Mohammad  Amin,  Parul  Jain,  Sanjay  Dubey,  Tarun  Kumar,  Trevor  Walker,  Utku  Irmak Product  :  Christian  posse,  Gyanda  Sachdeva,  Mike  Grishaver,  Parker  Barrile,  Sachit  Kamat,  Andrew  Hill
  54. 54. Thank  You! Questions? Contact: abhasin@linkedin.com h]p://engineering.linkedin.com/ 62

×