Successfully reported this slideshow.

추천시스템 이제는 돈이 되어야 한다.

80

Share

Loading in …3
×
1 of 134
1 of 134

More Related Content

추천시스템 이제는 돈이 되어야 한다.

  1. 1. PyCon Korea 2019 , . | 1
  2. 2. 2 / +
  3. 3. ?
  4. 4. ? 5
  5. 5. ? 6
  6. 6. ? 7
  7. 7. ? 8
  8. 8. ?
  9. 9. ?
  10. 10. .
  11. 11. 14 ( ) ? ?
  12. 12. 15 10 20 20
  13. 13. 16
  14. 14. . 17
  15. 15. ?
  16. 16. . - -
  17. 17. . - -
  18. 18. ?
  19. 19. 22 ( ) ? ?
  20. 20. : 23 ( ) ? ? ( ) .
  21. 21. 24 10 20 20
  22. 22. : . 25 10 20 20 20
  23. 23. 26
  24. 24. : . 27
  25. 25. .
  26. 26. ?
  27. 27. . 30 / TP.
  28. 28. .
  29. 29. . !32
  30. 30. 33 …
  31. 31. 34 - - ViewerEnd
  32. 32. • : 
 CTR(%) 
 • • MAB(Multi Armed Bandit) • User Clustering - !35
  33. 33. MAB(Multi Armed Bandit) • MAB = Exploration( ) and Exploitation( ) Trade-off • 10%( ) Feedback (impression, click) * ε-greedy MAB . !36
  34. 34. • Feedback CTR(%) . • CTR(%) = # of clicks / # of impressions Exploration 0.4% 4.0% 2.9% 7.3% 2.7% 8.7% 6.7% 1.0% 1.9% 8.1% 6.3% 3.6% 6.7% 8.0% 3.1% 3.6% 2.0% 4.4% 3.1% 7.3% 8.2% 2.7% 4.4% 8.1% 0.6% 5.9% 9.2% 7.3% 8.3% 8.6% 4.2% 9.9% 6.9% * ε-greedy MAB . !37 MAB(Multi Armed Bandit)
  35. 35. • CTR 90% ( ) • CTR 0.4% 4.0% 2.9% 7.3% 2.7% 8.7% 6.7% 1.0% 1.9% 8.1% 6.3% 3.6% 6.7% 8.0% 3.1% 3.6% 2.0% 4.4% 3.1% 7.3% 8.2% 2.7% 4.4% 8.1% 0.6% 5.9% 9.2% 7.3% 8.3% 8.6% 4.2% 9.9% 6.9% Exploitation8.0% 8.2% * ε-greedy MAB . !38 MAB(Multi Armed Bandit)
  36. 36. 0.4% 4.0% 2.9% 7.3% 2.7% 8.7% 6.7% 1.0% 1.9% 8.1% 6.3% 3.6% 6.7% 8.0% 3.1% 3.6% 2.0% 4.4% 3.1% 7.3% 8.2% 2.7% 4.4% 8.1% 0.6% 5.9% 9.2% 7.3% 8.3% 8.6% 4.2% 9.9% 6.9% Exploitation (10%) (90%) & ? : ? : . !39 MAB(Multi Armed Bandit)
  37. 37. 0.4% 4.0% 2.9% 7.3% 2.7% 8.7% 6.7% 1.0% 1.9% 8.1% 6.3% 3.6% 6.7% 8.0% 3.1% 3.6% 2.0% 4.4% 3.1% 7.3% 8.2% 2.7% 4.4% 8.1% 0.6% 5.9% 9.2% 7.3% 8.3% 8.6% 4.2% 9.9% 6.9% Exploitation Exploration(10%) Exploitation(90%) • MAB ? • TS-MAB ε-greedy UCB(Upper Confidence Bound) Lin-UCB Thompson Sampling NeuralBandit LinRel (Linear Associative Reinforcement Learning)  !40 MAB(Multi Armed Bandit)
  38. 38. ε-Greedy MAB ε=0.10 41 10M Impressions 10%(ε) 1M Impressions( ) 1 2 3 4 4 5 6 7 8 9 … 100
  39. 39. ε-Greedy MAB ε=0.10 42 10M Impressions 10%(ε) 1M Impressions( ) 1.1% 2.0% 8.2% 0.01% 4.6% 1.2% 5.2% 0.1% 0.2% 1.0% … 2.2% (100 ) 10k Impression
  40. 40. ε-Greedy MAB ε=0.10 43 10M Impressions 10%(ε) 1M Impressions( ) 1.1% 2.0% 8.2% 0.01% 4.6% 1.2% 5.2% 0.1% 0.2% 1.0% … 2.2% (100 ) 10k Impression CTR = 1.5% Best arm ( ) 3 8.2% 7 5.2% 4 4.6% 50 3.0%
  41. 41. ε-Greedy MAB ε=0.10 44 10M Impressions 10%(ε) 1M Impressions( ) 1.1% 2.0% 8.2% 0.01% 4.6% 1.2% 5.2% 0.1% 0.2% 1.0% … 2.2% Best arm ( ) 3 8.2% 7 5.2% 4 4.6% 50 3.0% 90%(1-ε) 9M Impressions (100 ) 10k Impression CTR = 1.5% CTR = 5.1% CTR 4.74%
  42. 42. ε-Greedy MAB ε=0.10 45 10M Impressions 10%(ε) 1M Impressions( ) 1.1% 2.0% 8.2% 0.01% 4.6% 1.2% 5.2% 1.5% 0.2% 1.0% … 2.2% Best arm ( ) 3 8.2% 7 5.2% 4 4.6% 90%(1-ε) 9M Impressions (100 ) 10k Impression CTR = 1.5% CTR = 5.1% CTR 4.74% 10k Impression CTR Impressions CTR (3σ)
  43. 43. ε-Greedy MAB ε=0.10 46 10M Impressions 10%(ε) 1M Impressions( ) 1.1% 2.0% 8.2% 0.01% 4.6% 1.2% 5.2% 1.5% 0.2% 10 Impressions CTR Impressions CTR
  44. 44. ε-Greedy MAB ε=0.10 47 10M Impressions 10%(ε) 1M Impressions( ) 1.1% 2.0% 8.2% 0.01% 4.6% 1.2% 5.2% 1.5% 0.2% 1.0% … 2.2% Best arm ( ) 3 8.2% 7 5.2% 4 4.6% 90%(1-ε) 9M Impressions (100 ) 10k Impression CTR = 1.5% CTR = 5.1% CTR 4.74% CTR Impressions 99.7%(3σ)
  45. 45. ε-Greedy MAB ε=0.10 48 10M Impressions 10%(ε) 1M Impressions( ) 1.1% 2.0% 8.2% 0.01% 4.6% 1.2% 5.2% 1.5% 0.2% 1.0% … 2.2% Best arm ( ) 3 8.2% 7 5.2% 4 4.6% 50 3.0% 90%(1-ε) 9M Impressions (100 ) 10k Impression CTR = 1.5% CTR = 5.1% CTR 4.74% CTR Impressions 99.7%(3σ) CTR 3.0% 3.0% 3.0% 3.0% 3.0% 3.0%3.0%
  46. 46. ε-Greedy MAB ε=0.10 49 10M Impressions 10%(ε) 1.1% 2.0% 5.2% 1.5% 0.2% 1.0% … 2.2% Best arm ( ) 90%(1-ε) (100 ) 10k Impression CTR = 1.5% CTR = 5.1% CTR 4.74% CTR Impressions 99.7%(3σ) 3.0% 3.0%3.0% Optimal Arm Impressions (regret )
  47. 47. Thompson Sampling MAB ?
  48. 48. Thompson Sampling MAB • (arm) CTR Beta(a,b) . ( a=click, b=unclick ) 51 1 10% Impressions : 10 50 100 200 1k 10k 2 25% Impressions : 10 50 100 200 1k 10k
  49. 49. Thompson Sampling MAB • (arm) CTR Beta(a,b) ( a=click, b=unclick ) 52 1 10% ( ) CTR 15% 1 (10%<15%) 100 Impressions trial . Impression -> Impressions : 10 50 100 200 1k 10k
  50. 50. Thompson Sampling MAB • (arm) CTR Beta(a,b) ( a=click, b=unclick ) 53 2 25% 2 CTR 25%>15% ( ) . Impressions : 10 50 100 200 1k 10k
  51. 51. • 1K Impressions 54 
 CTR
  52. 52. • 10K Impressions 55 
 CTR
  53. 53. • 50K Impressions 56 
 CTR
  54. 54. • 100K Impressions 57 
 CTR
  55. 55. • 500K Impressions 58 
 CTR
  56. 56. • 1M Impressions 59 
 CTR
  57. 57. • 1M Impressions 60 
 CTR CTR (Arm)
  58. 58. • 1M Impressions 61 
 CTR CTR ( ) . .
  59. 59. • 1M Impressions 62 
 CTR CTR ( ) . . TS-MAB & Trade-off Regret( ) .
  60. 60. User Clustering • CTR . CTR : 7.6% A : 25% (30 ) B : 2.1% ( ) C : 7.1% ( ) CTR !63
  61. 61. User Clustering • X CTR • CTR 200 8,000
  62. 62. User Clustering • • 8 8 8,000 64,000 CTR
  63. 63. Clustering ? CB(image,Text) Feature User Feature [0.628, 0.88, 0.376, 0.065, 0.849] [0.508, 0.268, 0.193, 0.125, 0.425] [0.431, 0.077, 0.012, 0.07, 0.037] [0.915, 0.294, 0.713, 0.851, 0.423] [0.508, 0.268, 0.193, 0.125, 0.425] [0.607, 0.639, 0.554, 0.092, 0.297] [0.587, 0.319, 0.094, 0.173, 0.177] [0.409, 0.458, 0.48, 0.319, 0.783] [0.479, 0.434, 0.618, 0.297, 0.752] [0.467, 0.206, 0.905, 0.7, 0.568] , , , , , , , , !66 1 2 3 4 5 6
  64. 64. Clustering ? 14 CB(image,Text) Feature User Feature [0.628, 0.88, 0.376, 0.065, 0.849] [0.508, 0.268, 0.193, 0.125, 0.425] [0.431, 0.077, 0.012, 0.07, 0.037] [0.915, 0.294, 0.713, 0.851, 0.423] [0.508, 0.268, 0.193, 0.125, 0.425] [0.607, 0.639, 0.554, 0.092, 0.297] [0.587, 0.319, 0.094, 0.173, 0.177] [0.409, 0.458, 0.48, 0.319, 0.783] [0.479, 0.434, 0.618, 0.297, 0.752] [0.467, 0.206, 0.905, 0.7, 0.568] , , , , , , , , 8 (#0~#7) ?
  65. 65. - • • #1, #5, #6, #7 • #0, #2 #3
  66. 66. - • • #1, #5, #6, #7 • #0, #2 #3 /
  67. 67. #1 , ,
  68. 68. #3 , ,
  69. 69. / #3 , ,
  70. 70. Tag
  71. 71. Tag
  72. 72. / .
  73. 73. ? 77 #2 #1 #3 User Clustering Targeting CTR 1 : CTR 9.1% 2 : CTR 8.8% 3 : CTR 8.0% 4 : CTR 7.8% 5 : CTR 7.1% 6 : CTR 6.8% 7 : CTR 6.7% … MAB Ranking
  74. 74. ? 78 #2 #3 User Clustering Targeting Ranker MAB Ranking Targeter
  75. 75. + MAB
  76. 76. -ViewerEnd • : 
 CTR(%) 
 • • (Item Feature) • MAB(Multi Armed Bandit) !80
  77. 77. ? 81 #2 #3 User Clustering Targeting Ranker MAB Ranking Targeter
  78. 78. ? 82 (Item) Targeter Feature Targeting Ranker MAB Ranking
  79. 79. -ViewerEnd (CF) (Text) (Image) !83 1 2 3
  80. 80. - Item Features / Image (1) Image Feature , Text (2) Text Feature Feedback (3) CF-Feature !84
  81. 81. ? !85
  82. 82. ? !86
  83. 83. ? !87
  84. 84. 1 3 41 1 92 1 1 3 41 1 92 1 1 3 41 1 92 1 Image Text( ) CF( ) !88 1 3 41 1 92 1Image Style
  85. 85. Image Text( ) CF( ) Image Style Style transfer network !89 1 3 41 1 92 1
  86. 86. Image Style Text( ) CF( ) Image Object Detection Task Image Object Pre-trained VGG19 Model !90 1 3 41 1 92 1
  87. 87. Image Style Image CF( ) , Keyword Word2Vec !91 1 3 41 1 92 1
  88. 88. Image Style Image Text( ) CF( ) Matrix Factorization(ALS) with implicit feedback (Feedback) Item-User !92 1 3 41 1 92 1
  89. 89. !93 1 3 41 1 92 1 1 3 41 1 92 1 1 3 41 1 92 1 1 3 41 1 92 1
  90. 90. “ ” (%) !94
  91. 91. 0.4% 4.0% 2.9% 7.3% 2.7% 8.7% 6.7% 1.0% 1.9% 8.1% 0.4% 4.0% 2.9% 7.3% 2.7% 8.7% 6.7% 1.0% 1.9% 8.1% 0.4% 4.4% 2.9% 7.3% 2.3% 8.7% 0.2% 1.0% 1.9% 8.1% 0.4% 6.0% 2.9% 7.3% 2.7% 5.6% 6.7% 1.0% 1.9% 8.1% MAB (%) !95
  92. 92. 0.4% 4.0% 2.9% 7.3% 2.7% 8.7% 6.7% 1.0% 1.9% 8.1% 0.4% 4.0% 2.9% 7.3% 2.7% 8.7% 6.7% 1.0% 1.9% 8.1% 0.4% 4.4% 2.9% 7.3% 2.3% 8.7% 0.2% 1.0% 1.9% 8.1% 0.4% 6.0% 2.9% 7.3% 2.7% 5.6% 6.7% 1.0% 1.9% 8.1% !96
  93. 93. ? 97 (Item) Targeting CTR 1 : CTR 9.1% 2 : CTR 8.8% 3 : CTR 8.0% 4 : CTR 7.8% 5 : CTR 7.1% 6 : CTR 6.8% 7 : CTR 6.7% … MAB Ranking
  94. 94. + MAB !98
  95. 95. !99
  96. 96. 100 1 2 3 4 5 6 … 89 90 (Clicks) Impression
  97. 97. 101 1 2 3 4 5 6 … 89 90 (Use Coin) Impression
  98. 98. 102 User Cluster + + MAB MAB RankerTargeter = =
  99. 99. 103 + MAB Conditional Bandit Exponential Smoothing Seen Decay Soft User Clustering Retention Model Unbiased Most Popular Feature Matching Targeter = = Ranker
  100. 100. 104 • MAB
 - Bandit Algorithm = Thompson Sampling(
 - Reward = Click (with Unclick )
 - Play Arms = Cluster Most Popular 
 - None Stationary = Exponential Decaying • 2 
 - = # of clicks / # of impressions 
 - = # of use_coins / # of impressions 1. MAB
  101. 101. 105 • MAB
 - Bandit Algorithm = Thompson Sampling
 - Reward = Click (with Unclick )
 - Play Arms = Cluster Most Popular 
 - None Stationary = Exponential Decaying • 2 
 - = # of clicks / # of impressions 
 - = # of use_coins / # of impressions 1. MAB Use Coin( ) MAB Reward Use Coin, Click + User Coin by @brandon.lim
  102. 102. 106 1. MAB ? (Beta) (Alpha) -20% —> ? ? MAB ? -20%
  103. 103. 2. Conditional Bandit 107 1 2 3 4 5 6 … 89 90 by @troye.kwon
  104. 104. 2. Conditional Bandit 108 1 2 3 4 5 6 … 89 90 Impressions Reward=Click( ) α=click, β=unclick MAB by @troye.kwon
  105. 105. 2. Conditional Bandit 109 1 2 3 4 5 6 … 89 Impressions Reward=Click( ) α=click, β=unclick MAB Reward=Use Coin( ) α=use-coin, β=click MAB by @troye.kwon
  106. 106. 2. Conditional Bandit ? 110 (Beta) (alpha) ? ? - MAB .
  107. 107. 3. Retention Model • : . 
 , 
 “ ” . • • MAB 111 by @jinny.k + MAB Targeter Ranker
  108. 108. 3. Retention Model ? 112 by @jinny.k (CTR) (CVR) ? (CTR) ?
  109. 109. 4. Seen decay • : Negative Feedback • click impression Ranker • : (alpha) (Beta) 113
  110. 110. -> CTR 114 • • Hard Clustering(k-Means) —> Soft Clustering (pLSI) • Feature Matching • Targetting Genre/Tag Matching • MAB non-stationary Exponential Smoothing • Targeting Unbiased Most Popular • MAB Hyper parameter Turning
  111. 111. (%) !115 Soft Clustering (pLSI) Feature Matching Conditional Bandit Retention Model Exponential Smoothing MABUnbiased Most Popular
  112. 112. ? ? !116
  113. 113. !117
  114. 114. ? 118
  115. 115. ? 119
  116. 116. ? 120
  117. 117. ? 121
  118. 118. ? 122
  119. 119. 0.1% 123 = 3.96% = 6.70% = 2.63% = 0.07%
  120. 120. -> 4.56 124 = 3.96% = 6.70% = 2.63% = 0.07% 4.56
  121. 121. -> 4.56 125 = 3.96% = 6.70% = 2.63% = 0.07% 4.56 AB (<0.001) Feedback (>4.56days)
  122. 122. / , ?
  123. 123. . 127 Base : Editor’s ( X) 1.9% Alpha : 1 4.8% Beta : 2 5.5% Gamma : 3 6.5% CTR 1 2 3 4 …
  124. 124. . 128 Base : Editor’s ( X) 1.9% Alpha : 1 4.8% Beta : 2 5.5% Gamma : 3 6.5% CTR + 242% + 42% 1 2 3 4 … CTR
  125. 125. . ?
  126. 126. / ? ?
  127. 127. / ? -> ? ->
  128. 128. 133 YOU ?
  129. 129. . !134 |

×