Realtime Recommender with Redis: Hands on

1,705 views
1,535 views

Published on

Published in: Education, Travel, Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,705
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
27
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Realtime Recommender with Redis: Hands on

  1. 1. Realtime Recommender Hands On Torben Brodt plista GmbH June 3rd, 2013 Berlin Buzzwords http://berlinbuzzwords.de/
  2. 2. Contents 1. How to feed a recommender? 2. How to pass data quickly? 3. How to build a recommender?
  3. 3. How to feed a recommender?
  4. 4. How to feed a recommender? ● ○ recommendations & advertising network ○ ~5k recs/ second ● integrated on publishers ○ welt.de, abendblatt, etc
  5. 5. How to feed a recommender? ● to show recommendations we integrated javascript ● we have URL + HTTP Headers ○ user agent ○ IP address -> geolocation src http://en.wikipedia.org/wiki/Pac-Man 5
  6. 6. How to pass data quickly?
  7. 7. How to pass data quickly? NOSQL! WHAT ELSE?
  8. 8. How to pass data quickly? Thrift Serialization
  9. 9. How to pass data quickly?
  10. 10. ● Data Types: String, Lists, Set, .. ● Hash ○ map between string fields and string values, very fast ○ HINCR complexity O(1) ● Sorted Set ○ ZINCR complexity: O(log(N)) where N is the number of elements in the sorted set. ○ Allows to limit number of result: ZREVRANGEBYSCORE ○ UNION + INTERSECT How to pass data quickly? p:welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 10
  11. 11. How to build a recommendation?
  12. 12. How to build a recommendation? Behavioral based on interaction between user and article ○ Most Popular ○ Collaborative Filtering ○ Item to Item Content based on the articles ○ Content Similarity ○ Latest Item Classification ● different recommender families
  13. 13. How to build a recommendation? welt.de/football/berlin_wins.html ● ZINCR "p:welt.de" berlin_wins ● ZREVRANGEBYSCORE p:welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 Live Read + Live Write = Real Time Recommendations
  14. 14. Most popular with timeseries welt.de/football/berlin_wins.html ● ZINCR "p:welt.de:1360007000" berlin_wins ● ZUNION ○ "p:welt.de:1360007000" ○ "p:welt.de:1360006000" ○ "p:welt.de:1360005000" ● ZREVRANGEBYSCORE p:welt.de:1360005000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360006000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360007000 berlin_wins 689 summer_is_coming 420 plista_best_company 135
  15. 15. Most popular with timeseries welt.de/football/berlin_wins.html ● ZINCR "p:welt.de:1360007000" berlin_wins ● ZUNION ... WEIGHTS ○ "p:welt.de:1360007000" .. 4 ○ "p:welt.de:1360006000" .. 2 ○ "p:welt.de:1360005000" .. 1 ● ZREVRANGEBYSCORE p:welt.de:1360005000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360006000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360007000 berlin_wins 689 summer_is_coming 420 plista_best_company 135 15
  16. 16. Most popular with timeseries :1360007000 -1h -2h -3h -4h -5h -6h -7h -8h :1360007000 :1360007000 4 2 1
  17. 17. Most popular to any context ● it's not only publisher, we use ~50 context attributes context attributes: ● publisher ● weekday ● geolocation ● demographics ● ... publisher = welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 weekday = sunday berlin_wins 400 +1 dortmund_wins 200 ... 100 geolocation = dortmund dortmund_wins 200 berlin_wins 10 +1 ... 5
  18. 18. Most popular to any context ZUNION ... WEIGHTS p:welt.de:1360007 4 p:welt.de:1360006 2 p:welt.de:1360005 1 w:sunday:1360007 4 w:sunday:1360006 2 w:sunday:1360005 1 g:dortmund:1360007 4 g:dortmund:1360006 2 g:dortmund:1360005 1 ● how it looks like in Redis publisher = welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 weekday = sunday berlin_wins 400 dortmund_wins 200 ... 100 geolocation = dortmund dortmund_wins 200 berlin_wins 10 ... 5
  19. 19. Even more Matrix Operations ;) ● Similarity Matrix ● Human Control Matrix ● Meta-learning Matrix ○ cooperation with ○ aided from ∏ Σ
  20. 20. More recommenders possible this was only about most popular ● other algorithms using redis ○ incremental collaborative filtering ○ article to article paths (~graph) ○ .. using external data sources srchttp://en.wikipedia.org/wiki/Flash_(comics) 20
  21. 21. What else in Redis? ● message bus ● many recommenders ● live statistics ● caching "One technology to rule them all"
  22. 22. Questions? www.plista.com torben.brodt@plista.com @torbenbrodt xing.com/profile/Torben_Brodt http://goo.gl/pvXm5 http://lnkd.in/MUXXuv

×