0
Realtime Recommender
Hands On
Torben Brodt
plista GmbH
June 3rd, 2013
Berlin Buzzwords
http://berlinbuzzwords.de/
Contents
1. How to feed a recommender?
2. How to pass data quickly?
3. How to build a recommender?
How to feed a recommender?
How to feed a recommender?
●
○ recommendations & advertising network
○ ~5k recs/ second
● integrated on publishers
○ welt....
How to feed a recommender?
● to show recommendations we integrated javascript
● we have URL + HTTP Headers
○ user agent
○ ...
How to pass data quickly?
How to pass data quickly?
NOSQL!
WHAT
ELSE?
How to pass data quickly?
Thrift
Serialization
How to pass data quickly?
● Data Types: String, Lists, Set, ..
● Hash
○ map between string fields and string values, very
fast
○ HINCR complexity O(...
How to build a recommendation?
How to build a recommendation?
Behavioral
based on interaction
between user and
article
○ Most Popular
○ Collaborative Fil...
How to build a recommendation?
welt.de/football/berlin_wins.html
● ZINCR "p:welt.de" berlin_wins
● ZREVRANGEBYSCORE
p:welt...
Most popular with timeseries
welt.de/football/berlin_wins.html
● ZINCR "p:welt.de:1360007000" berlin_wins
● ZUNION
○ "p:we...
Most popular with timeseries
welt.de/football/berlin_wins.html
● ZINCR "p:welt.de:1360007000" berlin_wins
● ZUNION ... WEI...
Most popular with timeseries
:1360007000
-1h -2h -3h -4h -5h -6h -7h -8h
:1360007000
:1360007000
4
2
1
Most popular to any context
● it's not only publisher, we use ~50 context
attributes
context attributes:
● publisher
● wee...
Most popular to any context
ZUNION ... WEIGHTS
p:welt.de:1360007 4
p:welt.de:1360006 2
p:welt.de:1360005 1
w:sunday:136000...
Even more Matrix Operations ;)
● Similarity Matrix
● Human Control Matrix
● Meta-learning Matrix
○ cooperation with
○ aide...
More recommenders possible
this was only about most popular
● other algorithms using redis
○ incremental collaborative fil...
What else in Redis?
● message bus
● many recommenders
● live statistics
● caching
"One technology to rule them all"
Questions?
www.plista.com
torben.brodt@plista.com
@torbenbrodt
xing.com/profile/Torben_Brodt
http://goo.gl/pvXm5
http://ln...
Realtime Recommender with Redis: Hands on
Upcoming SlideShare
Loading in...5
×

Realtime Recommender with Redis: Hands on

1,298

Published on

Published in: Education, Travel, Technology

Transcript of "Realtime Recommender with Redis: Hands on"

  1. 1. Realtime Recommender Hands On Torben Brodt plista GmbH June 3rd, 2013 Berlin Buzzwords http://berlinbuzzwords.de/
  2. 2. Contents 1. How to feed a recommender? 2. How to pass data quickly? 3. How to build a recommender?
  3. 3. How to feed a recommender?
  4. 4. How to feed a recommender? ● ○ recommendations & advertising network ○ ~5k recs/ second ● integrated on publishers ○ welt.de, abendblatt, etc
  5. 5. How to feed a recommender? ● to show recommendations we integrated javascript ● we have URL + HTTP Headers ○ user agent ○ IP address -> geolocation src http://en.wikipedia.org/wiki/Pac-Man 5
  6. 6. How to pass data quickly?
  7. 7. How to pass data quickly? NOSQL! WHAT ELSE?
  8. 8. How to pass data quickly? Thrift Serialization
  9. 9. How to pass data quickly?
  10. 10. ● Data Types: String, Lists, Set, .. ● Hash ○ map between string fields and string values, very fast ○ HINCR complexity O(1) ● Sorted Set ○ ZINCR complexity: O(log(N)) where N is the number of elements in the sorted set. ○ Allows to limit number of result: ZREVRANGEBYSCORE ○ UNION + INTERSECT How to pass data quickly? p:welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 10
  11. 11. How to build a recommendation?
  12. 12. How to build a recommendation? Behavioral based on interaction between user and article ○ Most Popular ○ Collaborative Filtering ○ Item to Item Content based on the articles ○ Content Similarity ○ Latest Item Classification ● different recommender families
  13. 13. How to build a recommendation? welt.de/football/berlin_wins.html ● ZINCR "p:welt.de" berlin_wins ● ZREVRANGEBYSCORE p:welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 Live Read + Live Write = Real Time Recommendations
  14. 14. Most popular with timeseries welt.de/football/berlin_wins.html ● ZINCR "p:welt.de:1360007000" berlin_wins ● ZUNION ○ "p:welt.de:1360007000" ○ "p:welt.de:1360006000" ○ "p:welt.de:1360005000" ● ZREVRANGEBYSCORE p:welt.de:1360005000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360006000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360007000 berlin_wins 689 summer_is_coming 420 plista_best_company 135
  15. 15. Most popular with timeseries welt.de/football/berlin_wins.html ● ZINCR "p:welt.de:1360007000" berlin_wins ● ZUNION ... WEIGHTS ○ "p:welt.de:1360007000" .. 4 ○ "p:welt.de:1360006000" .. 2 ○ "p:welt.de:1360005000" .. 1 ● ZREVRANGEBYSCORE p:welt.de:1360005000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360006000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360007000 berlin_wins 689 summer_is_coming 420 plista_best_company 135 15
  16. 16. Most popular with timeseries :1360007000 -1h -2h -3h -4h -5h -6h -7h -8h :1360007000 :1360007000 4 2 1
  17. 17. Most popular to any context ● it's not only publisher, we use ~50 context attributes context attributes: ● publisher ● weekday ● geolocation ● demographics ● ... publisher = welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 weekday = sunday berlin_wins 400 +1 dortmund_wins 200 ... 100 geolocation = dortmund dortmund_wins 200 berlin_wins 10 +1 ... 5
  18. 18. Most popular to any context ZUNION ... WEIGHTS p:welt.de:1360007 4 p:welt.de:1360006 2 p:welt.de:1360005 1 w:sunday:1360007 4 w:sunday:1360006 2 w:sunday:1360005 1 g:dortmund:1360007 4 g:dortmund:1360006 2 g:dortmund:1360005 1 ● how it looks like in Redis publisher = welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 weekday = sunday berlin_wins 400 dortmund_wins 200 ... 100 geolocation = dortmund dortmund_wins 200 berlin_wins 10 ... 5
  19. 19. Even more Matrix Operations ;) ● Similarity Matrix ● Human Control Matrix ● Meta-learning Matrix ○ cooperation with ○ aided from ∏ Σ
  20. 20. More recommenders possible this was only about most popular ● other algorithms using redis ○ incremental collaborative filtering ○ article to article paths (~graph) ○ .. using external data sources srchttp://en.wikipedia.org/wiki/Flash_(comics) 20
  21. 21. What else in Redis? ● message bus ● many recommenders ● live statistics ● caching "One technology to rule them all"
  22. 22. Questions? www.plista.com torben.brodt@plista.com @torbenbrodt xing.com/profile/Torben_Brodt http://goo.gl/pvXm5 http://lnkd.in/MUXXuv
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×