SlideShare a Scribd company logo
1 of 23
Download to read offline
Realtime Recommender
Hands On
Torben Brodt
plista GmbH
June 3rd, 2013
Berlin Buzzwords
http://berlinbuzzwords.de/
Contents
1. How to feed a recommender?
2. How to pass data quickly?
3. How to build a recommender?
How to feed a recommender?
How to feed a recommender?
●
○ recommendations & advertising network
○ ~5k recs/ second
● integrated on publishers
○ welt.de, abendblatt, etc
How to feed a recommender?
● to show recommendations we integrated javascript
● we have URL + HTTP Headers
○ user agent
○ IP address -> geolocation
src http://en.wikipedia.org/wiki/Pac-Man
5
How to pass data quickly?
How to pass data quickly?
NOSQL!
WHAT
ELSE?
How to pass data quickly?
Thrift
Serialization
How to pass data quickly?
● Data Types: String, Lists, Set, ..
● Hash
○ map between string fields and string values, very
fast
○ HINCR complexity O(1)
● Sorted Set
○ ZINCR complexity: O(log(N)) where N is the number
of elements in the sorted set.
○ Allows to limit number of result:
ZREVRANGEBYSCORE
○ UNION + INTERSECT
How to pass data quickly?
p:welt.de
berlin_wins 689 +1
summer_is_coming 420
plista_company 135
10
How to build a recommendation?
How to build a recommendation?
Behavioral
based on interaction
between user and
article
○ Most Popular
○ Collaborative Filtering
○ Item to Item
Content
based on the articles
○ Content Similarity
○ Latest Item
Classification
● different recommender families
How to build a recommendation?
welt.de/football/berlin_wins.html
● ZINCR "p:welt.de" berlin_wins
● ZREVRANGEBYSCORE
p:welt.de
berlin_wins 689 +1
summer_is_coming 420
plista_company 135
Live Read
+ Live Write
= Real Time Recommendations
Most popular with timeseries
welt.de/football/berlin_wins.html
● ZINCR "p:welt.de:1360007000" berlin_wins
● ZUNION
○ "p:welt.de:1360007000"
○ "p:welt.de:1360006000"
○ "p:welt.de:1360005000"
● ZREVRANGEBYSCORE
p:welt.de:1360005000
berlin_wins 420
summer_is_coming 135
plista_best_company 689
p:welt.de:1360006000
berlin_wins 420
summer_is_coming 135
plista_best_company 689
p:welt.de:1360007000
berlin_wins 689
summer_is_coming 420
plista_best_company 135
Most popular with timeseries
welt.de/football/berlin_wins.html
● ZINCR "p:welt.de:1360007000" berlin_wins
● ZUNION ... WEIGHTS
○ "p:welt.de:1360007000" .. 4
○ "p:welt.de:1360006000" .. 2
○ "p:welt.de:1360005000" .. 1
● ZREVRANGEBYSCORE
p:welt.de:1360005000
berlin_wins 420
summer_is_coming 135
plista_best_company 689
p:welt.de:1360006000
berlin_wins 420
summer_is_coming 135
plista_best_company 689
p:welt.de:1360007000
berlin_wins 689
summer_is_coming 420
plista_best_company 135
15
Most popular with timeseries
:1360007000
-1h -2h -3h -4h -5h -6h -7h -8h
:1360007000
:1360007000
4
2
1
Most popular to any context
● it's not only publisher, we use ~50 context
attributes
context attributes:
● publisher
● weekday
● geolocation
● demographics
● ...
publisher = welt.de
berlin_wins 689 +1
summer_is_coming 420
plista_company 135
weekday = sunday
berlin_wins 400 +1
dortmund_wins 200
... 100
geolocation = dortmund
dortmund_wins 200
berlin_wins 10 +1
... 5
Most popular to any context
ZUNION ... WEIGHTS
p:welt.de:1360007 4
p:welt.de:1360006 2
p:welt.de:1360005 1
w:sunday:1360007 4
w:sunday:1360006 2
w:sunday:1360005 1
g:dortmund:1360007 4
g:dortmund:1360006 2
g:dortmund:1360005 1
● how it looks like in Redis
publisher = welt.de
berlin_wins 689 +1
summer_is_coming 420
plista_company 135
weekday = sunday
berlin_wins 400
dortmund_wins 200
... 100
geolocation = dortmund
dortmund_wins 200
berlin_wins 10
... 5
Even more Matrix Operations ;)
● Similarity Matrix
● Human Control Matrix
● Meta-learning Matrix
○ cooperation with
○ aided from
∏
Σ
More recommenders possible
this was only about most popular
● other algorithms using redis
○ incremental collaborative filtering
○ article to article paths (~graph)
○ .. using external data sources
srchttp://en.wikipedia.org/wiki/Flash_(comics)
20
What else in Redis?
● message bus
● many recommenders
● live statistics
● caching
"One technology to rule them all"
Questions?
www.plista.com
torben.brodt@plista.com
@torbenbrodt
xing.com/profile/Torben_Brodt
http://goo.gl/pvXm5
http://lnkd.in/MUXXuv

More Related Content

Viewers also liked

Ahg microsoft stream_insight_queries
Ahg microsoft stream_insight_queriesAhg microsoft stream_insight_queries
Ahg microsoft stream_insight_queriesSteve Xu
 
Chicago AWS user group meetup - May 2014 at Cohesive
Chicago AWS user group meetup - May 2014 at CohesiveChicago AWS user group meetup - May 2014 at Cohesive
Chicago AWS user group meetup - May 2014 at CohesiveCloudCamp Chicago
 
AppSensor Near Real-Time Event Detection and Response - DevNexus 2016
AppSensor Near Real-Time Event Detection and Response - DevNexus 2016AppSensor Near Real-Time Event Detection and Response - DevNexus 2016
AppSensor Near Real-Time Event Detection and Response - DevNexus 2016jtmelton
 
Application Deployment at UC Riverside
Application Deployment at UC RiversideApplication Deployment at UC Riverside
Application Deployment at UC RiversideMichael Kennedy
 
concepto de colección local
concepto de colección localconcepto de colección local
concepto de colección localguestf488db7
 
EMC World 2015 - The Devops Toolkit
EMC World 2015 - The Devops ToolkitEMC World 2015 - The Devops Toolkit
EMC World 2015 - The Devops ToolkitJonas Rosland
 
Tech Chat – What's New in Sumo Logic
Tech Chat – What's New in Sumo LogicTech Chat – What's New in Sumo Logic
Tech Chat – What's New in Sumo LogicSumo Logic
 
AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)
AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)
AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)Amazon Web Services
 
Five pillars of Infrastructure Monitoring
Five pillars of Infrastructure MonitoringFive pillars of Infrastructure Monitoring
Five pillars of Infrastructure MonitoringDaniel Koller
 
Marian Marinov, 1H Ltd.
Marian Marinov, 1H Ltd.Marian Marinov, 1H Ltd.
Marian Marinov, 1H Ltd.Ontico
 
George Park Workshop 1 - Cosumnes CSD
George Park Workshop 1 - Cosumnes CSDGeorge Park Workshop 1 - Cosumnes CSD
George Park Workshop 1 - Cosumnes CSDCosumnes CSD
 
Hadoop / Spark on Malware Expression
Hadoop / Spark on Malware ExpressionHadoop / Spark on Malware Expression
Hadoop / Spark on Malware ExpressionMapR Technologies
 
Application Development on Metapod
Application Development on MetapodApplication Development on Metapod
Application Development on MetapodCisco DevNet
 
How to Keep Students Motivated During Winter
How to Keep Students Motivated During WinterHow to Keep Students Motivated During Winter
How to Keep Students Motivated During WinterRobert Peters, Ed.D
 

Viewers also liked (19)

OS17 Brochure
OS17 BrochureOS17 Brochure
OS17 Brochure
 
Composite çelik
Composite çelikComposite çelik
Composite çelik
 
Ahg microsoft stream_insight_queries
Ahg microsoft stream_insight_queriesAhg microsoft stream_insight_queries
Ahg microsoft stream_insight_queries
 
Introduction to ICS/SCADA security
Introduction to ICS/SCADA securityIntroduction to ICS/SCADA security
Introduction to ICS/SCADA security
 
Coniferous Forest
Coniferous ForestConiferous Forest
Coniferous Forest
 
Chicago AWS user group meetup - May 2014 at Cohesive
Chicago AWS user group meetup - May 2014 at CohesiveChicago AWS user group meetup - May 2014 at Cohesive
Chicago AWS user group meetup - May 2014 at Cohesive
 
AppSensor Near Real-Time Event Detection and Response - DevNexus 2016
AppSensor Near Real-Time Event Detection and Response - DevNexus 2016AppSensor Near Real-Time Event Detection and Response - DevNexus 2016
AppSensor Near Real-Time Event Detection and Response - DevNexus 2016
 
Application Deployment at UC Riverside
Application Deployment at UC RiversideApplication Deployment at UC Riverside
Application Deployment at UC Riverside
 
concepto de colección local
concepto de colección localconcepto de colección local
concepto de colección local
 
EMC World 2015 - The Devops Toolkit
EMC World 2015 - The Devops ToolkitEMC World 2015 - The Devops Toolkit
EMC World 2015 - The Devops Toolkit
 
Tech Chat – What's New in Sumo Logic
Tech Chat – What's New in Sumo LogicTech Chat – What's New in Sumo Logic
Tech Chat – What's New in Sumo Logic
 
AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)
AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)
AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)
 
Five pillars of Infrastructure Monitoring
Five pillars of Infrastructure MonitoringFive pillars of Infrastructure Monitoring
Five pillars of Infrastructure Monitoring
 
Marian Marinov, 1H Ltd.
Marian Marinov, 1H Ltd.Marian Marinov, 1H Ltd.
Marian Marinov, 1H Ltd.
 
George Park Workshop 1 - Cosumnes CSD
George Park Workshop 1 - Cosumnes CSDGeorge Park Workshop 1 - Cosumnes CSD
George Park Workshop 1 - Cosumnes CSD
 
Book of Fauna and Flora
Book of Fauna and FloraBook of Fauna and Flora
Book of Fauna and Flora
 
Hadoop / Spark on Malware Expression
Hadoop / Spark on Malware ExpressionHadoop / Spark on Malware Expression
Hadoop / Spark on Malware Expression
 
Application Development on Metapod
Application Development on MetapodApplication Development on Metapod
Application Development on Metapod
 
How to Keep Students Motivated During Winter
How to Keep Students Motivated During WinterHow to Keep Students Motivated During Winter
How to Keep Students Motivated During Winter
 

Similar to Realtime Recommender with Redis: Hands on

Living Labs Challenge Workshop
Living Labs Challenge WorkshopLiving Labs Challenge Workshop
Living Labs Challenge WorkshopTorben Brodt
 
Open recommendation platform
Open recommendation platformOpen recommendation platform
Open recommendation platformTorben Brodt
 
SIGIR 2013 BARS Keynote - the search for the best live recommender system
SIGIR 2013 BARS Keynote - the search for the best live recommender systemSIGIR 2013 BARS Keynote - the search for the best live recommender system
SIGIR 2013 BARS Keynote - the search for the best live recommender systemTorben Brodt
 
Content recommendations
Content recommendationsContent recommendations
Content recommendationsTorben Brodt
 
Website metadata - getting competitive intelligence on websites
Website metadata - getting competitive intelligence on websitesWebsite metadata - getting competitive intelligence on websites
Website metadata - getting competitive intelligence on websitesnetcomber
 
Crawling and Processing the Italian Corporate Web
Crawling and Processing the Italian Corporate WebCrawling and Processing the Italian Corporate Web
Crawling and Processing the Italian Corporate WebSpeck&Tech
 
Open Source BI Overview
Open Source BI Overview Open Source BI Overview
Open Source BI Overview Alex Meadows
 
[MongoDB.local Bengaluru 2018] Keynote
[MongoDB.local Bengaluru 2018] Keynote[MongoDB.local Bengaluru 2018] Keynote
[MongoDB.local Bengaluru 2018] KeynoteMongoDB
 
MongoDB Schema Design
MongoDB Schema DesignMongoDB Schema Design
MongoDB Schema DesignJoe Drumgoole
 
What's inside 09 2013: gstreamer: black magic and witchcraft
What's inside 09 2013: gstreamer: black magic and witchcraftWhat's inside 09 2013: gstreamer: black magic and witchcraft
What's inside 09 2013: gstreamer: black magic and witchcraftMarc Leeman
 
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI AI Frontiers
 
Logging and ranting / Vytis Valentinavičius (Lamoda)
Logging and ranting / Vytis Valentinavičius (Lamoda)Logging and ranting / Vytis Valentinavičius (Lamoda)
Logging and ranting / Vytis Valentinavičius (Lamoda)Ontico
 
Mapping french open data actors on the web with common crawl
Mapping french open data actors on the web with common crawlMapping french open data actors on the web with common crawl
Mapping french open data actors on the web with common crawldata publica
 
"Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ...
"Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ..."Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ...
"Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ...Dataconomy Media
 
Juraj vysvader - Python developer's CV
Juraj vysvader - Python developer's CVJuraj vysvader - Python developer's CV
Juraj vysvader - Python developer's CVJuraj Vysvader
 
Search at SoundCloud
Search at SoundCloudSearch at SoundCloud
Search at SoundCloudTomasz Elendt
 
Data Day Seattle 2017: Scaling Data Science at Stitch Fix
Data Day Seattle 2017: Scaling Data Science at Stitch FixData Day Seattle 2017: Scaling Data Science at Stitch Fix
Data Day Seattle 2017: Scaling Data Science at Stitch FixStefan Krawczyk
 
Building a Cross Channel Content Delivery Platform with MongoDB
Building a Cross Channel Content Delivery Platform with MongoDBBuilding a Cross Channel Content Delivery Platform with MongoDB
Building a Cross Channel Content Delivery Platform with MongoDBMongoDB
 

Similar to Realtime Recommender with Redis: Hands on (20)

Living Labs Challenge Workshop
Living Labs Challenge WorkshopLiving Labs Challenge Workshop
Living Labs Challenge Workshop
 
Open recommendation platform
Open recommendation platformOpen recommendation platform
Open recommendation platform
 
SIGIR 2013 BARS Keynote - the search for the best live recommender system
SIGIR 2013 BARS Keynote - the search for the best live recommender systemSIGIR 2013 BARS Keynote - the search for the best live recommender system
SIGIR 2013 BARS Keynote - the search for the best live recommender system
 
Content recommendations
Content recommendationsContent recommendations
Content recommendations
 
Website metadata - getting competitive intelligence on websites
Website metadata - getting competitive intelligence on websitesWebsite metadata - getting competitive intelligence on websites
Website metadata - getting competitive intelligence on websites
 
Crawling and Processing the Italian Corporate Web
Crawling and Processing the Italian Corporate WebCrawling and Processing the Italian Corporate Web
Crawling and Processing the Italian Corporate Web
 
Open Source BI Overview
Open Source BI Overview Open Source BI Overview
Open Source BI Overview
 
[MongoDB.local Bengaluru 2018] Keynote
[MongoDB.local Bengaluru 2018] Keynote[MongoDB.local Bengaluru 2018] Keynote
[MongoDB.local Bengaluru 2018] Keynote
 
MongoDB Schema Design
MongoDB Schema DesignMongoDB Schema Design
MongoDB Schema Design
 
What's inside 09 2013: gstreamer: black magic and witchcraft
What's inside 09 2013: gstreamer: black magic and witchcraftWhat's inside 09 2013: gstreamer: black magic and witchcraft
What's inside 09 2013: gstreamer: black magic and witchcraft
 
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
 
Logging and ranting / Vytis Valentinavičius (Lamoda)
Logging and ranting / Vytis Valentinavičius (Lamoda)Logging and ranting / Vytis Valentinavičius (Lamoda)
Logging and ranting / Vytis Valentinavičius (Lamoda)
 
Mapping french open data actors on the web with common crawl
Mapping french open data actors on the web with common crawlMapping french open data actors on the web with common crawl
Mapping french open data actors on the web with common crawl
 
"Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ...
"Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ..."Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ...
"Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ...
 
Juraj vysvader - Python developer's CV
Juraj vysvader - Python developer's CVJuraj vysvader - Python developer's CV
Juraj vysvader - Python developer's CV
 
Search at SoundCloud
Search at SoundCloudSearch at SoundCloud
Search at SoundCloud
 
Data Day Seattle 2017: Scaling Data Science at Stitch Fix
Data Day Seattle 2017: Scaling Data Science at Stitch FixData Day Seattle 2017: Scaling Data Science at Stitch Fix
Data Day Seattle 2017: Scaling Data Science at Stitch Fix
 
Building a Cross Channel Content Delivery Platform with MongoDB
Building a Cross Channel Content Delivery Platform with MongoDBBuilding a Cross Channel Content Delivery Platform with MongoDB
Building a Cross Channel Content Delivery Platform with MongoDB
 
Workflow Engines + Luigi
Workflow Engines + LuigiWorkflow Engines + Luigi
Workflow Engines + Luigi
 
Horneando apis
Horneando apisHorneando apis
Horneando apis
 

More from Torben Brodt

Recommender Trends 2014
Recommender Trends 2014Recommender Trends 2014
Recommender Trends 2014Torben Brodt
 
Paper the plista dataset
Paper  the plista datasetPaper  the plista dataset
Paper the plista datasetTorben Brodt
 
#TOA13 - Tech Opoen Air Recommender Hackathon
#TOA13 - Tech Opoen Air Recommender Hackathon#TOA13 - Tech Opoen Air Recommender Hackathon
#TOA13 - Tech Opoen Air Recommender HackathonTorben Brodt
 
Algorithmus, Good School, Camp Digital
Algorithmus, Good School, Camp DigitalAlgorithmus, Good School, Camp Digital
Algorithmus, Good School, Camp DigitalTorben Brodt
 
Recommender Hackathon @plista 2013/04
Recommender Hackathon @plista 2013/04Recommender Hackathon @plista 2013/04
Recommender Hackathon @plista 2013/04Torben Brodt
 
RecSys2012 inside the plista contest
RecSys2012   inside the plista contestRecSys2012   inside the plista contest
RecSys2012 inside the plista contestTorben Brodt
 
Webhacks am Beispiel PHP + MySQL
Webhacks am Beispiel PHP + MySQLWebhacks am Beispiel PHP + MySQL
Webhacks am Beispiel PHP + MySQLTorben Brodt
 
Collaborative Filtering.. für automatische Empfehlungen
Collaborative Filtering.. für automatische EmpfehlungenCollaborative Filtering.. für automatische Empfehlungen
Collaborative Filtering.. für automatische EmpfehlungenTorben Brodt
 
Google Web Toolkit
Google Web ToolkitGoogle Web Toolkit
Google Web ToolkitTorben Brodt
 
Geld Verdienen Mit Adsense
Geld Verdienen Mit AdsenseGeld Verdienen Mit Adsense
Geld Verdienen Mit AdsenseTorben Brodt
 
Web 2.0 - "Fluch oder Segen"
Web 2.0 - "Fluch oder Segen"Web 2.0 - "Fluch oder Segen"
Web 2.0 - "Fluch oder Segen"Torben Brodt
 

More from Torben Brodt (14)

Recommender Trends 2014
Recommender Trends 2014Recommender Trends 2014
Recommender Trends 2014
 
Paper the plista dataset
Paper  the plista datasetPaper  the plista dataset
Paper the plista dataset
 
Nrs2013 recap
Nrs2013 recapNrs2013 recap
Nrs2013 recap
 
#TOA13 - Tech Opoen Air Recommender Hackathon
#TOA13 - Tech Opoen Air Recommender Hackathon#TOA13 - Tech Opoen Air Recommender Hackathon
#TOA13 - Tech Opoen Air Recommender Hackathon
 
Algorithmus, Good School, Camp Digital
Algorithmus, Good School, Camp DigitalAlgorithmus, Good School, Camp Digital
Algorithmus, Good School, Camp Digital
 
Recommender Hackathon @plista 2013/04
Recommender Hackathon @plista 2013/04Recommender Hackathon @plista 2013/04
Recommender Hackathon @plista 2013/04
 
RecSys2012 inside the plista contest
RecSys2012   inside the plista contestRecSys2012   inside the plista contest
RecSys2012 inside the plista contest
 
Webhacks am Beispiel PHP + MySQL
Webhacks am Beispiel PHP + MySQLWebhacks am Beispiel PHP + MySQL
Webhacks am Beispiel PHP + MySQL
 
GIT / SVN
GIT / SVNGIT / SVN
GIT / SVN
 
Collaborative Filtering.. für automatische Empfehlungen
Collaborative Filtering.. für automatische EmpfehlungenCollaborative Filtering.. für automatische Empfehlungen
Collaborative Filtering.. für automatische Empfehlungen
 
Google Web Toolkit
Google Web ToolkitGoogle Web Toolkit
Google Web Toolkit
 
Geld Verdienen Mit Adsense
Geld Verdienen Mit AdsenseGeld Verdienen Mit Adsense
Geld Verdienen Mit Adsense
 
AJAX
AJAXAJAX
AJAX
 
Web 2.0 - "Fluch oder Segen"
Web 2.0 - "Fluch oder Segen"Web 2.0 - "Fluch oder Segen"
Web 2.0 - "Fluch oder Segen"
 

Recently uploaded

microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introductionMaksud Ahmed
 
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdfssuser54595a
 
Incoming and Outgoing Shipments in 1 STEP Using Odoo 17
Incoming and Outgoing Shipments in 1 STEP Using Odoo 17Incoming and Outgoing Shipments in 1 STEP Using Odoo 17
Incoming and Outgoing Shipments in 1 STEP Using Odoo 17Celine George
 
mini mental status format.docx
mini    mental       status     format.docxmini    mental       status     format.docx
mini mental status format.docxPoojaSen20
 
Organic Name Reactions for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions  for the students and aspirants of Chemistry12th.pptxOrganic Name Reactions  for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions for the students and aspirants of Chemistry12th.pptxVS Mahajan Coaching Centre
 
Crayon Activity Handout For the Crayon A
Crayon Activity Handout For the Crayon ACrayon Activity Handout For the Crayon A
Crayon Activity Handout For the Crayon AUnboundStockton
 
Solving Puzzles Benefits Everyone (English).pptx
Solving Puzzles Benefits Everyone (English).pptxSolving Puzzles Benefits Everyone (English).pptx
Solving Puzzles Benefits Everyone (English).pptxOH TEIK BIN
 
Employee wellbeing at the workplace.pptx
Employee wellbeing at the workplace.pptxEmployee wellbeing at the workplace.pptx
Employee wellbeing at the workplace.pptxNirmalaLoungPoorunde1
 
Science 7 - LAND and SEA BREEZE and its Characteristics
Science 7 - LAND and SEA BREEZE and its CharacteristicsScience 7 - LAND and SEA BREEZE and its Characteristics
Science 7 - LAND and SEA BREEZE and its CharacteristicsKarinaGenton
 
KSHARA STURA .pptx---KSHARA KARMA THERAPY (CAUSTIC THERAPY)————IMP.OF KSHARA ...
KSHARA STURA .pptx---KSHARA KARMA THERAPY (CAUSTIC THERAPY)————IMP.OF KSHARA ...KSHARA STURA .pptx---KSHARA KARMA THERAPY (CAUSTIC THERAPY)————IMP.OF KSHARA ...
KSHARA STURA .pptx---KSHARA KARMA THERAPY (CAUSTIC THERAPY)————IMP.OF KSHARA ...M56BOOKSTORE PRODUCT/SERVICE
 
Enzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdf
Enzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdfEnzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdf
Enzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdfSumit Tiwari
 
URLs and Routing in the Odoo 17 Website App
URLs and Routing in the Odoo 17 Website AppURLs and Routing in the Odoo 17 Website App
URLs and Routing in the Odoo 17 Website AppCeline George
 
Sanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdfSanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdfsanyamsingh5019
 
Presiding Officer Training module 2024 lok sabha elections
Presiding Officer Training module 2024 lok sabha electionsPresiding Officer Training module 2024 lok sabha elections
Presiding Officer Training module 2024 lok sabha electionsanshu789521
 
Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)eniolaolutunde
 
Hybridoma Technology ( Production , Purification , and Application )
Hybridoma Technology  ( Production , Purification , and Application  ) Hybridoma Technology  ( Production , Purification , and Application  )
Hybridoma Technology ( Production , Purification , and Application ) Sakshi Ghasle
 
Accessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impactAccessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impactdawncurless
 

Recently uploaded (20)

microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introduction
 
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf
 
Incoming and Outgoing Shipments in 1 STEP Using Odoo 17
Incoming and Outgoing Shipments in 1 STEP Using Odoo 17Incoming and Outgoing Shipments in 1 STEP Using Odoo 17
Incoming and Outgoing Shipments in 1 STEP Using Odoo 17
 
Staff of Color (SOC) Retention Efforts DDSD
Staff of Color (SOC) Retention Efforts DDSDStaff of Color (SOC) Retention Efforts DDSD
Staff of Color (SOC) Retention Efforts DDSD
 
mini mental status format.docx
mini    mental       status     format.docxmini    mental       status     format.docx
mini mental status format.docx
 
Organic Name Reactions for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions  for the students and aspirants of Chemistry12th.pptxOrganic Name Reactions  for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions for the students and aspirants of Chemistry12th.pptx
 
Crayon Activity Handout For the Crayon A
Crayon Activity Handout For the Crayon ACrayon Activity Handout For the Crayon A
Crayon Activity Handout For the Crayon A
 
Solving Puzzles Benefits Everyone (English).pptx
Solving Puzzles Benefits Everyone (English).pptxSolving Puzzles Benefits Everyone (English).pptx
Solving Puzzles Benefits Everyone (English).pptx
 
Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1
 
Employee wellbeing at the workplace.pptx
Employee wellbeing at the workplace.pptxEmployee wellbeing at the workplace.pptx
Employee wellbeing at the workplace.pptx
 
Science 7 - LAND and SEA BREEZE and its Characteristics
Science 7 - LAND and SEA BREEZE and its CharacteristicsScience 7 - LAND and SEA BREEZE and its Characteristics
Science 7 - LAND and SEA BREEZE and its Characteristics
 
KSHARA STURA .pptx---KSHARA KARMA THERAPY (CAUSTIC THERAPY)————IMP.OF KSHARA ...
KSHARA STURA .pptx---KSHARA KARMA THERAPY (CAUSTIC THERAPY)————IMP.OF KSHARA ...KSHARA STURA .pptx---KSHARA KARMA THERAPY (CAUSTIC THERAPY)————IMP.OF KSHARA ...
KSHARA STURA .pptx---KSHARA KARMA THERAPY (CAUSTIC THERAPY)————IMP.OF KSHARA ...
 
Enzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdf
Enzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdfEnzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdf
Enzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdf
 
URLs and Routing in the Odoo 17 Website App
URLs and Routing in the Odoo 17 Website AppURLs and Routing in the Odoo 17 Website App
URLs and Routing in the Odoo 17 Website App
 
Sanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdfSanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdf
 
Presiding Officer Training module 2024 lok sabha elections
Presiding Officer Training module 2024 lok sabha electionsPresiding Officer Training module 2024 lok sabha elections
Presiding Officer Training module 2024 lok sabha elections
 
Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)
 
Model Call Girl in Bikash Puri Delhi reach out to us at 🔝9953056974🔝
Model Call Girl in Bikash Puri  Delhi reach out to us at 🔝9953056974🔝Model Call Girl in Bikash Puri  Delhi reach out to us at 🔝9953056974🔝
Model Call Girl in Bikash Puri Delhi reach out to us at 🔝9953056974🔝
 
Hybridoma Technology ( Production , Purification , and Application )
Hybridoma Technology  ( Production , Purification , and Application  ) Hybridoma Technology  ( Production , Purification , and Application  )
Hybridoma Technology ( Production , Purification , and Application )
 
Accessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impactAccessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impact
 

Realtime Recommender with Redis: Hands on

  • 1. Realtime Recommender Hands On Torben Brodt plista GmbH June 3rd, 2013 Berlin Buzzwords http://berlinbuzzwords.de/
  • 2. Contents 1. How to feed a recommender? 2. How to pass data quickly? 3. How to build a recommender?
  • 3. How to feed a recommender?
  • 4. How to feed a recommender? ● ○ recommendations & advertising network ○ ~5k recs/ second ● integrated on publishers ○ welt.de, abendblatt, etc
  • 5. How to feed a recommender? ● to show recommendations we integrated javascript ● we have URL + HTTP Headers ○ user agent ○ IP address -> geolocation src http://en.wikipedia.org/wiki/Pac-Man 5
  • 6. How to pass data quickly?
  • 7. How to pass data quickly? NOSQL! WHAT ELSE?
  • 8. How to pass data quickly? Thrift Serialization
  • 9. How to pass data quickly?
  • 10. ● Data Types: String, Lists, Set, .. ● Hash ○ map between string fields and string values, very fast ○ HINCR complexity O(1) ● Sorted Set ○ ZINCR complexity: O(log(N)) where N is the number of elements in the sorted set. ○ Allows to limit number of result: ZREVRANGEBYSCORE ○ UNION + INTERSECT How to pass data quickly? p:welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 10
  • 11. How to build a recommendation?
  • 12. How to build a recommendation? Behavioral based on interaction between user and article ○ Most Popular ○ Collaborative Filtering ○ Item to Item Content based on the articles ○ Content Similarity ○ Latest Item Classification ● different recommender families
  • 13. How to build a recommendation? welt.de/football/berlin_wins.html ● ZINCR "p:welt.de" berlin_wins ● ZREVRANGEBYSCORE p:welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 Live Read + Live Write = Real Time Recommendations
  • 14. Most popular with timeseries welt.de/football/berlin_wins.html ● ZINCR "p:welt.de:1360007000" berlin_wins ● ZUNION ○ "p:welt.de:1360007000" ○ "p:welt.de:1360006000" ○ "p:welt.de:1360005000" ● ZREVRANGEBYSCORE p:welt.de:1360005000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360006000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360007000 berlin_wins 689 summer_is_coming 420 plista_best_company 135
  • 15. Most popular with timeseries welt.de/football/berlin_wins.html ● ZINCR "p:welt.de:1360007000" berlin_wins ● ZUNION ... WEIGHTS ○ "p:welt.de:1360007000" .. 4 ○ "p:welt.de:1360006000" .. 2 ○ "p:welt.de:1360005000" .. 1 ● ZREVRANGEBYSCORE p:welt.de:1360005000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360006000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360007000 berlin_wins 689 summer_is_coming 420 plista_best_company 135 15
  • 16. Most popular with timeseries :1360007000 -1h -2h -3h -4h -5h -6h -7h -8h :1360007000 :1360007000 4 2 1
  • 17. Most popular to any context ● it's not only publisher, we use ~50 context attributes context attributes: ● publisher ● weekday ● geolocation ● demographics ● ... publisher = welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 weekday = sunday berlin_wins 400 +1 dortmund_wins 200 ... 100 geolocation = dortmund dortmund_wins 200 berlin_wins 10 +1 ... 5
  • 18. Most popular to any context ZUNION ... WEIGHTS p:welt.de:1360007 4 p:welt.de:1360006 2 p:welt.de:1360005 1 w:sunday:1360007 4 w:sunday:1360006 2 w:sunday:1360005 1 g:dortmund:1360007 4 g:dortmund:1360006 2 g:dortmund:1360005 1 ● how it looks like in Redis publisher = welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 weekday = sunday berlin_wins 400 dortmund_wins 200 ... 100 geolocation = dortmund dortmund_wins 200 berlin_wins 10 ... 5
  • 19. Even more Matrix Operations ;) ● Similarity Matrix ● Human Control Matrix ● Meta-learning Matrix ○ cooperation with ○ aided from ∏ Σ
  • 20. More recommenders possible this was only about most popular ● other algorithms using redis ○ incremental collaborative filtering ○ article to article paths (~graph) ○ .. using external data sources srchttp://en.wikipedia.org/wiki/Flash_(comics) 20
  • 21. What else in Redis? ● message bus ● many recommenders ● live statistics ● caching "One technology to rule them all"
  • 22.