SlideShare a Scribd company logo
Realtime Recommender
Hands On
Torben Brodt
plista GmbH
June 3rd, 2013
Berlin Buzzwords
http://berlinbuzzwords.de/
Contents
1. How to feed a recommender?
2. How to pass data quickly?
3. How to build a recommender?
How to feed a recommender?
How to feed a recommender?
●
○ recommendations & advertising network
○ ~5k recs/ second
● integrated on publishers
○ welt.de, abendblatt, etc
How to feed a recommender?
● to show recommendations we integrated javascript
● we have URL + HTTP Headers
○ user agent
○ IP address -> geolocation
src http://en.wikipedia.org/wiki/Pac-Man
5
How to pass data quickly?
How to pass data quickly?
NOSQL!
WHAT
ELSE?
How to pass data quickly?
Thrift
Serialization
How to pass data quickly?
● Data Types: String, Lists, Set, ..
● Hash
○ map between string fields and string values, very
fast
○ HINCR complexity O(1)
● Sorted Set
○ ZINCR complexity: O(log(N)) where N is the number
of elements in the sorted set.
○ Allows to limit number of result:
ZREVRANGEBYSCORE
○ UNION + INTERSECT
How to pass data quickly?
p:welt.de
berlin_wins 689 +1
summer_is_coming 420
plista_company 135
10
How to build a recommendation?
How to build a recommendation?
Behavioral
based on interaction
between user and
article
○ Most Popular
○ Collaborative Filtering
○ Item to Item
Content
based on the articles
○ Content Similarity
○ Latest Item
Classification
● different recommender families
How to build a recommendation?
welt.de/football/berlin_wins.html
● ZINCR "p:welt.de" berlin_wins
● ZREVRANGEBYSCORE
p:welt.de
berlin_wins 689 +1
summer_is_coming 420
plista_company 135
Live Read
+ Live Write
= Real Time Recommendations
Most popular with timeseries
welt.de/football/berlin_wins.html
● ZINCR "p:welt.de:1360007000" berlin_wins
● ZUNION
○ "p:welt.de:1360007000"
○ "p:welt.de:1360006000"
○ "p:welt.de:1360005000"
● ZREVRANGEBYSCORE
p:welt.de:1360005000
berlin_wins 420
summer_is_coming 135
plista_best_company 689
p:welt.de:1360006000
berlin_wins 420
summer_is_coming 135
plista_best_company 689
p:welt.de:1360007000
berlin_wins 689
summer_is_coming 420
plista_best_company 135
Most popular with timeseries
welt.de/football/berlin_wins.html
● ZINCR "p:welt.de:1360007000" berlin_wins
● ZUNION ... WEIGHTS
○ "p:welt.de:1360007000" .. 4
○ "p:welt.de:1360006000" .. 2
○ "p:welt.de:1360005000" .. 1
● ZREVRANGEBYSCORE
p:welt.de:1360005000
berlin_wins 420
summer_is_coming 135
plista_best_company 689
p:welt.de:1360006000
berlin_wins 420
summer_is_coming 135
plista_best_company 689
p:welt.de:1360007000
berlin_wins 689
summer_is_coming 420
plista_best_company 135
15
Most popular with timeseries
:1360007000
-1h -2h -3h -4h -5h -6h -7h -8h
:1360007000
:1360007000
4
2
1
Most popular to any context
● it's not only publisher, we use ~50 context
attributes
context attributes:
● publisher
● weekday
● geolocation
● demographics
● ...
publisher = welt.de
berlin_wins 689 +1
summer_is_coming 420
plista_company 135
weekday = sunday
berlin_wins 400 +1
dortmund_wins 200
... 100
geolocation = dortmund
dortmund_wins 200
berlin_wins 10 +1
... 5
Most popular to any context
ZUNION ... WEIGHTS
p:welt.de:1360007 4
p:welt.de:1360006 2
p:welt.de:1360005 1
w:sunday:1360007 4
w:sunday:1360006 2
w:sunday:1360005 1
g:dortmund:1360007 4
g:dortmund:1360006 2
g:dortmund:1360005 1
● how it looks like in Redis
publisher = welt.de
berlin_wins 689 +1
summer_is_coming 420
plista_company 135
weekday = sunday
berlin_wins 400
dortmund_wins 200
... 100
geolocation = dortmund
dortmund_wins 200
berlin_wins 10
... 5
Even more Matrix Operations ;)
● Similarity Matrix
● Human Control Matrix
● Meta-learning Matrix
○ cooperation with
○ aided from
∏
Σ
More recommenders possible
this was only about most popular
● other algorithms using redis
○ incremental collaborative filtering
○ article to article paths (~graph)
○ .. using external data sources
srchttp://en.wikipedia.org/wiki/Flash_(comics)
20
What else in Redis?
● message bus
● many recommenders
● live statistics
● caching
"One technology to rule them all"
Questions?
www.plista.com
torben.brodt@plista.com
@torbenbrodt
xing.com/profile/Torben_Brodt
http://goo.gl/pvXm5
http://lnkd.in/MUXXuv

More Related Content

Viewers also liked

OS17 Brochure
OS17 BrochureOS17 Brochure
OS17 Brochure
Dominic Vogel
 
Composite çelik
Composite çelikComposite çelik
Composite çelik
Abdullah ÇELİK
 
Ahg microsoft stream_insight_queries
Ahg microsoft stream_insight_queriesAhg microsoft stream_insight_queries
Ahg microsoft stream_insight_queries
Steve Xu
 
Introduction to ICS/SCADA security
Introduction to ICS/SCADA securityIntroduction to ICS/SCADA security
Introduction to ICS/SCADA security
Cysinfo Cyber Security Community
 
Coniferous Forest
Coniferous ForestConiferous Forest
Coniferous Forest
Mariano Villarreal
 
Chicago AWS user group meetup - May 2014 at Cohesive
Chicago AWS user group meetup - May 2014 at CohesiveChicago AWS user group meetup - May 2014 at Cohesive
Chicago AWS user group meetup - May 2014 at Cohesive
CloudCamp Chicago
 
AppSensor Near Real-Time Event Detection and Response - DevNexus 2016
AppSensor Near Real-Time Event Detection and Response - DevNexus 2016AppSensor Near Real-Time Event Detection and Response - DevNexus 2016
AppSensor Near Real-Time Event Detection and Response - DevNexus 2016
jtmelton
 
Application Deployment at UC Riverside
Application Deployment at UC RiversideApplication Deployment at UC Riverside
Application Deployment at UC Riverside
Michael Kennedy
 
concepto de colección local
concepto de colección localconcepto de colección local
concepto de colección local
guestf488db7
 
EMC World 2015 - The Devops Toolkit
EMC World 2015 - The Devops ToolkitEMC World 2015 - The Devops Toolkit
EMC World 2015 - The Devops Toolkit
Jonas Rosland
 
Tech Chat – What's New in Sumo Logic
Tech Chat – What's New in Sumo LogicTech Chat – What's New in Sumo Logic
Tech Chat – What's New in Sumo Logic
Sumo Logic
 
AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)
AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)
AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)
Amazon Web Services
 
Five pillars of Infrastructure Monitoring
Five pillars of Infrastructure MonitoringFive pillars of Infrastructure Monitoring
Five pillars of Infrastructure Monitoring
Daniel Koller
 
Marian Marinov, 1H Ltd.
Marian Marinov, 1H Ltd.Marian Marinov, 1H Ltd.
Marian Marinov, 1H Ltd.
Ontico
 
George Park Workshop 1 - Cosumnes CSD
George Park Workshop 1 - Cosumnes CSDGeorge Park Workshop 1 - Cosumnes CSD
George Park Workshop 1 - Cosumnes CSD
Cosumnes CSD
 
Book of Fauna and Flora
Book of Fauna and FloraBook of Fauna and Flora
Book of Fauna and Flora
moisesrodriguezmateo
 
Hadoop / Spark on Malware Expression
Hadoop / Spark on Malware ExpressionHadoop / Spark on Malware Expression
Hadoop / Spark on Malware Expression
MapR Technologies
 
Application Development on Metapod
Application Development on MetapodApplication Development on Metapod
Application Development on Metapod
Cisco DevNet
 
How to Keep Students Motivated During Winter
How to Keep Students Motivated During WinterHow to Keep Students Motivated During Winter
How to Keep Students Motivated During Winter
Robert Peters, Ed.D
 

Viewers also liked (19)

OS17 Brochure
OS17 BrochureOS17 Brochure
OS17 Brochure
 
Composite çelik
Composite çelikComposite çelik
Composite çelik
 
Ahg microsoft stream_insight_queries
Ahg microsoft stream_insight_queriesAhg microsoft stream_insight_queries
Ahg microsoft stream_insight_queries
 
Introduction to ICS/SCADA security
Introduction to ICS/SCADA securityIntroduction to ICS/SCADA security
Introduction to ICS/SCADA security
 
Coniferous Forest
Coniferous ForestConiferous Forest
Coniferous Forest
 
Chicago AWS user group meetup - May 2014 at Cohesive
Chicago AWS user group meetup - May 2014 at CohesiveChicago AWS user group meetup - May 2014 at Cohesive
Chicago AWS user group meetup - May 2014 at Cohesive
 
AppSensor Near Real-Time Event Detection and Response - DevNexus 2016
AppSensor Near Real-Time Event Detection and Response - DevNexus 2016AppSensor Near Real-Time Event Detection and Response - DevNexus 2016
AppSensor Near Real-Time Event Detection and Response - DevNexus 2016
 
Application Deployment at UC Riverside
Application Deployment at UC RiversideApplication Deployment at UC Riverside
Application Deployment at UC Riverside
 
concepto de colección local
concepto de colección localconcepto de colección local
concepto de colección local
 
EMC World 2015 - The Devops Toolkit
EMC World 2015 - The Devops ToolkitEMC World 2015 - The Devops Toolkit
EMC World 2015 - The Devops Toolkit
 
Tech Chat – What's New in Sumo Logic
Tech Chat – What's New in Sumo LogicTech Chat – What's New in Sumo Logic
Tech Chat – What's New in Sumo Logic
 
AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)
AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)
AWS re:Invent 2016: Deploying Scalable SAP Hybris Clusters using Docker (CON312)
 
Five pillars of Infrastructure Monitoring
Five pillars of Infrastructure MonitoringFive pillars of Infrastructure Monitoring
Five pillars of Infrastructure Monitoring
 
Marian Marinov, 1H Ltd.
Marian Marinov, 1H Ltd.Marian Marinov, 1H Ltd.
Marian Marinov, 1H Ltd.
 
George Park Workshop 1 - Cosumnes CSD
George Park Workshop 1 - Cosumnes CSDGeorge Park Workshop 1 - Cosumnes CSD
George Park Workshop 1 - Cosumnes CSD
 
Book of Fauna and Flora
Book of Fauna and FloraBook of Fauna and Flora
Book of Fauna and Flora
 
Hadoop / Spark on Malware Expression
Hadoop / Spark on Malware ExpressionHadoop / Spark on Malware Expression
Hadoop / Spark on Malware Expression
 
Application Development on Metapod
Application Development on MetapodApplication Development on Metapod
Application Development on Metapod
 
How to Keep Students Motivated During Winter
How to Keep Students Motivated During WinterHow to Keep Students Motivated During Winter
How to Keep Students Motivated During Winter
 

Similar to Realtime Recommender with Redis: Hands on

Living Labs Challenge Workshop
Living Labs Challenge WorkshopLiving Labs Challenge Workshop
Living Labs Challenge Workshop
Torben Brodt
 
Open recommendation platform
Open recommendation platformOpen recommendation platform
Open recommendation platform
Torben Brodt
 
SIGIR 2013 BARS Keynote - the search for the best live recommender system
SIGIR 2013 BARS Keynote - the search for the best live recommender systemSIGIR 2013 BARS Keynote - the search for the best live recommender system
SIGIR 2013 BARS Keynote - the search for the best live recommender system
Torben Brodt
 
Content recommendations
Content recommendationsContent recommendations
Content recommendations
Torben Brodt
 
Website metadata - getting competitive intelligence on websites
Website metadata - getting competitive intelligence on websitesWebsite metadata - getting competitive intelligence on websites
Website metadata - getting competitive intelligence on websites
netcomber
 
Crawling and Processing the Italian Corporate Web
Crawling and Processing the Italian Corporate WebCrawling and Processing the Italian Corporate Web
Crawling and Processing the Italian Corporate Web
Speck&Tech
 
Open Source BI Overview
Open Source BI Overview Open Source BI Overview
Open Source BI Overview
Alex Meadows
 
WSO2Con USA 2015: An Introduction to the WSO2 Analytics Platform
WSO2Con USA 2015: An Introduction to the WSO2 Analytics PlatformWSO2Con USA 2015: An Introduction to the WSO2 Analytics Platform
WSO2Con USA 2015: An Introduction to the WSO2 Analytics Platform
WSO2
 
[MongoDB.local Bengaluru 2018] Keynote
[MongoDB.local Bengaluru 2018] Keynote[MongoDB.local Bengaluru 2018] Keynote
[MongoDB.local Bengaluru 2018] Keynote
MongoDB
 
MongoDB Schema Design
MongoDB Schema DesignMongoDB Schema Design
MongoDB Schema Design
Joe Drumgoole
 
What's inside 09 2013: gstreamer: black magic and witchcraft
What's inside 09 2013: gstreamer: black magic and witchcraftWhat's inside 09 2013: gstreamer: black magic and witchcraft
What's inside 09 2013: gstreamer: black magic and witchcraft
Marc Leeman
 
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
AI Frontiers
 
Logging and ranting / Vytis Valentinavičius (Lamoda)
Logging and ranting / Vytis Valentinavičius (Lamoda)Logging and ranting / Vytis Valentinavičius (Lamoda)
Logging and ranting / Vytis Valentinavičius (Lamoda)
Ontico
 
Mapping french open data actors on the web with common crawl
Mapping french open data actors on the web with common crawlMapping french open data actors on the web with common crawl
Mapping french open data actors on the web with common crawl
data publica
 
"Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ...
"Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ..."Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ...
"Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ...
Dataconomy Media
 
Juraj vysvader - Python developer's CV
Juraj vysvader - Python developer's CVJuraj vysvader - Python developer's CV
Juraj vysvader - Python developer's CV
Juraj Vysvader
 
Search at SoundCloud
Search at SoundCloudSearch at SoundCloud
Search at SoundCloud
Tomasz Elendt
 
Data Day Seattle 2017: Scaling Data Science at Stitch Fix
Data Day Seattle 2017: Scaling Data Science at Stitch FixData Day Seattle 2017: Scaling Data Science at Stitch Fix
Data Day Seattle 2017: Scaling Data Science at Stitch Fix
Stefan Krawczyk
 
Building a Cross Channel Content Delivery Platform with MongoDB
Building a Cross Channel Content Delivery Platform with MongoDBBuilding a Cross Channel Content Delivery Platform with MongoDB
Building a Cross Channel Content Delivery Platform with MongoDB
MongoDB
 
Horneando apis
Horneando apisHorneando apis
Horneando apis
Paradigma Digital
 

Similar to Realtime Recommender with Redis: Hands on (20)

Living Labs Challenge Workshop
Living Labs Challenge WorkshopLiving Labs Challenge Workshop
Living Labs Challenge Workshop
 
Open recommendation platform
Open recommendation platformOpen recommendation platform
Open recommendation platform
 
SIGIR 2013 BARS Keynote - the search for the best live recommender system
SIGIR 2013 BARS Keynote - the search for the best live recommender systemSIGIR 2013 BARS Keynote - the search for the best live recommender system
SIGIR 2013 BARS Keynote - the search for the best live recommender system
 
Content recommendations
Content recommendationsContent recommendations
Content recommendations
 
Website metadata - getting competitive intelligence on websites
Website metadata - getting competitive intelligence on websitesWebsite metadata - getting competitive intelligence on websites
Website metadata - getting competitive intelligence on websites
 
Crawling and Processing the Italian Corporate Web
Crawling and Processing the Italian Corporate WebCrawling and Processing the Italian Corporate Web
Crawling and Processing the Italian Corporate Web
 
Open Source BI Overview
Open Source BI Overview Open Source BI Overview
Open Source BI Overview
 
WSO2Con USA 2015: An Introduction to the WSO2 Analytics Platform
WSO2Con USA 2015: An Introduction to the WSO2 Analytics PlatformWSO2Con USA 2015: An Introduction to the WSO2 Analytics Platform
WSO2Con USA 2015: An Introduction to the WSO2 Analytics Platform
 
[MongoDB.local Bengaluru 2018] Keynote
[MongoDB.local Bengaluru 2018] Keynote[MongoDB.local Bengaluru 2018] Keynote
[MongoDB.local Bengaluru 2018] Keynote
 
MongoDB Schema Design
MongoDB Schema DesignMongoDB Schema Design
MongoDB Schema Design
 
What's inside 09 2013: gstreamer: black magic and witchcraft
What's inside 09 2013: gstreamer: black magic and witchcraftWhat's inside 09 2013: gstreamer: black magic and witchcraft
What's inside 09 2013: gstreamer: black magic and witchcraft
 
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
 
Logging and ranting / Vytis Valentinavičius (Lamoda)
Logging and ranting / Vytis Valentinavičius (Lamoda)Logging and ranting / Vytis Valentinavičius (Lamoda)
Logging and ranting / Vytis Valentinavičius (Lamoda)
 
Mapping french open data actors on the web with common crawl
Mapping french open data actors on the web with common crawlMapping french open data actors on the web with common crawl
Mapping french open data actors on the web with common crawl
 
"Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ...
"Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ..."Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ...
"Data Pipelines for Small, Messy and Tedious Data", Vladislav Supalov, CAO & ...
 
Juraj vysvader - Python developer's CV
Juraj vysvader - Python developer's CVJuraj vysvader - Python developer's CV
Juraj vysvader - Python developer's CV
 
Search at SoundCloud
Search at SoundCloudSearch at SoundCloud
Search at SoundCloud
 
Data Day Seattle 2017: Scaling Data Science at Stitch Fix
Data Day Seattle 2017: Scaling Data Science at Stitch FixData Day Seattle 2017: Scaling Data Science at Stitch Fix
Data Day Seattle 2017: Scaling Data Science at Stitch Fix
 
Building a Cross Channel Content Delivery Platform with MongoDB
Building a Cross Channel Content Delivery Platform with MongoDBBuilding a Cross Channel Content Delivery Platform with MongoDB
Building a Cross Channel Content Delivery Platform with MongoDB
 
Horneando apis
Horneando apisHorneando apis
Horneando apis
 

More from Torben Brodt

Recommender Trends 2014
Recommender Trends 2014Recommender Trends 2014
Recommender Trends 2014
Torben Brodt
 
Paper the plista dataset
Paper  the plista datasetPaper  the plista dataset
Paper the plista dataset
Torben Brodt
 
Nrs2013 recap
Nrs2013 recapNrs2013 recap
Nrs2013 recap
Torben Brodt
 
#TOA13 - Tech Opoen Air Recommender Hackathon
#TOA13 - Tech Opoen Air Recommender Hackathon#TOA13 - Tech Opoen Air Recommender Hackathon
#TOA13 - Tech Opoen Air Recommender Hackathon
Torben Brodt
 
Algorithmus, Good School, Camp Digital
Algorithmus, Good School, Camp DigitalAlgorithmus, Good School, Camp Digital
Algorithmus, Good School, Camp DigitalTorben Brodt
 
Recommender Hackathon @plista 2013/04
Recommender Hackathon @plista 2013/04Recommender Hackathon @plista 2013/04
Recommender Hackathon @plista 2013/04
Torben Brodt
 
RecSys2012 inside the plista contest
RecSys2012   inside the plista contestRecSys2012   inside the plista contest
RecSys2012 inside the plista contest
Torben Brodt
 
Webhacks am Beispiel PHP + MySQL
Webhacks am Beispiel PHP + MySQLWebhacks am Beispiel PHP + MySQL
Webhacks am Beispiel PHP + MySQL
Torben Brodt
 
GIT / SVN
GIT / SVNGIT / SVN
GIT / SVN
Torben Brodt
 
Collaborative Filtering.. für automatische Empfehlungen
Collaborative Filtering.. für automatische EmpfehlungenCollaborative Filtering.. für automatische Empfehlungen
Collaborative Filtering.. für automatische Empfehlungen
Torben Brodt
 
Google Web Toolkit
Google Web ToolkitGoogle Web Toolkit
Google Web Toolkit
Torben Brodt
 
Geld Verdienen Mit Adsense
Geld Verdienen Mit AdsenseGeld Verdienen Mit Adsense
Geld Verdienen Mit Adsense
Torben Brodt
 
AJAX
AJAXAJAX
Web 2.0 - "Fluch oder Segen"
Web 2.0 - "Fluch oder Segen"Web 2.0 - "Fluch oder Segen"
Web 2.0 - "Fluch oder Segen"
Torben Brodt
 

More from Torben Brodt (14)

Recommender Trends 2014
Recommender Trends 2014Recommender Trends 2014
Recommender Trends 2014
 
Paper the plista dataset
Paper  the plista datasetPaper  the plista dataset
Paper the plista dataset
 
Nrs2013 recap
Nrs2013 recapNrs2013 recap
Nrs2013 recap
 
#TOA13 - Tech Opoen Air Recommender Hackathon
#TOA13 - Tech Opoen Air Recommender Hackathon#TOA13 - Tech Opoen Air Recommender Hackathon
#TOA13 - Tech Opoen Air Recommender Hackathon
 
Algorithmus, Good School, Camp Digital
Algorithmus, Good School, Camp DigitalAlgorithmus, Good School, Camp Digital
Algorithmus, Good School, Camp Digital
 
Recommender Hackathon @plista 2013/04
Recommender Hackathon @plista 2013/04Recommender Hackathon @plista 2013/04
Recommender Hackathon @plista 2013/04
 
RecSys2012 inside the plista contest
RecSys2012   inside the plista contestRecSys2012   inside the plista contest
RecSys2012 inside the plista contest
 
Webhacks am Beispiel PHP + MySQL
Webhacks am Beispiel PHP + MySQLWebhacks am Beispiel PHP + MySQL
Webhacks am Beispiel PHP + MySQL
 
GIT / SVN
GIT / SVNGIT / SVN
GIT / SVN
 
Collaborative Filtering.. für automatische Empfehlungen
Collaborative Filtering.. für automatische EmpfehlungenCollaborative Filtering.. für automatische Empfehlungen
Collaborative Filtering.. für automatische Empfehlungen
 
Google Web Toolkit
Google Web ToolkitGoogle Web Toolkit
Google Web Toolkit
 
Geld Verdienen Mit Adsense
Geld Verdienen Mit AdsenseGeld Verdienen Mit Adsense
Geld Verdienen Mit Adsense
 
AJAX
AJAXAJAX
AJAX
 
Web 2.0 - "Fluch oder Segen"
Web 2.0 - "Fluch oder Segen"Web 2.0 - "Fluch oder Segen"
Web 2.0 - "Fluch oder Segen"
 

Recently uploaded

Mule event processing models | MuleSoft Mysore Meetup #47
Mule event processing models | MuleSoft Mysore Meetup #47Mule event processing models | MuleSoft Mysore Meetup #47
Mule event processing models | MuleSoft Mysore Meetup #47
MysoreMuleSoftMeetup
 
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptxPrésentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
siemaillard
 
Wound healing PPT
Wound healing PPTWound healing PPT
Wound healing PPT
Jyoti Chand
 
Pengantar Penggunaan Flutter - Dart programming language1.pptx
Pengantar Penggunaan Flutter - Dart programming language1.pptxPengantar Penggunaan Flutter - Dart programming language1.pptx
Pengantar Penggunaan Flutter - Dart programming language1.pptx
Fajar Baskoro
 
Temple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation resultsTemple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation results
Krassimira Luka
 
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptxBeyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
EduSkills OECD
 
Pharmaceutics Pharmaceuticals best of brub
Pharmaceutics Pharmaceuticals best of brubPharmaceutics Pharmaceuticals best of brub
Pharmaceutics Pharmaceuticals best of brub
danielkiash986
 
BBR 2024 Summer Sessions Interview Training
BBR  2024 Summer Sessions Interview TrainingBBR  2024 Summer Sessions Interview Training
BBR 2024 Summer Sessions Interview Training
Katrina Pritchard
 
math operations ued in python and all used
math operations ued in python and all usedmath operations ued in python and all used
math operations ued in python and all used
ssuser13ffe4
 
How to deliver Powerpoint Presentations.pptx
How to deliver Powerpoint  Presentations.pptxHow to deliver Powerpoint  Presentations.pptx
How to deliver Powerpoint Presentations.pptx
HajraNaeem15
 
HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.
deepaannamalai16
 
Nutrition Inc FY 2024, 4 - Hour Training
Nutrition Inc FY 2024, 4 - Hour TrainingNutrition Inc FY 2024, 4 - Hour Training
Nutrition Inc FY 2024, 4 - Hour Training
melliereed
 
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
PECB
 
مصحف القراءات العشر أعد أحرف الخلاف سمير بسيوني.pdf
مصحف القراءات العشر   أعد أحرف الخلاف سمير بسيوني.pdfمصحف القراءات العشر   أعد أحرف الخلاف سمير بسيوني.pdf
مصحف القراءات العشر أعد أحرف الخلاف سمير بسيوني.pdf
سمير بسيوني
 
Educational Technology in the Health Sciences
Educational Technology in the Health SciencesEducational Technology in the Health Sciences
Educational Technology in the Health Sciences
Iris Thiele Isip-Tan
 
Standardized tool for Intelligence test.
Standardized tool for Intelligence test.Standardized tool for Intelligence test.
Standardized tool for Intelligence test.
deepaannamalai16
 
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
GeorgeMilliken2
 
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem studentsRHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
Himanshu Rai
 
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptxBIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
RidwanHassanYusuf
 
skeleton System.pdf (skeleton system wow)
skeleton System.pdf (skeleton system wow)skeleton System.pdf (skeleton system wow)
skeleton System.pdf (skeleton system wow)
Mohammad Al-Dhahabi
 

Recently uploaded (20)

Mule event processing models | MuleSoft Mysore Meetup #47
Mule event processing models | MuleSoft Mysore Meetup #47Mule event processing models | MuleSoft Mysore Meetup #47
Mule event processing models | MuleSoft Mysore Meetup #47
 
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptxPrésentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
 
Wound healing PPT
Wound healing PPTWound healing PPT
Wound healing PPT
 
Pengantar Penggunaan Flutter - Dart programming language1.pptx
Pengantar Penggunaan Flutter - Dart programming language1.pptxPengantar Penggunaan Flutter - Dart programming language1.pptx
Pengantar Penggunaan Flutter - Dart programming language1.pptx
 
Temple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation resultsTemple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation results
 
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptxBeyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
 
Pharmaceutics Pharmaceuticals best of brub
Pharmaceutics Pharmaceuticals best of brubPharmaceutics Pharmaceuticals best of brub
Pharmaceutics Pharmaceuticals best of brub
 
BBR 2024 Summer Sessions Interview Training
BBR  2024 Summer Sessions Interview TrainingBBR  2024 Summer Sessions Interview Training
BBR 2024 Summer Sessions Interview Training
 
math operations ued in python and all used
math operations ued in python and all usedmath operations ued in python and all used
math operations ued in python and all used
 
How to deliver Powerpoint Presentations.pptx
How to deliver Powerpoint  Presentations.pptxHow to deliver Powerpoint  Presentations.pptx
How to deliver Powerpoint Presentations.pptx
 
HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.
 
Nutrition Inc FY 2024, 4 - Hour Training
Nutrition Inc FY 2024, 4 - Hour TrainingNutrition Inc FY 2024, 4 - Hour Training
Nutrition Inc FY 2024, 4 - Hour Training
 
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
 
مصحف القراءات العشر أعد أحرف الخلاف سمير بسيوني.pdf
مصحف القراءات العشر   أعد أحرف الخلاف سمير بسيوني.pdfمصحف القراءات العشر   أعد أحرف الخلاف سمير بسيوني.pdf
مصحف القراءات العشر أعد أحرف الخلاف سمير بسيوني.pdf
 
Educational Technology in the Health Sciences
Educational Technology in the Health SciencesEducational Technology in the Health Sciences
Educational Technology in the Health Sciences
 
Standardized tool for Intelligence test.
Standardized tool for Intelligence test.Standardized tool for Intelligence test.
Standardized tool for Intelligence test.
 
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
 
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem studentsRHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
 
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptxBIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
 
skeleton System.pdf (skeleton system wow)
skeleton System.pdf (skeleton system wow)skeleton System.pdf (skeleton system wow)
skeleton System.pdf (skeleton system wow)
 

Realtime Recommender with Redis: Hands on

  • 1. Realtime Recommender Hands On Torben Brodt plista GmbH June 3rd, 2013 Berlin Buzzwords http://berlinbuzzwords.de/
  • 2. Contents 1. How to feed a recommender? 2. How to pass data quickly? 3. How to build a recommender?
  • 3. How to feed a recommender?
  • 4. How to feed a recommender? ● ○ recommendations & advertising network ○ ~5k recs/ second ● integrated on publishers ○ welt.de, abendblatt, etc
  • 5. How to feed a recommender? ● to show recommendations we integrated javascript ● we have URL + HTTP Headers ○ user agent ○ IP address -> geolocation src http://en.wikipedia.org/wiki/Pac-Man 5
  • 6. How to pass data quickly?
  • 7. How to pass data quickly? NOSQL! WHAT ELSE?
  • 8. How to pass data quickly? Thrift Serialization
  • 9. How to pass data quickly?
  • 10. ● Data Types: String, Lists, Set, .. ● Hash ○ map between string fields and string values, very fast ○ HINCR complexity O(1) ● Sorted Set ○ ZINCR complexity: O(log(N)) where N is the number of elements in the sorted set. ○ Allows to limit number of result: ZREVRANGEBYSCORE ○ UNION + INTERSECT How to pass data quickly? p:welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 10
  • 11. How to build a recommendation?
  • 12. How to build a recommendation? Behavioral based on interaction between user and article ○ Most Popular ○ Collaborative Filtering ○ Item to Item Content based on the articles ○ Content Similarity ○ Latest Item Classification ● different recommender families
  • 13. How to build a recommendation? welt.de/football/berlin_wins.html ● ZINCR "p:welt.de" berlin_wins ● ZREVRANGEBYSCORE p:welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 Live Read + Live Write = Real Time Recommendations
  • 14. Most popular with timeseries welt.de/football/berlin_wins.html ● ZINCR "p:welt.de:1360007000" berlin_wins ● ZUNION ○ "p:welt.de:1360007000" ○ "p:welt.de:1360006000" ○ "p:welt.de:1360005000" ● ZREVRANGEBYSCORE p:welt.de:1360005000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360006000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360007000 berlin_wins 689 summer_is_coming 420 plista_best_company 135
  • 15. Most popular with timeseries welt.de/football/berlin_wins.html ● ZINCR "p:welt.de:1360007000" berlin_wins ● ZUNION ... WEIGHTS ○ "p:welt.de:1360007000" .. 4 ○ "p:welt.de:1360006000" .. 2 ○ "p:welt.de:1360005000" .. 1 ● ZREVRANGEBYSCORE p:welt.de:1360005000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360006000 berlin_wins 420 summer_is_coming 135 plista_best_company 689 p:welt.de:1360007000 berlin_wins 689 summer_is_coming 420 plista_best_company 135 15
  • 16. Most popular with timeseries :1360007000 -1h -2h -3h -4h -5h -6h -7h -8h :1360007000 :1360007000 4 2 1
  • 17. Most popular to any context ● it's not only publisher, we use ~50 context attributes context attributes: ● publisher ● weekday ● geolocation ● demographics ● ... publisher = welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 weekday = sunday berlin_wins 400 +1 dortmund_wins 200 ... 100 geolocation = dortmund dortmund_wins 200 berlin_wins 10 +1 ... 5
  • 18. Most popular to any context ZUNION ... WEIGHTS p:welt.de:1360007 4 p:welt.de:1360006 2 p:welt.de:1360005 1 w:sunday:1360007 4 w:sunday:1360006 2 w:sunday:1360005 1 g:dortmund:1360007 4 g:dortmund:1360006 2 g:dortmund:1360005 1 ● how it looks like in Redis publisher = welt.de berlin_wins 689 +1 summer_is_coming 420 plista_company 135 weekday = sunday berlin_wins 400 dortmund_wins 200 ... 100 geolocation = dortmund dortmund_wins 200 berlin_wins 10 ... 5
  • 19. Even more Matrix Operations ;) ● Similarity Matrix ● Human Control Matrix ● Meta-learning Matrix ○ cooperation with ○ aided from ∏ Σ
  • 20. More recommenders possible this was only about most popular ● other algorithms using redis ○ incremental collaborative filtering ○ article to article paths (~graph) ○ .. using external data sources srchttp://en.wikipedia.org/wiki/Flash_(comics) 20
  • 21. What else in Redis? ● message bus ● many recommenders ● live statistics ● caching "One technology to rule them all"
  • 22.