SlideShare a Scribd company logo
UV Logic
using Redis Bitmap
Jooyong Oh
awesome2828@gmail.com
What we want?
● Periodical UV
○ daily
○ weekly
○ monthly
○ total
● UV by dimension
○ country
○ service
before logic
● traditional logic
○ make and maintain daily UV list
○ make weekly/monthly UV list using daily UV
Daily Logs
Daily UV
Daily UV
Daily UV
Daily UV
Weekly
UV
Monthly
UV
Total UV
UV by
country
UV by
service
before logic
● size is too large !!!
○ if daily UV is 50 million, and store (Date, User-ID, country, service),
we need about 55GB
● cannot support real-time UV !!!
● problem is...
○ “maintain” daily UV list
○ aggregate from “whole” daily logs
new logic
● Concept :
○ use Redis HashSet & Bitmap
○ maintain whole user HashSet
○ maintain daily bitmap
○ calcurate weekly, monthly, total UV with Bit
operation
new logic
Conceptual
diagram
new logic
Diagram
with real command
new logic
● When we need UV for a day,
we can get it with BITCOUNT command
> BITCOUNT UV_{YYYYMMDD}
● If we need for UV for n-day,
we can get it with BITOP, BITCOUNT command
> BITOP OR dest UV_{YYYYMMDD1} UV_{YYYYMMDD2} … UV_
{YYYYMMDDn}
> BITCOUNT dest
new logic
● The Bitmap name “UV_{YYYYMMDD}” can be extended
to “UV_service_{YYYYMMDD}” and “UV_country_
{YYYYMMDD}”
● When we need UV for ServiceA and CountryA, we can
get it with BITOP command
> BITOP AND dest UV_serviceA_{YYYYMMDD} UV_countryA_
{YYYYMMDD}
> BITCOUNT dest
Benefit of the new logic
● We can get UV for elastic period (last 3day, 4day or temporary
period)
● We can get UV in real-time
● Time complexity of Redis Bitmap operation is just O(1)
● It need very small memory for UV (1.5GB when total user is 1
billion.
Future work
● Redis Bitmap can cover to 2^32 (about 4
billion)
● If user count increased, this logic need to be
improved (shard would be one candidate)
Jooyong Oh
awesome2828@gmail.com
End of Slide

More Related Content

Viewers also liked

HIgh Performance Redis- Tague Griffith, GoPro
HIgh Performance Redis- Tague Griffith, GoProHIgh Performance Redis- Tague Griffith, GoPro
HIgh Performance Redis- Tague Griffith, GoPro
Redis Labs
 
RedisConf 2016 talk - The Redis API: Simple, Composable, Powerful
RedisConf 2016 talk - The Redis API: Simple, Composable, PowerfulRedisConf 2016 talk - The Redis API: Simple, Composable, Powerful
RedisConf 2016 talk - The Redis API: Simple, Composable, Powerful
DynomiteDB
 
Using Redis as Distributed Cache for ASP.NET apps - Peter Kellner, 73rd Stre...
 Using Redis as Distributed Cache for ASP.NET apps - Peter Kellner, 73rd Stre... Using Redis as Distributed Cache for ASP.NET apps - Peter Kellner, 73rd Stre...
Using Redis as Distributed Cache for ASP.NET apps - Peter Kellner, 73rd Stre...
Redis Labs
 
Troubleshooting Redis- DaeMyung Kang, Kakao
Troubleshooting Redis- DaeMyung Kang, KakaoTroubleshooting Redis- DaeMyung Kang, Kakao
Troubleshooting Redis- DaeMyung Kang, Kakao
Redis Labs
 
Scalable Streaming Data Pipelines with Redis
Scalable Streaming Data Pipelines with RedisScalable Streaming Data Pipelines with Redis
Scalable Streaming Data Pipelines with Redis
Avram Lyon
 
Cloud Foundry for Data Science
Cloud Foundry for Data ScienceCloud Foundry for Data Science
Cloud Foundry for Data Science
Ian Huston
 
Back your App with MySQL & Redis, the Cloud Foundry Way- Kenny Bastani, Pivotal
Back your App with MySQL & Redis, the Cloud Foundry Way- Kenny Bastani, PivotalBack your App with MySQL & Redis, the Cloud Foundry Way- Kenny Bastani, Pivotal
Back your App with MySQL & Redis, the Cloud Foundry Way- Kenny Bastani, Pivotal
Redis Labs
 
Redis & MongoDB: Stop Big Data Indigestion Before It Starts
Redis & MongoDB: Stop Big Data Indigestion Before It StartsRedis & MongoDB: Stop Big Data Indigestion Before It Starts
Redis & MongoDB: Stop Big Data Indigestion Before It Starts
Itamar Haber
 
March 29, 2016 Dr. Josiah Carlson talks about using Redis as a Time Series DB
March 29, 2016 Dr. Josiah Carlson talks about using Redis as a Time Series DBMarch 29, 2016 Dr. Josiah Carlson talks about using Redis as a Time Series DB
March 29, 2016 Dr. Josiah Carlson talks about using Redis as a Time Series DB
Josiah Carlson
 
Redis High availability and fault tolerance in a multitenant environment
Redis High availability and fault tolerance in a multitenant environmentRedis High availability and fault tolerance in a multitenant environment
Redis High availability and fault tolerance in a multitenant environment
Iccha Sethi
 
Benchmarking Redis by itself and versus other NoSQL databases
Benchmarking Redis by itself and versus other NoSQL databasesBenchmarking Redis by itself and versus other NoSQL databases
Benchmarking Redis by itself and versus other NoSQL databases
Itamar Haber
 
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, HerokuPostgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
Redis Labs
 

Viewers also liked (12)

HIgh Performance Redis- Tague Griffith, GoPro
HIgh Performance Redis- Tague Griffith, GoProHIgh Performance Redis- Tague Griffith, GoPro
HIgh Performance Redis- Tague Griffith, GoPro
 
RedisConf 2016 talk - The Redis API: Simple, Composable, Powerful
RedisConf 2016 talk - The Redis API: Simple, Composable, PowerfulRedisConf 2016 talk - The Redis API: Simple, Composable, Powerful
RedisConf 2016 talk - The Redis API: Simple, Composable, Powerful
 
Using Redis as Distributed Cache for ASP.NET apps - Peter Kellner, 73rd Stre...
 Using Redis as Distributed Cache for ASP.NET apps - Peter Kellner, 73rd Stre... Using Redis as Distributed Cache for ASP.NET apps - Peter Kellner, 73rd Stre...
Using Redis as Distributed Cache for ASP.NET apps - Peter Kellner, 73rd Stre...
 
Troubleshooting Redis- DaeMyung Kang, Kakao
Troubleshooting Redis- DaeMyung Kang, KakaoTroubleshooting Redis- DaeMyung Kang, Kakao
Troubleshooting Redis- DaeMyung Kang, Kakao
 
Scalable Streaming Data Pipelines with Redis
Scalable Streaming Data Pipelines with RedisScalable Streaming Data Pipelines with Redis
Scalable Streaming Data Pipelines with Redis
 
Cloud Foundry for Data Science
Cloud Foundry for Data ScienceCloud Foundry for Data Science
Cloud Foundry for Data Science
 
Back your App with MySQL & Redis, the Cloud Foundry Way- Kenny Bastani, Pivotal
Back your App with MySQL & Redis, the Cloud Foundry Way- Kenny Bastani, PivotalBack your App with MySQL & Redis, the Cloud Foundry Way- Kenny Bastani, Pivotal
Back your App with MySQL & Redis, the Cloud Foundry Way- Kenny Bastani, Pivotal
 
Redis & MongoDB: Stop Big Data Indigestion Before It Starts
Redis & MongoDB: Stop Big Data Indigestion Before It StartsRedis & MongoDB: Stop Big Data Indigestion Before It Starts
Redis & MongoDB: Stop Big Data Indigestion Before It Starts
 
March 29, 2016 Dr. Josiah Carlson talks about using Redis as a Time Series DB
March 29, 2016 Dr. Josiah Carlson talks about using Redis as a Time Series DBMarch 29, 2016 Dr. Josiah Carlson talks about using Redis as a Time Series DB
March 29, 2016 Dr. Josiah Carlson talks about using Redis as a Time Series DB
 
Redis High availability and fault tolerance in a multitenant environment
Redis High availability and fault tolerance in a multitenant environmentRedis High availability and fault tolerance in a multitenant environment
Redis High availability and fault tolerance in a multitenant environment
 
Benchmarking Redis by itself and versus other NoSQL databases
Benchmarking Redis by itself and versus other NoSQL databasesBenchmarking Redis by itself and versus other NoSQL databases
Benchmarking Redis by itself and versus other NoSQL databases
 
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, HerokuPostgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
 

Similar to UV logic using redis bitmap

JHipster Code 2020 keynote
JHipster Code 2020 keynoteJHipster Code 2020 keynote
JHipster Code 2020 keynote
Julien Dubois
 
A real-time big data architecture for glasses detection using computer vision...
A real-time big data architecture for glasses detection using computer vision...A real-time big data architecture for glasses detection using computer vision...
A real-time big data architecture for glasses detection using computer vision...
Alberto Fernandez Villan
 
Approximate Vector Search at Scale, With Application to Image Search - SciPY ...
Approximate Vector Search at Scale, With Application to Image Search - SciPY ...Approximate Vector Search at Scale, With Application to Image Search - SciPY ...
Approximate Vector Search at Scale, With Application to Image Search - SciPY ...
Wakana Nogami
 
Jupyter Tips and Tricks
Jupyter Tips and TricksJupyter Tips and Tricks
Jupyter Tips and Tricks
NuSpace
 
Big data @ uber vu (1)
Big data @ uber vu (1)Big data @ uber vu (1)
Big data @ uber vu (1)Mihnea Giurgea
 
Devoxx : being productive with JHipster
Devoxx : being productive with JHipsterDevoxx : being productive with JHipster
Devoxx : being productive with JHipster
Julien Dubois
 
JHipster Conf 2019 English keynote
JHipster Conf 2019 English keynoteJHipster Conf 2019 English keynote
JHipster Conf 2019 English keynote
Julien Dubois
 
WebTech overview assignments
WebTech overview assignmentsWebTech overview assignments
WebTech overview assignmentsKatrien Verbert
 
"What we learned from 5 years of building a data science software that actual...
"What we learned from 5 years of building a data science software that actual..."What we learned from 5 years of building a data science software that actual...
"What we learned from 5 years of building a data science software that actual...
Dataconomy Media
 
Mikrotik metarouter
Mikrotik metarouterMikrotik metarouter
Mikrotik metarouter
Achmad Mardiansyah
 
Childhood Dream to Pi World Record by Emma Haruka, Developer Advocate @Google
Childhood Dream to Pi World Record by Emma Haruka, Developer Advocate @GoogleChildhood Dream to Pi World Record by Emma Haruka, Developer Advocate @Google
Childhood Dream to Pi World Record by Emma Haruka, Developer Advocate @Google
Paris Women in Machine Learning and Data Science
 
Automatise everyday work - Sara Krk
Automatise everyday work - Sara KrkAutomatise everyday work - Sara Krk
Automatise everyday work - Sara Krk
Institute of Contemporary Sciences
 
India Analytics and Big Data Summit 2015
India Analytics and Big Data Summit 2015India Analytics and Big Data Summit 2015
India Analytics and Big Data Summit 2015Kanwal Prakash Singh
 
India Analytics and Big Data Summit 2015
India Analytics and Big Data Summit 2015India Analytics and Big Data Summit 2015
India Analytics and Big Data Summit 2015Kanwal Prakash Singh
 

Similar to UV logic using redis bitmap (14)

JHipster Code 2020 keynote
JHipster Code 2020 keynoteJHipster Code 2020 keynote
JHipster Code 2020 keynote
 
A real-time big data architecture for glasses detection using computer vision...
A real-time big data architecture for glasses detection using computer vision...A real-time big data architecture for glasses detection using computer vision...
A real-time big data architecture for glasses detection using computer vision...
 
Approximate Vector Search at Scale, With Application to Image Search - SciPY ...
Approximate Vector Search at Scale, With Application to Image Search - SciPY ...Approximate Vector Search at Scale, With Application to Image Search - SciPY ...
Approximate Vector Search at Scale, With Application to Image Search - SciPY ...
 
Jupyter Tips and Tricks
Jupyter Tips and TricksJupyter Tips and Tricks
Jupyter Tips and Tricks
 
Big data @ uber vu (1)
Big data @ uber vu (1)Big data @ uber vu (1)
Big data @ uber vu (1)
 
Devoxx : being productive with JHipster
Devoxx : being productive with JHipsterDevoxx : being productive with JHipster
Devoxx : being productive with JHipster
 
JHipster Conf 2019 English keynote
JHipster Conf 2019 English keynoteJHipster Conf 2019 English keynote
JHipster Conf 2019 English keynote
 
WebTech overview assignments
WebTech overview assignmentsWebTech overview assignments
WebTech overview assignments
 
"What we learned from 5 years of building a data science software that actual...
"What we learned from 5 years of building a data science software that actual..."What we learned from 5 years of building a data science software that actual...
"What we learned from 5 years of building a data science software that actual...
 
Mikrotik metarouter
Mikrotik metarouterMikrotik metarouter
Mikrotik metarouter
 
Childhood Dream to Pi World Record by Emma Haruka, Developer Advocate @Google
Childhood Dream to Pi World Record by Emma Haruka, Developer Advocate @GoogleChildhood Dream to Pi World Record by Emma Haruka, Developer Advocate @Google
Childhood Dream to Pi World Record by Emma Haruka, Developer Advocate @Google
 
Automatise everyday work - Sara Krk
Automatise everyday work - Sara KrkAutomatise everyday work - Sara Krk
Automatise everyday work - Sara Krk
 
India Analytics and Big Data Summit 2015
India Analytics and Big Data Summit 2015India Analytics and Big Data Summit 2015
India Analytics and Big Data Summit 2015
 
India Analytics and Big Data Summit 2015
India Analytics and Big Data Summit 2015India Analytics and Big Data Summit 2015
India Analytics and Big Data Summit 2015
 

Recently uploaded

一比一原版(TWU毕业证)西三一大学毕业证成绩单
一比一原版(TWU毕业证)西三一大学毕业证成绩单一比一原版(TWU毕业证)西三一大学毕业证成绩单
一比一原版(TWU毕业证)西三一大学毕业证成绩单
ocavb
 
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
AbhimanyuSinha9
 
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
Subhajit Sahu
 
一比一原版(QU毕业证)皇后大学毕业证成绩单
一比一原版(QU毕业证)皇后大学毕业证成绩单一比一原版(QU毕业证)皇后大学毕业证成绩单
一比一原版(QU毕业证)皇后大学毕业证成绩单
enxupq
 
FP Growth Algorithm and its Applications
FP Growth Algorithm and its ApplicationsFP Growth Algorithm and its Applications
FP Growth Algorithm and its Applications
MaleehaSheikh2
 
【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】
【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】
【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】
NABLAS株式会社
 
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
pchutichetpong
 
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
u86oixdj
 
Machine learning and optimization techniques for electrical drives.pptx
Machine learning and optimization techniques for electrical drives.pptxMachine learning and optimization techniques for electrical drives.pptx
Machine learning and optimization techniques for electrical drives.pptx
balafet
 
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
nscud
 
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
slg6lamcq
 
Opendatabay - Open Data Marketplace.pptx
Opendatabay - Open Data Marketplace.pptxOpendatabay - Open Data Marketplace.pptx
Opendatabay - Open Data Marketplace.pptx
Opendatabay
 
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
oz8q3jxlp
 
一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单
一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单
一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单
ewymefz
 
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
v3tuleee
 
My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.
rwarrenll
 
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
ukgaet
 
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
Timothy Spann
 
Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)
TravisMalana
 
一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单
enxupq
 

Recently uploaded (20)

一比一原版(TWU毕业证)西三一大学毕业证成绩单
一比一原版(TWU毕业证)西三一大学毕业证成绩单一比一原版(TWU毕业证)西三一大学毕业证成绩单
一比一原版(TWU毕业证)西三一大学毕业证成绩单
 
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
 
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
 
一比一原版(QU毕业证)皇后大学毕业证成绩单
一比一原版(QU毕业证)皇后大学毕业证成绩单一比一原版(QU毕业证)皇后大学毕业证成绩单
一比一原版(QU毕业证)皇后大学毕业证成绩单
 
FP Growth Algorithm and its Applications
FP Growth Algorithm and its ApplicationsFP Growth Algorithm and its Applications
FP Growth Algorithm and its Applications
 
【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】
【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】
【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】
 
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
 
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
 
Machine learning and optimization techniques for electrical drives.pptx
Machine learning and optimization techniques for electrical drives.pptxMachine learning and optimization techniques for electrical drives.pptx
Machine learning and optimization techniques for electrical drives.pptx
 
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
 
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
 
Opendatabay - Open Data Marketplace.pptx
Opendatabay - Open Data Marketplace.pptxOpendatabay - Open Data Marketplace.pptx
Opendatabay - Open Data Marketplace.pptx
 
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
 
一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单
一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单
一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单
 
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
 
My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.
 
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
 
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
 
Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)
 
一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单
 

UV logic using redis bitmap

  • 1. UV Logic using Redis Bitmap Jooyong Oh awesome2828@gmail.com
  • 2. What we want? ● Periodical UV ○ daily ○ weekly ○ monthly ○ total ● UV by dimension ○ country ○ service
  • 3. before logic ● traditional logic ○ make and maintain daily UV list ○ make weekly/monthly UV list using daily UV Daily Logs Daily UV Daily UV Daily UV Daily UV Weekly UV Monthly UV Total UV UV by country UV by service
  • 4. before logic ● size is too large !!! ○ if daily UV is 50 million, and store (Date, User-ID, country, service), we need about 55GB ● cannot support real-time UV !!! ● problem is... ○ “maintain” daily UV list ○ aggregate from “whole” daily logs
  • 5. new logic ● Concept : ○ use Redis HashSet & Bitmap ○ maintain whole user HashSet ○ maintain daily bitmap ○ calcurate weekly, monthly, total UV with Bit operation
  • 8. new logic ● When we need UV for a day, we can get it with BITCOUNT command > BITCOUNT UV_{YYYYMMDD} ● If we need for UV for n-day, we can get it with BITOP, BITCOUNT command > BITOP OR dest UV_{YYYYMMDD1} UV_{YYYYMMDD2} … UV_ {YYYYMMDDn} > BITCOUNT dest
  • 9. new logic ● The Bitmap name “UV_{YYYYMMDD}” can be extended to “UV_service_{YYYYMMDD}” and “UV_country_ {YYYYMMDD}” ● When we need UV for ServiceA and CountryA, we can get it with BITOP command > BITOP AND dest UV_serviceA_{YYYYMMDD} UV_countryA_ {YYYYMMDD} > BITCOUNT dest
  • 10. Benefit of the new logic ● We can get UV for elastic period (last 3day, 4day or temporary period) ● We can get UV in real-time ● Time complexity of Redis Bitmap operation is just O(1) ● It need very small memory for UV (1.5GB when total user is 1 billion.
  • 11. Future work ● Redis Bitmap can cover to 2^32 (about 4 billion) ● If user count increased, this logic need to be improved (shard would be one candidate)