Redis data modeling example it is from "이것이 레디스다" 한빛미디어 press book

1. REDIS DATA MODEL SAMPLE
Terry’s Redis

2. 1.LogWriter
• Problem domain : Collect all logs from distributed server and merge it into single log file
Server
Server
Redis
Key:’WAS:log’
Value : Log (Single String)
Append log to String
Log File
Flush to log file
Problem
String append bring memory re-
allocation. So every log write makes a
memory relocation
Log String
Log String
Log String
Server
Server
Key:’WAS:log’
:
Log File
Redis
(List)
lpush
rpop
Solution
Use List data type and push the log
and pop & write the log into log file

3. 2.Visitor count
• Problem domain
– Count total event page visit #
– Count visit # per each event page
event:click:total
event:click:{event page# id}
visit #
visit #
event:click:{event page# id} visit #
:
Key
Value
(String Type)
incr
• Enhancement Request
– Count total event page visit # per day
– Count visit # per each event page per day
visit #
visit #
event:click:daily:total:{date}
event:click:daily:{date}:{event page# id}
event:click:daily:{date}:{event page# id} visit #
Key
Value
(String Type)
incr

4. 2.Visitor count
• Problem
– It cannot find event start & end date because of that it is hard to find “key name”
• Solution
– Use hash data type
– Sort by using java.util.SortedHashMap
event:click:total:hash date visit #
date visit #
date visit #
Key Value (Hash)
event:click:total:hash:{eventid} date visit #
date visit #
date visit #
Total event page visit per day
Daily visit # per day for each event
page
hincrBy
java.util.SortedHashMap Sorted by date
Redis

5. 3. Shopping Basket
• Problem domain
– Make shopping basket which can support
• add product
• remove product
• empty shopping basket
• list products in the shopping basket
• remove product which expires 3 days
{userNo}:cart:product
{
‘productNo’:’{productNo}’,
‘prodctName’:{productName}’,
‘quantity’:’{quantity}’
}
{userNo}:cart:productid:{productNo}
(개별상품 주문정보)
Value (String)
{ ‘productNo’,’productNo’,….}
{
‘productNo’:’{productNo}’,
‘prodctName’:{productName}’,
‘quantity’:’{quantity}’
}
{userNo}:cart:productid:{productNo}
(개별상품 주문정보)
setex(key,{EXPIRE
TIME(3days)},JSON VALUE);
Key
SimpleJson is used
org.json.simple

6. 3. Shopping Basket
• Problem
– getProductList()
for(productsNo){
json=jedis.get(product)
result+=json
}
{userNo}:cart:product { ‘productNo’,’productNo’,….}
{
‘productNo’:’{productNo}’,
‘prodctName’:{productName}’,
‘quantity’:’{quantity}’
}
{userNo}:cart:productid:{productNo}
(개별상품 주문정보)
It makes # of calls to redis
• Solution
– Use Redis pipeline call
– p = redis.pipelined()
getProductList()
for(productsNo){
p.get(product)
}
List<Object> redisResult = p.syncAndReturnAll();
for(item:redisResult){
json.add(item)
}

7. 4. Like it
• Problem domain
– Add Like to posing : sadd
– Remove Like from posting : srem
– Validate specific user’s Like :sismember
– Total count of Like in specific posting : scard
– Total count of Like in postings :pipleline (for postings) + scard
posting:like:{posting no}
Value (Set)
{userNo}
Key
{userNo}
{userNo}
:
Each value is unique in a Set
※ scard  160K scard/sec with pipe line
20개의 게시물별로 좋아요합을 출력하려면 160K/20 = 8000 TPS
If it needs more TPS, use read replica

8. 5. Count unique visitor per day (not page view)
• Problem domain
– Capacity : it has 10M users
– Count unique vistor # per day
1 2 3 4 10
M….Key = unique:vistors:{date}
Value(String/Bit)
Map eash 10M user into bit
10M bit required = 1.9M per day
Redis.setbit(Key,{userNo},true);
Jedis.bitOffSet
CountUnqueVisitor # per day = Jedis.bitCount(Key)

9. 5. Count unique visitor per day (not page view)
• Problem domain
– Capacity : it has 10M users
– Count unique vistor # per day
1 2 3 4 10
M….Key = unique:vistors:{date}
Value(String/Bit)
Map eash 10M user into bit
10M bit required = 1.9M per day
Redis.setbit(Key,{userNo},true);
Jedis.bitOffSet
CountUnqueVisitor # per day = Jedis.bitCount(Key)

10. 5. Count unique visitor per day (not page view)
• Enhancement request
– Count unique visitor who visits every day in a week
• Solution
– AND operation in 1Week data and count bit
1 2 3 4 10
M….Key = unique:vistors:{date}
Value(String/Bit)
1 2 3 4 10
MKey = unique:vistors:{date}
1 2 3 4 10
MKey = unique:vistors:{date}
1 2 3 4 10
M….Key = unique:vistors:{date}
1 2 3 4 10
MKey = unique:vistors:{date}
1 2 3 4 10
MKey = unique:vistors:{date}
1 2 3 4 10
MKey = unique:vistors:{date}
1W
AND
bitop(BitOP.AND,{key},[unique:vistors:day1,
unique:vistors:day2,…])
1 2 3 4 10
MKey = {key}
Result =bitcount({key})
bitop From Redis

11. 5. Count unique visitor per day (not page view)
• Enhancement request
– Get list of visitor who visited site every day.
– 근데 예제가 좀 이상함. AND 연산으로 구해서 1은 사용자만 구하면 될텐데.
• Solution
– Register Lua script and run it
• Register String sha1 = jedis.script.Load( (String)”LUA Script”);
• Run jedis.evalsha(sha1)
※ BitSet Order
– Bitset order between Redis and Program language(LUA) can be different (opposite direction)