Redis data modeling examples

9,739 views

Published on

Redis data modeling example
it is from "이것이 레디스다" 한빛미디어 press book

Published in: Engineering

Redis data modeling examples

  1. 1. REDIS DATA MODEL SAMPLE Terry’s Redis
  2. 2. 1.LogWriter • Problem domain : Collect all logs from distributed server and merge it into single log file Server Server Redis Key:’WAS:log’ Value : Log (Single String) Append log to String Log File Flush to log file Problem String append bring memory re- allocation. So every log write makes a memory relocation Log String Log String Log String Server Server Key:’WAS:log’ : Log File Redis (List) lpush rpop Solution Use List data type and push the log and pop & write the log into log file
  3. 3. 2.Visitor count • Problem domain – Count total event page visit # – Count visit # per each event page event:click:total event:click:{event page# id} visit # visit # event:click:{event page# id} visit # : Key Value (String Type) incr • Enhancement Request – Count total event page visit # per day – Count visit # per each event page per day visit # visit # event:click:daily:total:{date} event:click:daily:{date}:{event page# id} event:click:daily:{date}:{event page# id} visit # Key Value (String Type) incr
  4. 4. 2.Visitor count • Problem – It cannot find event start & end date because of that it is hard to find “key name” • Solution – Use hash data type – Sort by using java.util.SortedHashMap event:click:total:hash date visit # date visit # date visit # Key Value (Hash) event:click:total:hash:{eventid} date visit # date visit # date visit # Total event page visit per day Daily visit # per day for each event page hincrBy java.util.SortedHashMap Sorted by date Redis
  5. 5. 3. Shopping Basket • Problem domain – Make shopping basket which can support • add product • remove product • empty shopping basket • list products in the shopping basket • remove product which expires 3 days {userNo}:cart:product { ‘productNo’:’{productNo}’, ‘prodctName’:{productName}’, ‘quantity’:’{quantity}’ } {userNo}:cart:productid:{productNo} (개별상품 주문정보) Value (String) { ‘productNo’,’productNo’,….} { ‘productNo’:’{productNo}’, ‘prodctName’:{productName}’, ‘quantity’:’{quantity}’ } {userNo}:cart:productid:{productNo} (개별상품 주문정보) setex(key,{EXPIRE TIME(3days)},JSON VALUE); Key SimpleJson is used org.json.simple
  6. 6. 3. Shopping Basket • Problem – getProductList() for(productsNo){ json=jedis.get(product) result+=json } {userNo}:cart:product { ‘productNo’,’productNo’,….} { ‘productNo’:’{productNo}’, ‘prodctName’:{productName}’, ‘quantity’:’{quantity}’ } {userNo}:cart:productid:{productNo} (개별상품 주문정보) It makes # of calls to redis • Solution – Use Redis pipeline call – p = redis.pipelined() getProductList() for(productsNo){ p.get(product) } List<Object> redisResult = p.syncAndReturnAll(); for(item:redisResult){ json.add(item) }
  7. 7. 4. Like it • Problem domain – Add Like to posing : sadd – Remove Like from posting : srem – Validate specific user’s Like :sismember – Total count of Like in specific posting : scard – Total count of Like in postings :pipleline (for postings) + scard posting:like:{posting no} Value (Set) {userNo} Key {userNo} {userNo} : Each value is unique in a Set ※ scard  160K scard/sec with pipe line 20개의 게시물별로 좋아요합을 출력하려면 160K/20 = 8000 TPS If it needs more TPS, use read replica
  8. 8. 5. Count unique visitor per day (not page view) • Problem domain – Capacity : it has 10M users – Count unique vistor # per day 1 2 3 4 10 M….Key = unique:vistors:{date} Value(String/Bit) Map eash 10M user into bit 10M bit required = 1.9M per day Redis.setbit(Key,{userNo},true); Jedis.bitOffSet CountUnqueVisitor # per day = Jedis.bitCount(Key)
  9. 9. 5. Count unique visitor per day (not page view) • Problem domain – Capacity : it has 10M users – Count unique vistor # per day 1 2 3 4 10 M….Key = unique:vistors:{date} Value(String/Bit) Map eash 10M user into bit 10M bit required = 1.9M per day Redis.setbit(Key,{userNo},true); Jedis.bitOffSet CountUnqueVisitor # per day = Jedis.bitCount(Key)
  10. 10. 5. Count unique visitor per day (not page view) • Enhancement request – Count unique visitor who visits every day in a week • Solution – AND operation in 1Week data and count bit 1 2 3 4 10 M….Key = unique:vistors:{date} Value(String/Bit) 1 2 3 4 10 MKey = unique:vistors:{date} 1 2 3 4 10 MKey = unique:vistors:{date} 1 2 3 4 10 M….Key = unique:vistors:{date} 1 2 3 4 10 MKey = unique:vistors:{date} 1 2 3 4 10 MKey = unique:vistors:{date} 1 2 3 4 10 MKey = unique:vistors:{date} 1W AND bitop(BitOP.AND,{key},[unique:vistors:day1, unique:vistors:day2,…]) 1 2 3 4 10 MKey = {key} Result =bitcount({key}) bitop From Redis
  11. 11. 5. Count unique visitor per day (not page view) • Enhancement request – Get list of visitor who visited site every day. – 근데 예제가 좀 이상함. AND 연산으로 구해서 1은 사용자만 구하면 될텐데. • Solution – Register Lua script and run it • Register String sha1 = jedis.script.Load( (String)”LUA Script”); • Run jedis.evalsha(sha1) ※ BitSet Order – Bitset order between Redis and Program language(LUA) can be different (opposite direction)

×