Hot/Cold Data Transfer between Redis and Mongo concept

1. 徐熊瓶
Hot/Cold Data TransferHot/Cold Data Transfer
between Redis and Mongobetween Redis and Mongo

2. 2
Redis is anRedis is an open sourceopen source,, BSDBSD licensed,licensed,
advancedadvanced key-valuekey-value store.store.
It is often referred to as a data structureIt is often referred to as a data structure
server since keys can containserver since keys can contain stringsstrings,,
hasheshashes,, listslists,, setssets andand sorted setssorted sets..

3. 3
MongoDB is an MongoDB is an open sourceopen source
document databasedocument database,,
and the leading NoSQL database.and the leading NoSQL database.
Written in C++ Written in C++

4. 4
RedisDB is TemporarilyRedisDB is Temporarily
MongoDB is PersistenceMongoDB is Persistence

5. 5
MongoDBRedisDB

6. 6
TopDB
MongoDB
RedisDB

7. 7
TopDB is implementsTopDB is implements
Jedis-like interfaceJedis-like interface
Jedis is redis driver for JavaJedis is redis driver for Java

8. 8
TopDB support redis key typeTopDB support redis key type ::
a:b:ca:b:c
a:b:c:a:b:c:1234512345
a:b:c:a:b:c:1234512345::6789067890
a:b:c:a:b:c:1234512345::6789067890:d:e:f:d:e:f
prefixKey keyPair
recommend be shardkey
suffixKey

9. 9
Redis dbIndex will get a Mongo dbNameRedis dbIndex will get a Mongo dbName
Mapping TableMapping Table (Defined in TopDBModel)(Defined in TopDBModel)::
dbIndex 1 -> mdb00001dbIndex 1 -> mdb00001
dbIndex 2 -> mdb00002dbIndex 2 -> mdb00002
dbIndex 3 -> mdb00003dbIndex 3 -> mdb00003
dbIndex 4 -> mdb00004dbIndex 4 -> mdb00004
dbIndex 5 -> mdb00005dbIndex 5 -> mdb00005
Another will throw Exception likeAnother will throw Exception like ““dbIndex Error : 6dbIndex Error : 6””

10. 10
aaa:bbb:aaa:bbb:104150104150:ccc:ccc
↓↓
aaa:bbb:cccaaa:bbb:ccc
↓↓
aaa_bbb_cccaaa_bbb_ccc
Mongo Collection Name Defined in TopDBModelMongo Collection Name Defined in TopDBModel
if collection name doesnif collection name doesn’’t exists will throw Exception liket exists will throw Exception like
““TopDB vaild Mongo Collection Error!TopDB vaild Mongo Collection Error!”” ……etc.etc.
Mongo Collection Name defined ruleMongo Collection Name defined rule
Redis Key
Mongo Collection

11. 11
104150104150 must become linkedHashMapmust become linkedHashMap
Ex 1 :Ex 1 :
LinkedHashMap keyPairMap = new LinkeHashMap()LinkedHashMap keyPairMap = new LinkeHashMap()
keyPairMap.put(keyPairMap.put(““idid””,, ““104150104150””))
Mean 1 is :Mean 1 is : aaa:bbb:aaa:bbb:104150104150:ccc:ccc
Ex 2 :Ex 2 :
LinkedHashMap keyPairMap = new LinkeHashMap()LinkedHashMap keyPairMap = new LinkeHashMap()
keyPairMap.put(keyPairMap.put(““id1id1””,, ““104150104150””))
keyPairMap.put(keyPairMap.put(““id2id2””,, ““000001000001””))
Mean 2 is :Mean 2 is : aaa:bbb:aaa:bbb:104150104150::000001000001:ccc:ccc
keyPair is sortedkeyPair is sorted

12. 12
Sample CodeSample Code
topDB.hget(topDB.hget(
topDB.getTopDBModel(topDB.getTopDBModel(
1,1, ““aaa:bbbaaa:bbb””,, ““id:104150id:104150””,, ““cccccc””
))
, 123, 123
););

13. 13
TopDB workflowTopDB workflow
TopDB.hget(topDBModel, field)TopDB.hget(topDBModel, field)
↓ ↓
checkStringData() {checkStringData() {
If (redisDB.get()==null) {If (redisDB.get()==null) { // redis data doesn// redis data doesn’’t existst exists
mongoDB.select();mongoDB.select();
redisDB.save(mongoDB);redisDB.save(mongoDB);
}}
}}
↓ ↓
Return redisDB.hget(dbIndex, redisDBKey, field);Return redisDB.hget(dbIndex, redisDBKey, field);

14. Q & AQ & A