Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Hedis - GET HBase via Redis
Kewang
2
Live DEMO
https://www.youtube.com/watch?v=aAgJaqvgqLU
Who I am
●
王慕羣
● Java / Node.js / AngularJS
● SQL-like / HBase
GitHub: kewang
Facebook: kewangtw
Linkedin: kewangtw
Slides...
What Mitake is
三竹資訊
What Mitake is
三竹資訊
●
企業簡訊平台
What Mitake is
三竹資訊
●
企業簡訊平台
● 市佔 100% 的行動下單
What Mitake is
三竹資訊
●
企業簡訊平台
● 市佔 100% 的行動下單
● 市佔 70% 的行動銀行
What Mitake is
三竹資訊
●
企業簡訊平台
● 市佔 100% 的行動下單
● 市佔 70% 的行動銀行
●
企業內部應用、產壽險、金融相關政府機關
What Mitake is
三竹資訊
●
企業簡訊平台
● 市佔 100% 的行動下單
● 市佔 70% 的行動銀行
●
企業內部應用、產壽險、金融相關政府機關
● 雲端: Qmi 、 TechMe
What Mitake is
三竹資訊
●
企業簡訊平台
● 市佔 100% 的行動下單
● 市佔 70% 的行動銀行
●
企業內部應用、產壽險、金融相關政府機關
● 雲端: Qmi 、 TechMe
● 其他:兌彩券、台灣匯率、三竹小股王、行...
11
Agenda
● Redis brief
● Scenario
● Hedis (aka Hyper Redis)
● Benchmark
● TODO list
12
Redis brief
13
Redis brief
14
Redis brief
● Key-value pair
15
Redis brief
● Key-value pair
● Value data type: String, List, Sets, Ordered sets,
Hashes
16
Redis brief
● Key-value pair
● Value data type: String, List, Sets, Ordered sets,
Hashes
● Binary-safe
17
Redis brief
● Key-value pair
● Value data type: String, List, Sets, Ordered sets,
Hashes
● Binary-safe
● Single thread
18
Redis brief
● Key-value pair
● Value data type: String, List, Sets, Ordered sets,
Hashes
● Binary-safe
● Single thread
...
19
Scenario
Get hot data, etc.
20
Redis
21
Redis
Application server
Redis Database
Client
22
Redis - Send request to AP
Application server
Redis Database
Client
(1)
23
Redis - AP gets data from Redis
Application server
Redis Database
(2)
Client
(1)
24
Redis - Redis has no data
Application server
Redis Database
(2)
Client
(1)
(3)
25
Redis - Back to get data from DB
Application server
Redis Database
(4)
(2)
Client
(1)
(3)
26
Redis - AP got data
Application server
Redis Database
(4)
(5)
(2)
Client
(1)
(3)
27
Redis - Store data to Redis
Application server
Redis Database
(6)
(4)
(5)
(2)
Client
(1)
(3)
28
Redis - Return data to client
Application server
Redis Database
(6)
(4)
(5)
(2)
Client (7)
(1)
(3)
29
Hedis
30
Hedis
Application server
Hedis Database
Client
31
Hedis - Send request to AP
Application server
Hedis Database
Client
(1)
32
Hedis - AP gets data from Hedis
Application server
Hedis Database
Client
(2)
(1)
33
Hedis - Turns to get from DB
Application server
Hedis Database
Client
(2)
(3)
(1)
34
Hedis - Store data to Hedis
Application server
Hedis Database
Client
(2)
(3)
(4)
(1)
35
Hedis - Return data to AP
Application server
Hedis Database
Client
(2)
(3)
(4)
(5)
(1)
36
Hedis - Return data to client
Application server
Hedis Database
Client
(2)
(3)
(4)
(5)
(1)
(6)
37
Hedis
Hyper Redis
38
Architecture
39
Architecture
Hedis
40
Architecture
Redis
Hedis
41
Architecture
Redis Connector
Hedis
42
Architecture
Redis Connector
MySQLHedis
43
Architecture
Redis Connector
MySQL
HBase
Hedis
44
Architecture
Redis Connector
MySQL
HBase
others
Hedis
45
Architecture
Redis Connector
MySQL
HBase
others
Hedis
46
Architecture
Redis Connector
MySQL
HBase
others
Hedis
47
Architecture
Redis Connector
MySQL
HBase
others
Hedis
48
Architecture
Redis Connector
MySQL
HBase
others
Hedis
49
Architecture
● Core
● Connector
● Database
50
Architecture
● Core - Redis
● Connector
● Database
51
Architecture
● Core - Redis
● Connector - connect Redis and database together
● Database
52
Architecture
● Core - Redis
● Connector - connect Redis and database together
● Database - any database (includes RDBMS...
53
Configuration
54
Configuration
55
How to start Hedis up ?
56
How to start Hedis up ?
57
Commands - only GET currently
58
Commands - only GET currently
● GET "cdh1://user@kewang"
59
Commands - only GET currently
● GET "cdh1://user@kewang"
– No exists: turns to get "kewang" rowkey at "user" table
on "...
60
Commands - only GET currently
● GET "cdh1://user@kewang"
– No exists: turns to get "kewang" rowkey at "user" table
on "...
61
Commands - another GET
62
Commands - another GET
● GET "cdh1://!user@kewang"
63
Commands - another GET
● GET "cdh1://!user@kewang"
64
Commands - another GET
● GET "cdh1://!user@kewang"
– Always gets "kewang" rowkey at "user" table on "cdh1"
connector di...
65
Commands - another GET
● GET "cdh1://!user@kewang"
– Always gets "kewang" rowkey at "user" table on "cdh1"
connector di...
66
Benchmark
67
Latency - Lower is better
68
Latency - Lower is better
69
Request - Higher is better
70
Request - Higher is better
71
TODO list
72
TODO list
73
TODO list
● Fix memory leak
74
TODO list
● Fix memory leak
● Support multiple Redis version
75
TODO list
● Fix memory leak
● Support multiple Redis version
● Support more commands
76
TODO list
● Fix memory leak
● Support multiple Redis version
● Support more commands
● Support more databases
77
TODO list
● Fix memory leak
● Support multiple Redis version
● Support more commands
● Support more databases
● Support...
78
TODO list
● Fix memory leak
● Support multiple Redis version
● Support more commands
● Support more databases
● Support...
79
TODO list
● Fix memory leak
● Support multiple Redis version
● Support more commands
● Support more databases
● Support...
80
Live DEMO
81
References
● http://hedis.io
● http://redis.io
82
WE'RE HIRING !!!
Backend, System,Web, ... etc.
83
Upcoming SlideShare
Loading in …5
×

Hedis - GET HBase via Redis

3,555 views

Published on

Hedis stands for "Hyper Redis". It can retrieve data from **ANY** database directly and stores to itself.

Published in: Technology
  • Be the first to comment

Hedis - GET HBase via Redis

  1. 1. Hedis - GET HBase via Redis Kewang
  2. 2. 2 Live DEMO https://www.youtube.com/watch?v=aAgJaqvgqLU
  3. 3. Who I am ● 王慕羣 ● Java / Node.js / AngularJS ● SQL-like / HBase GitHub: kewang Facebook: kewangtw Linkedin: kewangtw Slideshare: kewang Mail: cpckewang@gmail.com
  4. 4. What Mitake is 三竹資訊
  5. 5. What Mitake is 三竹資訊 ● 企業簡訊平台
  6. 6. What Mitake is 三竹資訊 ● 企業簡訊平台 ● 市佔 100% 的行動下單
  7. 7. What Mitake is 三竹資訊 ● 企業簡訊平台 ● 市佔 100% 的行動下單 ● 市佔 70% 的行動銀行
  8. 8. What Mitake is 三竹資訊 ● 企業簡訊平台 ● 市佔 100% 的行動下單 ● 市佔 70% 的行動銀行 ● 企業內部應用、產壽險、金融相關政府機關
  9. 9. What Mitake is 三竹資訊 ● 企業簡訊平台 ● 市佔 100% 的行動下單 ● 市佔 70% 的行動銀行 ● 企業內部應用、產壽險、金融相關政府機關 ● 雲端: Qmi 、 TechMe
  10. 10. What Mitake is 三竹資訊 ● 企業簡訊平台 ● 市佔 100% 的行動下單 ● 市佔 70% 的行動銀行 ● 企業內部應用、產壽險、金融相關政府機關 ● 雲端: Qmi 、 TechMe ● 其他:兌彩券、台灣匯率、三竹小股王、行動股市 ... 等
  11. 11. 11 Agenda ● Redis brief ● Scenario ● Hedis (aka Hyper Redis) ● Benchmark ● TODO list
  12. 12. 12 Redis brief
  13. 13. 13 Redis brief
  14. 14. 14 Redis brief ● Key-value pair
  15. 15. 15 Redis brief ● Key-value pair ● Value data type: String, List, Sets, Ordered sets, Hashes
  16. 16. 16 Redis brief ● Key-value pair ● Value data type: String, List, Sets, Ordered sets, Hashes ● Binary-safe
  17. 17. 17 Redis brief ● Key-value pair ● Value data type: String, List, Sets, Ordered sets, Hashes ● Binary-safe ● Single thread
  18. 18. 18 Redis brief ● Key-value pair ● Value data type: String, List, Sets, Ordered sets, Hashes ● Binary-safe ● Single thread ● In-memory database
  19. 19. 19 Scenario Get hot data, etc.
  20. 20. 20 Redis
  21. 21. 21 Redis Application server Redis Database Client
  22. 22. 22 Redis - Send request to AP Application server Redis Database Client (1)
  23. 23. 23 Redis - AP gets data from Redis Application server Redis Database (2) Client (1)
  24. 24. 24 Redis - Redis has no data Application server Redis Database (2) Client (1) (3)
  25. 25. 25 Redis - Back to get data from DB Application server Redis Database (4) (2) Client (1) (3)
  26. 26. 26 Redis - AP got data Application server Redis Database (4) (5) (2) Client (1) (3)
  27. 27. 27 Redis - Store data to Redis Application server Redis Database (6) (4) (5) (2) Client (1) (3)
  28. 28. 28 Redis - Return data to client Application server Redis Database (6) (4) (5) (2) Client (7) (1) (3)
  29. 29. 29 Hedis
  30. 30. 30 Hedis Application server Hedis Database Client
  31. 31. 31 Hedis - Send request to AP Application server Hedis Database Client (1)
  32. 32. 32 Hedis - AP gets data from Hedis Application server Hedis Database Client (2) (1)
  33. 33. 33 Hedis - Turns to get from DB Application server Hedis Database Client (2) (3) (1)
  34. 34. 34 Hedis - Store data to Hedis Application server Hedis Database Client (2) (3) (4) (1)
  35. 35. 35 Hedis - Return data to AP Application server Hedis Database Client (2) (3) (4) (5) (1)
  36. 36. 36 Hedis - Return data to client Application server Hedis Database Client (2) (3) (4) (5) (1) (6)
  37. 37. 37 Hedis Hyper Redis
  38. 38. 38 Architecture
  39. 39. 39 Architecture Hedis
  40. 40. 40 Architecture Redis Hedis
  41. 41. 41 Architecture Redis Connector Hedis
  42. 42. 42 Architecture Redis Connector MySQLHedis
  43. 43. 43 Architecture Redis Connector MySQL HBase Hedis
  44. 44. 44 Architecture Redis Connector MySQL HBase others Hedis
  45. 45. 45 Architecture Redis Connector MySQL HBase others Hedis
  46. 46. 46 Architecture Redis Connector MySQL HBase others Hedis
  47. 47. 47 Architecture Redis Connector MySQL HBase others Hedis
  48. 48. 48 Architecture Redis Connector MySQL HBase others Hedis
  49. 49. 49 Architecture ● Core ● Connector ● Database
  50. 50. 50 Architecture ● Core - Redis ● Connector ● Database
  51. 51. 51 Architecture ● Core - Redis ● Connector - connect Redis and database together ● Database
  52. 52. 52 Architecture ● Core - Redis ● Connector - connect Redis and database together ● Database - any database (includes RDBMS & NoSQL)
  53. 53. 53 Configuration
  54. 54. 54 Configuration
  55. 55. 55 How to start Hedis up ?
  56. 56. 56 How to start Hedis up ?
  57. 57. 57 Commands - only GET currently
  58. 58. 58 Commands - only GET currently ● GET "cdh1://user@kewang"
  59. 59. 59 Commands - only GET currently ● GET "cdh1://user@kewang" – No exists: turns to get "kewang" rowkey at "user" table on "cdh1" connector
  60. 60. 60 Commands - only GET currently ● GET "cdh1://user@kewang" – No exists: turns to get "kewang" rowkey at "user" table on "cdh1" connector – Exists: gets directly
  61. 61. 61 Commands - another GET
  62. 62. 62 Commands - another GET ● GET "cdh1://!user@kewang"
  63. 63. 63 Commands - another GET ● GET "cdh1://!user@kewang"
  64. 64. 64 Commands - another GET ● GET "cdh1://!user@kewang" – Always gets "kewang" rowkey at "user" table on "cdh1" connector directly
  65. 65. 65 Commands - another GET ● GET "cdh1://!user@kewang" – Always gets "kewang" rowkey at "user" table on "cdh1" connector directly – Use ruby's bang sign ( ! ) concept, it will modify original value
  66. 66. 66 Benchmark
  67. 67. 67 Latency - Lower is better
  68. 68. 68 Latency - Lower is better
  69. 69. 69 Request - Higher is better
  70. 70. 70 Request - Higher is better
  71. 71. 71 TODO list
  72. 72. 72 TODO list
  73. 73. 73 TODO list ● Fix memory leak
  74. 74. 74 TODO list ● Fix memory leak ● Support multiple Redis version
  75. 75. 75 TODO list ● Fix memory leak ● Support multiple Redis version ● Support more commands
  76. 76. 76 TODO list ● Fix memory leak ● Support multiple Redis version ● Support more commands ● Support more databases
  77. 77. 77 TODO list ● Fix memory leak ● Support multiple Redis version ● Support more commands ● Support more databases ● Support git
  78. 78. 78 TODO list ● Fix memory leak ● Support multiple Redis version ● Support more commands ● Support more databases ● Support git ● Add dockerfile
  79. 79. 79 TODO list ● Fix memory leak ● Support multiple Redis version ● Support more commands ● Support more databases ● Support git ● Add dockerfile ● Redesign official website
  80. 80. 80 Live DEMO
  81. 81. 81 References ● http://hedis.io ● http://redis.io
  82. 82. 82 WE'RE HIRING !!! Backend, System,Web, ... etc.
  83. 83. 83

×