The document discusses various strategies for designing row keys in HBase. It covers techniques for avoiding hotspotting like salting, hashing, or reversing keys. It also discusses refining IDs by using shorter hashed values. Other sections provide examples for row keys that enable authentication with access tokens, comments linked to posts in a tree structure, access control lists for controlling data visibility, and organizing statistical data by time period and user. The overall goal of the document is to provide best practices for row key design in HBase to optimize performance and functionality.
55. 55
Rowkey design - Rice dumpling
Id ME00000024AC
Title Announce
Content We are hiring
56. 56
Rowkey design - Rice dumpling
Id ME00000024AC
Title Announce
Content We are hiring
Id ME00000024AC.ME00000037ZZ
Title (n/a)
Content I want to join your team !!!
57. 57
Rowkey design - Rice dumpling
Id ME00000024AC
Title Announce
Content We are hiring
Id ME00000024AC.ME00000037ZZ
Title (n/a)
Content I want to join your team !!!
63. 63
Rowkey design - Access controlling
● When post a message (Write)
– Generate ACL Id
64. 64
Rowkey design - Access controlling
● When post a message (Write)
– Generate ACL Id
– Put ACL Id to message, and reader's ACLs
65. 65
Rowkey design - Access controlling
● When post a message (Write)
– Generate ACL Id
– Put ACL Id to message, and reader's ACLs
● When read my messages (Read)
66. 66
Rowkey design - Access controlling
● When post a message (Write)
– Generate ACL Id
– Put ACL Id to message, and reader's ACLs
● When read my messages (Read)
– Scan my ACLs, and all messages
67. 67
Rowkey design - Access controlling
● When post a message (Write)
– Generate ACL Id
– Put ACL Id to message, and reader's ACLs
● When read my messages (Read)
– Scan my ACLs, and all messages
– If my ACLs contains message's ACL Id, can SHOW it
70. 70
Rowkey design - Access controlling
ACL hash hash(A, B, K)+C+R
ACL Id AI0070AD
Message Id ME00000024AC
Title Announce
Content We are hiring
ACL Id AI0070AD
Write
79. 79
Rowkey design - Statistics
● Variety of types
– e.g., Likes, Comments, Registrations
● By unit
– i.e., hourly,daily,weekly,monthly,yearly
80. 80
Rowkey design - Statistics
● Variety of types
– e.g., Likes, Comments, Registrations
● By unit
– i.e., hourly,daily,weekly,monthly,yearly
● By user
82. 82
Rowkey design - Statistics
● Sum counts from 2014/9/7 to 2014/9/20 group by
user or counting type
Unit+Time Base+User Id+Type D+201409+AAA+Like
02 20
08 52
09 41
... ...
20 55
83. 83
Rowkey design - Statistics
● Sum counts from 2014/9/7 to 2014/9/20 group by
user or counting type
Unit+Time Base+User Id+Type D+201409+AAA+Like
02 20
08 52
09 41
... ...
20 55
84. 84
Rowkey design - Statistics
● Sum AAA's counts from 2014/9/7 to 2014/9/20
group by counting type
Unit+Time Base+User Id+Type D+201409+AAA+Like
02 20
08 52
09 41
... ...
20 55
85. 85
Rowkey design - Statistics
● Sum AAA's like counts from 2014/9/7 to 2014/9/20
Unit+Time Base+User Id+Type D+201409+AAA+Like
02 20
08 52
09 41
... ...
20 55
102. 102
No. 2 ONE table, MULTI domains
● In RDBMS
–
–
–
● In NoSQL
–
–
–
103. 103
No. 2 ONE table, MULTI domains
● In RDBMS (at design time)
–
–
–
● In NoSQL
–
–
–
104. 104
No. 2 ONE table, MULTI domains
● In RDBMS (at design time)
–
–
–
● In NoSQL (at runtime)
–
–
–
105. 105
No. 2 ONE table, MULTI domains
● In RDBMS (at design time)
– Primary key affects only one column
–
–
● In NoSQL (at runtime)
–
–
–
106. 106
No. 2 ONE table, MULTI domains
● In RDBMS (at design time)
– Primary key affects only one column
–
–
● In NoSQL (at runtime)
– Rowkey always changes
–
–
107. 107
No. 2 ONE table, MULTI domains
● In RDBMS (at design time)
– Primary key affects only one column
– Schema is fixed
–
● In NoSQL (at runtime)
– Rowkey always changes
–
–
108. 108
No. 2 ONE table, MULTI domains
● In RDBMS (at design time)
– Primary key affects only one column
– Schema is fixed
–
● In NoSQL (at runtime)
– Rowkey always changes
– Schema always changes
–
109. 109
No. 2 ONE table, MULTI domains
● In RDBMS (at design time)
– Primary key affects only one column
– Schema is fixed
– DAO serves one domain
● In NoSQL (at runtime)
– Rowkey always changes
– Schema always changes
–
110. 110
No. 2 ONE table, MULTI domains
● In RDBMS (at design time)
– Primary key affects only one column
– Schema is fixed
– DAO serves one domain
● In NoSQL (at runtime)
– Rowkey always changes
– Schema always changes
– DAO serves many domains
111. 111
No. 2 ONE table, MULTI domains
User Id ID0000001A3B
Access Token d66e3b70-3666-11e4-8c21-0800200c9a66
Expired Time 1410077636
112. 112
No. 2 ONE table, MULTI domains
User Id ID0000001A3B
Access Token d66e3b70-3666-11e4-8c21-0800200c9a66
Expired Time 1410077636
User Id+ACL Id ID0000001A3B+AI0070AD
Create 1
Read 1
Update 0
Delete 0
113. 113
No. 2 ONE table, MULTI domains
User Id ID0000001A3B
Access Token d66e3b70-3666-11e4-8c21-0800200c9a66
Expired Time 1410077636
User Id+ACL Id ID0000001A3B+AI0070AD
Create 1
Read 1
Update 0
Delete 0
114. 114
No. 2 ONE table, MULTI domains
● A DAO maps to a domain in RDBMS
115. 115
No. 2 ONE table, MULTI domains
● A DAO maps to a domain in RDBMS
DB DAO Domain A
116. 116
No. 2 ONE table, MULTI domains
● A DAO maps to multiple domains in NoSQL
117. 117
No. 2 ONE table, MULTI domains
● A DAO maps to multiple domains in NoSQL
DB DAO
Domain A1
Domain A2
Domain A3
118. 118
No. 2 ONE table, MULTI domains
● A DAO maps to multiple domains in NoSQL
● Build a middle layer to translate multiple domains
DB DAO
Domain A1
Domain A2
Domain A3
119. 119
No. 2 ONE table, MULTI domains
● A DAO maps to multiple domains in NoSQL
● Build a middle layer to translate multiple domains
DB DAO
Domain A1
Domain A2
Domain A3
Schema
120. 120
No. 2 ONE table, MULTI domains
● A DAO maps to multiple domains in NoSQL
● Build a middle layer to translate multiple domains
DB DAO
Domain A1
Domain A2
Domain A3
Schema
121. 121
No. 2 ONE table, MULTI domains
private String checkDomainType(Result result) {
if (result.isEmpty()) {
return null;
} else {
String rowkey = Bytes.toString(result.getRow());
String[] splitKey = rowkey.split(DIVIDER);
if (splitKey.length == 1) {
}
}
}
122. 122
No. 2 ONE table, MULTI domains
private String checkDomainType(Result result) {
if (result.isEmpty()) {
return null;
} else {
String rowkey = Bytes.toString(result.getRow());
String[] splitKey = rowkey.split(DIVIDER);
if (splitKey.length == 1) {
return DOMAIN_TYPE_VALIDATION1;
}
}
}
123. 123
No. 2 ONE table, MULTI domains
private String checkDomainType(Result result) {
if (result.isEmpty()) {
return null;
} else {
String rowkey = Bytes.toString(result.getRow());
String[] splitKey = rowkey.split(DIVIDER);
if (splitKey.length == 1) {
return DOMAIN_TYPE_VALIDATION1;
} else if (splitKey.length == 2) {
return DOMAIN_TYPE_VALIDATION2;
}
}
}
134. 134
API Blueprint - Introduction
● Web API Language
● Pure Markdown
● Design for Humans
135. 135
API Blueprint - Introduction
● Web API Language
● Pure Markdown
● Design for Humans
● Understandable by Machines
136. 136
API Blueprint - Introduction
● Web API Language
● Pure Markdown
● Design for Humans
● Understandable by Machines
● Powerful Tooling
137. 137
API Blueprint - Introduction
● Web API Language
● Pure Markdown
● Design for Humans
● Understandable by Machines
● Powerful Tooling
● Easy Lifecycle