标题:
Architecture and Practice for DAL (5) Data Sharding
Architecture and Practice for Data Access Layer (5) Data Sharding
联动优势数据访问层DAL架构和实践之五:分片数据分片
说明:
How to implement a dalet to access sharding databases.
和已有DAL软件(如许超前DAL手机之家、陈思儒Amoeba/贺贤懋Cobar等)不一样,在前端访问方式的选择上,抛弃JDBC方式,而是为同一个dalet数据服务,同时提供自定义TCP长连接和HTTP长连接两种接口。
因而通过抛弃JDBC可以获得多方面的好处——
1)可减少S端协议解析和查询分析的开销;
2)也简化C端编程。
3)后端存储就不再限于RDB了,而可以是任意NOSQL、文件、缓存、甚至是Tuxedo等在线服务。
4)可以实现无状态了,更容易横向扩展。
5)从接口上就可消除join等关键字的误用,避免引起服务端负担过重。
The document is a collection of images and text on the topic of data visualization. It discusses the history of visualization tools like pie charts and scatterplot matrices. It also covers modern visualization techniques for different data types such as trees, networks, text, and trajectories. Challenges in the field like dealing with large datasets and visualizing on mobile devices are mentioned. Applications discussed include sensemaking for intelligence analysis, detecting financial crimes, understanding social networks, and visualizing seismic and traffic data.
标题:
Architecture and Practice for DAL (5) Data Sharding
Architecture and Practice for Data Access Layer (5) Data Sharding
联动优势数据访问层DAL架构和实践之五:分片数据分片
说明:
How to implement a dalet to access sharding databases.
和已有DAL软件(如许超前DAL手机之家、陈思儒Amoeba/贺贤懋Cobar等)不一样,在前端访问方式的选择上,抛弃JDBC方式,而是为同一个dalet数据服务,同时提供自定义TCP长连接和HTTP长连接两种接口。
因而通过抛弃JDBC可以获得多方面的好处——
1)可减少S端协议解析和查询分析的开销;
2)也简化C端编程。
3)后端存储就不再限于RDB了,而可以是任意NOSQL、文件、缓存、甚至是Tuxedo等在线服务。
4)可以实现无状态了,更容易横向扩展。
5)从接口上就可消除join等关键字的误用,避免引起服务端负担过重。
The document is a collection of images and text on the topic of data visualization. It discusses the history of visualization tools like pie charts and scatterplot matrices. It also covers modern visualization techniques for different data types such as trees, networks, text, and trajectories. Challenges in the field like dealing with large datasets and visualizing on mobile devices are mentioned. Applications discussed include sensemaking for intelligence analysis, detecting financial crimes, understanding social networks, and visualizing seismic and traffic data.
This document appears to contain charts showing smartphone operating system market share over time and in a particular quarter. The line chart shows market share percentages for Symbian, Windows Mobile, Android and iPhone between 2010 and 2011, with Android increasing dramatically. The pie chart for a single quarter breaks down market share as 35% Android, 29% Symbian, 11% iPhone and smaller shares for others.
4. 系统架构
客 户 机(Client) 数 据 流
应用程序 控 制 流
C/P ython /
P H P/J a v a/...
客 户 机(Client)
J a v a程 序
查 询 服 务 器(Q S) D D B JD BC 管理服务器 D BA
QuryServer D BI Ma ste r 管理工具
数 据 库 节 点(D B N) 数 据 库 节 点(D B N) 数 据 库 节 点(D B N)
M yS Q L/ M yS Q L/ M yS Q L/
O ra c le O ra c le O ra c le
5. Sharding实现
Sharding实现
• 均衡字段:用来定位记录所在DBN的表字段。
• 均衡策略:均衡函数、桶、存储映射表。
• 表 --> 均衡策略:多对一
哈希函数 存储映射表
均衡函数
(固 定) 桶
哈 希 表 (可 调 整)
逻 辑I D
均衡字段值 H SD B1N1
N
S NB 2
D N2
负载信息
6. 查询处理流程
SELECT * FROM T 查询结果
WHERE a = 1 语 法
ORDER BY b
LIMIT 100 OFFSET 1 0 0
解 析
LIMIT 1 0 0
存储映射表 LIM IT OFFSET 1 0 0
发送给后台的
T, a=1 查 询 计 划 S Q L 句,可 能
L语
计 划 MERGE
ORDER BY b 与 原 语 句 不 同
生 成 SOR T
S N1, S N2
SELECT * F R OM T SELECT * F R OM T
WHERE a = 1 WHERE a = 1
ORDER BY b LIMIT 2 0 0 ORDER BY b LIMIT 2 0 0
根 据“存 储 映 射 表” 计 划
与 查 询 条 件,可 决 定 执 行 各后台数据库查询结
相关存储节点 果,按b排 好 序
b
D B N1
SN 1 D B N2
SN 2
10. 读写分离
• 支持对Master和各Slave节点的读操作设置权重
• 限制从延迟过大的Slave上读取数据
• 通过hint指定select语句的数据来源和延迟限制
D BI
– MasterOnly:只在Master上执行(默认)
– SlaveOnly:只在Slave上执行
Write
Read
R e
d
R ea
– SlavePerfer:优先在Slave上执行
ad
– LoadBalance:根据权重选择节点执行
Master
/*LOADBALANCE(TYPE=slaveonly,DELAY=60)*/ select …
S la ve1 S la ve2