Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
MySQL Sharding
vitess
research report
SWAT-ARCH
potian
20150202
DB simple expand to clusters
ò  服务化之后,DB系统还有哪些风险点
ò  大促:大流量读写操作,高可用高性能冲刺GMV
ò  非大促:日积月累数据量超过磁盘上限
Pain points
ò  DataSize Utilization
ò  QPS & TPS
ò  UpTime Crash recovery
ò  Client Connections
ò  Sharding: Complex ...
vitess project introduce
ò  vitesse is a sports CAR
ò  youtube/google内部2011+大规模使用vtocc
ò  独特点: DB多角色划分OLTP/OLAP/BACKUP
...
vitess components block diagram
vitess query life
vttablet modules
vitess core functions
ò  zk存储分片元数据信息(基于range分区)
ò  MySQL连接池(vttablet代理实现)
ò  Row Cache与复用(vttablet内部集成memcache实现)
ò  F...
vitess core routines
ò  Sharding and resharding
ò  Distribition lock service and topology http server
ò  Failover and s...
vitess dependencies
ò  MariaDB-10/MySQL-5.6 GTID/gMySQL-5.1 branch
ò  zookeeper
ò  Go1.2
ò  Python 2.7
ò  Other misc ...
vitess in production
ò  Youtube since 2011 service all mysql traffic
ò  10K~40K connections
ò  Vast increase in MySQL s...
vitess system analyze
ò  劣势
ò  项目庞大,部署复杂(但所有sharding系统都有这个问题)
ò  代码量大,成熟度还不够,新功能开发时间<2years
ò  目前无大规模部署先例,据称有不少坑
ò  G...
Upcoming SlideShare
Loading in …5
×

Vitess调研报告

2,016 views

Published on

mysql sharding solution vitess research report

Published in: Technology
  • Be the first to comment

Vitess调研报告

  1. 1. MySQL Sharding vitess research report SWAT-ARCH potian 20150202
  2. 2. DB simple expand to clusters ò  服务化之后,DB系统还有哪些风险点 ò  大促:大流量读写操作,高可用高性能冲刺GMV ò  非大促:日积月累数据量超过磁盘上限
  3. 3. Pain points ò  DataSize Utilization ò  QPS & TPS ò  UpTime Crash recovery ò  Client Connections ò  Sharding: Complex client/Resharding/Track/Failover/ Latency/Stats ò  OLAP/BACKUP ò  Muitiple logical databases ò  Lookup database
  4. 4. vitess project introduce ò  vitesse is a sports CAR ò  youtube/google内部2011+大规模使用vtocc ò  独特点: DB多角色划分OLTP/OLAP/BACKUP ò  暂未解决 OLTP: 分布式事务/跨分片的INDEX与AGGREGATIONS/跨DB的JOIN OLAP: MapReduce/resumable streaming query
  5. 5. vitess components block diagram
  6. 6. vitess query life
  7. 7. vttablet modules
  8. 8. vitess core functions ò  zk存储分片元数据信息(基于range分区) ò  MySQL连接池(vttablet代理实现) ò  Row Cache与复用(vttablet内部集成memcache实现) ò  Failover(vtgate与zk通信实现) ò  Sharding & resharding(vtgate与zk), vertical sharding? ò  SQL parser/rewrite and limit/route/result aggregation(vtgate) ò  Vtctl/vtctld 分片管理与schema管理工具 ò  Sql blacklist/Sql killer/限流/主从复制加速/olap引流与过滤 ò  负载均衡/跟踪sharding状态信息/失败重试/0宕机时间重启 ò  Client语言支持 go/python/java BSON协议/gorpc
  9. 9. vitess core routines ò  Sharding and resharding ò  Distribition lock service and topology http server ò  Failover and sql route ò  Sql lifetime
  10. 10. vitess dependencies ò  MariaDB-10/MySQL-5.6 GTID/gMySQL-5.1 branch ò  zookeeper ò  Go1.2 ò  Python 2.7 ò  Other misc linux tools,prefer Ubuntu10/Fedora… ò  Challenges: Ø  Started in 2010 Q1 Ø  rpc performance Ø  cgo calls Ø  garbage collector Ø  network I/O Ø  runtime scheduler Ø  memory leaks
  11. 11. vitess in production ò  Youtube since 2011 service all mysql traffic ò  10K~40K connections ò  Vast increase in MySQL serving capacity ò  Config, logging, stats & monitoring ò  Rowcache getting ready to launch
  12. 12. vitess system analyze ò  劣势 ò  项目庞大,部署复杂(但所有sharding系统都有这个问题) ò  代码量大,成熟度还不够,新功能开发时间<2years ò  目前无大规模部署先例,据称有不少坑 ò  Golang的gc不确定性能如何 ò  优势 ò  Golang实现简单快捷易上手,公司也可因此做go语言技术储备 ò  代码完全开源,技术支持服务较好,DevOps Team有9persons ò  功能齐全,该有的都有,暂不需要的也有 ò  权衡要素:自研代价,工期考量,性能与稳定性质量保障 ò  与其他方案比较(主要是DDB,TDDL)

×