2010 ChinaDatabaseCon "Intorduction to Distributed Infrastructure"

667 views

Published on

Published in: Technology
1 Comment
1 Like
Statistics
Notes
No Downloads
Views
Total views
667
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
0
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

2010 ChinaDatabaseCon "Intorduction to Distributed Infrastructure"

  1. 1. Intorduction to Distributed Infrastructure Intorduction to Distributed Infrastructure Guang Feng dudu@hunantv.com April 29, 2010
  2. 2. Intorduction to Distributed Infrastructure Topic 当今分布式技术主要有以下几个研究方向
  3. 3. Intorduction to Distributed Infrastructure Topic 当今分布式技术主要有以下几个研究方向 分布式文件系统
  4. 4. Intorduction to Distributed Infrastructure Topic 当今分布式技术主要有以下几个研究方向 分布式文件系统 分布式计算框架
  5. 5. Intorduction to Distributed Infrastructure Topic 当今分布式技术主要有以下几个研究方向 分布式文件系统 分布式计算框架 分布式存储引擎
  6. 6. Intorduction to Distributed Infrastructure Topic 当今分布式技术主要有以下几个研究方向 分布式文件系统 分布式计算框架 分布式存储引擎 数据库会议的一些个人思考
  7. 7. Intorduction to Distributed Infrastructure Topic 当今分布式技术主要有以下几个研究方向 分布式文件系统 分布式计算框架 分布式存储引擎 数据库会议的一些个人思考 Language Tower
  8. 8. Intorduction to Distributed Infrastructure Topic 当今分布式技术主要有以下几个研究方向 分布式文件系统 分布式计算框架 分布式存储引擎 数据库会议的一些个人思考 Language Tower 数据库性能监控与分析
  9. 9. Intorduction to Distributed Infrastructure Topic 当今分布式技术主要有以下几个研究方向 分布式文件系统 分布式计算框架 分布式存储引擎 数据库会议的一些个人思考 Language Tower 数据库性能监控与分析 DBA团队建设与职责
  10. 10. Intorduction to Distributed Infrastructure Distributed File System 解决的问题
  11. 11. Intorduction to Distributed Infrastructure Distributed File System 解决的问题 大容量文件存储
  12. 12. Intorduction to Distributed Infrastructure Distributed File System 解决的问题 大容量文件存储 本地文件系统的I/O瓶颈
  13. 13. Intorduction to Distributed Infrastructure Distributed File System 解决的问题 大容量文件存储 本地文件系统的I/O瓶颈 灾难备份
  14. 14. Intorduction to Distributed Infrastructure Distributed File System 解决的问题 大容量文件存储 本地文件系统的I/O瓶颈 灾难备份 主要产品
  15. 15. Intorduction to Distributed Infrastructure Distributed File System 解决的问题 大容量文件存储 本地文件系统的I/O瓶颈 灾难备份 主要产品 面向大文件存储 Google GFS like...
  16. 16. Intorduction to Distributed Infrastructure Distributed File System 解决的问题 大容量文件存储 本地文件系统的I/O瓶颈 灾难备份 主要产品 面向大文件存储 Google GFS like... 面向小文件存储 MogileFS
  17. 17. Intorduction to Distributed Infrastructure Distributed Computing Framework 概念
  18. 18. Intorduction to Distributed Infrastructure Distributed Computing Framework 概念 MapReduce
  19. 19. Intorduction to Distributed Infrastructure Distributed Computing Framework 概念 MapReduce 并行处理的是数据,不是任务(计算能力)
  20. 20. Intorduction to Distributed Infrastructure Distributed Computing Framework 概念 MapReduce 并行处理的是数据,不是任务(计算能力) 产品
  21. 21. Intorduction to Distributed Infrastructure Distributed Computing Framework 概念 MapReduce 并行处理的是数据,不是任务(计算能力) 产品 Hadoop
  22. 22. Intorduction to Distributed Infrastructure Distributed Computing Framework 概念 MapReduce 并行处理的是数据,不是任务(计算能力) 产品 Hadoop *Gearman
  23. 23. Intorduction to Distributed Infrastructure Distributed Computing Framework Gearman Figure: Gearman Arch
  24. 24. Intorduction to Distributed Infrastructure Distributed Computing Framework Gearman work in parallel Figure: Gearman Arch
  25. 25. Intorduction to Distributed Infrastructure Distributed Computing Framework Gearman work in parallel background task Figure: Gearman Arch
  26. 26. Intorduction to Distributed Infrastructure Distributed Storage Engine Colunmn-Based Storage
  27. 27. Intorduction to Distributed Infrastructure Distributed Storage Engine Colunmn-Based Storage Google BigTable
  28. 28. Intorduction to Distributed Infrastructure Distributed Storage Engine Colunmn-Based Storage Google BigTable Amazon Dynamo
  29. 29. Intorduction to Distributed Infrastructure Distributed Storage Engine Colunmn-Based Storage Google BigTable Amazon Dynamo Not Only SQL
  30. 30. Intorduction to Distributed Infrastructure Distributed Storage Engine Colunmn-Based Storage Google BigTable Amazon Dynamo Not Only SQL Tokyo Cabinet
  31. 31. Intorduction to Distributed Infrastructure Distributed Storage Engine Colunmn-Based Storage Google BigTable Amazon Dynamo Not Only SQL Tokyo Cabinet Relational Database
  32. 32. Intorduction to Distributed Infrastructure Distributed Storage Engine Colunmn-Based Storage Google BigTable Amazon Dynamo Not Only SQL Tokyo Cabinet Relational Database on-Proxy
  33. 33. Intorduction to Distributed Infrastructure Distributed Storage Engine Colunmn-Based Storage Google BigTable Amazon Dynamo Not Only SQL Tokyo Cabinet Relational Database on-Proxy Cluster
  34. 34. Intorduction to Distributed Infrastructure Colunmn-Based Storage 优势
  35. 35. Intorduction to Distributed Infrastructure Colunmn-Based Storage 优势 动态增加存储列,水平切分伸缩性强
  36. 36. Intorduction to Distributed Infrastructure Colunmn-Based Storage 优势 动态增加存储列,水平切分伸缩性强 每单元(Tracker)中记录统一数据的多个版本
  37. 37. Intorduction to Distributed Infrastructure Colunmn-Based Storage 优势 动态增加存储列,水平切分伸缩性强 每单元(Tracker)中记录统一数据的多个版本 数据聚集性强
  38. 38. Intorduction to Distributed Infrastructure Colunmn-Based Storage 优势 动态增加存储列,水平切分伸缩性强 每单元(Tracker)中记录统一数据的多个版本 数据聚集性强 缺点
  39. 39. Intorduction to Distributed Infrastructure Colunmn-Based Storage 优势 动态增加存储列,水平切分伸缩性强 每单元(Tracker)中记录统一数据的多个版本 数据聚集性强 缺点 不适合频繁CRUD
  40. 40. Intorduction to Distributed Infrastructure Colunmn-Based Storage 优势 动态增加存储列,水平切分伸缩性强 每单元(Tracker)中记录统一数据的多个版本 数据聚集性强 缺点 不适合频繁CRUD 不支持条件查询
  41. 41. Intorduction to Distributed Infrastructure Colunmn-Based Storage 优势 动态增加存储列,水平切分伸缩性强 每单元(Tracker)中记录统一数据的多个版本 数据聚集性强 缺点 不适合频繁CRUD 不支持条件查询 不适合OLTP(On-Line Analytical Processing)
  42. 42. Intorduction to Distributed Infrastructure NoSQL:Tokyo Cabinet 特性
  43. 43. Intorduction to Distributed Infrastructure NoSQL:Tokyo Cabinet 特性 键值对存储
  44. 44. Intorduction to Distributed Infrastructure NoSQL:Tokyo Cabinet 特性 键值对存储 insert 0.4s/1M records & search 0.33s/1M records
  45. 45. Intorduction to Distributed Infrastructure NoSQL:Tokyo Cabinet 特性 键值对存储 insert 0.4s/1M records & search 0.33s/1M records 支持丰富的语言调用(HTTP协议)
  46. 46. Intorduction to Distributed Infrastructure NoSQL:Tokyo Cabinet 特性 键值对存储 insert 0.4s/1M records & search 0.33s/1M records 支持丰富的语言调用(HTTP协议) 不足
  47. 47. Intorduction to Distributed Infrastructure NoSQL:Tokyo Cabinet 特性 键值对存储 insert 0.4s/1M records & search 0.33s/1M records 支持丰富的语言调用(HTTP协议) 不足 上亿的数据存储后,性能下降很快
  48. 48. Intorduction to Distributed Infrastructure NoSQL:Tokyo Cabinet 特性 键值对存储 insert 0.4s/1M records & search 0.33s/1M records 支持丰富的语言调用(HTTP协议) 不足 上亿的数据存储后,性能下降很快 很难扩展到多机
  49. 49. Intorduction to Distributed Infrastructure RDB:Amoeba Proxy Architecture(代理风格)
  50. 50. Intorduction to Distributed Infrastructure RDB:Amoeba Proxy Architecture(代理风格) 特性
  51. 51. Intorduction to Distributed Infrastructure RDB:Amoeba Proxy Architecture(代理风格) 特性 提供规则数据切分
  52. 52. Intorduction to Distributed Infrastructure RDB:Amoeba Proxy Architecture(代理风格) 特性 提供规则数据切分 降低db 与客户端的连接数
  53. 53. Intorduction to Distributed Infrastructure RDB:Amoeba Proxy Architecture(代理风格) 特性 提供规则数据切分 降低db 与客户端的连接数 读写分离
  54. 54. Intorduction to Distributed Infrastructure RDB:Amoeba Proxy Architecture(代理风格) 特性 提供规则数据切分 降低db 与客户端的连接数 读写分离 未来打算支持异构数据库
  55. 55. Intorduction to Distributed Infrastructure RDB:EnterpriseDB Cluster 特性
  56. 56. Intorduction to Distributed Infrastructure RDB:EnterpriseDB Cluster 特性 内置连接池,内存共享
  57. 57. Intorduction to Distributed Infrastructure RDB:EnterpriseDB Cluster 特性 内置连接池,内存共享 读请求负载均衡
  58. 58. Intorduction to Distributed Infrastructure RDB:EnterpriseDB Cluster 特性 内置连接池,内存共享 读请求负载均衡 所有节点并行查询
  59. 59. Intorduction to Distributed Infrastructure RDB:EnterpriseDB Cluster 特性 内置连接池,内存共享 读请求负载均衡 所有节点并行查询 查询缓存
  60. 60. Intorduction to Distributed Infrastructure Language Tower 分布式基础架构
  61. 61. Intorduction to Distributed Infrastructure Language Tower 分布式基础架构 语言无关的通讯协议
  62. 62. Intorduction to Distributed Infrastructure Language Tower 分布式基础架构 语言无关的通讯协议 动态脚本语言实现业务逻辑
  63. 63. Intorduction to Distributed Infrastructure Thinking... 数据库性能监控与分析
  64. 64. Intorduction to Distributed Infrastructure Thinking... 数据库性能监控与分析 结合主机负载与数据库指标进行监控
  65. 65. Intorduction to Distributed Infrastructure Thinking... 数据库性能监控与分析 结合主机负载与数据库指标进行监控 数据库日志(二进制日志、慢查询)分析
  66. 66. Intorduction to Distributed Infrastructure Thinking... 数据库性能监控与分析 结合主机负载与数据库指标进行监控 数据库日志(二进制日志、慢查询)分析 根据业务特点进行针对优化
  67. 67. Intorduction to Distributed Infrastructure Thinking... 数据库性能监控与分析 结合主机负载与数据库指标进行监控 数据库日志(二进制日志、慢查询)分析 根据业务特点进行针对优化 DBA团队建设与职责
  68. 68. Intorduction to Distributed Infrastructure Thinking... 数据库性能监控与分析 结合主机负载与数据库指标进行监控 数据库日志(二进制日志、慢查询)分析 根据业务特点进行针对优化 DBA团队建设与职责 日常维护数据
  69. 69. Intorduction to Distributed Infrastructure Thinking... 数据库性能监控与分析 结合主机负载与数据库指标进行监控 数据库日志(二进制日志、慢查询)分析 根据业务特点进行针对优化 DBA团队建设与职责 日常维护数据 Top Query的分析与解决
  70. 70. Intorduction to Distributed Infrastructure Thinking... 数据库性能监控与分析 结合主机负载与数据库指标进行监控 数据库日志(二进制日志、慢查询)分析 根据业务特点进行针对优化 DBA团队建设与职责 日常维护数据 Top Query的分析与解决 SQL由谁来写?
  71. 71. Intorduction to Distributed Infrastructure Thank you! Q&A

×