Sth About SSD

2,313 views

Published on

分享

Published in: Technology
  • Be the first to comment

Sth About SSD

  1. 1. Sth. About SSD @w维西 MySQL_DBA rezic7@gmail.com
  2. 2. Contents FusionIO Intel SSD+LSI Raid FIO+Flashcache安装备注 FIO性能测试 OS/Percona的参数设定 Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  3. 3. FusionIO-SSD_Raid Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  4. 4. FusionIO-SSD_Raid Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  5. 5. Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  6. 6. ≈FlashcacheWeek 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  7. 7. 2nd gen 6Gb/s Raid Controller 上限提升近3倍 Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  8. 8. FastPath 最适合OLTP FastPath的工作原理是通过MegaRAID控制卡的 Firmware来开创一条延迟非常低的I/O路径,以此来充 分优化SSD的性能。 ≈HBA? Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  9. 9. HBA Host Bus Adapter 在服务器和存储间进行I/O处理和物理连接的适配器,形成直接 的磁盘通道;减轻CPU在数据存储和检索任务的负载,提高 IO性能 几乎没有performance limitation, 性能基本是单盘性能x盘数; 而RAID性能则受RAID形式以及RAID卡本身的限制; Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  10. 10. HBA 一般HBA的弱点: 1. 基本不算真正意义的硬件raid,现在的硬件raid,物理磁盘上都会带 有一些ddf信息,而HBA组建的raid可能不带这些信息 2. raid的检验和纠错能力会比较弱 3. 不带cache,对随机写性能影响很大 性能测试表现突出的原因 挂上多块SSD后,随机读写的缺陷被填补上,但是数据效验这块还是短板,推 荐不用在严格生产系统中 淘宝Oceanbase收藏夹应用的当前架构 青岛机房 : (HBA+单320GB-SSD )*10台 流量60% 杭州机房 : (raid 10 + 10块SAS)*22台 流量 40% Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  11. 11. Intel SSD New Series——320 2011.Q1-Q2 Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  12. 12. 下一代SSD主控芯片? 2012年推出,最大支持5TB容量的SSD,随机写入 IOPS最高可达400K 目前SandForce SF-2000系列最高随机写入IOPS数据 大概在60K Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  13. 13. 4K IOPS 70kWeek 3 Rezic7@Gmail.com 8K IOPS 50k Tuesday, November 22, 2011
  14. 14. Fusion-IO结构 Fusion-IO,只能做软raid或者LVM 新测试机是双卡,可通过 sudo mdadm –D /dev/md0查看 Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  15. 15. Fusion-IO比其他SSD方案更好?Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  16. 16.  Fusion IO等PCIe卡在全随机写方面有一定的性能优 势,但其所标称的性能一般以优化配置情形下的数据 为准,如512Byte文件大小,做50% over-provision.另 一些厂商,则对典型workload做特别算法优化,比如 4kB文件大小性能最好,而2kB/8kB等则较差;建议针 对实际情形测试 Intel SSD in RAID模式,在read intensive workload或 者是读写混合(如3:1 R/W)下,其性能能达到甚至 远超过(如全随机读)主流的PCIe卡方案,同时具有 其他方面的优势,如:数据安全性(可以配置RAID), 断电保护,容量/性能性价比,可置换性 Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  17. 17. Contents FusionIO Intel SSD+LSI Raid FusionIO+Flashcache安装备注 FIO磁盘性能测试工具 OS/Percona的参数设定 Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  18. 18. Fsuion‐io ioDrive 安装指南 系统内存要求:ioDrive 需要系统提供足够的内存来保 存元数据,大部分情况下iodrive 对 于内存的最小要求是:320G ‐‐‐1.7G 640G ‐‐‐3.4G 系统类型版本Fusion‐io 只支持64 位的操作系统, 目前Fusion‐io ioDrive 不是可启动 OS 的设备。OS要安装在原有的硬 盘设备上。 Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  19. 19. IO driver驱动 命令行工具 Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  20. 20. FlashCache details Write-through & write-back 多模式 先进先出和LRU的快管理 需自行编译核心模块 也适用于ibdata1/ib_logs Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  21. 21. Fio+Flashcache安装TIPS 查看现有的设备状态sudo fio-status (fct0,fct1) sudo fdisk -l (fioa,fiob)sudo fdisk -l /dev/md0 确认文件系统 安装编译Flashcache1.编译内核tree mkdir -p ~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS} echo %_topdir %(echo $HOME)/rpmbuild > .rpmmacros 安装源码RPM后, SPECS下生成kernel.spec增加 %define _default_patch_fuzz 2 (某些新版本rpmbuild默认为0) Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  22. 22. Fio+Flashcache安装TIPS2.拷贝配置 cp /boot/config-2.6.32….x86_64 ~/rpmbuild/BUILD/kernel-/linux-/ cd ~/rpmbuild/BUILD/kernel-...el6/linux-...el6.x86_64/ make oldconfig && make prepare && make modules_prepare cp /usr/src/kernels/2.6…/Module.symvers ~/rpmbuild/BUILD/kernel-/linux-/3.编译 根据手册修改Makefile make KERNEL_TREE=~/rpmbuild/BUILD/kernel-/linux-…/4.加载验证 sudo insmod ./src/flashcache.ko sudo lsmod | grep flash5.搭建挂载flashcachesudo PATH=/sbin ./flashcache_create -b 4k -p back cachea /dev/md0 /dev/sda12 sudo mkfs.ext3 /dev/mapper/cachea Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  23. 23. Contents FusionIO Intel SSD+LSI Raid FusionIO+Flashcache安装备注 FIO磁盘性能测试工具 OS/Percona的参数设定 Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  24. 24. FIO编译及使用 sudo yum install libaio-devel make && sudo make install sudo fio -filename=innodb.dat -direct=1 - ioengine=psync -directory=/u01 -rw=randrw -bs=16k -size=5G -numjobs=32 -runtime=10 -group_reporting -name=mysqlio Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  25. 25. 参数介绍 readwrite=str, rw=str read #连续读 write #连续写 randread #随机读 randwrite #随机写 rw #混合连续读写 randrw #混合随机读写 rwmixread = 90 混合读写比例,参考ICDB 10:1 direct = 1 不使用io设备buffer directory=/u01 测试目录,用于指定磁盘 runtime=10 测试时长(秒) name=io_test 作业名 filename= innodb.dat 文件名 bs= 16k 块大小 bssplit=4k/40:8k/60 可分比例 Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  26. 26. 参数介绍:IO Engines fio支持: sync psync libaio posixaio mmap splice syslet-rw sg null net netsplice cpuio guasi MySQL先前是自写的 Fake AIO(xiyu) innodb走sync较多,即使启用aio, 做过统计真正 走aio的5%不到(chuba) InnoDB Plugin 1.1 supports native AIO on Linux http://blogs.innodb.com/wp/2010/04/innodb-performance-aio-linux/ 建议每次测试前低格fioa,fiob;重做flashcache Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  27. 27. 支持参数文件[global] [innodb-log] [innodb-rdrw] stonewalldirect=1 directory=/u02 stonewallsize=2G bsrange=512-2048 directory=/u01bs=16k ioengine=sync bs=16K rw=writetimeout=30 numjobs=64 ioengine=psync rw=randrw[bin-log] [innodb-rdread] rwmixread=90 stonewalldirectory=/u02 size=200G directory=/u01rw=write numjobs=32 bs=16Kioengine=sync ioengine=psyncbsrange=512-1024 rw=randreadnumjobs=64 size=200G numjobs=32 Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  28. 28. Contents FusionIO Intel SSD+LSI Raid FusionIO+Flashcache安装备注 FIO磁盘性能测试工具 SSD+Percona的参数设定 Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  29. 29. CPU软中断 淘宝核心系统和Intel都出了新的patch echo 2 > /sys/block/sdX/queue/rq_affinity  在RHEL6 + Dell H700时,关闭rq-affinity, 小于7万iops,跑满一个core;打开后10万iops  RHEL6 + Dell H200 + HBA时,打开rq-affinity, 22.5万iops,打开严格模式可达26.5万iops Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  30. 30. Percona参数设定 – innodb_thread_concurrency=0 – innodb_read_ahead=0 – innodb_adaptive_flushing=1 – innodb_adaptive_checkpoint=keep_average – innodb_max_dirty_pages_pct=60 – innodb_flush_method=O_DIRECT – innodb_io_capacity=10000 – Innodb_flush_neighbor_pages=0 – innodb_read_io_threads=8 (16 for writes) – innodb_page_size = 4K | 8K | 16K(Use carefully) – innodb_flush_neighbor_pages= ON | OFF – innodb_log_block_size = 512 | 4096 – innodb_doublewrite_file – innodb_log_file_size > 4G FusionIO工程师在Percona New York展会上的分享Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011
  31. 31. Fusion-IO@Percona Live 2011 NYC Week 3 Rezic7@Gmail.com Tuesday, November 22, 2011

×